Flush Data

Description

Flush all buffers, store any pending results and block until they are ready to query. Programmatically it is possible to follow a call to flush with a call to get results and be certain that all possible results will be available for the data uploaded prior to the flush.

When requesting a flush it is possible to specify that interim results for the most recent bucket be calculated. Alternatively, it is possible to state that all data up to a given time has been received, and final results up to that time should be created.

Flushing a closed job has no effect.

Definition

http://localhost:8080/engine/v2/data/<jobId>/flush
jobId:The identifier of the job to be flushed.

Parameters

calcInterim:Calculates interim results for the most recent bucket. If a latency window is specified, then calculates interim results for all buckets within the latency window. May be used in conjunction with start and end to calculate interim results for a particular bucket or range of buckets. Defaults to false if not specified.
start & end:When used in conjunction with calcInterim=true, specifies the range of buckets on which to calculate interim results. If only start is specified, then will calculate interim results for that individual bucket.
advanceTime:Advanced configuration option. Setting this tells the Engine API that no data prior to advanceTime is expected. This cannot be undone. Results are finalized up to this time, and any subsequent input data with a timestamp earlier than advanceTime will be ignored. The parameter value is expected to be a ISO-8601 timestamp.

Method

POST

Returns

Status code 200 (OK). The flush request does not return until all buffers are flushed and results committed to the data store.

Errors

See the Error Codes documentation for the full list of errors that may be returned by the API.

EXAMPLE RESPONSE

For example, using the cURL command line client, the command to flush the job data buffers is:

curl -X POST http://localhost:8080/engine/v2/data/<jobId>/flush

If successful the API acknowledges the flush action:

{
  "acknowledged":true
}

NOTES

The job should not be explicitly flushed unless there is a need to query results immediately after doing so. Regularly flushing a job unnecessarily will degrade performance. Closing a job also ensures all buffers are flushed and results committed to the data store. Thus there is no need to flush a job immediately before closing it unless you want to trigger calculation of interim results for the final bucket of data that was uploaded.

To view interim results for records or buckets using the Results Endpoint, the ?includeInterim=true query parameter is required.