Preview Endpoint

The preview endpoint is provided as an aid to debugging complex transforms or chains of transforms. Using this endpoint, you can preview the output from the transforms for a sample-set of data without performing any of the analytics.

POST a sample of your data to the job’s preview endpoint exactly as you would with the data endpoint. The output of the transforms is returned, stripped of and fields not used in the analysis. The timestamp will be in Unix epoch format.

Example

In this sample data set the date and time fields need to be merged into a single field we can achieve this using the concat transform:

date,       time,     event type
2015-03-31, 08:00:00, logon event
2015-03-31, 08:00:27, logon event
2015-03-31, 08:00:31, directory service event
...

The full job configuration below defines a single detector to find rare occurrences of event type, the concat transform creates a new field datetime from date and time and the dataDescription is set to read that datetime field with the appropriate timeFormat.

{
    "analysisConfig" : {
        "bucketSpan":3600,
        "detectors" :[{"function":"rare","byFieldName":"event type"}]
    },
    "transforms":[
        {
            "transform" : "concat",
            "inputs" : ["date", "time"],
            "outputs" : "datetime"
        }
    ],
    "dataDescription" : {
        "fieldDelimiter":",",
        "timeField":"datetime",
        "timeFormat":"yyyy-MM-ddHH:mm:ss"
    }
}

To test the configuration we will pipe a few rows of data into cURL and POST it to the preview endpoint. The ‘-‘ option tells cURL to take stdin as the input.

echo "date,time,event type
2015-03-31,08:00:00,logon event
2015-03-31,08:00:27,logon event
2015-03-31,08:00:31,directory service event" | curl -X POST -T - "http://localhost:8080/engine/v2/preview/<jobId>"

The response contains the datetime field created by the concat transform which has been parsed and converted to an epoch value, and the event type field used by the rare detector. Only fields used in the analysis are previewed.

datetime,event type
1427785200,logon event
1427785227,logon event
1427785231,directory service event

The response is always plain text/csv even regardless of the data format. If the example data above was JSON formatted with the appropriate changes to the job’s dataDescription then the following:

echo '{"date" : "2015-03-31", "time" : "08:00:00", "event type" : "logon event"}
{"date" : "2015-03-31", "time" : "08:00:27", "event type" : "logon event"}
{"date" : "2015-03-31", "time" : "08:00:31", "event type" : "directory service event"}' | curl -X POST -T - "http://localhost:8080/engine/v2/preview/<jobId>"

would produce exactly the same preview.

datetime,event type
1427785200,logon event
1427785227,logon event
1427785231,directory service event