API Introduction

IT is a Big Data world. Performance and security data from mixed cloud and premise based infrastructures, complex applications and agile DevOps environments result in an overwhelming volume of data. Traditional monitoring, troubleshooting and forensic approaches just won’t scale.

To find problems before they have user impact across large data sets, an alternative approach is needed. Highly scalable, unsupervised machine learning algorithms can be used to detect and diagnose these developing performance issues and security threats before they have an impact.

Prelert’s Behavioral Analytics uses artificial intelligence in the form of unsupervised machine learning and advanced computational mathematics to process huge volumes of streaming data. It automatically learns normal behavior patterns represented by the data, then identifies and cross-correlates the anomalies.

Prelert Engine API

What are anomalies

Anomalies are items, events or observations that do not conform to an expected pattern or other items in a dataset. For example:

  • Significant deviations in time series metrics
  • Significant changes in message rates
  • Rare or unusual log messages
  • Unusual user behavior compared to typical user’s behavior

If the expected or normal patterns of behavior are known, anomalies can be identified.

Who is the Engine API for

The Engine API is ideal for anyone who needs a simple and effective method to gain insight into large amounts of varied data.

You don’t need to be a data or a rocket scientist to use the probabilistic mathematics and adaptive statistical modeling in the Engine API. We have done the hard work for you.

We expect users of the Engine API to have a programming or a scripting background. You’ll have already solved the problem of how to _store_ your data and just want a better way to _analyze_ it.

Whilst anomaly detection is applicable to any data, this document provides examples that focus on machine data. People with knowledge of (and responsibility for) managing the performance or security of complex IT systems will benefit most from the tutorial examples.

Key Features

  • Anomaly detection - finds data anomalies using probabilistic mathematics and adaptive statistical modeling
  • Dashboard - installs with an open source Kibana dashboard that allows a graphical view of results
  • Highly scalable - able to work on big datasets, as well as small, routinely processing millions of data points in real-time
  • Highly optimized - able to process ~3TB across 30,000 metrics per day on 1 core of a laptop
  • Easy to consume - simply feed the data in, anomalies are identified, highlighted and summarized
  • Simple to implement and maintain - self-learning so no rules, models or thresholds to configure and maintain, provides rapid insight without continued configuration
  • Easy integration - use any language to call the RESTful Engine API

Getting started with the API

After downloading the API and installing, we recommend that you work through the tutorials.

Please also visit our GitHub site which contains a variety of examples of how to use the API.

Fair use terms

Behavioral Analytics for the Elastic Stack and the Engine API, as well as examples and code samples in this documentation, on GitHub or on the Prelert website, are made available free of charge, on a limited function basis, for evaluation, development and test purposes only. Please upgrade to a full license for use in production.

We kindly request attribution where significant re-use of code has occurred, or if analytical results attributable to Prelert are published or presented at a conference or where publicity in some form is being sought.

If in doubt as to whether or not your usage requires permission, please contact support@prelert.com.

Our full license terms are available to view here.

About Prelert ®

Prelert is the anomaly detection company. Its automated behavioral analytics make it easy for users and developers to uncover real-time insights into the operational opportunities and risks hidden in massive data sets. By using unsupervised machine learning technology, Prelert enables non-data scientists to go beyond the limits of search to quickly derive value from their organization’s data. To learn more, please visit prelert.com or follow @prelert.