Known Issues

User Interface

Timecharts appear to misrepresent the time and size of an anomaly

Data is modeled and anomalies are detected according to bucketed time intervals. So, if we are analyzing data using a 10 min bucket and an anomaly occurs at 10:33, we will report that it occurred in the time bucket 10:30. This means that it occurred between 10:30:00 and 10:39:59.

When viewing the results, Splunk timecharts will apply additional time bucketing which automatically adjusts depending upon the number of items to plot. If the Splunk timechart applied a 1hr time bucketing strategy then the anomaly above would be labelled as occuring at 10:00. Also, if the timechart is displaying the average of a value, then a maximum peak may not be evident.

This is not wrong, it is just a practical aggregation of data. Please be aware that this can happen. If it does, then select a smaller time range over which to view the detailed results.

Creating an Insight takes some time

When manually creating an Insight, information is stored in the Splunk prelertresults index and the KV store. The command to create the Insight will wait until this information is available and can be read from Splunk. Depending upon your environment, there may be a short time to wait before the Insight creation is completed.

Dashboard panel displays “Waiting for data” message

On dashboards with multiple search panels, some panels may display a “Waiting for data” message indefinitely. If you inspect the search in question you can see that it has completed; however the dashboard panel is unable to access these results. Refreshing the panel will re-run the search and should now display the results.

This has been seen with Splunk 6.2.x and early 6.3.x builds, and is more likely to occur on resource constrained systems and on Splunk installations on Windows. The Splunk bug number is SPL-110384, and this is fixed in Splunk version 6.3.4.

If you observe dashboard panels displaying “Waiting for data” indefinitely it is recommended that you upgrade to at least version 6.3.4 of Splunk.

Results summary fails for fields containing backslashes or double quotes on Splunk versions prior to 6.2.3

Splunk 6.2.3 changed the interpretation of backslashes in the macros.conf configuration file. As a result, the results summary view does not correctly escape backslashes and double quotes when navigating between panels.

If the fields in your data contain backslashes or double quotes it is recommended that you upgrade to at least version 6.2.3 of Splunk.

Home page does not show Insight monitors that are running on historical data

Anomaly Detective App for Splunk 4.2 displays an embedded custom dashboard on its home page. Upon install, this dashboard contains four pre-configured panels. One of these panels shows a list of Insight Monitors that are currently running. These are Insight Monitors that are running continuously in real-time.

By design, this panel does not show Insight Monitors that are running on historical data, sometimes known as running a LookBack.

Anomaly Detection

Time functions are not supported in Evaluation Mode

time_of_day and time_of_week functions are not supported when using Evaluation Mode as the data is analyzed in reverse time order.

Time functions do not account for Daylight Savings changes

time_of_day and time_of_week functions are not aware of changes due to Daylight Savings. When this transition occurs, please expect to see anomalies raised. After a day or so, this will become the new learned behavior.

Evaluation Mode does not return any results

In Evaluation Mode, if bucketspan has not been explicitly specified, Prelert will attempt to calculate a sensible bucketspan based upon characteristics of the data. If this calculation fails, then the prelertautodetect command appears to complete successfully, however, no records are returned.

This will occur if there are less than 10,000 records in the input data. As a workaround, please specify a bucketspan. Using a smaller bucketspan is more likely to yield results, such as:

index=applicationlog | prelertautodetect bucketspan=300 count by errorcode

Insights

Anomaly scores in an Insight are lower than expected

Insights are created based upon their initial score. i.e. the score is only based on data that has been seen prior to the start time of the Insight. When looking at an Insight in the Insight View, the anomaly scores displayed have been re-normalized. i.e. they take more recent data into account. As such, when the decision to create the Insight was made, the anomaly scores would have been different. This effect is most evident during start up or after significant changes in state, when the models have had less time to learn.

Anomalies have not been added to an Insight eventhough they occur within the configured append window

For anomalies to be appended by an Insight Monitor, they must have had the potential to have been created as an Insight in their own right. This is by design.

So, for example, an influencer based Insight Monitor is running continuously with an append window set to 1 day and a maximum Insight span of 1 week. The influencer field is username. Insight-1 is created for user Bob who has been committing suspicious login activity. This contains multiple anomalies from Windows security event logs. Four hours later, Insight-2 is identified for the same user, as Bob has been exfiltrating unusually large amounts of data to rare domains.

Rather than create Insight-2 separately, its anomalies will be appended to Insight-1 as it occurs within 1 day of Bob’s previous insight and the total insight span is still less than 1 week.

Configuration

Unable to create a new Anomaly Search due to a duplicate name

When you delete an Anomaly Search, its results are deleted from the Splunk prelertresults index and the KV store. If this deletion did not complete successfully, then it is possible that results remain in the prelertresults index or KV store. If this happens, an error will occur if attempting to create an Anomaly Search with the same name.

To workaround, please use an alternative name. To resolve this issue, the results can be manually deleted. This would require administrative access and should be done with guidance from support@prelert.com.

This page

Browse

You are here