This blog post is the second of a two-part tutorial on using Sumo Logic’s LogReduce as a means of pattern recognition in machine data on the Sumo Logic platform. In the first blog post, we introduced LogReduce and what is necessary to adequately use this feature on your preferred machine data. This tutorial will discuss how we can interpret the results given by the LogReduce feature on the Sumo Logic platform.
In the first tutorial, we used Sumo Logic’s LogReduce feature to find patterns in the logs of the MacOS I use. We found a list of signatures representing groups of messages in my macOS system for a given time period.
What is visible now? How can we analyze these patterns using Sumo Logic?
The Signatures table
First, let’s have a look at two tabs—Messages and Signatures. Signatures are the patterns found in the messages as seen in the MacOS log.
The Signatures tab shows a table with several columns and icons. There is also a Signature column with Signature Details.
What do these details say?
I’m no expert on MacOS logs, but I see recurring items in the signatures related to Sumo Logic.
I notice the signatures start with the $DATE sign (intended to ignore timestamps occurring in the MacOS log—and replace them with the $DATE sign. This enables more readability).
The same applies to URLs found in the MacOS log. These are replaced with the $URL sign.
OK, but what about the wildcards (***) in the signatures?
These wildcards are applied because they represent fields that vary in the logs. Like the $DATE and $URL signs, they are used to develop the patterns and make them more readable.
Great—Now I can imagine how patterns are formed from the MacOS logs using the LogReduce feature. But what about the other columns (Select, Count, Relevance, Actions and Edit)? What do they represent?
In the Select column, you can select a checkbox and click the View Details button at the right.
Now Sumo Logic runs the LogReduce algorithm on the signatures with the details operator, and then displays the resulting sub-signatures, as seen below.
Notice that the query at the top has changed. Details and the signature code are added.
Next to this, the amount of sub-signatures is equal to 2,912, as seen in the Count column for the signature used. Clicking this number in the Count column gives the same result.
We have discussed the Select and Count columns, but what about the Relevance column?
The LogReduce Relevance column shows a numerical score for a signature, predicting which signatures could be most meaningful to a user. The value is calculated by using your history of feedback (Thumbs Up and Thumbs Down), and the instances when you’ve chosen to view signature details.
By default, LogReduce results are displayed in descending order of the Relevance value.
LogReduce uses the similarity of signature content (the words in a signature) to predict relevance for signatures. For example, if a user has promoted a number of signatures (clicking the Thumbs up icon) that contain the word “kernel,” then new signatures containing “kernel” will be scored higher. The Relevance value will be between 0 and 10, depending on the level of promotion.
This demotion/promotion brings us to the Actions column. We’ve covered the Thumbs icons, but what about the Split icon?
Splitting will make the results more granular: fewer wildcard asterisks will appear. Instead, specific values are included in the signatures.
This is illustrated below.
Notice the signatures have the same value for Relevance (9.53), which is logical, because it’s still the same signature, only split.
Finally, there’s the Edit button.
When clicking this button, a popup will appear showing the details of the signature. Here, you can edit the signature as seen in the Signature column. This way, you can add wildcards and so on as discussed above. (How to do this is part of another possible tutorial in the future.)
Wrap-up
This ends my tutorial on the analysis of signatures generated with the Sumo Logic LogReduce feature.
We’ve seen how we can interpret the results by using the columns and icons in the Signatures table.
Mind you, this is just the tip of the iceberg. Plenty can still be explored using Sumo Logic.
I hope you enjoyed this tutorial. If you have any questions, I am happy to answer them.
