Understanding Security Log Analysis
Introduction
When you open your phone app and see all of those you’ve called, or those that have called you, that is an example of a computer log. Logs are simply records of events that has happened on a system or network.
These logs can be anything from a simple call log to a complex jargon error log seen at the back-end of applications and even bugs encountered when running a program.
In general, a computer log not just gives us information on what is going on with a computer, it shows a more detailed and comprehensive one.
Now that we know what computer logs are, let’s get to see what security logs are and why they are important.
Security logs are also computer logs, they can be seen as a subset of computer logs that store security events in a system or network for further analysis and investigation.
Logs aren’t just meant to be stored away from active access. They are stored so that they can be reviewed for diagnosis and this is through a method called Log analysis.
Log Analysis
Log analysis is the processing of carefully inspecting log data to make meaningful information out of it. It is a process that requires expertise in data analysis, critical thinking and security, if you are analyzing security logs.
These logs provide information about the behavior of a system, its functioning state and even get meaningful intel on the activities that’s happening on a system or network.
The steps involved in analyzing logs differ based on the type of logs being analysed, but regardless, they all follow a very similar pattern of steps which I will also cover before this article is over.
Why Log Analysis.
It is one thing to store logs, it is another to analyse them, at least before they are deleted or archived depending on the case. But it is very necessary that log monitoring tools are employed by organizations and even individuals for periodic frequent analysis for a number of reasons.
- It is a proactive method against security issues. A lot of people don’t seem to understand that log monitoring and analysis helps to detect potential security issues before they escalate.
For instance, a security network admin can analyze the traffic coming from an organization’s network and if they notice unusual traffic from a particular endpoint, they can escalate the situation to the appropriate team, or if their job role permits, make necessary blocks.
- It helps in the recovery stage after a cyber breach. Since logs are generated for every single security event that occurs on a system, it becomes an important approach to analyze logs as a post-incident remediation method for forensic purposes. This can be useful in finding out what went wrong in the system or network.
This indicates that log analysis isn’t just a proactive method, it is also a reactive method.
- Compliance. This is a very big reason why organizations must analyze and store logs. It helps them determine their compliance level according to industry standards and often provides a bedrock for security audits.
- Detection and Troubleshooting. A lot of times, when security analysts or other IT professionals are troubleshooting a system or network, one of the first places they check is the appropriate log for what they are troubleshooting.
If for instance, they are troubleshooting system problems in a Linux endpoint, they could simply use the journalctl command to view logs of what has happened. Sometimes, certain applications automatically generate logs when they crash and direct the user to view those logs to have an insight of what went wrong.
- Investigation. Logs are used to verify that a specific action was carried out in a system/network ensuring that there is evidence to fall back on if there is ever a denial of action.
How to Analyze Logs.
Here is the real job. Logs are often displayed in different formats. Some in binary, some in plain-text. While the ones in plain-text might seem to be easier to analyze, the reverse is often the case as analyzing a log is an entirely different thing and far from merely reading logs. Even plain-text logs can be confusing to understand.
Analyzing logs, like I wrote earlier, requires expertise in certain areas and is a process that requires the ability to deduce accurate and meaningful data from the analyzed logs.
Log analysis takes two method:
The manual method and the automatic method.
Manual log analysis involves the collection of data into a single place, formatting them for easy comprehension, manual inspection of each log instance for correlation and pattern recognition or at this stage, using an analysis tool of your choice to assist with the analysis.
Manually analyzing logs is not without its own problems, as it shares many problems associated with the manual working of data–errors, a work that’s not neat, time consuming, etc.
Automatic Log analysis: While a human is still required at certain stages, automatic analysis of logs still remains the best. With an automatic log analysis set up, all events happening on a system or network are automatically collected and aggregated into a single place. Sub-tools can be set up to automatically scan these logs, raise alarms if certain defined parameters are found and even merged with other security tools to act on the logged data.
With automatic log analysis, large amounts of log data can be worked on in shorter times with better accuracy than manual method, leaving humans with less work to find specific issues in the logs.
Though, in automatic log analysis, there is still the possibility of false positives and true negatives, but they are very minimal in occurrence frequency.
With this being said, how come there is still a need for humans to have log analysis knowledge? Well, it is because log analysis is not reading an article like this, you need to be able to interpret what you see, isn’t it?
If you are interested in understanding how to read logs, you can check out this Tryhackme room.
Log Analysis Tools.
There are different tools used to view and analyze logs, some of them are simple tools, others are comprehensive tools providing many features like integration with other tools. I’ll give a short info of the tools I’ll below.
Splunk: if you are in cyber security, you’ll be familiar with this name. It is a Security Information and Event Management tool from Cisco, and as a SIEM tool, it is mandatory that it comes with a robust log analysis feature.
Syslog-ng: An open source log analysis tool that can collect and aggregate log from a variety of sources.
Sumo Logic: Primarily a cloud based log analysis tool, it also has the ability to gather logs from physical devices.
Windows Event Manager: A built-in windows tool used to collect system logs of events in Windows hosts. While not considered a full time log analysis tool by some, its ability to give meaningful insight into what’s going on in a windows machine makes it a good log tool.
Logic Monitor: An automatic log monitoring tool, providing both automatic monitoring of connected endpoints and analysis of them.
Graylog: A user-friendly open source log collector platform that helps organizations store logs.
Elastic: Another popular, open source log analysis tool that combines three apps in its suite to assist in the management of logs files.
There are lots of other log analysis tools out there, you can research further to find yours, if you want a comprehensive one, then Splunk is my shot, but for pricing, Syslog-ng should be your go-to.
Conclusion
In summary, if you use a computer, you can’t do without having log analysis knowledge, especially security logs. One IT expert called security the reason logs exist in the first place.
Understanding how security logs work is a necessary skill and I believe that this article has done a good job in explaining that.