By Prasanna Abeysekera

Event logging

In today's digital age, the importance of cybersecurity cannot be overstated. With countless threats lurking in the shadows of the internet, organisations must remain vigilant in their efforts to protect sensitive data and maintain the trust of their customers. One crucial aspect of this defence is the processing of security logs.

This article explores the potential consequences of failing to process security logs and presents specific examples to illustrate these risks. Let's take a moment to explore some real-life examples that illustrate the risks associated with neglecting the processing of security logs. These instances serve as cautionary tales.

Information about the risks of not processing security logs:

• Increased risk of data breaches: The heightened susceptibility to data breaches can be attributed to the utilisation of security logs, which can furnish significant insights regarding potential instances of unauthorised access attempts, abnormal traffic patterns, and data exfiltration. Failure to analyse security logs can result in organisations being unaware of these incidents and potentially unable to implement measures to mitigate a data breach.
• Increased risk of malware infections: Security logs are a valuable resource offering insights into malware infections. They contain information regarding the various types of malware detected, the specific systems that have been compromised, and even the individuals who may be accountable for these infections. Organisations need to understand that they need to properly process security logs to avoid difficulties in detecting and responding to malware infections. This lack of awareness can potentially result in additional harm to their systems and data.
• Increased risk of denial-of-service attacks: Denial-of-service (DoS) attacks occur when a system is flooded with excessive traffic, rendering it inaccessible to genuine users. Security logs are a valuable resource that can offer insights into Denial of Service (DoS) attacks. These logs contain essential details, including the origin of the attack, the specific type of attack employed, and the specific systems that are being targeted. By analysing these logs, security professionals can better understand the nature and scope of the attack, which can help in devising appropriate countermeasures. Organisations must understand the significance of processing security logs to effectively detect and respond to DoS attacks. Without this crucial step, they may face increased vulnerability to such attacks, potentially resulting in financial losses and reputational damage.
• Increased risk of insider threats: Insider threats refer to a specific category of threats originating from individuals within an organisation. Insider threats are a significant concern in organisations and can arise from various sources. These sources include employees who act maliciously, dissatisfied or unhappy employees, and unintentional errors made by employees. Organisations must be aware of these potential threats and take appropriate measures to mitigate them. Security logs are a valuable resource offering insights into potential insider threats. These logs contain important details about various aspects, including the users who have accessed sensitive data, the specific systems that have been accessed, and the specific activities that have been carried out. By analysing these logs, organisations can better understand any potential risks or vulnerabilities within their systems. Organisations need to understand that by correctly processing security logs, they may avoid difficulties in detecting and responding to insider threats. Insider threats can lead to negative consequences such as financial losses and damage to reputation.

Several instances demonstrate the potential consequences of improper utilisation of Security information and event management(SIEM) systems.
This encompasses the following:

• The failure to implement practical SIEM use cases for threat detection.
• It is imperative to gather, ingest, and analyse the requisite logs.
• The potential for security logs to be disregarded.

The following examples serve as illustrative instances that highlight the potential risks associated with the incorrect processing of security logs.

• The SolarWinds Supply Chain Attack in 2020 was a cyber attack that targeted SolarWinds' software updates. This allowed hackers to gain unauthorised access to the networks of various organisations, including government agencies and private companies. The attackers tried to avoid being detected by altering and bypassing security logs and other signs of compromise, making it difficult to pinpoint the intrusion.
• In 2017, Equifax, one of the largest credit reporting agencies in the United States, experienced a significant data breach that exposed the personal information of approximately 143 million individuals. The breach went undetected for several months before it was discovered. It is believed that the attackers exploited a vulnerability in Equifax's security logs to gain access to the company's systems and retrieve sensitive information. This breach had far-reaching consequences and highlighted the importance of robust cybersecurity measures and timely detection of breaches.
• The Target data breach took place in November 2013. During this incident, Target suffered a significant breach of customer data, compromising personal and financial information belonging to around 40 million customers. The breach at Target occurred when cybercriminals infected the point-of-sale systems with malware, enabling them to steal credit and debit card information. The malware employed in the attack could bypass Target's security systems and evade detection in Target's security logs. The existing security systems could not accurately analyse the logs and detect malicious activity. The breach went undetected for a significant period, which allowed the attackers to access and exfiltrate personal and financial information belonging to millions of Target customers.

The examples above demonstrate the impact on security log processing, or the lack thereof.

It is generally recommended to prioritise installing and configuring a Security Information and Event Management (SIEM) system before implementing applications and servers. The presence of a SIEM system before the deployment of new systems can enhance the efficacy of identifying and addressing security threats.
There are several advantages associated with installing a SIEM system before implementing applications and servers. Some advantages of deploying a SIEM ahead of other infrastructure components include:

• Early detection of threats: A SIEM system is designed to assist in the early detection of security threats, thereby preventing potential damage. A SIEM system can gather and evaluate security logs from various systems, such as newly implemented applications and servers.
• Improved incident response: A SIEM system can significantly enhance your ability to effectively respond to security incidents. A SIEM system offers numerous benefits, including a centralised view of all your security logs. This centralised view simplifies identifying the origin of a security incident and enables you to promptly take appropriate measures to minimise any potential harm.
• Reduced false positives: A SIEM system is designed to assist in reducing the occurrence of false positives you may encounter. A SIEM system can effectively identify and separate non-threatening events from potentially harmful ones by using advanced machine learning techniques.
• Improved compliance: A SIEM system can enhance your organisation's adherence to security regulations. A SIEM system is valuable because it can gather and retain security logs in a centralised location. This feature simplifies the showcasing adherence to regulations such as the Payment Card Industry Data Security Standard (PCI DSS) and the Health Insurance Portability and Accountability Act (HIPAA).

Installing a SIEM system before properly implementing applications and servers can present certain challenges. To illustrate, it is essential to understand that configuring the SIEM system is necessary to effectively gather logs from your newly implemented systems. In addition, it is vital to train your staff to effectively utilise the SIEM system. Installing a SIEM system before implementing applications and servers brings numerous advantages that surpass the associated challenges.

Is it possible to manually analyse log files? 

Image designed by Freepik

Indeed, there is no requirement for you to utilise any specialised tools. I have analysed log files of varying sizes, ranging from 100 to 10,000 events. I used Python-based filtering or an Excel sheet for this analysis. Having tools to automate the process would be highly beneficial. If you don't have access to specialised tools, there's no need to worry. You can still analyse nearly any log files using MS Excel, albeit through a manual process that may require additional time. However, with practice and learning, you can become proficient in creating Excel macros to the point where you can rival commercially available SIEM tools. Trust me when I say this. With time, you can normalise events and improve your filtering skills. If you are a digital forensic expert or aspiring to become one, you will understand the subject matter I am referring to.

The following examples demonstrate processing multiple log files without using advanced tools.

Once upon a time, a renowned consultancy and telecommunications service provider in Australia found itself in the midst of a distributed Concord's eHealth upgrade. They sought expertise to identify specific modifications in their system and application files. The project served as a source of inspiration, offering an opportunity to navigate the intricacies of their technological infrastructure and make meaningful contributions to their overall success.

Among the challenges encountered was setting up a couple of old laptops for a specific purpose. The responsibility fell upon configuring six laptops to run text-processing scripts. The objective was clear—to analyse multiple log files and carefully identify and assess modifications made during the upgrade process. The question arose: Should a Cloud platform be utilised for this endeavour? Considerations about SFTP, normalising, loading, confidentiality, and other related topics followed. Contemplations on whether it was better to have complete control over the security aspects ensued.

As the project commenced, excitement and apprehension coursed through the air. Would the aging hardware be capable of tackling the demanding task? Only time would unravel the truth. Determination ignited to breathe new life into these relics from a bygone era. The careful inspection confirmed the laptops were in good working order. Installing in-house developed text processing software and selecting necessary Python scripts ensued, preparing the ground for the log file analysis.

The logs required for analysis were gathered and examined from the entirety of the distributed eHealth system. This represents the initial state(Baseline) or current condition before implementing enhancements or modifications. Each file in this environment undergoes a checksum verification process, which is documented by a scanning script and stored in an Excel database. The information was spread out over various sheets in Excel. However, there were times when Excel needed help handling the amount of data, either exceeding the maximum number of rows or crashing the program.

The image shown below presents an easy way to inspect a file's checksum.

For more than eight days, the laptops worked nonstop, processing events around the clock every day. Due to high CPU utilisation and internal temperature, some laptops power down abruptly. Turn on full blasts on three table fans and place them in front of the computer desks. After much tenacity, a complete log of what transpired to cause file changes during the update was produced.

Following the completion of the analysis, the next crucial step involved arranging for a professional company to handle data destruction, ensuring the secure obliteration of all the hard drives. This meticulous approach allowed for a peaceful state of mind, knowing that the destruction process could be witnessed and managed responsibly.

The consultancy and telecommunications service provider could effectively analyse and understand the complex modifications that took place during their eHealth upgrade, resulting in a successful project conclusion. The text highlights the importance of several key factors when working with ageing hardware to analyse log files. These factors include meticulous analysis, data security, and adaptability. This success bolstered the idea that log data could be processed manually and insightful insights extracted without professional equipment.

In conclusion, while installing a SIEM system before fully implementing applications and servers may pose certain challenges, it is a strategic decision that brings many advantages. By configuring the SIEM system to gather logs from the newly implemented systems, organisations can establish a robust foundation for comprehensive security monitoring and incident response. Furthermore, training the staff to effectively utilise the SIEM system maximises its capabilities, enabling proactive threat detection and swift remediation actions. Ultimately, the benefits derived from early implementation of a SIEM system far outweigh the challenges, leading to enhanced security posture, improved risk management, and greater peace of mind for organisations venturing into the digital landscape.

In our upcoming blog post, we will delve into the world of efficiently processing security logs and provide valuable insights and recommendations for organisations. Stay tuned as we explore this crucial aspect of maintaining a robust security system.

Prasanna Abeysekera, a writer for GABEY, primarily focuses on cybersecurity and the changing nature of online threats.

Legal Disclaimer