Contents
Over the past two years, the annual number of information security incident investigations Incident investigation is a process of exploring and analyzing events in order to promptly identify the source of the threat, localize traces of an attacker in the compromised infrastructure, and assist in the restoration of disrupted business processes. conducted by the Incident Response (IR) team at the Positive Technologies Expert Security Center (PT ESC) has consistently increased. Just in 2022, there was a 50% increase. In the first nine months of 2023, compared to the figures for the entire 2022, the number of investigation projects grew by 76%. We assume that such a surge could have been triggered by an increase in the number of cybersecurity incidents due to recent geopolitical and economic events worldwide.
For this study, we analyzed information obtained from more than 100 projects of two types: incident investigation and retrospective analysis of infrastructure Retrospective analysis of company infrastructure involves a detailed examination of available data sets (such as system images, OS event logs, memory dumps, network traffic, malware samples, and information security logs) over a given period to identify traces of possible compromise. In most cases, the maximum available period is used, depending on the log retention depth and existing forensic artifacts. . These projects were conducted from Q1 2021 to Q3 2023 in various companies in the Russian Federation and CIS. Most of these organizations (69%) are government agencies and industrial, financial, and IT companies. A significant portion of the attacks we investigated were targeted, meaning the attackers were aiming to harm specific companies.
20% of the organizations are in the RAEX-600 rating for the largest companies in Russia by the volume of product sales for 2022, and 21% are in the Expert-400 ranking for 2021.
Summary
By analyzing the incident investigation projects and retrospective studies of company infrastructures, we established the following:
- 40% of incidents were related to the activities of known APT groups. APT groups are organized cybercrime structures with a high level of technical expertise, possessing resources and capabilities for long-term covert presence in the infrastructures of victim companies.
- The remaining 60% of incidents were related to the activities of APT groups that couldn't be clearly identified at the time of the investigation, and other unidentified threat actors, whose primary motives were financial gain and/or hacktivism, including for political objectives.
Since the beginning of 2022, we've noticed an increase in politically motivated incidents (as of the publication, they constitute 9% of the total number for the period under investigation). Previously, such motives in attacks by threat actors were almost non-existent.
- In 25% of completed projects dedicated to retrospective analysis of company infrastructures, traces of APT group activities were identified. These groups often operated in the networks of victim companies for quite a long time (from six months to a year at the time of analysis) without being detected.
- APT group attacks most commonly targeted government agencies (34%), industrial enterprises (30%), IT companies (7%), media (5%), and telecom (5%).
- Threat actors remained in the victim company's infrastructure for a median average of 37 days until their discovery (Time to Detect, TTD), with the longest stay in an infrastructure being 3 years.
- The average time from the start of investigation to writing the final report (Time to Response, TTR) was 21 days. We often received initial results within just a few hours after the investigation request.
Based on the initial investigation results, organizational and technical measures can be swiftly taken to counter the attacker.
More details on how to quickly obtain significant initial investigation results can be found in the section "What factors can influence investigation time?"
- During the first three quarters of 2023, the number of incidents caused by supply chain A cyberattack in which attackers gain unauthorized access to corporate infrastructure by compromising the suppliers of the victim company's software or equipment. For example, the attackers can inject malicious functionality into a software product's source code or distribute malicious updates to infect the target organization's infrastructure. and trusted relationship A cyberattack in which attackers gain unauthorized access to corporate infrastructure by compromising a "trusted third party with whom the victim company has established communication channels. For example, compromising a subsidiary through which attackers gain access to the corporate infrastructure of the main organization. attacks doubled compared to the whole of 2022.
As a result of the incidents investigated, affected companies most often faced:
- Internal business process disruptions (32%)
- Cyberespionage involving prolonged attacker presence in the victim's infrastructure, typically with the aim of continuous extraction of confidential information (32%)
- Direct extraction of confidential information (26%)
We analyzed the most crucial aspects of these incidents, including security system flaws in companies, and compiled a list of recommendations to minimize Unfortunately, completely eliminating all attacks is impossible, as attackers may, for example, exploit zero-day vulnerabilities (vulnerabilities nobody is aware of yet). the number of successful attacks.
Research findings
What's the difference between retrospective infrastructure analysis and cybersecurity incident response projects?
Retrospective infrastructure analysis is typically conducted to identify traces of potential compromise over the maximum possible period. This period is determined by the log retention depth and available forensic artifacts. During retrospective analysis, experts do the following:
- Conduct signature scanning of file systems and RAM on infrastructure hosts to search for potential malware and analyze it.
- Collect and thoroughly analyze other relevant data (such as OS event logs, traces of user activity, user session information, traces of software execution on endpoints, and logs from all available information protection tools).
- Collect network artifacts and analyze them using the PT ESC reputation base.
Retrospective projects constitute 18% of our sample.
The average project duration is 30 days.
An investigation into a security incident is typically initiated for one of the following reasons:
- Suspicious or directly malicious activity is detected in the victim company's infrastructure (using existing security systems), or notification is received based on threat intelligence information (66% of cases).
- The affected company receives information from attackers about the encryption or erasure of data within the corporate infrastructure, often accompanied by a ransom demand (15%).
- Attackers disclose information about their intrusion into the company's internal network or publish stolen confidential information on the Internet or dark web (10%).
The average time from the start of an investigation to the completion of the final report was 21 days. The overall distribution can be seen in Figure 4. If a company promptly contacts PT Expert Security Center's IR experts upon detecting an incident and provides everything necessary to begin the investigation, significant initial results This includes primary information about the timeline and causes of the incident and recommended organizational and technical measures to counter the attacker, such as instructions for isolating affected hosts. can be obtained within a few hours.
What factors can influence the investigation time?
- Failure to collect information necessary for the investigation in a timely manner.
If the required data is not collected immediately after the incident is detected (for example, only after a few days or even later), there is increased likelihood that the logs relevant for analysis Log rotation is the process of overwriting logs after a set time to optimize the disk space provided for their storage. will be rotated out.
- Untimely provision of collected data, refusal to provide information in the required scope.
The longer this stage takes, the longer the attackers can remain in the company's infrastructure and the more damage they can inflict on the business.
- Mistaken actions by information security and technology specialists within the victim company.
As a result of incorrect (albeit intuitively understandable) actions by on-site IT and cybersecurity professionals, malware samples necessary for further investigation, for example, to obtain indicators of compromise, can be permanently deleted, RAM contents may be destroyed (due to computer reboot), and other crucial information needed for a thorough investigation can be lost.
Once the attack's development is stopped or the acute phase is mitigated (usually within 2–3 days), PT Expert Security Center IR experts move into the active investigation phase. On average, this phase takes 5–7 working days. During this time, it's typically possible to determine the incident's scale, the attackers' TTP Tactics, Techniques, and Procedures (TTP) are a description of the cybercriminals' actions during an attack. Understanding the TTP empowers the creation of both a more effective high-level description of the cyberattack and a more detailed analysis of it. , main tools used, and the controlling network infrastructure. This information enables the swift development and implementation of effective organizational and technical measures to localize the incident and eliminate the attackers' presence in the infrastructure—in other words, to draw up and execute an incident response plan.
Remote response in Positive Technologies
From the beginning of the COVID-19 pandemic until today, the majority (95%) of incidents reported to the IR experts at the PT Expert Security Center have been investigated remotely. This format gives faster initial results without needing to travel to the affected company's location. However, if a company's policy strictly prohibits remote interaction, the experts are always prepared to carry out the work directly on-site.
PT Expert Security Center's IR experts have developed a toolkit for incident response and investigation, covering numerous popular operating systems, including Windows, Linux, and MacOS.
For a more detailed understanding of the remote interaction format and the tools used by our experts, refer to the article "Remote incident response: how COVID-19 challenges gave rise to new practices".
During incident investigations, IR experts collaborate with the technical staff from the affected company to do the following:
- Analyze available information security logs, samples of malware, network indicators, RAM dumps, network traffic samples, and other data sets from key infrastructure hosts, both affected and unaffected in the incident.
- Compile a list compromised hosts and user accounts.
- Determine the initial attack vector and reconstruct the incident timeline.
- Compile a list of detected malware and host-based and network-based indicators of compromise.
- Conduct scans of the entire available infrastructure to detect identified indicators of compromise.
- Eliminate the presence of attackers, including their communication channels, within the infrastructure.
How do attackers operate?
The average time from the moment attackers compromise the infrastructure until they are stopped (or contained) was 45 days (median value), with the longest attack lasting for 5 years. Details of the distribution can be found in Figure 5.
As the initial penetration vector, attackers most commonly (63%) exploited vulnerabilities in publicly accessible web applications used by the victim. Specifically, according to our experience, the most frequently attacked web applications were Microsoft Exchange mail servers (50% of all attacks where vulnerable web applications were the initial vector), Bitrix web servers (13%), and Atlassian products (7%), such as Confluence and Jira. The second most frequently successful penetration method was phishing emails.
In the majority of cases (96%), when the investigation was initiated, the attackers were still present within the compromised infrastructure or at least had a pre-established communication channel available. The median time from the moment of compromise to the detection of the attackers (TTD) was 37 days, with the longest being 3 years. Details of the distribution can be found in Figure 7.
Among the most common incidents in recent times (21% among the total incidents investigated during the period under study) are incidents related to full encryption or erasure of information on the customer's infrastructure hosts. The complete list of corresponding malware It's important to note that the table includes software that may also be used for legitimate purposes (BitLocker, DiskCryptor). identified during the cybersecurity incident investigations is presented in Table 1.
Table 1. List of information encryption/erasure malware identified in cybersecurity incident investigations
Omerta | TinyCrypt | Babuk and its modifications, like BadWeather | Filecoder.MY |
BitLocker | Phobos | CrossLock | Locker |
Zeppelin | DiskCryptor | BlackCocaine | CaddyWiper and its modified versions |
VoidCrypt | LockBit | RCRU64 |
Threat actor attribution is a complex process that doesn't always give reliable results. Over the last three years, IR experts at PT Expert Security Center have researched incidents involving 15 known APT groups, identified based on the tools used, network infrastructure, and TTP. The complete list is presented in Table 2.
Table 2. List of identified APT groups
APT35 | CloudAtlas | OldGremlin |
APT41 (Winnti) | Cobalt | Sneaking Leprechaun |
BlackTie (TwistedPanda) | DarkRiver | SpacePirates |
BronzeUnion | Gallium | TA428 (TaskMasters) |
ChamelGang | Lazarus | TokyoHotel |
The share of incidents definitively linked to the activities of specific publicly known APT groups during the examined period amounted to 40%. The distribution of victims of such attacks (as well as attacks involving unidentified APT groups) by industry is presented below in Figure 8.
APT groups typically employ unique malware which provides the attackers access to the company's infrastructure after the initial compromise. Examples can be found in Table 3.
Table 3. Malware used in APT attacks
App_global | Daxin | FaceFish | Msdaprst Backdoor | ShadowPad |
Backdoor Leiosaurid | Decoy Dog | Fasol RAT | MsmRAT | Sidewalk |
BeaconLoader | DeedRAT | Gh0st | owowa | SPINNER |
Bisonal | DiskTool | IP He1per (PingPull) | PlugX | TaskMasters Backdoor |
CobInt | DNSep | Kitsune | ProjC | TaskMasters Backdoor PowerShell |
CotxRat | DonutHole | Matadoor | PwShell | TgRAT |
DarkPulsar (2017) | Doorme | Microcin | Remshell | ThreatNeedle |
TinyFluff | TinyNode | TokyoX | Voidoor | XDSpy |
Nevertheless, both APT groups and less skilled attackers use auxiliary software, which in the vast majority of cases is publicly available on the Internet. We analyzed this software and mapped it on the MITRE ATT&CK® heatmap by the task performed and the frequency of detection in projects.
From time to time, attackers exploit publicly available legitimate tools for remote host control (7% of the total detected software in projects), brought into the infrastructure of the affected company by the criminals themselves or already in use by its employees (such as system administrators) at the time of the attack. The complete list of such utilities identified in incident investigations over the observed period can be found in Table 4.
Table 4. List of utilities for remote host control (by frequency of use by attackers)
RemoteUtilities (RMS) | TeamViewer | Mesh Agent |
Dameware | AnyDesk | AmmyyAdmin |
Radmin | WireGuard | Atera |
Remote Utilities | Pulseway | Splashtop |
OpenVPN |
The attackers don't hesitate to use the living-off-the-land techniques. This involves using legitimate software and operating system components for malicious actions. A complete list of such utilities can be found on the LOLBAS project page. The living-off-the-land method allows criminals to disguise malicious activity, thus evading detection by the cybersecurity systems and incident response specialists of the affected company.
In all our cybersecurity incident investigation projects, attackers utilized built-in command-line interpreters such as cmd, PowerShell, and bash.
Interestingly, in some cases, attackers unconventionally employed certutil and msbuild utilities: certutil was used to download malware from the Internet, and msbuild, to execute the Cobalt Strike Beacon payload.
As we expected, the analysis of all the projects made it clear that attackers most frequently compromised hosts running Windows OS. However, it’s also worth mentioning that a considerable number of compromised hosts were running Linux. This once again demonstrates the error in believing that this operating system is completely invulnerable to attacks, or at least highly resilient.
If your infrastructure includes Linux hosts, pay special attention to their protection.
Attack consequences
During the investigations, IR experts from PT Expert Security Center assess the impact of the incident on the company's infrastructure and business processes. In most cases, such consequences can be categorized as follows:
- Intelligence activities, espionage
Characterized by a prolonged presence of attackers in the victim’s infrastructure (median value of 35 days). During this time, the attackers continuously have operational access to confidential information.
- Data exfiltration
One of the markers of data exfiltration by attackers is the tools used, notably publicly available file archivers like WinRAR and 7z. In recent years, there has also been an increase of cases of data exfiltration to public file-sharing or cloud resources using utilities like Rclone and MEGASync.
Since 2022, there has been an increase in incidents where attackers obtain specific user data to gain access to employees' Telegram Desktop messenger accounts. These accounts often contain up-to-date authentication data for accessing corporate information systems, as well as other confidential information. In 2022, such incidents accounted for 9% of all cases in which confidential data was extracted, and for the first three quarters of 2023, this figure reached 63%.
- Impact on business processes
Characterized by disruptions to the operation of key infrastructure hosts of the victim company. This is a common consequence of attackers launching malware to encrypt or erase data.
The financial damage from attacks was not assessed, as this requires tracking companies' financial indicators over several years. This method provides the most accurate assessment of the impact of cyberattacks on business indicators. For example, following the cyberattack in late July 2023, the shares of Estee Lauder at the time of the present study have not yet recovered in price.
According to IBM experts, in 2023, the average damage from an attack resulting in a leak of confidential data amounted to USD 4.45 million. And the average damage from a successful attack using ransomware, according to Embroker, amounted to USD 1.85 million.
Examples of detected attacks
Attack scenarios observed during incident investigations often start similarly, regardless of the attackers' goals and motives. As a preparatory stage, attackers always conduct reconnaissance, gathering and studying publicly available information about the company, its employees, services, and available resources.
The attackers then attempt to gain initial access to the company's infrastructure—for example, by discovering and exploiting a vulnerable service in the company's network perimeter or conducting a successful phishing campaign targeting company employees.
Another way to access a company's internal network is to use social engineering methods on its employees. For example, attackers might send phishing emails containing malicious attachments or links to fraudulent websites.
More details about these and other methods of penetrating a company's internal network can be found in the relevant section of the MITRE ATT&CK® matrix.
After gaining initial access, attackers try to deploy malware in the infrastructure to establish remote connection to their command-and-control servers and ensure persistence. Notable examples of such malware are the Sliver and Cobalt Strike frameworks. Typically, at this stage, the attackers must already deploy evasion measures to conceal their malicious activity and avoid detection. For this purpose (Defense Evasion), a variety of software is used, primarily aimed at counteracting static analysis.
You can see the full list of tools used by attackers at different stages of the attack in Figure 9.
In the vast majority of cases, after infiltrating an infrastructure, attackers conduct host and network/domain reconnaissance (Discovery), typically for as long as they remain in the victim's infrastructure. As a result, they may, for example, obtain lists of processes, users and their privileges, intercept requests, and gain access to databases containing confidential information, information about the structure of Active Directory objects, and other data.
Having established themselves in the infrastructure, cybercriminals seek to escalate privileges, for example, by gaining access to a domain administrator account. This can be achieved in various ways, such as extracting password hashes from the lsass.exe process memory. These hashes can be used in a Pass the Hash attack to gain access to the domain controller. Privileged access enables cybercriminals to move freely within the company’s infrastructure (Lateral Movement) and propagate malware according to their goals.
After ensuring efficient spreading of their presence in the target infrastructure through the privileges obtained, attackers begin to display various behaviors depending on their goals:
- Attackers distributing malware for data encryption/erasure (in a number of investigations we discovered previously compromised security solutions installed in the company used for that purpose) may load and execute their malware (Impact) at this stage, or they may start by exfiltrating the confidential information gathered during the host and network/domain reconnaissance.
We've been seeing this trend of exfiltrating stolen company data before launching malware since 2020. This move allows attackers to demand ransom for restoring access to the infrastructure or for not disclosing the stolen information.
- Attackers conducting espionage campaigns generally behave less conspicuously. Their aim is to remain within the company's infrastructure for as long as possible, swiftly gathering confidential information such as corporate documents, tables, presentations, and databases, often containing authentication data stored in cleartext.
To swiftly detect attackers conducting cyberespionage, it’s essential to establish monitoring processes within the infrastructure. For certain attacker techniques encountered in incidents, we have already proposed a set of detection measures. You can find them in this article
Summing up this part of the research, we should emphasize once again that, regardless of attackers' motives, they often perform roughly the same actions in the initial stages of an attack. By making effort to detect TTP and tools characteristic of these stages, it's possible to intercept an attack in time and prevent the possible realization of a non-tolerable event.
Conclusion
Our experience shows that many incidents can be avoided or at least mitigated. To do this, we recommend the following:
- Use the latest versions of software and operating systems and establish processes for vulnerability management and remediation. Track trending vulnerabilities on your assets and set a 24-hour SLA for their remediation.
- Apply best practices, especially when setting the company's password policy.
- Enforce two-factor authentication for all public-facing services at the perimeter and critical elements within the company’s internal network.
- Apply the principle of least privilege.
- Configure network infrastructure segmentation and restrict access between segments according to your business processes; eliminate uncontrolled network connectivity at all infrastructure levels.
- Set up a regular backup process for all domain hosts and organize the storage of these backups on a host isolated from the main network.
- Conduct ongoing perimeter audits for vulnerabilities and public-facing services.
- Configure cybersecurity event monitoring for timely incident detection and response.
We also recommend using proven cybersecurity technologies and tools, such as:
- Privileged account management systems
- Vulnerability management (VM) systems
- Security incident and event management (SIEM) systems
- Network traffic behavioral analysis (NTA) systems
- Next-generation firewalls (NGFW) for channel protection, network segmentation, and network route hardening
- Web application firewalls (WAF)
- Static and dynamic analysis systems for detecting malicious objects (sandboxes)
- Antivirus software and endpoint detection and response systems (EDR), as well as the more advanced XDR solutions.