CitrixBleed isn’t going away: Security experts struggle to control critical vulnerability
U.S. authorities are struggling to contain a critical vulnerability in Citrix Netscaler Application Delivery Controller and Netscaler Gateway, widely used networking appliances that help companies enable secure remote access.
Thousands of organizations worldwide use the technology, and researchers have seen attacks targeting a wide range of industries, including financial services companies, defense contractors, law firms, technology providers and government agencies.
While there are two vulnerabilities, threat actors have widely exploited CVE-2023-4966, a critical buffer overflow vulnerability researchers dubbed CitrixBleed. Thus far, the CVE has been linked to ransomware attacks and other malicious activity by multiple threat groups, including LockBit 3.0 and AlphV/BlackCat.
The speed and scale of the attacks has challenged even the most experienced cybersecurity experts. So far, the attacks have impacted some of the most sophisticated and highly regulated companies in the world.
With so many moving parts, here’s what you need to know about CitrixBleed:
What’s going on?
Citrix released a security bulletin for vulnerabilities in Netscaler ADC and Netscaler Gateway on Oct. 10, listed as CVE-2023-4966 and CVE-2023-4967. The technology is used for load balancing, to help applications run faster and for secure remote access, according to security researchers.
Threat groups have widely exploited CVE-2023-4966 for about two months, even in instances where patching was done. Mandiant and other threat researchers warned that threat actors can bypass the existing patch in cases where previous user sessions have not been deleted.
Cybersecurity and Infrastructure Security Agency joined the chorus of security stakeholders warning organizations of the vulnerabilities, and later added CVE-2023-4966 to its Known Exploited Vulnerability list.
Does the patch work?
Exploitation of CitrixBleed has escalated for several weeks despite a patch being issued Oct. 10.
Citrix confirmed session hijacking in an Oct. 23 blog post and said it received “credible reports” of targeted attacks exploiting the vulnerability. While Citrix maintains it was not aware of any exploitation prior to the Oct. 10 patch, it urged customers to install recommended builds.
In mid-October, Mandiant shared an urgent warning about observed exploitation dating back to August where hackers were able to hijack authenticated sessions and bypass multifactor authentication. Session data was also stolen prior to patch deployment and later used by threat actors.
“We observed session hijacking at organizations who had updated their Netscaler devices,” Charles Carmakal, CTO at Mandiant Consulting, Google Cloud, said via email.
Mandiant warned users to terminate all active or persistent sessions to prevent future attacks.
Data from Palo Alto Networks show on Oct. 8 — two days before the patch was released — there were 20,750 systems running potentially vulnerable versions of Gateway/ADC. That number fell to 7,984 unpatched versions by Oct. 15.
Mandiant said that prior to Oct. 10, it was investigating cases where a threat actor was taking over Netscaler sessions through an unknown means, according to a Nov. 2 blog post.
“In October 2023, we were investigating an intrusion and there was some activity that didn’t make sense based on available evidence,” Carmakal said via email. “After CVE-2023-4966 was disclosed, we were able to use the information Citrix published to determine that exploit of CVE-2023-4966 was the initial access vector.”
However, Citrix has said repeatedly that nobody informed the company of any prior exploitation, saying the patch was developed by its internal team.
Where the name came about
A key development in the evolution of the vulnerability came Oct. 25, when Assetnote released a proof of concept, along with research. The Australia-based firm, at the time, dubbed the vulnerability CitrixBleed in reference to the historic HeartBleed vulnerability.
Successful exploitation of a two-week old vulnerability that had been patched highlights the issues and the challenges of vendor security management, according to Assetnote executives.
“Third-party vendor software and appliances represent a blind spot for most organizations and in our opinion are the real ‘shadow IT’ problem,” Michael Gianarakis, Assetnote co-founder and CEO, said via email. “These systems are widely deployed inside companies and there are very few avenues for these organizations to get effective and proactive visibility into the security risks they present.”
Who was affected?
Thus far, several high-profile security incidents are tied to the CitrixBleed vulnerabilities.
In late October, Boeing began working with law enforcement as part of a formal investigation into a ransomware claim from the LockBit threat group. The ransomware group leaked almost 45 gigabytes of data reportedly stolen from Boeing.
Boeing later voluntarily shared TTPs and IOCs stemming from the LockBit attack with authorities.
Boeing is not alone, however. CitirixBleed also caught two financial services firms, disrupting bank operations.
ICBC Financial Services, a subsidiary of the Industrial and Commercial Bank of China, disclosed a ransomware attack in early November, which threat researcher Kevin Beaumont later linked to CitrixBleed.
Beaumont also linked the CitrixBleed vulnerability to a cyberattack against Ongoing Operations, a subsidiary of Trellance Cooperative Holdings, which led to outages at 60 credit unions.
Satnam Narang, senior staff research engineer at Tenable, said the attacks have largely been “smash and grab” operations by opportunistic ransomware groups, as opposed to methodical espionage operations targeting specific companies.
What are the feds doing?
The New York State Department of Financial Services on Nov. 14 warned all regulated entities to take immediate actions to mitigate CitrixBleed. The agency, which just enacted newly amended disclosure rules for regulated entities at the beginning of November, warned that the vulnerability could lead to ransomware deployment, data theft and disrupt operations.
Just before Thanksgiving, CISA, FBI, MS-ISAC and Australian Signals Directorate issued a joint bulletin regarding exploitation of CitrixBleed by Lockbit 3.0. CISA, which released mitigation guidance for the vulnerability on Nov. 7, says it has warned about 300 organizations they were running vulnerable instances through its ransomware warning program.
In late November, the U.S. Department of Health and Human Services issued a bulletin warning healthcare organizations about the risk of CitrixBleed.
Stemming the flow
Security researchers warn that malicious attacks are likely to continue as long as Netscaler instances remain unpatched and users fail to take additional mitigation steps, particularly the deletion of active sessions.
Data from Palo Alto Networks shows the number of vulnerable systems has fallen to 811, representing a 96% decline from the pre-patch figures.
Shadowserver data shows there are 1,801 unpatched systems, down from more than 20,000 on Oct. 12.
CISA officials repeated earlier concerns about manufacturers failing to use memory-safe languages, which have contributed to CitrixBleed exploitation.