98% of companies surveyed view some of their AI models as vital for business success, and 77% have experienced breaches in their AI systems over the past year, according to HiddenLayer.

The report surveyed 150 IT security and data science leaders to shed light on the biggest vulnerabilities impacting AI today, their implications for commercial and federal organizations, and cutting-edge advancements in security controls for AI in all its forms.

Researchers revealed the extensive use of AI in modern businesses, noting an average of 1,689 AI models actively used by companies. This has made AI security a top priority, with 94% of IT leaders dedicating funds to safeguard their AI in 2024.

However, confidence in these investments is mixed, as only 61% express high confidence in their budget allocation. Furthermore, 92% are in the process of devising a strategy to address this novel threat. These results underscore the growing demand for assistance establishing robust AI security measures.

AI risks

Adversaries can leverage a variety of methods to utilize AI to their advantage. The most common risks of AI usage include:

• Manipulation to give biased, inaccurate, or harmful information.
• Creation of harmful content, such as malware, phishing, and propaganda.
• Development of deep fake images, audio, and video.
• Leveraged by malicious actors to provide access to dangerous or illegal information.

Major types of attacks on AI and security challenges

• Adversarial machine learning attacks: Target AI algorithms with the aim to alter AI’s behavior, evade AI-based detection, or steal the underlying technology.
• Generative AI system attacks: Threaten AI’s filters and restrictions, intended to generate content deemed harmful or illegal.
• Supply chain attacks: Attack ML artifacts and platforms with the intention of arbitrary code execution and delivery of traditional malware.

While industries are reaping the benefits of increased efficiency and innovation thanks to AI, many organizations do not have proper security measures in place to ensure safe use. Some of the biggest challenges reported by organizations in securing their AI include:

• 61% of IT leaders acknowledge shadow AI, solutions that are not officially known or under the control of the IT department, as a problem within their organizations.
• 89% express concern about security vulnerabilities associated with integrating third-party AIs, and 75% believe third-party AI integrations pose a greater risk than existing threats.

Best practices for securing AI

Discovery and asset management: Begin by identifying where AI is already used in your organization. What applications has your organization already purchased that use AI or have AI-enabled features?

Risk assessment and threat modeling: Perform threat modeling to understand the potential vulnerabilities and attack vectors that could be exploited by malicious actors to complete your understanding of your organization’s AI risk exposure.

Data security and privacy: Go beyond the typical implementation of encryption, access controls, and secure data storage practices to protect your AI model data. Evaluate and implement security solutions that are purpose-built to provide runtime protection for AI models.

Model robustness and validation: Regularly assess the robustness of AI models against adversarial attacks. This involves pen-testing the model’s response to various attacks, such as intentionally manipulated inputs.

Secure development practices: Incorporate security into your AI development lifecycle. Train your data scientists, data engineers, and developers on the various attack vectors associated with AI.

Continuous monitoring and incident response: Implement continuous monitoring mechanisms to detect anomalies and potential security incidents in real-time for your AI, and develop a robust AI incident response plan to quickly and effectively address security breaches or anomalies.

More to consider: