Runtime attacks can turn profitable AI into budget holes
4 months ago
2
Read more about this special issue.
AI is a promising technology, but its security costs are blinding. New attacks on AI’s operational side are quietly increasing budgets, jeopardizing compliance with regulations, and eroding customers’ trust. All of these factors threaten ROI and total cost ownership for enterprise AI deployments.
AI’s potential to deliver game-changing insights and efficiency improvements has captured the enterprise. As organizations rush to operationalize models, a sobering truth is emerging: the inference stage is under attack. This is where AI transforms investment into real-time value for business. This critical moment is driving the total cost ownership (TCO), in ways that initial business case calculations failed to predict.
Security executives, CFOs and others who approved AI projects because of their transformative potential are now faced with the hidden costs of defending them. Inference is the place where AI “comes to life” for a company, and where it can do the most damage. Cost inflation is the result. Breach containment in regulated industries can be as high as $5 million per incident, compliance retrofits can cost hundreds of thousands of dollars and trust failures may lead to stock losses or contract cancellations which can decimate AI ROI. AI becomes a budget wildcard if cost containment is not considered.
The unseen battleground: AI inference, exploding TCO and more
Cristian Rodriguez, CTO of the Americas field office at the time, said that AI inference was rapidly becoming the next insider risk. CrowdStrikewas announced to the audience at RSAC 2025 ().
Others technology leaders echo this view and see a blind spot in enterprise strategies. Vineet arora, CTO of Steffen Schreier is the SVP of Product and Portfolio at Telesigna Proximus Global Company, is “the assumption” that third-party models have been thoroughly vetted, and are inherently safe for deployment.
However, he warned that, in reality, these models “often haven’t be evaluated against an organizations specific threat landscape or regulatory needs,” which could lead to harmful or incompliant outputs, which can erode brand confidence. Schreier told VentureBeat “inference-time vulnerability — such as prompt injection, output manipulation or context leakage — could be exploited to produce harmful or biased outputs.” This can be a serious risk, especially in regulated sectors, and can quickly erode trust in brands.
The fallout from a compromised inference affects TCO on multiple fronts. Cybersecurity budgets are spiraling, regulatory compliance is in jeopardy and customer trust is eroding. This growing concern is reflected in the executive sentiment. CrowdStrike’s In the State of AI in Cybersecurity Surveyonly 39% of respondents thought that generative AI’s benefits clearly outweighed its risks, while 40% deemed them comparable. This ambivalence highlights a crucial finding: safety and privacy controls are now top requirements for new-gen AI initiatives. A striking 90% of organizations have implemented or developed policies to govern AI adoption. The top concerns are not abstract anymore; 26% cite the exposure of sensitive data and 25% fear adversarial attack as key risks.
Security leaders exhibit mixed sentiments regarding the overall safety of gen AI, with top concerns centered on the exposure of sensitive data to LLMs (26%) and adversarial attacks on AI tools (25%).
Anatomy of a inference attack
Adversaries are aggressively probing the unique attack surface revealed by running AI models. Schreier says that to defend against this, it is important to treat each input as a possible hostile attack. Frameworks such as the OWASP Top 10 Large Language Model Applicationscatalog these threats which are no more theoretical but active attack vectors affecting the enterprise:
Injection of prompts (LLM01) or insecure output handlings (LLM02) Attackers manipulate models using inputs or outputs. Malicious inputs may cause the model ignore instructions or reveal proprietary code. Insecure output handling occurs if an application blindly relies on AI responses. This allows attackers to inject malicious code into downstream systems.
Model poisoning and training data poisoning (LLM03). Attackers corrupt data by introducing tainted samples or hidden triggers. A seemingly innocent input can later unleash malicious outputs.
A model denial of service can be caused by adversaries using complex inputs. This will slow or crash the AI models, resulting in a direct revenue loss.
Supply chains and plugin vulnerabilities (LLM05, LLM07 and LLM08): AI ecosystems are built on shared components. As an example, a A vulnerability in the Flowise LLM Tool exposed sensitive data and private AI dashboards, including GitHub tokens, OpenAI API keys and other sensitive data on 438 servers.
Disclosure of sensitive information (LLM06). Clever queries can extract confidential data from an AI model, if the information was part of the training data.
Excessive Agency (LLM08) or Overreliance (LLM09). Giving an AI agent unchecked access to perform trades or modify databases can be a recipe for disaster.
Theft of proprietary models (LLM10) : An organization’s proprietary model can be stolen using sophisticated extraction techniques. This is a direct attack on its competitive edge.
These threats are rooted in fundamental security failures. Adversaries log in using leaked credentials. According to the report, in early 2024, 35% cloud intrusions used valid user credentials. New, unattributed cloud attacks also spiked at 26%. CrowdStrike Global Threat Report A deepfake resulted in a fraudulent $25.6 million transferwhile AI-generated emails have shown a click-through rate of 54%, which is more than four-times higher than those written manually.
The OWASP framework illustrates how various LLM attack vectors target different components of an AI application, from prompt injection at the user interface to data poisoning in the training models and sensitive information disclosure from the datastore.
Back to basics: Fundamental security for a New Era
To secure AI, we must return to the fundamentals of security — but with a modern perspective. “I believe that we need a step-back and to ensure that the foundations and fundamentals of security still apply,” Rodriguez argued. “The same approach that you would use to secure an OS is also the approach you would use to secure this AI model.”
To lock down cloud environments, where most AI workloads are located, it’s important to enforce unified protection, including rigorous data governance, robust Cloud Security posture Management (CSPM), identity-first security via cloud infrastructure entitlement management, and cloud infrastructure entitlement management. AI systems need to be protected with the same access controls and runtime protections that are applied to other business-critical cloud assets.
Shadow AI or the unauthorised use of AI by employees creates an unknown, massive attack surface. Financial analysts who use a free online LLM to store confidential documents may accidentally leak proprietary information. As Rodriguez warned, queries made to public models could “become someone else’s answer.” To address this, a combination of a clear policy, employee training, and technical controls such as AI security posture management will be needed to discover and assess any AI assets, whether sanctioned or unsanctioned.