Why would a chatbot that refuses to do something harmful when you ask it directly turn around and do exactly that a moment later? The answer, in a growing number of real cases, is that the chatbot was never asked by the person using it at all. It was asked by something else: a hidden line of text buried in a webpage, an email, or a document the AI was told to read. That technique has a name, prompt injection, and as AI tools move from answering questions to taking real actions on a user’s behalf, it has become the risk security researchers worry about most.
What Prompt Injection Is
The OWASP GenAI Security Project, an open community effort that tracks risks specific to AI applications, lists prompt injection as the first entry on its current risk list for large language model applications. Its own definition is direct: a prompt injection vulnerability “occurs when user prompts alter the LLM’s behavior or output in unintended ways,” and the project notes that the input causing this “do not need to be human-visible/readable, as long as the content is parsed by the model.” In plain terms, an AI model cannot reliably tell the difference between an instruction its developer gave it and an instruction that showed up disguised as ordinary content. If text tells the model to do something, the model may try to do it, regardless of where that text came from.
Direct Injection vs. Indirect Injection
OWASP splits the problem into two shapes. A direct prompt injection happens when the person typing into the chat is the one trying to manipulate the model, feeding it a cleverly worded message designed to override its guidelines. That version overlaps heavily with what people usually call “jailbreaking,” and OWASP treats jailbreaking as one form of prompt injection rather than a separate thing.
The more unsettling version is indirect prompt injection, where the person using the AI tool is completely innocent and the attack arrives through content the model is simply asked to process. OWASP’s own example is a webpage summary: a page can contain hidden instructions, invisible to a human reader but perfectly readable to the model doing the summarizing, that try to redirect what the AI does next. The same idea applies to an email an assistant is asked to triage, a shared document an agent is asked to read, or a search result a browsing tool pulls in automatically.
Why AI Agents Raise the Stakes
A chatbot that only replies with text is a limited target: the worst an injection usually achieves is a weird or inappropriate answer. The calculation changes once that same model is wired up to take real actions, the pattern behind AI agents and the growing list of tools that connect models to outside systems through standards like the Model Context Protocol. An agent that can send an email, submit a form, or approve a purchase has something worth hijacking, and an attacker doesn’t need to guess a password to try, they just need to get a few lines of hidden text in front of the model at the right moment.
This isn’t a hypothetical risk raised only by security researchers. The makers of today’s AI browsers acknowledge it directly in their own product documentation. OpenAI’s materials for its ChatGPT Atlas browser describe agents as carrying risk that includes “susceptibility to hidden malicious instructions,” and Anthropic’s page for its Claude in Chrome extension describes a dedicated safety layer built specifically to catch “instructions hidden by the site” before an action is allowed to run. Both companies are describing the same underlying problem covered in our explainer on how AI browsers work, and neither claims to have fully solved it.
How Companies Are Trying to Defend Against It
Anthropic’s own developer documentation on mitigating jailbreaks and prompt injections lays out the current best practices. They’re aimed mainly at engineers, but they shape how safe a given AI tool is in practice. The guidance recommends keeping outside content, like a fetched webpage or a tool’s results, clearly separated from a model’s core instructions rather than blending them into the same text block, limiting what an agent is allowed to do so a successful injection “can do minimal damage,” and in some systems, running a smaller model as a screening step on tool outputs before the main model acts on them. None of this is framed as a complete fix. It’s defense in depth: several imperfect safeguards stacked together, because no single one reliably stops every attack.
What This Means If You Use AI Tools
You don’t need to understand the technical defenses to take away the practical lesson: the more autonomy you hand an AI tool, and the more of your own accounts it can touch, the more this problem matters to you personally. Giving a chatbot a document to summarize is low risk. Giving an agent your email, your calendar, and the ability to click through checkout pages is a meaningfully bigger decision, one worth making deliberately rather than by default. A few habits help: start with lower-stakes tasks before handing an agent anything sensitive, check whether a tool asks for explicit confirmation before financial or irreversible actions, and treat “it can browse the web for me” as a feature with a real, currently unsolved tradeoff attached, not a free upgrade.
The Bottom Line
Prompt injection isn’t a bug that a future software update quietly closes. It follows from how today’s AI models work: text is text to them, whether it comes from a trusted developer or a stranger’s webpage, and that ambiguity is exactly what makes AI agents useful and exploitable at the same time. The companies building these tools are open about that tension rather than pretending it’s resolved, which is itself useful information when deciding how much to trust an AI agent with.
FAQs
Is prompt injection the same thing as jailbreaking an AI model?
They overlap but aren’t identical. OWASP’s own framework treats jailbreaking as one specific form of prompt injection, where the person using the AI directly tries to override its guidelines. Prompt injection more broadly also covers indirect attacks, where the instructions arrive hidden inside content the AI is asked to process, with no involvement from the person using the tool.
Can prompt injection steal my data or money?
It can, depending on what the AI tool is connected to. If an agent can send messages, submit forms, or complete purchases, a successful injection could try to direct those abilities toward something the user never asked for. That’s why companies building agentic tools add approval steps for sensitive, hard-to-reverse actions.
Can I protect myself from prompt injection as a regular user?
Not completely, since the underlying weakness lives in how the AI model processes text, not in anything you personally do wrong. You can lower your risk by limiting how much access and autonomy you grant an AI tool, especially over financial accounts, and by favoring products that require explicit confirmation before high-stakes actions.
Have companies fixed prompt injection yet?
No major AI company currently claims the problem is fully solved. Anthropic’s own guidance frames its recommended protections as reducing risk rather than eliminating it, and both OpenAI and Anthropic acknowledge the issue directly in their documentation for agentic products rather than presenting it as resolved.
Does prompt injection only affect chatbots?
No. It affects any system built around a language model that reads text and acts on it, including AI browsers, email assistants, coding tools, and anything built on standards like the Model Context Protocol that connect a model to outside data and tools. The risk scales with what the AI is able to do, not just how it answers questions.












Discussion about this post