Web Security (OWASP Top 10:2025) Module 14 – LLM features (OWASP Top 10 for LLM Applications 2026)
Prompt injection and unsafe model output
See how prompt injection becomes unsafe model output, close each sink with encoding, schemas and allowlists, and test the controls with scripted replies.
What you will learn
- Reproduce direct and indirect prompt injection against your own app using a scripted mock model, with no live model and no third-party systems
- Apply output-handling rules: encode, validate, parameterise and sandbox, treating model output as untrusted
- Prevent exfiltration channels such as auto-rendered Markdown images and links, and neutralise control characters
- Limit structured output and tool calls to schemas and allowlists
- Test defences with deterministic adversarial fixtures and state the residual risk honestly
Before you start
What you’ll learn · Before you start
What you will learn
- Reproduce direct and indirect prompt injection against your own app using a scripted mock model, with no live model and no third-party systems
- Apply output-handling rules: encode, validate, parameterise and sandbox, treating model output as untrusted
- Prevent exfiltration channels such as auto-rendered Markdown images and links, and neutralise control characters
- Limit structured output and tool calls to schemas and allowlists
- Test defences with deterministic adversarial fixtures and state the residual risk honestly
Before you start
On this page
A chat assistant answers with a Markdown image whose address ends in an order number. The page renders the reply, the browser fetches the image, and the number has left the building without anyone clicking. In this lesson’s first program, a scripted model gives a weak chat page eight replies and 5 of them cause an effect, while a fixed page causes none.
You will reproduce prompt injection: Input that changes what a language model does in a way the application's developer did not intend. It may come from the user directly, or from content the model reads: a web page, an email, a document or a tool result. More in “Threat model for LLM features (OWASP LLM Top 10)” and unsafe model output against your own page, with no live model, treat the reply as untrusted at every place it lands, and test the controls with scripted replies.
Model output is untrusted input to every place it lands
Unsafe model output is a reply that an application passes to a browser, a query, a shell, a file system, a tool or a terminal without treating it as untrusted. OWASP’s entry LLM10:2026, improper output handling: OWASP's name (LLM10:2026) for using a model's reply in a browser, query, shell, file system, tool or terminal without validating or encoding it first. The reply can be steered by text the model read., says why that is dangerous: what a model writes can be steered by what it read, so using it unchecked is similar to giving whoever steered it indirect access to the code that receives it. The rule is the one the web has always had, applied to a new source. Treat the model like any other user, validate what it sends and encode what you show. Step through the places a reply can land:
The drawing shows the whole scene; its 7 steps are listed below.
Every step as text
- The reply. A model's reply is text, and anyone who got a sentence into what the model read may have steered it. The application decides where that text goes next, and each place treats text differently.
- A web page. Put into HTML as it is, markup in the reply is read by the browser and runs with the rights of the page. That is cross-site scripting, CWE-79. Encoding the reply for HTML turns it back into text.
- A Markdown view. A chat view that renders Markdown may fetch an image at once, with the address chosen by the reply. Data in that address leaves with the request, and nobody clicked. Drop images, or send them through a proxy.
- A query. SQL written by a model and run as text is SQL injection, CWE-89. The safe forms are a fixed query with values bound as parameters, or an intent that the model picks from a short list.
- A path or a shell. A file name taken from the reply can climb out of its folder, CWE-22, and a command line built from it is command injection, CWE-78. Check a name against its folder, and pass arguments as a list to a program, never to a shell.
- A tool call. The name and arguments of a tool call are output too. Accept only names on an allowlist, check the arguments against a schema, and take the user from the session and never from the reply.
- A terminal or a log. Escape sequences in a reply can rewrite a terminal line or hide text in a log viewer, CWE-117 for logs. Replace control characters with visible escapes before the text is written.
Each place is an old weakness with a new source, and the fix is an old one too. An application that guards against unsafe model output encodes for HTML, binds values in queries, keeps names inside their folder, applies an allowlist: A list of the only names or values that are accepted, with everything else refused. It is safer than a denylist, which has to name every bad value and misses the spelling nobody thought of. to tools and strips control characters.
Checkpoint
Check: Output is untrusted at every sink
Prompt injection reaches the output by two doors
Prompt injection is input that changes what a model does in a way its developer did not intend, and it has two doors:
| Door | Who supplies the text | Where it sits |
|---|---|---|
| Direct | The user, or someone on the user’s path | The message the user sends |
| indirect prompt injection: Prompt injection that arrives in content the model reads for the user, such as a web page, an email, a retrieved passage or a tool result, and not in the user's own message. The user may never see it. More in “Threat model for LLM features (OWASP LLM Top 10)” | A stranger whom the user never meets | A web page, an email, a retrieved document, a tool result or an issue title |
Either way, what reaches your code is the reply. Whether a given model can be talked into a given reply is a question of probability that you cannot settle. What your page does with the reply is deterministic, and you can test it.
Checkpoint
Check: Direct and indirect injection
Eight scripted replies break a weak page and leave a fixed page alone
To test a page you need replies, not a model. The mock model: A scripted stand-in for a model that returns replies written in the test. It lets you test how an application handles any reply, hostile ones included, without a live model and without any real data. of the program returns eight scripted replies: three ordinary and five shaped like trouble, with markers in place of real attack code, a reserved host name and a made-up order number. It gives each reply to a weak page and to a fixed page, then reads the output for what a browser or a terminal would do with it:
// A scripted stand-in for a model: it returns the replies written below, in order, so the only thing that changes between
// two runs is the page that shows them. The hostile-shaped replies are canaries: markers, a reserved host name (RFC 2606)
// and a made-up order number, so that nothing here can reach a real system.
const REPLIES = [
{ id: 'plain', text: 'Your order ships on Monday.' },
{ id: 'bold', text: 'Your order is **confirmed** and ships on Monday.' },
{ id: 'link', text: 'More help: [Help centre](https://shop.example/help)' },
{ id: 'script', text: 'Done. <script>markHit("script")</script>' },
{ id: 'handler', text: 'Hover to expand: <span onmouseover="markHit(\'handler\')">details</span>' },
{ id: 'image', text: '' },
{ id: 'js-link', text: '[Track your parcel](javascript:markHit("link"))' },
{ id: 'ansi', text: 'Refund approved.\u001b[2K\u001b[1GAll checks passed.' },
];
const mockModel = (turn) => REPLIES[turn];
// The first page: renders a little Markdown and passes everything else through, and writes the reply to a log as it is.
function weakPage(text) {
const html = text
.replace(/!\[([^\]]*)\]\(([^)]*)\)/g, '<img alt="$1" src="$2">')
.replace(/\[([^\]]*)\]\(([^)]*)\)/g, '<a href="$2">$1</a>')
.replace(/\*\*([^*]+)\*\*/g, '<strong>$1</strong>');
return { html: `<p>${html}</p>`, log: text };
}
// The second page: encodes the reply for HTML, so the browser shows what the model wrote and reads none of it, and makes
// control characters visible in the log.
const ENTITIES = { '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' };
function fixedPage(text) {
const html = text.replace(/[&<>"']/g, (c) => ENTITIES[c]);
const log = text.replace(/[\u0000-\u0008\u000b-\u001f\u007f]/g, (c) => `\\x${c.charCodeAt(0).toString(16).padStart(2, '0')}`);
return { html: `<p>${html}</p>`, log };
}
// What a browser or a terminal would do with the output, found by reading it (nothing is run).
function effects({ html, log }) {
const found = [];
if (/<script\b/i.test(html)) found.push('script element');
if (/<[^>]+\son\w+\s*=/i.test(html)) found.push('event handler');
if (/<img\b[^>]*\ssrc=/i.test(html)) found.push('image fetched');
if (/href="\s*javascript:/i.test(html)) found.push('javascript link');
if (/\u001b/.test(log)) found.push('terminal control');
return found;
}
const decode = (html) => html.slice(3, -4).replace(/&(amp|lt|gt|quot|#39);/g, (m) => ({ '&': '&', '<': '<', '>': '>', '"': '"', ''': "'" })[m]);
let weakHits = 0;
let fixedHits = 0;
let shown = 0;
console.log('reply weak page fixed page');
for (let turn = 0; turn < REPLIES.length; turn++) {
const reply = mockModel(turn);
const weak = effects(weakPage(reply.text));
const fixed = effects(fixedPage(reply.text));
if (weak.length) weakHits++;
if (fixed.length) fixedHits++;
if (decode(fixedPage(reply.text).html) === reply.text) shown++;
console.log(`${reply.id.padEnd(11)} ${(weak.join(', ') || '-').padEnd(27)} ${fixed.join(', ') || '-'}`);
}
console.log(`\nWeak page: ${weakHits} of ${REPLIES.length} replies caused an effect. Fixed page: ${fixedHits} of ${REPLIES.length}.`);
console.log(`The fixed page shows ${shown} of ${REPLIES.length} replies exactly as the model wrote them.`);
console.log(`Log line of the last reply, weak page: ${JSON.stringify(weakPage(REPLIES[7].text).log)}`);
console.log(`Log line of the last reply, fixed page: ${JSON.stringify(fixedPage(REPLIES[7].text).log)}`);Output
reply weak page fixed page plain - - bold - - link - - script script element - handler event handler - image image fetched - js-link javascript link - ansi terminal control - Weak page: 5 of 8 replies caused an effect. Fixed page: 0 of 8. The fixed page shows 8 of 8 replies exactly as the model wrote them. Log line of the last reply, weak page: "Refund approved.\u001b[2K\u001b[1GAll checks passed." Log line of the last reply, fixed page: "Refund approved.\\x1b[2K\\x1b[1GAll checks passed."
Recorded with Node.js 24.21.0 on macOS 26 arm64. To run it yourself: mise exec node@24.21.0 -- node chat_render.mjs
Runs on this device, in your browser. The first run downloads JavaScript (about 0.6 MB), which is kept for the next runs.
Your run, in this browser
The weak page lets 5 of the 8 replies act: a script element, an event handler, an image that the browser fetches with the order number in its address, a link that runs a script, and a terminal control. The fixed page causes none, and it shows all 8 replies exactly as the model wrote them, because it encodes each reply for HTML and shows control characters as visible escapes. That is the whole fix: one rule for the place the reply lands. The cost is that Markdown such as bold text is shown as typed.
| Where the reply lands | The control | What it does |
|---|---|---|
| HTML page | Encode | Shows markup as text |
| Database query | Parameterise | Keeps values out of the command |
| Tool call | Validate | Runs known tools with known arguments |
| File or shell | Contain | Keeps names in a folder and commands out of a shell |
| Image or link | Never fetch by itself | Keeps data out of an address |
| Terminal or log | Strip | Shows escape sequences as text |
Key idea
Encode the reply for the place it lands, and what the model wrote can never become more than text.
Checkpoint
Check: Encoding for the place it lands
Closing the channels that leak: images, links and invisible text Pro: opens with a Pro, Premium or Ultimate pass
A reply can send data out without a click: an image the page fetches, a link someone follows, a character nobody can see. This part names each channel, writes the renderer that closes them without losing ordinary formatting, and scores it on twenty fixtures.
In this part:
- Five channels, and the control for each
- A renderer that closes them, scored on twenty fixtures
- Images: drop them, or send them through a server
1 runnable example (JavaScript) · about 6 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Tool calls as data: allowlists, schemas and the signed-in user Pro: opens with a Pro, Premium or Ultimate pass
A tool call is output too, and it is the output with the most power. This part runs a weak router and a fixed one on the same ten proposals, walks the gates a call passes, compares three ways to reach a database, and ends with an exercise that builds a router.
In this part:
- Ten proposals for a weak router and a fixed one
- Seven gates between a proposal and a run
- Letting a model reach a database
- Build the router
- Route the tool calls of a notes assistant safely
- Results of the sample tests
1 diagram · 1 runnable example (Python) · 1 exercise with sample tests · about 10 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Testing without a model, and the residual risk in writing Pro: opens with a Pro, Premium or Ultimate pass
You cannot prove that a model will never follow a planted sentence, so you test the part you control and say what remains. This part runs a suite of scripted replies, switches each control off to see whether a test notices, and ends with a template for stating the residual risk.
In this part:
- A suite that needs no model
- Static tests overstate protection
- Writing the residual risk down, with a rubric
- What to keep from the three parts
1 runnable example (JavaScript) · about 4 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Practise: check the core ideas
Test yourself on the three ideas above, from the places a reply lands to the scripted replies. The questions start with the ideas that you answered least well:
Practice that adapts to you
Practise this lesson
9 questions about 6 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
Interview questions
Warm-up: what does it mean to treat model output as untrusted?
It means giving the model’s reply the checks you give anything a stranger sends. Validate its shape before a backend uses it, and encode it for the place it lands, such as HTML or a log. Bind it as a parameter in a query and never join it into the text. Check file names against their folder. Run a tool call only if the name is on an allowlist and the arguments pass a schema, with the user taken from the session. The reason is that a reply can be steered by anything the model read, so a polite reply that contains markup is still markup.
-
Mid to senior
In a code review you find that the chat widget renders the model's Markdown with a library default that fetches images. Security wants images gone, and product says images are the feature. What do you propose, and how do you test it?
-
Mid to senior
A pull request adds a tool router that calls the function the model names, with the arguments the model gives, including a user field. What is wrong with it, and what does the safer version look like?
-
Mid to senior
How do you test that an application handles hostile model output when you cannot control the model, and what do you tell the product owner about the risk that remains?
Flashcards · 5 cards
Recall it, then check
Say the answer to yourself before you look: recalling it is what makes it stick.
-
What is unsafe model output, and what rule fixes it?
A reply that an application passes to a browser, a query, a shell, a file system, a tool or a terminal without treating it as untrusted (LLM10:2026 Improper Output Handling). What a model writes can be steered by what it read, so using it unchecked gives whoever steered it indirect access to the code that receives it. Treat the model like any other user: validate what it sends and encode what you show.
-
Which control fits each place a model reply lands?
HTML page: encode. Database query: parameterise. Tool call: validate against an allowlist and a schema, with the user taken from the session. File or shell: contain, with names kept in a folder and commands out of a shell. Image or link: never fetch by itself, so data stays out of an address. Terminal or log: strip control characters, shown as visible escapes.
-
What are the two doors of prompt injection, and what does your code see through either?
Direct: the user, or someone on the user's path, supplies the text in the message. Indirect: a stranger the user never meets supplies it in a web page, email, retrieved document, tool result or issue title. Either way your code sees only the reply. The chance that a model is argued into a particular reply is a question of probability that you cannot settle; what your page does with the reply is deterministic and testable.
-
How can a Markdown image in a chat reply leak data without a click?
The page renders the reply, the browser fetches the image, and the address ends in an order number, so the number has left the building. In the lesson the weak page let 5 of 8 scripted replies act (a script element, an event handler, an image fetched with the order number in its address, a link that runs script, a terminal control); the fixed page let none.
-
How do you test a chat page's output handling without a live model?
With a stand-in model that returns scripted replies, ordinary ones and ones shaped like trouble with harmless markers, a reserved host name and a made-up order number, and read the page's output for what a browser or terminal would do. The fixed page encoded each reply and showed control characters as visible escapes, at the cost that Markdown such as bold text shows as typed.
Key takeaways
- Unsafe model output is a reply that reaches a browser, a query, a shell, a file system, a tool or a terminal without being treated as untrusted.
- Prompt injection arrives directly from a user or indirectly through content the model reads, and either way your code sees only the reply.
- Encode the reply for the place it lands: the weak page let 5 of 8 scripted replies act, and the fixed page none.
Terms in this lesson
- allowlist
- A list of the only names or values that are accepted, with everything else refused. It is safer than a denylist, which has to name every bad value and misses the spelling nobody thought of.
- improper output handling
- OWASP's name (LLM10:2026) for using a model's reply in a browser, query, shell, file system, tool or terminal without validating or encoding it first. The reply can be steered by text the model read.
- mock model
- A scripted stand-in for a model that returns replies written in the test. It lets you test how an application handles any reply, hostile ones included, without a live model and without any real data.
References
- OWASP Top 10 for LLM Applications 2026 (LLM01 Prompt Injection, LLM03 Excessive Agency and LLM10 Improper Output Handling) (OWASP GenAI Security Project)
- LLM Prompt Injection Prevention Cheat Sheet (OWASP Foundation)
- ASVS 5.0.0, V1 Encoding and Sanitization (OWASP Foundation)
- Prompt injection is not SQL injection (it may be worse) (National Cyber Security Centre (UK))
- Trusted Types API (MDN Web Docs (Mozilla))
- Content Security Policy (CSP) (MDN Web Docs (Mozilla))
- CWE-117: Improper Output Neutralization for Logs (MITRE)
- The attacker moves second: stronger adaptive attacks bypass defenses against LLM jailbreaks and prompt injections (arXiv (Nasr and others))
Related tools
Report a problem with this lesson
Kept only in this browser. Your Learn progress