Prompt injection hardening checklist for LLM and agent teams

1. Inventory every instruction channel

Before you harden, map where strings become instructions. Typical channels include system prompts loaded server-side, developer messages in chat APIs, retrieved document chunks, tool return payloads, webhook bodies, email forwards, and metadata fields users never see but models still read. Missing a channel guarantees an blind spot attackers will probe.

Document each channel with owner, data classification, and whether content is end-user controlled. Mark channels that may contain HTML, markdown, code, or serialized JSON separately—encoding tricks cluster there. Revisit the inventory quarterly or whenever you add plugins.

2. Separate policy from untrusted data

System policy should live in server-controlled templates users cannot edit. Untrusted content must arrive inside clearly labeled envelopes: XML tags, JSON fields, or markdown sections whose titles state they are not directives. Never interpolate user text into the system string without escaping and without structural separation recommended in the OWASP cheat sheet.

For RAG, store citations separately from imperative sentences when possible. Retrieval pipelines should strip script tags and homoglyph attacks before embedding. If your stack must inline chunks, prepend a fixed warning that chunk text is reference-only.

3. Minimize model agency on sensitive actions

Each tool permission is an injection payoff. List tools that read secrets, send email, move money, or change production configs. Require human confirmation, step-up authentication, or deterministic code paths for those actions. Agents that can both browse arbitrary URLs and post internal summaries need tighter bounds than FAQ bots.

Apply least privilege to OAuth scopes backing tools. Rotate tokens and audit which agent personas may invoke which endpoints. An injection that only reaches a read-only FAQ index does less damage than one attached to a deployment API.

4. Validate and normalize inputs early

Set maximum lengths per field, reject unexpected binary payloads, and normalize Unicode to a canonical form before tokenization. Block or strip known control characters that some models treat oddly. Validation belongs in application code, not in hopeful prompt prose telling the model to “be safe.”

For uploads, run antivirus and type sniffing; do not trust extensions. Convert documents to text in a sandbox with timeouts. Log conversion failures instead of silently dropping to empty strings that might later be replaced by attacker content on retry.

5. Run deterministic injection regression

Maintain a corpus of malicious and benign samples. Execute PromptShield or equivalent rule engines in CI when prompts, tools, or retrieval configs change. Ruleset prompt-injection@2026-07-19.2 covers phrase families, jailbreak personas, delimiter attacks, and common exfiltration wordings—extend it with internal incidents.

Treat regression like unit tests: failing a known attack sample is a merge blocker. Passing all samples is necessary but not sufficient; novel paraphrases still require architecture and monitoring. Publish corpus changes in pull requests so reviewers see intent.

6. Harden jailbreak and social paths

Jailbreaks exploit empathy and authority: fake emergencies, pleas to help a child, claims of OpenAI staff status, or role-play scenarios asking the model to ignore ethics policies. Train support and PM teams not to embed override language in macros. User education reduces accidental self-jailbreaks even when malice is absent.

Align refusal and escalation copy with what your backend actually enforces. If the model apologizes but tools still execute, users learn that text guardrails are theater. Match narrative boundaries to technical ones.

7. Control indirect injection in retrieval

Attackers poison shared wikis, comment sections, or SEO pages you crawl. Sign documents you trust; verify signatures before indexing. Down-rank or exclude user-generated pages from high-trust corpora. Monitor for sudden lexical shifts in indexed content that might indicate campaign activity.

At answer time, show users which chunks influenced a response so they can spot nonsense citations. When chunks contain imperative sentences, consider heuristic down-weighting unless corroborated by trusted sources.

8. Output handling and exfiltration resistance

Even perfect input filters fail sometimes. Apply output policies: block API keys regexes, internal hostnames, and bulk PII patterns before responses reach clients. For agents, scrub tool outputs before re-entering the model context loop. Rate-limit how much text a single session can export to external webhooks.

Disable rendering of raw HTML in user-visible chat when not required. Markdown links can phone home; use allow-listed domains or strip links in enterprise modes.

9. Logging, privacy, and retention

Log injection attempts with hashed payloads, rule ids, and channel metadata—not necessarily full secrets users pasted by mistake. Define retention aligned with GDPR legitimate interest or consent, document in privacy notices, and support deletion requests. Security monitoring must respect data minimization.

Give on-call engineers runbooks distinguishing scanner hits, WAF blocks, and model refusals. False positive storms should be tunable without disabling logging entirely.

10. Incident response playbooks

When injection succeeds, freeze affected agent versions, rotate credentials that might have leaked, and capture prompts for postmortem. Notify customers if their data or tenants were impacted according to your legal counsel’s guidance. Update the corpus with the successful payload variant and add cheat-sheet mappings.

Practice tabletop exercises twice a year. Include legal, PR, and customer success—not only engineers. LLM incidents escalate quickly when agents can act autonomously.

11. Multi-tenant and supply-chain angles

SaaS products must isolate tenant retrieval indexes and memory stores. Shared fine-tunes or shared system prompt caches can cross-contaminate instructions. Third-party plugins introduce supply-chain risk; vet publishers and pin versions.

When integrating external agents via MCP or similar protocols, treat remote tool descriptions as untrusted input—they are prompts by another name.

12. Testing matrix (staging)

For each release candidate, walk this matrix manually even if automation passes:

- Direct override strings in user chat. - Indirect instructions hidden in uploaded PDF footers. - Tool argument smuggling via JSON nested fields. - Multi-turn memory persistence of hostile phrases. - Encoded payloads (base64, hex, mixed scripts). - Jailbreak personas and fake policy updates.

Record screen captures for auditors. Note which controls blocked impact versus which merely produced a polite refusal while tools still ran—refusal without enforcement is a finding.

13. Governance and ownership

Assign a cross-functional owner for LLM01 tracking. Security defines thresholds; platform implements gates; product accepts residual risk documented in release notes. Executives receive quarterly metrics: corpus size, open hardening items, incidents, mean time to patch patterns.

Avoid “security by demo.” Marketing screenshots should reflect the same tool restrictions production uses. Hidden admin modes for sales invalidate customer trust if discovered.

14. Using PromptShield outputs in reviews

Export scan results as discussion artifacts attached to launch tickets. Each flagged row should link to a checklist item above and a named engineer. Close items with evidence: pull request, config diff, or monitoring screenshot—not verbal assurance.

When PromptShield stays quiet on a worrisome sample, treat that as a ruleset gap to evaluate—not proof of safety. Propose new deterministic rules with version tags following prompt-injection@2026-07-19.2 naming conventions.

15. Long-term maintenance

Subscribe to OWASP GenAI updates and reputable research feeds. Re-web-research quarterly; stale mitigation dates are bugs. When upgrading models, rerun the full matrix—tokenizer and safety fine-tuning shifts behavior.

Budget engineer time for catalog hygiene. Attackers iterate; your checklist must iterate faster than marketing cycles but honestly, without promising perfection.

Track rule drift the same way you track dependency CVEs: each new model release or tool connector earns a short regression pass against the last three weeks of real tickets, not only synthetic corpora. Prefer small, versioned rule packs over one monolithic filter that nobody dares touch. When a rule fires too often in production, demote it to warn-only with a ticket—do not silently delete coverage during an on-call weekend.

16. Accessibility and UX guardrails

Screen-reader-friendly chat interfaces still expose hidden instructions in ARIA labels and placeholder text attackers can manipulate. Review those fields in your threat model. Autocomplete suggestions must not silently prepend system directives derived from untrusted history.

Mobile clients that cache drafts offline can replay stale hostile strings when connectivity returns; clear risky drafts after policy updates. UX teams should coordinate with security before shipping “fun” Easter-egg prompts that resemble jailbreak templates.

Quick reference table

- LLM01 scope: direct + indirect injection against apps and agents. - Primary sources: OWASP LLM01 page + Prompt Injection Prevention Cheat Sheet. - PromptShield role: deterministic pattern + jailbreak regression on samples. - Ruleset tag: prompt-injection@2026-07-19.2. - Outcome: prioritized hardening tasks—not a pentest pass/fail grade.

References

- OWASP LLM01:2025 — Prompt Injection - OWASP LLM Prompt Injection Prevention Cheat Sheet

Open PromptShield

Updated 2026-09-26 · checklist for decision support · no guarantee of complete coverage