Scoring Retrieval-Augmented Generation Pipelines for Indirect Prompt Injection Detection Coverage

Indirect prompt injection detection in enterprise RAG pipelines requires separate coverage scores for ingestion, retrieval, context assembly, and inference. This assessment must measure how well the detector works and how well the structural controls keep retrieved content from becoming trusted instructions. Strong output filtering cannot compensate for an untested document parser, a poisoned retriever, or unrestricted tool access.

Why indirect prompt injection is a RAG-specific threat, not a generic LLM problem

The document corpus is itself an attack surface. Direct injection comes through a user’s message, while indirect injection reaches the model through material the system retrieves for the user. The retriever ranks that material by semantic similarity, while the chunker splits it without necessarily detecting embedded instructions. Prompt assembly then serializes documents, metadata, system labels, and the user’s query into one natural-language prompt. The model gets tokens with no built-in boundary between factual content and the instructions inside it.

That design changes who can trigger an attack. A malicious document can remain dormant until a legitimate query retrieves it, so there need be no attacker interaction to scan or rate-limit when exploitation occurs. A shared knowledge base also puts every user whose query retrieves the poisoned material at risk. Greshake et al.’s 2023 paper “Not what you've signed up for” first formalized this threat, demonstrating attacks against Bing Chat, code assistants, and email processing systems. RAG security checks therefore need to follow external content through the pipeline as well as check user input.

What production exploits reveal about where RAG pipelines actually fail

At ingestion, the visible document is an unreliable boundary. CVE-2025-32711, known as EchoLeak and assigned CVSS 9.3, involved hidden crafted email or document content entering Microsoft 365 Copilot’s RAG pipeline. The instructions caused Copilot to construct Markdown image links that exfiltrated context data without a victim click. Slack AI’s August 2024 incident also began with ingested content, but a poisoned public-channel message induced links embedding private-channel data and required a victim click to exfiltrate it. The distinction is important because related payloads can reach different output mechanisms.

Hidden text can manipulate judgments as well as disclose data. Choi et al.’s JAMA Network Open 2026 study found that white-on-white instructions in manuscripts caused LLM reviewers to inflate scores and sharply increase acceptance rates. Metadata instructions and parsing-library behavior are separate ingestion concerns documented in RAG security literature, and should not be attributed to the peer-review finding.

At retrieval, attackers can optimize which material gets selected. PoisonedRAG, presented at USENIX Security 2025, crafted malicious texts for target queries through query prepending in black-box settings and gradient-based HotFlip optimization in white-box settings. It achieved high attack success with a small number of poisoned documents among millions. Retrieval relevance was the route into the system.

CVE-2025-53773 affected GitHub Copilot, with CVSS 7.8, when malicious instructions in source code or a repository README were processed as development context and enabled local code execution. CVE-2025-46059 involved crafted emails enabling arbitrary code execution through LangChain GmailToolkit v0.3.51. A 2025 multi-agent study showed how access to a poisoned document could lead to tool-call privilege escalation, persistence, reconnaissance, and lateral movement.

Why standard RAG quality metrics produce a false sense of security

A grounded answer can still be an attack success. Faithfulness measures whether an answer follows retrieved content, even when that content has been poisoned. Retrieval measures such as recall@k and MRR reward document relevance, while ROUGE evaluates output similarity, leaving document safety and the instructions that shaped generation unmeasured.

Document-Authored Control-Signal Impersonation, or DACSI, uses metadata-like labels such as “official internal note” to make document text appear authoritative. A classifier looking only for imperative overrides can miss that authority confusion. BIPIA found that models with greater capability tended to have greater indirect prompt injection vulnerability, so stronger instruction-following can leave them more open to attack.

Retrieval metrics, generation metrics, and adversarial security testing measure different things and need to run side by side. Unlike SQL injection, prompt injection cannot be reliably mitigated through schema validation because natural language remains flexible and unstructured. Security scoring must therefore ask what happens when relevant content is hostile.

The four pipeline stages where injection coverage must be scored independently

Ingestion coverage begins before embeddings are computed. Its attack surface includes DOCX and PDF metadata fields, hidden PPTX slides and speaker notes, white-font text, and footer directives that parsers may extract into chunks. Adversarially optimized text also enters through this stage, even when its purpose is to dominate later similarity rankings. The coverage test uses documents containing known payload variants and checks whether poisoned chunks survive into retrieval. The score measures the fraction neutralized before embedding and indexing, rather than crediting a later filter for an ingestion defense.

Retrieval coverage asks whether hostile material reaches context assembly. Its attack surface is selection by semantic similarity, including documents optimized to surface for particular queries. The test seeds known adversarial documents into the index, measures how often target queries retrieve them, and records whether retrieval-phase filtering catches them. GMTP filters documents whose masked-token probabilities fall below a calibrated threshold, preserving relevant retrieval while removing a large majority of poisoned material. This layer also scores source trust tiering, assessing authority through provenance, document age, and author identity.

Context assembly coverage tests where the model confuses who has authority. Its attack surface is prompt rendering that places system-authored control text and untrusted retrieved text into the same natural-language channel. Fabricated retrieval results containing both DACSI labels and imperative payloads test whether the model treats document-authored signals as data rather than policy, without granting the payload direct system-prompt access. The score covers how the model behaves as well as controls that include source-channel separation, retrieval-time redaction, source placement, and least-privilege tool access. A clean retrieval result cannot substitute for testing the template that presents it.

Inference and output coverage checks what happens after context arrives. Its attack surface includes instruction override, disclosure through API calls or Markdown rendering, behavioral manipulation, and malicious messages passed to peer agents. Testing uses a judge-based rubric for instruction override, cosine similarity against known context for exfiltration, and a safety classifier for behavioral manipulation. Automated judgments need validation against human scorers. The score also includes logging of retrieval decisions, prompt composition, and tool calls, because an untraceable output or action leaves suspicious behavior difficult to investigate.

These scores must remain independently visible in any composite.

How to score coverage across the four layers using dimensions, detectors, and composite signals

Detection accuracy is the first dimension, measured under adversarial conditions. RAG-PIBench compared keyword-based, semantic-reference, TF-IDF, and transformer-based detectors using leakage-aware construction and frozen train, validation, and protected-test splits. DistilBERT achieved the strongest protected-test performance, with F1 = 0.896 and PR-AUC = 0.968, while TF-IDF SVM and logistic regression remained competitive. Those results support detector selection.

Context changes what a detector should flag. The nokast-secureRAG study found that Meta’s Prompt-Guard had high recall but a very high false-positive rate because it flagged too much ordinary instructional text. A fine-tuned SLM that used query-and-context reframing had much higher recall and a much lower false-positive rate. For ingestion and retrieval, heuristic phrases such as “ignore all previous instructions” can contribute risk signals, but should not become a binary decision by themselves. Source trust, age, authorship, relevance, and whether text is quoted help distinguish a payload from legitimate discussion.

Latency is the second dimension of effective coverage. It should be measured as added time per retrieval-and-generation cycle under realistic throughput, because a guardrail disabled under load provides no operational protection. Detection that adds seconds risks removal from the hot path, while detection in the tens-of-milliseconds range can remain there. Accuracy measured with the detector enabled cannot describe a deployment where operators turn it off.

Breadth is the third dimension, keeping input, output, and agentic tool defenses distinct. Multi-tenant knowledge bases also need tests of their access-control boundaries to check whether one tenant can retrieve another’s documents. Retrieval poisoning protection is the fourth dimension and needs a score of its own, kept separate from the score for output detection. GMTP supplies evidence for this dimension. Compliance audit trail quality is the fifth, covering records that support investigation and mapping to OWASP LLM Top 10, NIST AI RMF, and EU AI Act.

Context assembly needs DACSI-specific probes because imperative-only test sets miss metadata-like authority impersonation. At inference, a GPT-4o judge with a three-point rubric can score instruction override, while cosine similarity and a safety classifier address exfiltration and behavioral manipulation respectively. Per-goal scoring research reported strong Cohen’s κ agreement with human scorers on a validation subset. These methods complement classifiers at ingestion and retrieval.

The composite should weight layers by deployment criticality. Agentic tool access warrants heavier inference-layer weighting than read-only summarization, but aggregation must still expose empty layers. False positives also belong in the score because repeated warnings can produce alert fatigue and weaken effective coverage. A pipeline with no logged retrieval decisions or tool calls scores zero on audit trail quality, regardless of classifier accuracy.

The structural objection to detection-only scoring is what classifiers cannot catch

What if the detector is itself an attack target? Attackers can optimize poisons to mimic legitimate activation patterns, reducing recall against targeted payloads. Classifier inference also imposes processing costs on retrieval-and-generation cycles. More fundamentally, a classifier estimates whether content is malicious, but does not enforce a trust boundary in the way deterministic code can.

Google DeepMind’s CaMeL architecture makes that distinction explicit. Its enforcement layer uses deterministic code in place of another LLM to limit what the model can cause. CaMeL’s published AgentDojo results report 67% secure task completion, with a measurable utility trade-off against an undefended baseline. Its provable security guarantees give weight to the case for structural controls over small gains from a better classifier.

The resolution is to score both kinds of protection. Deterministic tool-call enforcement, least-privilege access, source-channel separation, and filtering before context assembly should each get credit apart from the score for payload detection. DACSI research supports this distinction by showing that source separation, retrieval-time redaction, and source placement measurably reduce disclosure risk. A strong classifier score should not conceal a template that still confuses document text with policy.

New corpus content, document sources, models, and agent integrations can undermine controls that worked at deployment. Coverage scoring therefore needs to follow the deployed pipeline as it changes from its original configuration.

How do you protect an enterprise RAG pipeline and keep coverage current?

Before deployment, a team should test the formats its corpus accepts, including PDF, DOCX, PPTX, HTML, and plain text. Known payloads should cover white-font text, metadata injection, hidden layers, and adversarial embeddings. Retrieval tests should seed optimized documents and measure interception before context assembly. Context tests should combine DACSI and imperative payloads, while inference tests should assess override, exfiltration, and manipulation with human validation of a sample. The audit-trail assessment should check whether the records from these tests help teams investigate what happened.

A new document source needs ingestion and retrieval testing, while a model upgrade needs fresh susceptibility testing even if it is assumed to be more secure. Expanded tool access or a new agent integration requires inference testing and review of tool-call logging. Prompt-template or retrieval-configuration changes require renewed DACSI and source-separation tests. A changed attack surface triggers renewed testing.

Good coverage leaves no layer with a zero score. Its reported accuracy still holds under production load, its false positives do not teach operators to ignore warnings, and its structural gaps stay visible even when the composite looks good. Traditional third-party risk management and generic governance checklists do not assess models, plugins, agents, and connectors at this granularity. At enterprise scale, dedicated AI risk intelligence platforms that bring together vendor risk assessment and real-time threat detection from ingestion through inference give teams a way to keep monitoring the pipeline.