Procurement Criteria for Evaluating Plugin and Connector Adversarial Instruction Detection Before Vendor Selection

Indirect prompt injection detection in enterprise RAG pipelines requires more than a secure model or a vendor certificate. Before selecting a vendor, procurement teams should require production-representative detection results, controls that separate retrieved data from trusted instructions, and AI-specific audit evidence. This article sets out those criteria and explains why ongoing connector monitoring must keep them enforceable after deployment.

Why generic security questionnaires cannot assess plugin adversarial instruction risk

Traditional security controls assume that trusted instructions and untrusted data are in separate channels. The LLM plugin layer weakens that assumption because both can arrive as natural-language text without a structural distinction between them. A retrieved document may contain useful information and an instruction intended to redirect the agent. Security assessment must ask whether access was authorized and whether content obtained through that access can become an instruction.

A certificate cannot answer that question by itself. Nor can it establish whether a plugin checks the integrity of conversation history transmitted between a browser and an LLM, or distinguishes trusted from untrusted content during retrieval-augmented generation, or RAG. These are properties of the actual data flow. A questionnaire that never checks them can make the system look safe on paper while leaving the path from retrieved content to an instruction untested. Procurement therefore needs a separate assessment of that data flow as a required part of ordinary due diligence.

How do plugins and connectors expose an enterprise LLM to adversarial instructions?

Connectors and protocol servers can have broad, undifferentiated permissions. When an injected instruction redirects an agent, it can inherit the access intended for legitimate work, creating a trust-authorization mismatch. Authentication does not resolve that mismatch on its own. The agent may be allowed to use a tool while the action it has been induced to take falls outside the user’s intent.

The route into that position determines what needs testing. Direct injection can turn input that an attacker controls into forged high-privilege messages when conversation history has no integrity checks. Indirect injection arrives through material the agent retrieves autonomously, including emails, web pages, documents and customer reviews. Stored injection persists in indexed knowledge bases or long-term memory and activates during later, unrelated queries.

Model-level hardening makes the model more resistant to attacks but cannot fix every flaw in transport or ingestion. Instruction hierarchy training and safety alignment also become less effective under iterative attack. The Five Eyes joint guidance, dated April 30 and published May 1, 2026, identifies prompt injection as a core mechanism for manipulating agents and states that no single safeguard is sufficient. Procurement consequently needs evidence about both the model and the surrounding system.

What adversarial instruction attacks at the plugin layer actually look like in production environments

The EchoLeak vulnerability against Microsoft 365 Copilot achieved zero-click data exfiltration when Copilot read a malicious email during normal background processing, without the user opening it. For procurement, the case establishes that testing only content deliberately submitted by users leaves an autonomous retrieval path unexamined.

CVE-2025-54135 targeted Cursor’s AI Agent through indirect prompt injection that made it create a malicious MCP configuration without the user’s approval, which allowed remote code execution. Procurement evaluation must cover configuration changes, execution authority and the safety of generated answers.

For buyers, this extends the relevant test scope to developer automation and supply chain workflows alongside conversational interfaces.

These cases involve different consequences and different entry points. Their common feature is that adversarial content reached an AI system through a workflow the organization already used. When the system can act, the result need not stop at a misleading answer. Authorized data access, API calls and file writes can become the means of executing an attacker’s instruction without another authentication step.

Why detection precision against production traffic is the first procurement filter, not an afterthought

What does a vendor mean by “detects prompt injection”? The first screening question should be the false positive rate, or FPR, against production-representative traffic. A detector that repeatedly flags legitimate work may be operationally unusable even if it catches attacks. Buyers should therefore ask for the traffic distribution behind the number, including whether the tests cover noisy, multilingual and domain-specific inputs. A narrow synthetic test does not establish performance on those workloads.

AgentDojo provides an open benchmark for examining attack success together with useful task completion. Its three core metrics are Benign Utility, Utility Under Attack and Attack Success Rate, but it has no built-in way to calibrate the threshold to a fixed reduction in utility. Procurement teams should ask for these scores along with internal test results and tradeoff curves. Otherwise, a low attack success rate can be difficult to interpret without knowing how much legitimate work the system prevented.

PVDetector reports a false negative rate below 1% across multiple LLMs and datasets, with minimal auxiliary overhead. False negatives describe missed attacks, while false positives describe legitimate traffic incorrectly flagged. A vendor that cannot show its false positives under realistic conditions should not pass the first screening based on detection claims alone.

Runtime detection capability — what vendors must demonstrate at the plugin and connector boundary

Procurement teams should ask vendors to show how pre-filtering works before context assembly. This layer should catch adversarial patterns in raw input, including malicious strings supplied by users and forged message history. An acceptable answer shows where interception occurs relative to the connector boundary and supplies latency benchmarks under production load, because operationally significant delay creates pressure to disable protection.

Vendors should demonstrate semantic and contextual analysis that distinguishes trusted first-party content from untrusted third-party material and detects attempts to hijack the agent’s goal. When AI Meets the Web: Prompt Injection Risks in Third-Party AI Chatbot Plugins, accepted to IEEE S&P 2026, found that nearly all plugins studied failed to make the trust distinction. Tests should cover retrieved documents, tool outputs and inter-agent communications as well as raw input. PVDetector’s analysis of hidden activation states offers a reference for examining reasoning-layer detection, rather than only input-output patterns, without prescribing a single implementation.

Behavioral monitoring must then address what input checks miss. An authenticated agent should trigger a detection signal if it tries to access sensitive files or send data outside the system in ways that fall outside its established intent. Vendors should demonstrate this in real time at the connector boundary. A post-hoc audit log is useful evidence for reconstruction, but it is not a substitute for runtime detection.

Architectural controls — the criteria that determine whether detection is even possible

What prevents a missed instruction from exercising all the connector’s permissions? Detection remains a cat-and-mouse problem, while privilege separation at the protocol level addresses the authority available to an injected instruction. Procurement should score both independently.

The architecture assessment should examine least-privilege token scoping for each task and read-only modes for retrieval-only connectors. It should also establish whether instruction and data channels are structurally tagged or reach the model as an undifferentiated text stream. The IEEE S&P study found that many of the plugins sent conversation history without integrity checks. Vendors should therefore demonstrate that manipulating network requests cannot forge that history. The required diagram should substantiate any “secure” component label by locating instruction-data separation in the actual flow.

A classifier that receives every prompt through an external API also receives the data contained in those prompts. For healthcare and financial buyers, that transfer can create potential compliance violations. Vendors must clarify whether detection runs in-process or through an external service, so Privacy and Legal teams can assess the processing path directly.

Framework mapping and audit requirements — translating detection claims into verifiable compliance evidence

Framework mapping makes technical claims comparable, but only when the mapping points to evidence. Relevant references include OWASP, MITRE ATLAS, NIST AI 600-1, the EU AI Act and ISO 42001. GDPR and NIS2 introduce conditional notification duties when an attack results in a data breach or significant incident. These references connect the tested controls to governance obligations and possible regulatory exposure.

NIST AI 600-1 addresses prompt injection in its Information Security category. Buyers should require documented inclusion of prompt injection defense in the MEASURE evaluation protocol and MANAGE control set. The Cloud Security Alliance’s AI-CAIQ spans 247 control objectives across 18 domains, with mappings to NIST AI 600-1, ISO/IEC 42001 and the EU AI Act. Procurement should ask for AI-CAIQ responses or an equivalent mapping so it can compare vendors on the same terms. A proprietary questionnaire without that mapping makes equivalent coverage harder to establish.

AI-specific penetration testing puts those responses to the test. Its scope should include direct injection, indirect injection through RAG, stored injection in indexed memory and connector-level privilege escalation. Tests should also cover multi-vector chains alongside isolated probes. A generic network or application penetration test provides no evidence about these capabilities unless its scope explicitly includes them.

Continuous monitoring after deployment — why pre-selection evaluation is not sufficient on its own

A successful assessment describes the system as it was when it was tested for selection. New connectors, changed permissions and evolving attacks can separate that assessed system from the one operating in production. Repeated adversarial attempts also weaken the protection offered by model-level hardening. Monitoring must therefore preserve the procurement standard as the deployment changes, rather than merely record that an assessment once occurred.

The Five Eyes guidance calls for “rigorous monitoring and human oversight” and advises organizations to “assume that agentic AI systems may behave unexpectedly and plan deployments accordingly, prioritising resilience, reversibility and risk containment”. Procurement can make that expectation contractual. Requirements should include real-time alerts for connector-level permission-boundary violations and vendor notification when new attack patterns or vulnerabilities affect the connector layer. These obligations address different gaps, because a current runtime alert and notice of a newly identified vulnerability are not interchangeable.

Audit logs should make it possible to piece together an injection event by tracing agent reasoning and tool invocations. Periodic re-evaluation should then use the same AgentDojo and production-representative FPR benchmarks applied at selection. Comparability is the point of retaining those tests. Without it, a later performance claim may describe different traffic or different testing conditions rather than demonstrate that the original standard still holds.

Putting the criteria together — a structured scorecard for pre-selection evaluation of plugin adversarial instruction detection

A usable scorecard separates four dimensions, namely detection capability, architectural soundness, framework compliance and audit traceability. Each needs its own evidence before vendor selection. For detection, the buyer’s question is whether the supplied tradeoff curves, realistic-traffic FPR, latency results and demonstrations establish useful protection at both the input and semantic or reasoning layers. An architecture diagram cannot answer that performance question.

For architectural soundness, the question reverses. Detection numbers cannot demonstrate least-privilege token scoping, read-only connector modes, conversation-history integrity or instruction-data separation. Buyers need the diagram and demonstrations that locate those controls, with a clear statement about external classifier use. This dimension establishes what access an attack could inherit and where sensitive prompt data would travel.

The compliance evidence should connect AI-specific penetration tests, including multi-vector chains, to recognized framework mappings and the documented MEASURE and MANAGE controls. Traceability should show whether alerts, reconstructable logs, notification duties and repeat evaluations will keep the commitment observable after deployment. A SOC 2 reference in place of AI-CAIQ mapping leaves that requested evidence unanswered.

At enterprise scale, administering this framework across vendors creates a coordination burden for TPRM, InfoSec, Privacy and Legal teams. A dedicated AI risk intelligence platform can combine vendor assessment with real-time AI threat detection, providing continuity that point-in-time reviews and generic governance frameworks do not supply alone. The platform’s role is to keep the evidence usable. Procurement teams can then compare vendors on demonstrated capability, identify unsupported claims and keep the selection standard enforceable as connectors and risks change.