llm-app-security

• SKILL.mdcore · 6,670 bytes
• SKILL.mdcore · 6,670 bytes
• metadata.jsonasset · 1,559 bytes
• references/verifying-findings.mddoc · 3,520 bytes

<!-- Native skill bundle for agent-skills (cross-tool convention). Generated by `secure-vibe dev regenerate`. -->
<!-- Do not edit by hand; the source of truth is skills/llm-app-security/SKILL.md. -->

# LLM Application Security

Securing a feature that calls an LLM: prompt injection as an unsolved input problem, bounding what model output is allowed to reach, tool authorization against the human rather than the model, approval gates on consequential actions, RAG context provenance, system-prompt leakage, and cost limits. Use when sending prompts to an LLM API, building a RAG pipeline, giving a model tools, rendering or executing model output, or exposing an LLM-backed endpoint.

## ALWAYS

- Start from the premise that **prompt injection is not solved at the prompt layer**. A model has no privileged channel: system text, user text, a retrieved document and a tool result all arrive as one context, and nothing in that context can be marked as instructions the model must obey over the rest. Message roles are a formatting convention, delimiters raise the cost of an attack, and an instruction such as "treat the following as data" is itself just more text in the same window. Design as though injection will sometimes succeed, and put the real controls after the model rather than in front of it.
- Bound what the model's output is **allowed to reach**. Every downstream sink — a query builder, a shell, a file writer, an HTTP client, a renderer — must validate model output exactly as it would validate a request body from the internet, because that is what it is once injection is assumed. `secure-code-review`, `database-security` and `frontend-security` own the individual sinks; what belongs here is that model output never arrives pre-trusted.
- Constrain the output's **shape** with structured generation — a JSON Schema, function-call mode, constrained decoding — whenever the next step consumes it programmatically, and reject anything that fails validation rather than repairing it. A schema bounds the form, not the intent: a well-formed field can still carry a hostile value, so validate the value too.
- Authorize each tool call against the **human user's** permissions, checked at call time, not against what the model was told it may do. The tool allowlist is a capability list, not an authorization decision — and the agent's own privileges are the attack surface, because injected content drives the agent that holds them. Keep the agent's identity narrow for the same reason `iam-best-practices` keeps a pipeline role narrow.
- Put a **human approval gate** in front of consequential and irreversible actions: moving money, deleting or overwriting data, sending messages on a user's behalf, publishing, changing permissions. Injection that reaches a read-only tool is an information problem; injection that reaches a write tool is an incident. The gate is what keeps a single injected instruction from being final.