AI Guardrails for LLMs and Agents: Types, Open-Source Tools and Limits

ASA AI Team · Updated 2026-11-08 · 4 min read

AI guardrails are runtime controls placed around a large language model or AI agent to check what goes in and what comes out — blocking prompt injection, jailbreaks, sensitive data leaks, off-topic requests and unauthorised tool calls. They are the production counterpart of red teaming: red teaming finds the weaknesses, guardrails enforce the rules.

Key takeaways

Where guardrails sit

CheckpointWhat it checksTypical action
Input railsUser prompts: prompt injection, jailbreak patterns, abusive or off-topic requests, personal dataBlock, rephrase, redact or route to a human
Retrieval and tool-content railsDocuments, web pages and tool outputs before they reach the model (indirect prompt injection)Strip or quarantine suspicious instructions; flag the source
Output railsModel answers: secrets, personal data, harmful or non-compliant content, hallucinated facts, unsafe codeRedact, block, add a disclaimer or regenerate
Execution (tool-call) railsWhich tool an agent wants to call, with which parameters, for which userAllow, deny, or require human approval

Detection techniques

Open-source guardrail tools

ToolMaintainerApproachStrong for
NeMo GuardrailsNVIDIAProgrammable rails (input, output, dialog, retrieval, execution) described in configuration and the Colang languageConversation flows, topical control, orchestrating several checks
Guardrails AIGuardrails AI (open source)Python framework of validators applied to model inputs and outputsStructured output, format and content validation
Llama GuardMetaOpen-weight safety classifier model for prompts and responsesContent-safety classification against a taxonomy
LLM GuardProtect AILibrary of input and output scanners (prompt injection, secrets, PII, toxicity…)Quick, broad scanning in a Python pipeline

Commercial cloud services offer similar capabilities (for example prompt-injection shields and content-safety APIs). For regulated or sensitive data, running guardrails on-premises keeps prompts and responses inside your perimeter.

Limits you must plan for

A deployment checklist

  1. Write the policy first: what is forbidden, for whom, and what happens on detection.
  2. Combine layers: deterministic rules, a classifier and tool-call policies.
  3. Scan untrusted content (RAG documents, tool outputs) as carefully as user input.
  4. Log every decision with the reason, and send blocks to your SIEM.
  5. Measure detection and false-positive rates on your red-team suite after every change.
  6. Keep least privilege and human approval for high-impact actions, whatever the guardrails say.

Frequently asked questions

What are AI guardrails?

Runtime controls around an LLM or AI agent that check inputs, retrieved content, outputs and tool calls against a policy, and block, redact or escalate what violates it.

Is NeMo Guardrails open source?

Yes. NVIDIA NeMo Guardrails is an open-source toolkit for adding programmable input, output, dialog, retrieval and execution rails to LLM applications.

Can guardrails stop prompt injection completely?

No. They significantly reduce it, but determined attackers can find bypasses. Combine guardrails with least privilege, isolation of untrusted content, human approval for sensitive actions and continuous red teaming.

Should guardrails run in the cloud or on-premises?

Both work. On-premises deployment keeps prompts and responses — which often contain sensitive data — inside your own infrastructure, which matters for regulated sectors.

Related guides