Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/build-plugins/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Use these guide links to move from the overview into task-specific instructions.
- [Basic Guide: Validate Plugin Configuration](validate-configuration.md) covers JSON-compatible config, validation rules, and structured diagnostics.
- [Basic Guide: Register Plugin Behavior](register-behavior.md) shows how to initialize config and install subscribers or middleware through `PluginContext`.
- [Advanced Guide: Design Plugin Configuration](advanced-configuration.md) covers validation rules, advanced configuration patterns, rollout controls, and `PluginContext` usage.
- [NeMo Guardrails Example Plugin](nemoguardrails.md) shows an external Python plugin that applies NeMo Guardrails checks around NeMo Flow LLM and tool calls.
- [Code Examples](code-examples.md) provides patterns for dynamic header injection, subscriber-oriented export, multi-surface bundles, and framework-facing plugins.

Start by deciding which runtime surfaces the plugin owns: middleware,
Expand Down
231 changes: 231 additions & 0 deletions docs/build-plugins/nemoguardrails.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,231 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

# NeMo Guardrails Example Plugin

This example shows how to write a Python NeMo Flow plugin that calls the NeMo
Guardrails Python API.

The example lives under `examples/nemoguardrails`. The single-file plugin
implementation, runnable agent, and Guardrails config artifacts are under
`example`.
It is not part of the
`nemo_flow` Python package, and NeMo Flow does not depend on `nemoguardrails`.
Applications that use the example install NeMo Guardrails in their own
environment and import or vendor the example plugin.

## Install

Install NeMo Flow normally, then install NeMo Guardrails in the application or
example environment that activates the plugin:

```bash
pip install nemoguardrails
```

The bundled example config uses NeMo Guardrails' `nvidia_ai_endpoints` model
engine. Install the NVIDIA LangChain provider when you want to run that config
as-is:

```bash
pip install langchain-nvidia-ai-endpoints
```

## Configure

Guardrails stay in native NeMo Guardrails config. Point the plugin at a
Guardrails config directory, or pass inline YAML content.

```python
import asyncio

import nemo_flow
import plugin as nemoguardrails_plugin


async def main() -> None:
nemoguardrails_plugin.register()
try:
config = nemo_flow.plugin.PluginConfig(
components=[
nemo_flow.plugin.ComponentSpec(
kind=nemoguardrails_plugin.DEFAULT_KIND,
config={
"config_path": "./rails",
"codec": "openai_chat",
},
)
]
)
await nemo_flow.plugin.initialize(config)
finally:
nemo_flow.plugin.clear()
nemoguardrails_plugin.deregister()


asyncio.run(main())
```

The `config_path` directory is a normal NeMo Guardrails config directory. For
example:

```yaml
# rails/config.yml
models:
- type: main
engine: nvidia_ai_endpoints
model: meta/llama-3.1-8b-instruct

rails:
input:
flows:
- self check input
output:
flows:
- self check output

prompts:
- task: self_check_input
content: |-
You are checking whether a NeMo Flow request should be allowed.
The input may be plain user text or a JSON object with tool_name and
arguments fields.
User input: {{ user_input }}
Should this request be blocked? Answer only Yes or No.

- task: self_check_output
content: |-
You are checking whether a NeMo Flow response should be returned.
The output may be assistant text or a JSON object with tool_name,
arguments, and result fields.
Model output: {{ bot_response }}
Should this response be blocked? Answer only Yes or No.
```

The plugin config accepts these fields:

- `config_path`: Path to a NeMo Guardrails config directory.
- `config_yaml`: Inline NeMo Guardrails YAML config.
- `colang_content`: Optional inline Colang content. This can only be used with
`config_yaml`.
- `codec`: One of `openai_chat`, `openai_responses`, or
`anthropic_messages`. This is required when `input` or `output` is enabled.
- `input`: Whether to run input rails around LLM calls. Defaults to `true`.
- `output`: Whether to run output rails around LLM calls. Defaults to `true`.
- `tool_input`: Whether to check managed tool arguments before execution.
Defaults to `false`.
- `tool_output`: Whether to check managed tool results after execution.
Defaults to `false`.
- `priority`: Execution-intercept priority. Defaults to `100`.

Exactly one of `config_path` or `config_yaml` is required.

## Example Agent

The example includes
[`agent_example.py`](../../examples/nemoguardrails/example/agent_example.py), a
concrete example agent that initializes the plugin, checks a managed
`tools.execute(...)` call, and checks a managed `llm.execute(...)` call against
live NVIDIA-hosted inference.

Run it from a checkout where NeMo Flow and NeMo Guardrails are installed. The
default lane uses a passthrough Guardrails config and the `current_time` tool.
This is the fastest live validation path because it exercises the real plugin,
real `nemoguardrails` initialization, tool execution, and LLM execution without
running model-backed self-check rails:

```bash
export NVIDIA_API_KEY="<your-key>"
python examples/nemoguardrails/example/agent_example.py
```

To run the inline self-check rails example, load
[`example_config.yml`](../../examples/nemoguardrails/example/example_config.yml)
from `example` and pass it as inline `config_yaml`:

```bash
python examples/nemoguardrails/example/agent_example.py --guardrails-config inline
```

The config directory lane uses the bundled
`examples/nemoguardrails/example/rails/config.yml` by default. It
contains the same input and output self-check rails as `example/example_config.yml`:

```bash
python examples/nemoguardrails/example/agent_example.py --guardrails-config path
```

Use `--tool weather` when you want the example to use a weather tool instead
of the default `current_time` tool:

```bash
python examples/nemoguardrails/example/agent_example.py --tool weather
```

Pass `--config-path` when you want the example agent to use your own native
NeMo Guardrails config directory:

```bash
python examples/nemoguardrails/example/agent_example.py \
--guardrails-config path \
--config-path ./rails
```

## Runtime Behavior

For non-streaming `llm.execute(...)` calls, the plugin checks the user input
before the model call and checks the assistant text after the model call.
Guardrails can pass, block, or rewrite input. For output, this example supports
pass and block; modified output raises because NeMo Flow response codecs are
decode-only and the example does not rewrite provider-shaped responses.

For managed `tools.execute(...)` calls, the plugin can also check serialized
tool arguments before execution and serialized tool results after execution.
When Guardrails rewrites tool arguments or results, the rewritten content must
be valid JSON.

The bundled config uses the same NeMo Guardrails input and output self-check
rails for both LLM messages and tool payloads. The plugin makes tool calls
visible to Guardrails by serializing managed tool arguments and results as JSON
message content.

This behavior changes the real execution path. It is not an observability-only
sanitize guardrail.

## Supported Codecs

The example is intentionally limited to NeMo Flow's built-in LLM codec shapes:

- `openai_chat` for OpenAI Chat Completions-style requests and responses.
- `openai_responses` for OpenAI Responses API-style requests and responses.
- `anthropic_messages` for Anthropic Messages-style requests and responses.
Comment on lines +201 to +203
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial | 💤 Low value

Consider varying sentence structure for better readability.

Three consecutive sentences begin with "for," which reduces scannability. Consider restructuring for variety.

✨ Suggested alternative
 The example is intentionally limited to NeMo Flow's built-in LLM codec shapes:
 
-- `openai_chat` for OpenAI Chat Completions-style requests and responses.
-- `openai_responses` for OpenAI Responses API-style requests and responses.
-- `anthropic_messages` for Anthropic Messages-style requests and responses.
+- `openai_chat`: OpenAI Chat Completions-style requests and responses
+- `openai_responses`: OpenAI Responses API-style requests and responses
+- `anthropic_messages`: Anthropic Messages-style requests and responses
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- `openai_chat` for OpenAI Chat Completions-style requests and responses.
- `openai_responses` for OpenAI Responses API-style requests and responses.
- `anthropic_messages` for Anthropic Messages-style requests and responses.
The example is intentionally limited to NeMo Flow's built-in LLM codec shapes:
- `openai_chat`: OpenAI Chat Completions-style requests and responses
- `openai_responses`: OpenAI Responses API-style requests and responses
- `anthropic_messages`: Anthropic Messages-style requests and responses
🧰 Tools
🪛 LanguageTool

[style] ~203-~203: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...s and responses. - anthropic_messages for Anthropic Messages-style requests and r...

(ENGLISH_WORD_REPEAT_BEGINNING_RULE)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/build-plugins/nemoguardrails.md` around lines 201 - 203, Rewrite the
three bullet lines so they vary sentence structure and avoid all starting with
"for"; specifically update the entries for openai_chat, openai_responses, and
anthropic_messages to describe what each plugin handles using different lead-ins
(e.g., "Handles OpenAI Chat Completions-style requests and responses," "Supports
OpenAI Responses API-style requests and responses," "Provides support for
Anthropic Messages-style requests and responses") while keeping the same intent
and keywords.


Provider-specific payloads outside those codecs need a NeMo Flow codec and a
response text replacement strategy before a production plugin can apply
modified output safely.

## Limitations

This example calls NeMo Guardrails `check_async`, not `generate_async`. It
checks around NeMo Flow LLM and tool execution calls, but it does not let NeMo
Guardrails take over generation or agent orchestration.

The example does not support:

- Streaming LLM calls.
- Dialog rails, retrieval rails, execution rails, or generation rails that
require NeMo Guardrails to orchestrate the full generation flow.
- Arbitrary provider payloads beyond the three supported NeMo Flow codecs.
- Applying modified LLM output back into provider responses.
- Rewriting tool-call arguments inside model responses before an application
turns those model tool calls into managed `tools.execute(...)` calls.

Tool checks use serialized JSON and NeMo Guardrails input/output checks. They
are NeMo Flow tool middleware checks powered by Guardrails, not a full
`generate_async` agent-loop integration.

`config_path` points at native NeMo Guardrails configuration. Guardrails config
can load project code such as actions, so treat that path as trusted
application code.
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,7 @@ Basic Guide: Define a Plugin <build-plugins/basic-guide>
Basic Guide: Validate Plugin Configuration <build-plugins/validate-configuration>
Basic Guide: Register Plugin Behavior <build-plugins/register-behavior>
Advanced Guide: Design Plugin Configuration <build-plugins/advanced-configuration>
NeMo Guardrails Example Plugin <build-plugins/nemoguardrails>
Code Examples <build-plugins/code-examples>
```

Expand Down
Loading
Loading