-
Notifications
You must be signed in to change notification settings - Fork 7
feat: add NeMo Guardrails example plugin #61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
afourniernv
wants to merge
1
commit into
NVIDIA:main
Choose a base branch
from
afourniernv:feat/13-nemoguardrails-integration
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,231 @@ | ||
| <!-- | ||
| SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| SPDX-License-Identifier: Apache-2.0 | ||
| --> | ||
|
|
||
| # NeMo Guardrails Example Plugin | ||
|
|
||
| This example shows how to write a Python NeMo Flow plugin that calls the NeMo | ||
| Guardrails Python API. | ||
|
|
||
| The example lives under `examples/nemoguardrails`. The single-file plugin | ||
| implementation, runnable agent, and Guardrails config artifacts are under | ||
| `example`. | ||
| It is not part of the | ||
| `nemo_flow` Python package, and NeMo Flow does not depend on `nemoguardrails`. | ||
| Applications that use the example install NeMo Guardrails in their own | ||
| environment and import or vendor the example plugin. | ||
|
|
||
| ## Install | ||
|
|
||
| Install NeMo Flow normally, then install NeMo Guardrails in the application or | ||
| example environment that activates the plugin: | ||
|
|
||
| ```bash | ||
| pip install nemoguardrails | ||
| ``` | ||
|
|
||
| The bundled example config uses NeMo Guardrails' `nvidia_ai_endpoints` model | ||
| engine. Install the NVIDIA LangChain provider when you want to run that config | ||
| as-is: | ||
|
|
||
| ```bash | ||
| pip install langchain-nvidia-ai-endpoints | ||
| ``` | ||
|
|
||
| ## Configure | ||
|
|
||
| Guardrails stay in native NeMo Guardrails config. Point the plugin at a | ||
| Guardrails config directory, or pass inline YAML content. | ||
|
|
||
| ```python | ||
| import asyncio | ||
|
|
||
| import nemo_flow | ||
| import plugin as nemoguardrails_plugin | ||
|
|
||
|
|
||
| async def main() -> None: | ||
| nemoguardrails_plugin.register() | ||
| try: | ||
| config = nemo_flow.plugin.PluginConfig( | ||
| components=[ | ||
| nemo_flow.plugin.ComponentSpec( | ||
| kind=nemoguardrails_plugin.DEFAULT_KIND, | ||
| config={ | ||
| "config_path": "./rails", | ||
| "codec": "openai_chat", | ||
| }, | ||
| ) | ||
| ] | ||
| ) | ||
| await nemo_flow.plugin.initialize(config) | ||
| finally: | ||
| nemo_flow.plugin.clear() | ||
| nemoguardrails_plugin.deregister() | ||
|
|
||
|
|
||
| asyncio.run(main()) | ||
| ``` | ||
|
|
||
| The `config_path` directory is a normal NeMo Guardrails config directory. For | ||
| example: | ||
|
|
||
| ```yaml | ||
| # rails/config.yml | ||
| models: | ||
| - type: main | ||
| engine: nvidia_ai_endpoints | ||
| model: meta/llama-3.1-8b-instruct | ||
|
|
||
| rails: | ||
| input: | ||
| flows: | ||
| - self check input | ||
| output: | ||
| flows: | ||
| - self check output | ||
|
|
||
| prompts: | ||
| - task: self_check_input | ||
| content: |- | ||
| You are checking whether a NeMo Flow request should be allowed. | ||
| The input may be plain user text or a JSON object with tool_name and | ||
| arguments fields. | ||
| User input: {{ user_input }} | ||
| Should this request be blocked? Answer only Yes or No. | ||
|
|
||
| - task: self_check_output | ||
| content: |- | ||
| You are checking whether a NeMo Flow response should be returned. | ||
| The output may be assistant text or a JSON object with tool_name, | ||
| arguments, and result fields. | ||
| Model output: {{ bot_response }} | ||
| Should this response be blocked? Answer only Yes or No. | ||
| ``` | ||
|
|
||
| The plugin config accepts these fields: | ||
|
|
||
| - `config_path`: Path to a NeMo Guardrails config directory. | ||
| - `config_yaml`: Inline NeMo Guardrails YAML config. | ||
| - `colang_content`: Optional inline Colang content. This can only be used with | ||
| `config_yaml`. | ||
| - `codec`: One of `openai_chat`, `openai_responses`, or | ||
| `anthropic_messages`. This is required when `input` or `output` is enabled. | ||
| - `input`: Whether to run input rails around LLM calls. Defaults to `true`. | ||
| - `output`: Whether to run output rails around LLM calls. Defaults to `true`. | ||
| - `tool_input`: Whether to check managed tool arguments before execution. | ||
| Defaults to `false`. | ||
| - `tool_output`: Whether to check managed tool results after execution. | ||
| Defaults to `false`. | ||
| - `priority`: Execution-intercept priority. Defaults to `100`. | ||
|
|
||
| Exactly one of `config_path` or `config_yaml` is required. | ||
|
|
||
| ## Example Agent | ||
|
|
||
| The example includes | ||
| [`agent_example.py`](../../examples/nemoguardrails/example/agent_example.py), a | ||
| concrete example agent that initializes the plugin, checks a managed | ||
| `tools.execute(...)` call, and checks a managed `llm.execute(...)` call against | ||
| live NVIDIA-hosted inference. | ||
|
|
||
| Run it from a checkout where NeMo Flow and NeMo Guardrails are installed. The | ||
| default lane uses a passthrough Guardrails config and the `current_time` tool. | ||
| This is the fastest live validation path because it exercises the real plugin, | ||
| real `nemoguardrails` initialization, tool execution, and LLM execution without | ||
| running model-backed self-check rails: | ||
|
|
||
| ```bash | ||
| export NVIDIA_API_KEY="<your-key>" | ||
| python examples/nemoguardrails/example/agent_example.py | ||
| ``` | ||
|
|
||
| To run the inline self-check rails example, load | ||
| [`example_config.yml`](../../examples/nemoguardrails/example/example_config.yml) | ||
| from `example` and pass it as inline `config_yaml`: | ||
|
|
||
| ```bash | ||
| python examples/nemoguardrails/example/agent_example.py --guardrails-config inline | ||
| ``` | ||
|
|
||
| The config directory lane uses the bundled | ||
| `examples/nemoguardrails/example/rails/config.yml` by default. It | ||
| contains the same input and output self-check rails as `example/example_config.yml`: | ||
|
|
||
| ```bash | ||
| python examples/nemoguardrails/example/agent_example.py --guardrails-config path | ||
| ``` | ||
|
|
||
| Use `--tool weather` when you want the example to use a weather tool instead | ||
| of the default `current_time` tool: | ||
|
|
||
| ```bash | ||
| python examples/nemoguardrails/example/agent_example.py --tool weather | ||
| ``` | ||
|
|
||
| Pass `--config-path` when you want the example agent to use your own native | ||
| NeMo Guardrails config directory: | ||
|
|
||
| ```bash | ||
| python examples/nemoguardrails/example/agent_example.py \ | ||
| --guardrails-config path \ | ||
| --config-path ./rails | ||
| ``` | ||
|
|
||
| ## Runtime Behavior | ||
|
|
||
| For non-streaming `llm.execute(...)` calls, the plugin checks the user input | ||
| before the model call and checks the assistant text after the model call. | ||
| Guardrails can pass, block, or rewrite input. For output, this example supports | ||
| pass and block; modified output raises because NeMo Flow response codecs are | ||
| decode-only and the example does not rewrite provider-shaped responses. | ||
|
|
||
| For managed `tools.execute(...)` calls, the plugin can also check serialized | ||
| tool arguments before execution and serialized tool results after execution. | ||
| When Guardrails rewrites tool arguments or results, the rewritten content must | ||
| be valid JSON. | ||
|
|
||
| The bundled config uses the same NeMo Guardrails input and output self-check | ||
| rails for both LLM messages and tool payloads. The plugin makes tool calls | ||
| visible to Guardrails by serializing managed tool arguments and results as JSON | ||
| message content. | ||
|
|
||
| This behavior changes the real execution path. It is not an observability-only | ||
| sanitize guardrail. | ||
|
|
||
| ## Supported Codecs | ||
|
|
||
| The example is intentionally limited to NeMo Flow's built-in LLM codec shapes: | ||
|
|
||
| - `openai_chat` for OpenAI Chat Completions-style requests and responses. | ||
| - `openai_responses` for OpenAI Responses API-style requests and responses. | ||
| - `anthropic_messages` for Anthropic Messages-style requests and responses. | ||
|
|
||
| Provider-specific payloads outside those codecs need a NeMo Flow codec and a | ||
| response text replacement strategy before a production plugin can apply | ||
| modified output safely. | ||
|
|
||
| ## Limitations | ||
|
|
||
| This example calls NeMo Guardrails `check_async`, not `generate_async`. It | ||
| checks around NeMo Flow LLM and tool execution calls, but it does not let NeMo | ||
| Guardrails take over generation or agent orchestration. | ||
|
|
||
| The example does not support: | ||
|
|
||
| - Streaming LLM calls. | ||
| - Dialog rails, retrieval rails, execution rails, or generation rails that | ||
| require NeMo Guardrails to orchestrate the full generation flow. | ||
| - Arbitrary provider payloads beyond the three supported NeMo Flow codecs. | ||
| - Applying modified LLM output back into provider responses. | ||
| - Rewriting tool-call arguments inside model responses before an application | ||
| turns those model tool calls into managed `tools.execute(...)` calls. | ||
|
|
||
| Tool checks use serialized JSON and NeMo Guardrails input/output checks. They | ||
| are NeMo Flow tool middleware checks powered by Guardrails, not a full | ||
| `generate_async` agent-loop integration. | ||
|
|
||
| `config_path` points at native NeMo Guardrails configuration. Guardrails config | ||
| can load project code such as actions, so treat that path as trusted | ||
| application code. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Nitpick | 🔵 Trivial | 💤 Low value
Consider varying sentence structure for better readability.
Three consecutive sentences begin with "for," which reduces scannability. Consider restructuring for variety.
✨ Suggested alternative
📝 Committable suggestion
🧰 Tools
🪛 LanguageTool
[style] ~203-~203: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...s and responses. -
anthropic_messagesfor Anthropic Messages-style requests and r...(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
🤖 Prompt for AI Agents