Merged
Conversation
rlundeen2
approved these changes
Mar 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR improves the PyRIT CLI experience by making registry naming consistent, fixing bugs, removing dead code, and updating all user-facing text (help, examples, docs) to match the new naming conventions.
SQLite in-memory fix
pyrit_shell, and the SQLAlchemy default pooling method isSingletonThreadPoolfor :memory: databases, which meant the background and main threads maintain their own connections to the DB. So, when the background thread created the DB tables, the main thread had no tables in its connection. Fix: UseStaticPoolwithcheck_same_thread=Falsewhendb_path == ":memory:"to prevent SQLAlchemyProgrammingErrorwhen multiple threads access the same in-memory database.Interrupt feature for
pyrit_shellpyrit_shell.py): Addedexcept KeyboardInterruptindo_runso Ctrl+C during a scenario run returns to the shell prompt instead of crashing the process.Registry Naming Consistency (BREAKING name changes for running commands in CLI)
The initializer and scenario names that were shown to users when running
list-initializersorlist-scenariosdid not match the registry names that were actually needed for running initializers or scenarios.scenario_registry.py): Registry keys are now computed consistently using dotted module paths (e.g.garak.encoding,foundry.red_team_agent). Added collision detection that keeps the first registration and logs a warning for duplicates. Also skips registration of deprecated aliases likeLeakageScenario.initializer_registry.py): InitializerRegistry keys are now computed viaclass_name_to_snake_case(cls.__name__, suffix="Initializer")instead of using filenames. This changestargets→target,scorers→scorer,objective_list→scenario_objective_list, etc.registry_namefield toClassRegistryEntry(base.py): Both registries now pass the suffix-stripped name asregistry_nameto metadata, which is used for display headers informat_scenario_metadataandformat_initializer_metadata. This provides a centralized place to access registry name.snake_class_nameproperty (base.py): No longer needed sinceregistry_namehandles display naming.Dead Code Removal
get_default_initializer_discovery_path()(frontend_core.py): This function was hardcoded to returninitializers/scenarios/, which only contained 3 of 7 initializers. All callers now use the context's initializer registry which discovers all initializers automatically.discovery_pathparameter fromlist_initializers_asyncandprint_initializers_list_async(frontend_core.py), and simplified callers inpyrit_scan.py,pyrit_shell.py, andpyrit_backend.py.initializer_registry.py): Deleted_initializer_pathsdict,resolve_initializer_paths()method, andshort_name/file_pathparameters from_register_initializer. Removed stale file-dedup check in_process_file.Initializer and Scenario Name Renames
pyrit_scan.py,pyrit_shell.py,_banner.py,_cli_args.py, andfrontend_core.py..pyrit_conf_exampleand config env filebuild_scripts/env_local_integration_test.Shell Banner Updates (
_banner.py)list-targetsandclearcommands to the banner command list.Tests and Documentation
New Tests
test_sqlite_memory.py): Verifies that in-memorySQLiteMemoryengines useStaticPooltest_pyrit_shell.py):test_do_run_keyboard_interrupt_returns_to_shell— verifies Ctrl+C prints an interrupted message and does not add the run to history.Updated Tests
ClassRegistryEntryconstructions intest_frontend_core.pynow include theregistry_namefield to match the updated dataclass.TestGetDefaultInitializerDiscoveryPathtest class andtest_list_initializers_with_discovery_pathtest — the function and parameter they tested no longer exist.test_do_list_initializers_with_pathfromtest_pyrit_shell.pyand simplifiedtest_do_list_initializers—discovery_pathparameter was removed.get_default_initializer_discovery_pathmocking fromtest_pyrit_scan.py(test_main_list_initializers,test_main_list_initializers_integration).targets→target,scorers→scorer) intest_frontend_core.pyparse tests andtest_scenarios.py.Documentation Updates
doc/code/front_end/1_pyrit_scan.ipynb/1_pyrit_scan.py: All examples updated to use proper scenario names and initializer names.doc/code/front_end/2_pyrit_shell.md: All shell examples updated; addedlist-targetscommand to command table; removed--databaseoverride from examples.doc/code/registry/1_class_registry.ipynb/1_class_registry.py: Registry usage examples updated from dotted names (e.g.garak.encoding) to short names (e.g.encoding); outputs refreshed.doc/getting_started/pyrit_conf.md: Initializer reference table and config examples updated (targets→target,scorers→scorer).