-
Notifications
You must be signed in to change notification settings - Fork 23
feat(AWML): introduce new deployment and onnx/tensorRT evaluation pipeline #115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
vividf
wants to merge
31
commits into
main
Choose a base branch
from
feat/new_deployment_and_evaluation_pipeline
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
c77cf92
chore: init
vividf 117dead
chore: add deployment framework intro
vividf 41a3e9a
feat(deployment):prototype deployment framework (#137)
vividf d80db0a
feat(deployment): restructure deployment framework (#160)
vividf a1edc32
chore: add missing pipeline directory and docstring
vividf a362b27
feat(deployment): refactor config and clean code (#180)
vividf d99296c
chore: refactor base config
vividf e23c210
chore: clean up code: device spec, remove unused fucntion .etc
vividf dd46052
chore: refactor export compenent
vividf 2b28f60
chore: fix more Device spec
vividf 85f0d64
chore: enable only trt evaluation
vividf e7439a6
chore: fix verification
vividf ca6f730
chore: fix main
vividf d150f5e
chore: fix base evaluator
vividf 23df89b
chore: clean up base
vividf 36d7568
chore: add base logger
vividf 5bd5b76
chore: update docs
vividf 0eed1f1
chore: add more readme
vividf d706ff7
chore: base add -> None for __init__
vividf 9dfb6a1
chore: remove task profile
vividf a75d739
chore: fix matrix report base
vividf efa71d1
chore: clean metrics
vividf 0c7c57c
chore: update project registry base
vividf ddc5106
feat(deployment): centerpoint deployment integration (#181)
vividf d0b491d
chore: improve docs
vividf 5080b40
chore: clean up docs
vividf a46f933
chore: clean up code
vividf b7b6954
chore: fix verification
vividf f873e85
chore: fix verification 2
vividf 9329b11
chore: clean up __init__.py
vividf 1d4c27f
chor: clean code
vividf File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| # AWML Deployment Framework | ||
|
|
||
| The `deployment/` package is the shared path from trained PyTorch checkpoints to ONNX and TensorRT artifacts, with verification and evaluation built into the same run. Shared runtime code lives in the framework packages, while task-specific logic lives under `deployment/projects/<name>/`. | ||
|
|
||
| ## Core workflow | ||
|
|
||
| ```text | ||
| Load checkpoint -> Export -> Verify -> Evaluate | ||
| ``` | ||
|
|
||
| ## Quick start | ||
|
|
||
| ```bash | ||
| python -m deployment.cli.main <project_name> <deploy_cfg.py> <model_cfg.py> [--log-level INFO] | ||
|
|
||
| # Example (CenterPoint) | ||
| python -m deployment.cli.main centerpoint \ | ||
| deployment/projects/centerpoint/config/deploy_config.py \ | ||
| <model_cfg.py> \ | ||
| --rot-y-axis-reference | ||
| ``` | ||
|
|
||
| ## What to read | ||
|
|
||
| | If you want to... | Start here | | ||
| | --- | --- | | ||
| | Run deployment today | [docs/runbook.md](docs/runbook.md) | | ||
| | Edit or author deploy config | [docs/configuration.md](docs/configuration.md) | | ||
| | Understand the framework structure | [docs/architecture.md](docs/architecture.md) | | ||
| | Troubleshoot or tune runs | [docs/operations.md](docs/operations.md) | | ||
| | Add a new project bundle | [docs/contributing.md](docs/contributing.md) | | ||
| | Run the current shipped project | [projects/centerpoint/README.md](projects/centerpoint/README.md) | | ||
|
|
||
| ## Current status | ||
|
|
||
| - Current first-class project: [CenterPoint](projects/centerpoint/README.md) | ||
| - Shared framework responsibilities: CLI, typed config, exporters, runtime orchestration, verification, evaluation, and pipeline creation | ||
|
|
||
| ## Repository layout | ||
|
|
||
| ```text | ||
| deployment/ | ||
| ├── cli/ # Unified CLI | ||
| ├── configs/ # Typed deploy config schema | ||
| ├── core/ # Shared types, evaluators, verification mixins | ||
| ├── exporters/ # ONNX / TensorRT exporters and export pipeline bases | ||
| ├── pipelines/ # Inference pipelines and global factory | ||
| ├── runtime/ # BaseDeploymentRunner, orchestrators, ArtifactManager | ||
| └── projects/ # Per-task bundles | ||
| ``` | ||
|
|
||
| ## License | ||
|
|
||
| See `LICENSE` at the repository root. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| """Autoware ML Deployment Framework. | ||
|
|
||
| Task-agnostic export, verification, and evaluation across backends (ONNX, TensorRT). | ||
| Import from concrete submodules (for example ``deployment.configs.base``, ``deployment.runtime.runner``). | ||
| """ | ||
|
|
||
| __version__ = "1.0.0" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| """Deployment CLI package.""" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| """ | ||
| CLI helpers: argument parsing and logging setup. | ||
|
|
||
| Config schema does not depend on argparse/logging; this module is the single place for CLI concerns. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import argparse | ||
| import logging | ||
| import os | ||
| from typing import Optional | ||
|
|
||
| _LOG_FORMAT = "%(levelname)s:%(name)s:%(message)s" | ||
|
|
||
|
|
||
| def setup_logging(level: str = "INFO") -> logging.Logger: | ||
| """ | ||
| Setup logging configuration. | ||
|
|
||
| Args: | ||
| level: Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) | ||
|
|
||
| Returns: | ||
| Configured logger instance | ||
| """ | ||
| logging.basicConfig(level=getattr(logging, level), format=_LOG_FORMAT) | ||
| return logging.getLogger("deployment") | ||
|
|
||
|
|
||
| def add_deployment_file_logging(log_file_path: str) -> None: | ||
| """ | ||
| Append a UTF-8 file handler to the root logger so all log records are also written to disk. | ||
|
|
||
| Idempotent for the same absolute path. Creates parent directories when needed. | ||
|
|
||
| Args: | ||
| log_file_path: Absolute or resolved path to the log file. | ||
| """ | ||
| path = os.path.abspath(os.path.expanduser(log_file_path)) | ||
| parent = os.path.dirname(path) | ||
| if parent: | ||
| os.makedirs(parent, exist_ok=True) | ||
|
|
||
| root = logging.getLogger() | ||
| for h in root.handlers: | ||
| if isinstance(h, logging.FileHandler): | ||
| if os.path.abspath(getattr(h, "baseFilename", "")) == path: | ||
| return | ||
|
|
||
| fh = logging.FileHandler(path, mode="a", encoding="utf-8") | ||
| fh.setFormatter(logging.Formatter(_LOG_FORMAT)) | ||
| fh.setLevel(root.level) | ||
| root.addHandler(fh) | ||
|
|
||
|
|
||
| def parse_base_args( | ||
| parser: Optional[argparse.ArgumentParser] = None, | ||
| ) -> argparse.ArgumentParser: | ||
| """ | ||
| Create argument parser with common deployment arguments. | ||
|
|
||
| Args: | ||
| parser: Optional existing ArgumentParser to add arguments to | ||
|
|
||
| Returns: | ||
| ArgumentParser with deployment arguments | ||
| """ | ||
| if parser is None: | ||
| parser = argparse.ArgumentParser( | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why |
||
| description="Deploy model to ONNX/TensorRT", | ||
| formatter_class=argparse.ArgumentDefaultsHelpFormatter, | ||
| ) | ||
|
|
||
| parser.add_argument("deploy_cfg", help="Deploy config path") | ||
| parser.add_argument("model_cfg", help="Model config path") | ||
| # Optional overrides | ||
| parser.add_argument( | ||
| "--log-level", | ||
| default="INFO", | ||
| choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"], | ||
| help="Logging level", | ||
| ) | ||
|
|
||
| return parser | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| """ | ||
| Single deployment entrypoint. | ||
|
|
||
| Usage: | ||
| python -m deployment.cli.main <project> <deploy_cfg.py> <model_cfg.py> [project-specific args] | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import argparse | ||
| import importlib | ||
| import pkgutil | ||
| import sys | ||
| import traceback | ||
| from typing import List | ||
|
|
||
| import deployment.projects as projects_pkg | ||
| from deployment.cli.args import parse_base_args | ||
| from deployment.projects.registry import project_registry | ||
|
|
||
|
|
||
| def _discover_project_packages() -> List[str]: | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| """Discover project package names under deployment.projects (without importing them).""" | ||
|
|
||
| names: List[str] = [] | ||
| for mod in pkgutil.iter_modules(projects_pkg.__path__): | ||
| if not mod.ispkg: | ||
| continue | ||
| if mod.name.startswith("_"): | ||
| continue | ||
| names.append(mod.name) | ||
| return sorted(names) | ||
|
|
||
|
|
||
| def _import_and_register_project(project_name: str) -> None: | ||
| """Import project package, which should register itself into project_registry.""" | ||
| importlib.import_module(f"deployment.projects.{project_name}") | ||
|
|
||
|
|
||
| def build_parser() -> argparse.ArgumentParser: | ||
| """Build the deployment CLI parser. | ||
|
|
||
| This discovers `deployment.projects.<name>` bundles, imports them to trigger | ||
| registration into ``deployment.projects.registry.project_registry``, then creates a | ||
| subcommand per registered project. | ||
| """ | ||
| parser = argparse.ArgumentParser( | ||
| description="AWML Deployment CLI", | ||
| formatter_class=argparse.ArgumentDefaultsHelpFormatter, | ||
| ) | ||
|
|
||
| subparsers = parser.add_subparsers(dest="project", required=True) | ||
|
|
||
| # Discover projects and import them so they can contribute args. | ||
| failed_projects: List[str] = [] | ||
| for project_name in _discover_project_packages(): | ||
| try: | ||
| _import_and_register_project(project_name) | ||
| except Exception as e: | ||
| tb = traceback.format_exc() | ||
| failed_projects.append(f"- {project_name}: {e}\n{tb}") | ||
| continue | ||
|
|
||
| try: | ||
| adapter = project_registry.get(project_name) | ||
| except KeyError: | ||
| continue | ||
|
|
||
| sub = subparsers.add_parser(project_name, help=f"{project_name} deployment") | ||
| parse_base_args(sub) # adds deploy_cfg, model_cfg, --log-level | ||
| adapter.add_args(sub) | ||
| sub.set_defaults(_adapter_name=project_name) | ||
|
|
||
| if not project_registry.list_projects(): | ||
| details = "\n".join(failed_projects) if failed_projects else "(no project packages discovered)" | ||
| raise RuntimeError( | ||
| "No deployment projects were registered. This usually means project imports failed.\n" f"{details}" | ||
| ) | ||
|
|
||
| return parser | ||
|
|
||
|
|
||
| def main(argv: List[str] | None = None) -> int: | ||
| """CLI entrypoint. | ||
|
|
||
| Args: | ||
| argv: Optional argv list (without program name). If None, uses `sys.argv[1:]`. | ||
|
|
||
| Returns: | ||
| Process exit code (0 for success). | ||
| """ | ||
| argv = sys.argv[1:] if argv is None else argv | ||
| parser = build_parser() | ||
| args = parser.parse_args(argv) | ||
|
|
||
| adapter = project_registry.get(args._adapter_name) | ||
| return adapter.run(args) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| raise SystemExit(main()) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| """Deployment configuration package (enums, schema, base container). | ||
|
|
||
| Import from concrete modules, e.g. ``deployment.configs.base``, ``deployment.configs.schema``, ``deployment.configs.enums``. | ||
| """ |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use
pathlibinstead