Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,24 +22,24 @@ jobs:
with:
python-version: "3.10"

- name: Install linting dependencies
- name: Install dependencies
run: |
pip install --upgrade pip
pip install ".[aimodel,huggingface]"
pip install . --group lint --group test
pip install --group lint --group test

- name: Ruff linter
- name: Ruff check
run: |
ruff check src/ tests/
ruff check examples/ src/ tests/

- name: Pylint
run: |
pylint src/ tests/
pylint examples/ src/ tests/

- name: Flake8
run: |
flake8 src/ tests/
flake8 examples/ src/ tests/

- name: Ruff formatter check
run: |
ruff format --check src/ tests/
ruff format --check examples/ src/ tests/
56 changes: 9 additions & 47 deletions .github/workflows/typecheck.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,56 +24,18 @@ jobs:

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install . --group typecheck
pip install --upgrade pip
pip install ".[aimodel,huggingface]"
pip install --group test --group typecheck

- name: Run mypy
- name: mypy
run: |
mypy src/pitloom --ignore-missing-imports --no-strict-optional
continue-on-error: true
mypy examples/ src/ tests/

pyright:
name: pyright
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.10"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pyright
pip install -e .

- name: Run pyright
run: |
pyright src/pitloom
continue-on-error: true

pyrefly:
name: pyrefly (via pyflakes)
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.10"

- name: Install dependencies
- name: pyright
run: |
python -m pip install --upgrade pip
pip install pyflakes
pip install -e .
pyright examples/ src/ tests/

- name: Run pyflakes
- name: pyrefly
run: |
pyflakes src/pitloom
continue-on-error: true
pyrefly check examples/ src/ tests/
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,21 @@ and this project adheres to
- Full release notes: <https://github.com/bact/pitloom/releases>
- Commit history: <https://github.com/bact/pitloom/compare/v0.7.0...v0.7.1>

## [UNRELEASED]

### Added

- A complete example for SBOM fragments generation and aggregation

### Changed

- Rename context manager method from `loom.shoot` to `loom.run`;
make consistent with MLflow

### Fixed

- Fix wrong fickling import in in PyTorch extractor

## [0.7.1] - 2026-05-14

### Changed
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ abstract: Automated transparency, woven from the ground up. SBOM generation for
repository-code: "https://github.com/bact/pitloom"
type: software
doi: 10.5281/zenodo.19243681
version: 0.7.1
version: 0.8.0
license-url: "https://spdx.org/licenses/Apache-2.0"
keywords:
- sbom
Expand All @@ -21,4 +21,4 @@ keywords:
- gguf
- onnx
- safetensors
date-released: 2026-05-14
date-released: 2026-05-25
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,14 +273,14 @@ external SBOM fragments that Pitloom will merge during the build process:
from pitloom import loom

# Use as a function decorator...
@loom.shoot(output_file="fragments/sentiment_model.json")
@loom.run(output_file="fragments/sentiment_model.json")
def train_model():
loom.set_model("sentiment-clf")
loom.add_dataset("imdb-reviews", dataset_type="text")
# ... training logic ...

# ...or use as a context manager
with loom.shoot(output_file="fragments/sentiment_model.json"):
with loom.run(output_file="fragments/sentiment_model.json"):
loom.set_model("sentiment-clf")
loom.add_dataset("imdb-reviews", dataset_type="text")
```
Expand All @@ -305,6 +305,8 @@ The generated SBOM will include:
- Creator and creation timestamp information
- **Metadata provenance** tracking for transparency

See a more complete example in [examples/`](./examples/) directory.

## Metadata provenance

Pitloom tracks the source of each metadata field in the SBOM using the SPDX 3
Expand Down
6 changes: 3 additions & 3 deletions codemeta.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@
"codeRepository": "https://github.com/bact/pitloom",
"copyrightYear": 2026,
"dateCreated": "2026-03-27",
"dateModified": "2026-05-14",
"datePublished": "2026-05-14",
"dateModified": "2026-05-25",
"datePublished": "2026-05-25",
"description": "Automated transparency, woven from the ground up. SBOM generation for Python & AI projects. Extract metadata from GGUF, ONNX, PyTorch, and Safetensors models with native Hatchling build-hook support.",
"developmentStatus": "active",
"downloadUrl": "https://github.com/bact/pitloom/releases",
Expand Down Expand Up @@ -54,5 +54,5 @@
"programmingLanguage": "Python 3",
"readme": "https://github.com/bact/pitloom/blob/main/README.md",
"url": "https://github.com/bact/pitloom",
"version": "0.7.1"
"version": "0.8.0"
}
2 changes: 1 addition & 1 deletion docs/design/format-neutral-representation.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Extractors Core model Serializers / Assemblers
read_pyproject() ─┐
read_setuptools() ─┤
read_ai_model() ─┤─-> DocumentModel ─-> Spdx3Assembler -> SPDX 3 JSON-LD
loom.shoot() (fragments) ─┘ (pitloom.core) [future] CycloneDXAssembler -> CycloneDX JSON
loom.run() (fragments) ─┘ (pitloom.core) [future] CycloneDXAssembler -> CycloneDX JSON
[future] AidocRenderer -> AIDOC markdown
[future] TechOpsDoc -> documentation
```
Expand Down
2 changes: 1 addition & 1 deletion docs/design/mlflow-extractor.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ with mlflow.start_run():
mlflow.log_param("learning_rate", 3e-4)
mlflow.log_metric("accuracy", 0.95)

with loom.shoot("fragment.spdx3.json") as run:
with loom.run("fragment.spdx3.json") as run:
run.set_model("my-transformer")
# The same facts, typed again
```
Expand Down
36 changes: 18 additions & 18 deletions docs/design/sbom-fragments.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ for each object. This approach:

### 3. `loom.py` SDK: sparse API surface

The current `Shoot` context manager supports only `set_model` and
The current `Run` context manager supports only `set_model` and
`add_dataset`. This is far less expressive than ML tracking SDKs that
practitioners already use daily. Key missing capabilities:

Expand All @@ -158,7 +158,7 @@ practitioners already use daily. Key missing capabilities:
- No serialisation of individual dataset elements with schema, provenance,
or curation notes.
- No model evaluation / scoring records.
- Error is raised if `loom.*` functions are called outside a `Shoot` block;
- Error is raised if `loom.*` functions are called outside a `Run` block;
notebook workflows need a more lenient persistent-session mode.

### 4. No W&B Weave integration
Expand Down Expand Up @@ -326,33 +326,33 @@ in `docs/design/mlflow-extractor.md`.
from pitloom import loom

# --- Context-managed fragment recording (existing, enhanced) ---
with loom.shoot("fragments/bert-v3.spdx3.json") as shot:
shot.set_model("my-bert", type_of_model="transformer")
with loom.run("fragments/bert-v3.spdx3.json") as run:
run.set_model("my-bert", model_type="transformer")

# MLflow-compatible logging functions
shot.log_param("learning_rate", 3e-4)
shot.log_param("batch_size", 32)
shot.log_metric("accuracy", 0.91)
shot.log_metric("f1_score", 0.88)
shot.log_tag("domain", "natural_language_processing")
shot.log_tag(stav.INFO_TRAINING, "Fine-tuned on FLORES-200")
run.log_param("learning_rate", 3e-4)
run.log_param("batch_size", 32)
run.log_metric("accuracy", 0.91)
run.log_metric("f1_score", 0.88)
run.log_tag("domain", "natural_language_processing")
run.log_tag(stav.INFO_TRAINING, "Fine-tuned on FLORES-200")

# Dataset documentation
ds = shot.add_dataset("flores-200", dataset_type="text")
ds = run.add_dataset("flores-200", dataset_type="text")
ds.set_size(rows=5_000_000)
ds.set_license("CC-BY-4.0")
ds.set_source_url("https://huggingface.co/datasets/facebook/flores")
ds.set_preprocessing("tokenized, lowercased, de-duplicated")
ds.log_tag("language_count", "200")

# Evaluation results (maps to SPDX Annotation)
shot.log_evaluation("flores-dev", {"accuracy": 0.91, "bleu": 42.3})
run.log_evaluation("flores-dev", {"accuracy": 0.91, "bleu": 42.3})

# --- Persistent session mode (for notebooks) ---
loom.start_session("fragments/notebook-run.spdx3.json")

# ... cell 1 ...
loom.set_model("incremental-model", type_of_model="classifier")
loom.set_model("incremental-model", model_type="classifier")
loom.log_param("epochs", 10)

# ... cell 2 (appends to same session) ...
Expand All @@ -367,12 +367,12 @@ loom.end_session()
### Accumulation mode for notebooks

The persistent session is backed by an `_ActiveSession` object (distinct
from `_ActiveShot`) that persists in module-level state and writes a
from `_ActiveRun`) that persists in module-level state and writes a
checkpoint file to disk on each `loom.save_session()` call. If the kernel
restarts, `loom.resume_session("fragments/notebook-run.spdx3.json")` reads
the last checkpoint and continues accumulating.

Key difference from `Shoot`: a session does **not** discard partial output
Key difference from `Run`: a session does **not** discard partial output
on exception; it preserves whatever has been recorded up to the crash.

### IPython magic integration
Expand Down Expand Up @@ -659,11 +659,11 @@ all earlier items; each can be delivered independently.

### Phase 2: SDK improvements (notebook and ML workflow ergonomics)

1. **`log_param`, `log_metric`, `log_tag` on `_ActiveShot`** -- expands the
existing `Shoot` API without breaking changes.
1. **`log_param`, `log_metric`, `log_tag` on `_ActiveRun`** -- expands the
existing `Run` API without breaking changes.
2. **`add_dataset` builder object** -- replace the current `add_dataset(name,
type)` with a fluent builder that supports `set_size`, `set_license`, etc.
3. **`log_evaluation` on `_ActiveShot`** -- maps to SPDX `Annotation` elements.
3. **`log_evaluation` on `_ActiveRun`** -- maps to SPDX `Annotation` elements.
4. **Persistent session mode** -- `loom.start_session()` / `loom.end_session()`.
5. **IPython magic** -- `%%pitloom_record` cell magic; optional, only activated
if `ipython` is installed.
Expand Down
2 changes: 1 addition & 1 deletion docs/implementation/demo.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ dynamically from training loops into your main SBOM seamlessly:
```python
from pitloom import loom

@loom.shoot("fragments/model.json")
@loom.run("fragments/model.json")
def run():
loom.set_model("my-classifier")
loom.add_dataset("my-text-data")
Expand Down
4 changes: 2 additions & 2 deletions docs/implementation/summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ SPDX 3.0 compliant SBOMs in JSON-LD format.
- See [docs/design/metadata-provenance.md](../design/metadata-provenance.md)

8. **ML tracking SDK** (`src/pitloom/loom.py`)
- Dual-syntax ContextDecorator (`@loom.shoot` and `with loom.shoot`)
- Dual-syntax ContextDecorator (`@loom.run` and `with loom.run`)
- Emits SPDX 3 SBOM fragments automatically during ML executions
- Seamlessly ingested into project SBOMs using `[tool.pitloom.fragments]` config

Expand Down Expand Up @@ -229,7 +229,7 @@ pitloom/
│ ├── __about__.py # Package version (__version__)
│ ├── __init__.py
│ ├── __main__.py # CLI entry point (loom / python -m pitloom)
│ ├── loom.py # ML tracking SDK (Shoot context manager / decorator)
│ ├── loom.py # ML tracking SDK (Run context manager / decorator)
│ └── py.typed # PEP 561 marker
├── tests/
│ ├── fixtures/
Expand Down
8 changes: 8 additions & 0 deletions examples/sentimentdemo-aibom/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Build output
dist/
*.egg-info/

# Generated artefacts - regenerate by re-running the pipeline.
data/processed/
models/
fragments/
Loading
Loading