HOWTO.md

Observer Python HOWTO

This manual explains how to use Observer's Python-facing provider library.

It is written for projects that want Python-native test authoring while still exposing deterministic provider contracts to Observer.

Quick Start: First 5 Commands

If you want the fastest path to the Python provider model, start in starter/ and run:

make list
make inventory
cat tests.inv
make run
make verify

That shows the whole path in order:

• raw provider host discovery
• derived canonical inventory
• the exact execution contract Observer will run against
• a real suite execution with the human console
• hash and JSONL verification against checked-in expected artifacts

If your application already owns its CLI, use starter-embedded/ instead. That path keeps normal application behavior outside an explicit observe routing point.

1. What This Library Is

This library is the Python-facing micro-SDK for Observer's provider contract.

It lets Python authors write tests with a human-first surface:

• describe(...)
• test(...)
• it(...)
• expect(...)
• ctx.observe().metric/vector/tag

Underneath that surface, Observer still enforces its own rules:

• deterministic collection
• explicit canonical identity
• duplicate validation
• bounded observation
• standard provider host transport for list, run, and observe

2. Smallest Useful Shape

from observer import collect_tests, describe, expect, test


def build_tests():
    with describe("math"):
        @test("adds two numbers")
        def add(ctx):
            ctx.stdout("ok\n")
            expect(2 + 3).to_be(5)


tests = collect_tests(build_tests)

3. Deterministic Identity

If id is omitted, Observer derives the canonical identity from suite path plus test title.

Examples:

• suite path pkg, title smoke test -> pkg :: smoke test
• duplicate derived title in the same suite -> pkg :: smoke test #2

If a test needs a refactor-stable identity, give it an explicit id=:

@test("smoke test", id="pkg::smoke")
def smoke(ctx):
    expect(True).to_be_truthy()

That explicit id becomes both canonical name and target in this first cut.

4. Author Context

Each test receives one context object.

The common author operations are:

• ctx.stdout(...)
• ctx.stderr(...)
• ctx.fail(...)
• ctx.observe()

Telemetry remains observational and bounded:

observer = ctx.observe()
assert observer.metric("wall_time_ns", 104233.0)
assert observer.vector("request_latency_ns", [1000.0, 1100.0, 980.0])
assert observer.tag("resource_path", "fixtures/config.json")

5. Expectations

The first-cut matcher surface includes:

• expect(value).to_be(expected)
• expect(value).to_equal(expected)
• expect(value).to_contain(expected)
• expect(value).to_match(pattern)
• expect(value).to_be_truthy()
• expect(value).to_be_falsy()
• expect(value).not_.to_be(...)

Failures become nonzero test exits with stderr evidence through the standard Observer host path.

6. Provider Host Transport

The Python library owns the standard provider host transport.

That means the host surface is:

• list
• run --target <target> --timeout-ms <u32>
• observe --target <target> --timeout-ms <u32>

Direct host example:

from observer import collect_tests, describe, expect, observer_host_main, test


def build_tests():
    with describe("pkg"):
        @test("smoke test", id="pkg::smoke")
        def smoke(ctx):
            ctx.stdout("ok\n")
            expect(True).to_be_truthy()


if __name__ == "__main__":
    observer_host_main("python", collect_tests(build_tests))

7. Embedded Observe Integration

If a Python application already owns main(), it can expose an embedded observe namespace:

if len(sys.argv) > 1 and sys.argv[1] == "observe":
    observer_host_dispatch_embedded("python", "observe", tests)
else:
    app_main(sys.argv)

That matches the same own-main integration pattern already used in the Rust and TypeScript surfaces.

8. Starter Workflow

The runnable starter in starter/ shows the real adoption path:

1. write ordinary Python code under test
2. author Observer tests in Python
3. expose those tests through a small provider host script
4. derive canonical inventory through observer derive-inventory
5. run a suite against that inventory
6. verify canonical hashes and report output against checked-in snapshots

The failing companion in starter-failure/ keeps the same flow but preserves one intentional failure so the deterministic failure report path stays easy to inspect.

9. Embedded Starter Workflow

The runnable starter in starter-embedded/ shows the application-owned CLI path:

1. keep normal app behavior in main()
2. route observe ... through observer_host_dispatch_embedded(...)
3. configure Observer with args = ["observe"]
4. derive canonical inventory through observer derive-inventory
5. run the same suite shape against the embedded provider boundary
6. verify canonical hashes and report output against checked-in snapshots

That is the preferred model when a Python application already uses its own subcommands and should not give the top-level command namespace to Observer.

10. Library Self-Tests And Packaging

The SDK now carries a stdlib-only self-test suite under tests/:

python3 -m unittest discover -s lib/python/tests -p 'test_*.py'

It also carries minimal packaging metadata in pyproject.toml, so the module can be installed directly from the repo copy when needed:

python3 -m pip install ./lib/python