Problem Description
While conformance/fixtures/ exists with JSON files (e.g., acl_evaluation.json, normalize_id.json), there isn't a standardized, automated runner to verify these across all SDKs (Python, TypeScript, Rust, Go, etc.) to ensure 100% behavioral parity.
Why solve it
Consistency is a core promise of apcore. A module defined in one language must behave identically in another. Manual verification is error-prone as the spec grows.
- Interoperability: Ensuring consistent execution across different platforms.
- Regression Prevention: Catching bugs early when updating SDKs or the core spec.
- Trust: Providing a reliable 'Conformance Suite' for third-party SDK developers.
How to solve it
- Runner Framework: Create a test runner (likely in Rust or Python) that consumes
conformance/fixtures/*.json.
- Standardized Interface: Define a 'Conformance Driver' (e.g., a simple CLI or JSON-RPC wrapper) for each SDK to implement.
- Automated CI: Integrate the runner into the CI pipelines of all official SDK repositories.
- Coverage: Expand the fixtures to cover more complex scenarios like the 11-step execution pipeline, streaming, and error propagation.
Problem Description
While
conformance/fixtures/exists with JSON files (e.g.,acl_evaluation.json,normalize_id.json), there isn't a standardized, automated runner to verify these across all SDKs (Python, TypeScript, Rust, Go, etc.) to ensure 100% behavioral parity.Why solve it
Consistency is a core promise of apcore. A module defined in one language must behave identically in another. Manual verification is error-prone as the spec grows.
How to solve it
conformance/fixtures/*.json.