Missing `spec/` Directory and `generate_spec.sh` Script

[dcguim]

Summary

The sdk/python/generate_models.sh and generate_ts_schema_types.js scripts both expect a spec/ directory containing pre-resolved JSON schemas. However:

1. spec/ is gitignored (line 3 of .gitignore)
2. No script exists to generate spec/ from source/schemas/
3. Running the generators fails with FileNotFoundError: /path/to/spec

Background

UCP schemas in source/schemas/ contain ucp_request and ucp_response annotations that control field visibility per direction and operation. These annotations must be resolved before generating SDK models.

Per CONTRIBUTING.md:

"Schemas live in source/ and are published with ucp_* annotations intact. Agents use ucp-schema to resolve annotations for specific operations at runtime."

Current State of Scripts

Script	Input	Output	Status
main.py (MkDocs)	source/schemas/	Resolves at runtime	Works - calls ucp-schema resolve on-demand
generate_ts_schema_types.js	spec/	generated/schema-types.ts	Broken - spec/ doesn't exist
sdk/python/generate_models.sh	../../spec/	src/ucp_sdk/models/	Broken - spec/ doesn't exist

How main.py Resolves Schemas (lines 88-137)

cmd = [
    "ucp-schema",
    "resolve",
    str(schema_path),
    "--request" if direction == "request" else "--response",
    "--op",
    operation,
]
if bundle:
    cmd.append("--bundle")

result = subprocess.run(cmd, capture_output=True, text=True, check=False)

This works for documentation because it resolves at runtime. But the SDK generators need pre-resolved schemas in spec/.

Missing Models

Comparing source/schemas/shopping/ with models/schemas/shopping/:

UCP Schema	Models Generated
cart.json	None - completely missing

All other schemas (checkout.json, discount.json, fulfillment.json, buyer_consent.json, ap2_mandate.json, order.json, payment.json) have corresponding models.

Proposed Solution: generate_spec.sh

Create a script that populates spec/ from source/schemas/ using ucp-schema resolve. This script should:

1. Resolve annotated schemas for all directions/operations
2. Bundle $ref pointers for self-contained schemas
3. Mirror the structure expected by existing generators

Suggested Implementation

#!/bin/bash
# generate_spec.sh - Generate resolved schemas into spec/ directory
#
# This script uses ucp-schema to resolve UCP annotations from source/schemas/
# and outputs self-contained JSON schemas to spec/ for SDK generators.

set -e

cd "$(dirname "$0")"

# Directories
SOURCE_DIR="source/schemas"
SPEC_DIR="spec"

# Schemas with UCP annotations
ANNOTATED_SCHEMAS=(
    "shopping/checkout.json"
    "shopping/cart.json"
    "shopping/discount.json"
    "shopping/fulfillment.json"
    "shopping/ap2_mandate.json"
    "shopping/buyer_consent.json"
)

# Schemas without UCP annotations (just bundle refs)
PLAIN_SCHEMAS=(
    "shopping/order.json"
    "shopping/payment.json"
)

# Operations for request direction (matching existing models)
REQUEST_OPS=("create" "update")

# Check for ucp-schema
if ! command -v ucp-schema &> /dev/null; then
    echo "Error: ucp-schema not found."
    echo "Install with: cargo install ucp-schema"
    exit 1
fi

# Setup directories
echo "Setting up spec/ directory..."
rm -rf "$SPEC_DIR"
mkdir -p "$SPEC_DIR/schemas/shopping"
mkdir -p "$SPEC_DIR/handlers"

# Resolve annotated schemas
echo "Resolving annotated schemas..."
for schema in "${ANNOTATED_SCHEMAS[@]}"; do
    basename=$(basename "$schema" .json)
    echo "  Processing $basename..."

    # Response schema (read operation)
    ucp-schema resolve "$SOURCE_DIR/$schema" \
        --response --op read --bundle --pretty \
        > "$SPEC_DIR/schemas/shopping/${basename}_resp.json"

    # Request schemas for each operation
    for op in "${REQUEST_OPS[@]}"; do
        ucp-schema resolve "$SOURCE_DIR/$schema" \
            --request --op "$op" --bundle --pretty \
            > "$SPEC_DIR/schemas/shopping/${basename}_${op}_req.json"
    done
done

# Bundle plain schemas
echo "Bundling plain schemas..."
for schema in "${PLAIN_SCHEMAS[@]}"; do
    basename=$(basename "$schema" .json)
    echo "  Bundling $basename..."
    ucp-schema resolve "$SOURCE_DIR/$schema" \
        --response --op read --bundle --pretty \
        > "$SPEC_DIR/schemas/shopping/${basename}.json"
done

# Copy types (no resolution needed)
echo "Copying type schemas..."
mkdir -p "$SPEC_DIR/schemas/shopping/types"
cp "$SOURCE_DIR/shopping/types/"*.json "$SPEC_DIR/schemas/shopping/types/"

echo ""
echo "Done! Resolved schemas written to $SPEC_DIR/"
find "$SPEC_DIR" -name "*.json" | wc -l | xargs echo "Total JSON files:"

Updated Workflow

After adding generate_spec.sh:

source/schemas/  --(generate_spec.sh)-->  spec/  --(generate_models.sh)-->  models/
                         |                  |
                         |                  +--(generate_ts_schema_types.js)--> generated/
                         |
                   uses ucp-schema resolve

CI Integration

Add to .github/workflows/docs.yml or create a new workflow:

- name: Generate spec/ from source/
  run: ./generate_spec.sh

- name: Generate Python models
  run: bash sdk/python/generate_models.sh

- name: Generate TypeScript types
  run: node generate_ts_schema_types.js

Previous Attempt

I attempted to create a workaround script at /Users/dguim/localwork/ucp/generate_pydantic_models.sh that:

1. Used ucp-schema resolve to generate resolved schemas
2. Ran datamodel-code-generator on the output

However, this produced problematic Pydantic models. For example, in the generated response/checkout.py:

class Instrument(Checkout):
    """A payment instrument with selection state."""

    model_config = ConfigDict(
        extra="allow",
    )
    selected: bool | None = None

This incorrectly has Instrument inheriting from Checkout, which doesn't make semantic sense - a payment instrument should not be a full checkout object. This appears to be an artifact of how datamodel-code-generator interprets the allOf composition in the bundled JSON Schema.

This suggests that either:

1. The schema bundling strategy needs adjustment
2. datamodel-code-generator options need tuning
3. The spec/ structure should be different from what I attempted

Questions for Maintainers

1. Is this the intended approach? Should spec/ be generated from source/ using ucp-schema?

2. Naming convention: Should resolved schemas use {name}_resp.json / {name}_{op}_req.json format, or a different structure?

3. Why is Cart missing? cart.json exists in source/schemas/shopping/ but has no generated models.

4. Handler schemas: Should source/handlers/ also be resolved into spec/handlers/?

Related Files

• .gitignore line 3: /spec/
• main.py lines 88-137: Runtime resolution for docs
• generate_ts_schema_types.js line 5: SOURCE_ROOT = path.resolve(__dirname, 'spec')
• sdk/python/generate_models.sh line 11: SCHEMA_DIR="../../spec/"
• ucp-schema CLI: https://github.com/universal-commerce-protocol/ucp-schema

[dcguim]

Okay, there was two issues here, the one regarding the checkout inheritance seems to be solved with the latest version (v1.1.0) of ucp-schema:

$ ucp-schema resolve source/schemas/shopping/checkout.json --request --op update --bundle
Error: invalid annotation at /properties/id/update: expected string or object, got object
$  git pull origin main & cargo install --path . --force 
... 
$ ucp-schema resolve source/schemas/shopping/checkout.json --request --op update --bundle
{"$schema":"https://json-schema.org/draft/2020-12/schema" ..

What’s still unclear is the intended workflow for regenerating the Python SDK models from source/schemas. Right now spec/ is git‑ignored and there’s no checked‑in script (e.g. generate_spec.sh or scripts/ci_check_models.sh mentioned in CONTRIBUTING.md) showing:

• how spec/ is meant to be built from source/schemas using ucp-schema,
• which capabilities/operations are officially supported by the Python SDK at a point in time, and
• how to keep sdk/python/src/ucp_sdk/models in sync with the schemas.

For example, I’ve been using this local generate_spec.sh:

#!/usr/bin/env bash
# generate_spec.sh
#
# *Attempt* to generate resolved UCP shopping schemas into spec-test/ in a layout
# which is compatible with sdk/python/generate_models.sh.
#
# Output layout (for shopping):
#   spec-test/schemas/shopping/<name>_resp.json
#   spec-test/schemas/shopping/<name>_create_req.json
#   spec-test/schemas/shopping/<name>_update_req.json
#   spec-test/schemas/shopping/types/*.json

set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

# Shopping source and target dirs
SOURCE_DIR="$ROOT_DIR/source/schemas/shopping"
TARGET_ROOT="$ROOT_DIR/spec-test"
TARGET_SHOPPING="$TARGET_ROOT/schemas/shopping"
TARGET_TYPES="$TARGET_SHOPPING/types"

# Shopping schemas we want as top-level resources
BASE_SCHEMAS=(
  "checkout"
  "discount"
  "fulfillment"
  "buyer_consent"
  "ap2_mandate"
  "order"
  "payment"
)

# Request operations we care about for resources
REQUEST_OPS=("create" "update")

if [[ ! -d "$SOURCE_DIR" ]]; then
  echo "Error: shopping schemas directory does not exist: $SOURCE_DIR" >&2
  exit 1
fi

if ! command -v ucp-schema >/dev/null 2>&1; then
  echo "Error: ucp-schema not found on PATH." >&2
  echo "Install it with: cargo install ucp-schema" >&2
  exit 1
fi

echo "Rebuilding spec-test/ for shopping schemas..."

rm -rf "$TARGET_ROOT"
mkdir -p "$TARGET_SHOPPING" "$TARGET_TYPES"

echo ""
echo "Resolving top-level shopping resources..."
for name in "${BASE_SCHEMAS[@]}"; do
  SRC_PATH="$SOURCE_DIR/${name}.json"
  if [[ ! -f "$SRC_PATH" ]]; then
    echo "  WARN: schema not found, skipping: $SRC_PATH"
    continue
  fi

  echo "  Processing $name..."

  # Response (read) schema
  ucp-schema resolve "$SRC_PATH" \
    --response --op read --bundle --pretty \
    > "$TARGET_SHOPPING/${name}_resp.json"

  # Request schemas for create/update
  for op in "${REQUEST_OPS[@]}"; do
    ucp-schema resolve "$SRC_PATH" \
      --request --op "$op" --bundle --pretty \
      > "$TARGET_SHOPPING/${name}_${op}_req.json"
  done
done

echo ""
echo "Copying type schemas..."
if [[ -d "$SOURCE_DIR/types" ]]; then
  cp "$SOURCE_DIR/types/"*.json "$TARGET_TYPES/" 2>/dev/null || true
else
  echo "  WARN: no types directory at $SOURCE_DIR/types"
fi

echo
echo "Done. Resolved schemas written under:"
echo "  $TARGET_ROOT"

I can now successfully generate resolved JSON schemas into spec/schemas/shopping/{name}_{op}.json using a local generate_spec.sh, but when I run sdk/python/generate_models.sh against spec/, the Python layout diverges from what’s checked in. For example, the current SDK has flat files like

• sdk/python/src/ucp_sdk/models/schemas/shopping/buyer_consent_resp.py,

but the regenerated models end up as packages like sdk/python/src/ucp_sdk/models-test/schemas/shopping/buyer_consent_resp/__init__.py (with nested modules under that folder). The same pattern appears for other resources (fulfillment_, discount_, etc.). It would be very helpful to know the exact spec/ layout and datamodel-code-generator invocation (version + flags, and any post‑processing) you use so we can reproduce the flat buyer_consent_*.py style structure instead of having to tweak your generate_models.sh