Add frame_format parameter to VideoScene for PNG support

[vinay553]

• Add FRAME_FORMAT_KEY constant
• Add frame_format parameter to VideoScene dataclass
• Update to_payload() to include frame_format when set
• Supports 'jpeg' (default) or 'png' for lossless frame extraction
• Bump version to 0.17.12
• Remove broken test

Greptile Summary

This PR adds an optional frame_format parameter to VideoScene to support PNG (lossless) frame extraction when uploading videos via video_location, defaulting to JPEG. It also bumps the version to 0.17.12 and removes a broken integration test.

• Bug: VideoScene.from_json() was not updated to deserialize the new frame_format field, so round-tripping through to_payload() → from_json() will silently drop it.
• The validate() method does not enforce that frame_format is one of the documented valid values ("jpeg" or "png").
• No unit tests were added for the new frame_format parameter (e.g., verifying serialization/deserialization or validation of invalid values).
• The removed test (test_job_listing_and_retrieval) is unrelated to the feature being added.

Confidence Score: 2/5

• This PR has a deserialization bug in from_json() that will cause data loss when round-tripping VideoScenes with frame_format set.
• The from_json() method was not updated alongside to_payload(), creating an asymmetry where frame_format is serialized but never deserialized. This is a functional bug that will affect any code path that reconstructs a VideoScene from JSON (e.g., the existing test fixtures use from_json). Additionally, there are no new tests for the feature.
• nucleus/scene.py — the from_json() method needs to be updated to include frame_format, and validate() should enforce valid values.

Important Files Changed

Filename	Overview
nucleus/scene.py	Adds frame_format field and serialization in to_payload(), but from_json() does not deserialize it, breaking round-trip consistency. Also lacks input validation for the new parameter.
nucleus/constants.py	Adds FRAME_FORMAT_KEY constant in alphabetical order. Clean, no issues.
CHANGELOG.md	Documents the new frame_format parameter with a usage example. Well-formatted.
pyproject.toml	Version bump from 0.17.11 to 0.17.12. Straightforward change.
tests/test_jobs.py	Removes test_job_listing_and_retrieval integration test described as broken. No replacement test added for the new frame_format feature.

Sequence Diagram

sequenceDiagram
    participant User
    participant VideoScene
    participant Payload as JSON Payload
    participant API as Nucleus API

    User->>VideoScene: Create with frame_format="png"
    VideoScene->>VideoScene: validate()
    VideoScene->>Payload: to_payload()
    Note over Payload: Includes frame_format: "png"
    Payload->>API: Upload scene

    API-->>Payload: Return scene JSON
    Payload->>VideoScene: from_json(payload)
    Note over VideoScene: ⚠️ frame_format is lost<br/>(not extracted from payload)

Loading

_{Last reviewed commit: 6341865}

[vinay553]

To be merged after https://github.com/scaleapi/scaleapi/pull/132674.

[edwinpav]

Looks good.
Since I was the only one who has redeployed a new SDK version in a while, lmk if help is needed for that part once this is merged.

Worth to take a look at greptile's comment as well

[edwinpav]

Why is this considered broken?

[vinay553]

CLIENT.get_job(fetch_id)

Throws a 401 error. Honestly no one uses Nucleus jobs anymore though perhaps you'll use it for deduping.

[edwinpav]

@greptileai review this PR

[greptile-apps]

_{5 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

nucleus/scene.py
from_json doesn't restore frame_format

to_payload() correctly serializes frame_format into the payload (line 703-704), but from_json() does not extract it when deserializing. This means a round-trip through to_payload() → from_json() will silently drop the frame_format field. The existing test fixture video_scenes in tests/test_video_scene.py uses from_json to reconstruct scenes, so any scene with a frame_format set will lose that value.

        return cls(
            reference_id=payload[REFERENCE_ID_KEY],
            frame_rate=payload.get(FRAME_RATE_KEY, None),
            items=items,
            metadata=payload.get(METADATA_KEY, {}),
            video_location=payload.get(VIDEO_URL_KEY, None),
            tracks=tracks,
            frame_format=payload.get(FRAME_FORMAT_KEY, None),
        )

Prompt To Fix With AI

This is a comment left during a code review.
Path: nucleus/scene.py
Line: 672-679

Comment:
**`from_json` doesn't restore `frame_format`**

`to_payload()` correctly serializes `frame_format` into the payload (line 703-704), but `from_json()` does not extract it when deserializing. This means a round-trip through `to_payload()` → `from_json()` will silently drop the `frame_format` field. The existing test fixture `video_scenes` in `tests/test_video_scene.py` uses `from_json` to reconstruct scenes, so any scene with a `frame_format` set will lose that value.

```suggestion
        return cls(
            reference_id=payload[REFERENCE_ID_KEY],
            frame_rate=payload.get(FRAME_RATE_KEY, None),
            items=items,
            metadata=payload.get(METADATA_KEY, {}),
            video_location=payload.get(VIDEO_URL_KEY, None),
            tracks=tracks,
            frame_format=payload.get(FRAME_FORMAT_KEY, None),
        )
```

How can I resolve this? If you propose a fix, please make it concise.