Skip to content

[ntuple] Add kTypeS3 locator type for S3 storage backend#22351

Open
JasMehta08 wants to merge 1 commit into
root-project:masterfrom
JasMehta08:ntuple-s3-locator-type
Open

[ntuple] Add kTypeS3 locator type for S3 storage backend#22351
JasMehta08 wants to merge 1 commit into
root-project:masterfrom
JasMehta08:ntuple-s3-locator-type

Conversation

@JasMehta08
Copy link
Copy Markdown
Contributor

This Pull Request

(Is a part of the GSoC 2026 project S3 Backend for RNTuple.)

The S3 backend supports two addressing modes: Mode A packs multiple pages into a single S3 object and uses a new locator type to encode both the object ID and byte offset, while Mode B stores one page per object and reuses the existing Object64 locator. This PR adds the on-disk format support needed for Mode A.

This PR:

  • Adds kTypeS3 = 0x03 to the ELocatorType enum in RNTupleTypes.hxx
  • Wires the new type code into SerializeLocator() and DeserializeLocator() in RNTupleSerialize.cxx, reusing the existing SerializeLocatorPayloadObject64 / DeserializeLocatorPayloadObject64
  • Adds round-trip serialization tests for the new locator type, verifying that the 64-bit payload passes the serialize-deserialize cycle with the correct type code

The 64-bit payload is interpreted differently depending on the locator type: 0x03 (Mode A) bit-splices upper 32 bits as object ID and lower 32 bits as byte offset, while 0x02 (Mode B) uses the full value as a plain object ID. The interpretation is handled by the backend, not the serializer. The serializer just stores and retrieves the raw 64-bit value.

Checklist

  • Tested changes locally
  • Locator round-trip tests pass for both kTypeS3 (0x03) and kTypeDAOS (0x02), locally.

@JasMehta08 JasMehta08 requested a review from jblomer as a code owner May 20, 2026 10:09
@JasMehta08 JasMehta08 marked this pull request as draft May 20, 2026 10:09
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 20, 2026

Test Results

    21 files      21 suites   3d 10h 4m 18s ⏱️
 3 857 tests  3 857 ✅ 0 💤 0 ❌
73 345 runs  73 345 ✅ 0 💤 0 ❌

Results for commit bee8ce4.

♻️ This comment has been updated with latest results.

@JasMehta08 JasMehta08 marked this pull request as ready for review May 21, 2026 13:07
@JasMehta08 JasMehta08 force-pushed the ntuple-s3-locator-type branch from bee8ce4 to 9365f1f Compare May 22, 2026 05:22
@JasMehta08
Copy link
Copy Markdown
Contributor Author

@jblomer
I have rebased the commits into one and also changed the kTestLocatorType to 5, as suggested. Please have a look and let me know if i should change something.

Thanks!

Copy link
Copy Markdown
Contributor

@jblomer jblomer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle looks good to me.

I wonder, however, if it would be better to encode the two modes in two different locator types. We would then consolidate the DAOS with the mode B locator (e.g. kTypeObject64) and create another locator type for mode A (e.g. kTypeMulti or similar). To be discussed, possibly for a follow-up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants