Skip to content

[net] Add SendPutReq to RCurlConnection for S3 uploads#22376

Open
JasMehta08 wants to merge 2 commits into
root-project:masterfrom
JasMehta08:curl-send-put-req
Open

[net] Add SendPutReq to RCurlConnection for S3 uploads#22376
JasMehta08 wants to merge 2 commits into
root-project:masterfrom
JasMehta08:curl-send-put-req

Conversation

@JasMehta08
Copy link
Copy Markdown
Contributor

This Pull Request:

(Is a part of the GSoC 2026 project S3 Backend for RNTuple.)

The S3 backend needs the ability to upload objects via HTTP PUT. This PR adds a single synchronous PUT method to RCurlConnection, following the same pattern as the existing SendHeadReq() and SendRangesReq() methods. SigV4 signing is handled automatically by the credentials already configured on the curl handle.

This PR:

  • Adds SendPutReq(const unsigned char *data, std::size_t length) to RCurlConnection
  • Uses CURLOPT_UPLOAD with a read callback (CURLOPT_READFUNCTION / CURLOPT_READDATA) and CURLOPT_INFILESIZE_LARGE for Content-Length
  • Adds a wire-level test using the TServerSocket pattern from the existing test suite, verifying the PUT method, Content-Length header, and request body
  • Adds an integration test gated on S3 credentials (GTEST_SKIP() when S3_ACCESS_KEY / S3_SECRET_KEY are not set) that does a PUT, HEAD, and GET round-trip against a real S3 endpoint

No connection pooling or retry logic is included in this PR. Those are planned for Phase 4 (Weeks 10-11) as SendMultiGetReq / SendMultiPutReq and a retry wrapper respectively.

Checklist

  • Tested changes locally
  • Wire-level test passes (TServerSocket captures correct PUT request)
  • Integration test passes against local MinIO
Local test output
=== SendPutReq Wire-Level Test ===
Testing against locally built ROOT + RCurl library

[Test 1] Basic PUT request
  PASS: SendPutReq returns success
  PASS: HTTP method is PUT
  PASS: Content-Length header matches payload size
  PASS: body matches payload

[Test 2] Empty PUT (zero-length body)
  PASS: empty SendPutReq returns success
  PASS: HTTP method is PUT for empty body
  PASS: body is empty

[Test 3] Large PUT (64 KB payload)
  PASS: large SendPutReq returns success
  PASS: Content-Length matches 64KB
  PASS: received body size == 64KB
  PASS: 64KB body content matches sent payload

[Test 4] HEAD after PUT (sticky-option reset)
  PASS: initial PUT succeeds
  PASS: first request is PUT
  PASS: HEAD after PUT succeeds
  PASS: second request is HEAD (not PUT — sticky options were reset)
  PASS: HEAD returns correct Content-Length

[Test 5] PUT after HEAD (reverse sticky-option check)
  PASS: first request is HEAD
  PASS: PUT after HEAD succeeds
  PASS: second request is PUT (NOBODY was reset)
  PASS: body arrives correctly after HEAD

=== Results ===
Passed: 20
Failed: 0

ALL TESTS PASSED
Local test output (MinIO integration)
=== SendPutReq MinIO Integration Test ===
Endpoint: http://127.0.0.1:9000
Bucket:   test-ntuple

[Test 1] PUT small object, then HEAD to verify
  PASS: PUT small object succeeds
  PASS: HEAD after PUT succeeds
  PASS: HEAD reports correct Content-Length

[Test 2] PUT object, then GET (range read) to verify content
  PASS: PUT content-verify object succeeds
  PASS: GET after PUT succeeds
  PASS: received full payload length
  PASS: GET content matches PUT payload

[Test 3] PUT empty object (zero bytes)
  PASS: PUT empty object succeeds
  PASS: HEAD after empty PUT succeeds
  PASS: empty object has size 0

[Test 4] PUT large object (256 KB) with content verification
  PASS: PUT 256KB object succeeds
  PASS: HEAD after large PUT succeeds
  PASS: HEAD reports 256KB
  PASS: GET 256KB object succeeds
  PASS: received full 256KB
  PASS: 256KB content matches (byte-for-byte)

[Test 5] PUT overwrite (same key, different content)
  PASS: first PUT succeeds
  PASS: overwrite PUT succeeds
  PASS: HEAD reflects overwritten size
  PASS: GET returns overwritten content

[Test 6] Same connection: PUT then HEAD (sticky-option reset)
  PASS: PUT on shared conn succeeds
  PASS: HEAD on same conn after PUT succeeds (sticky reset works)
  PASS: HEAD on same conn reports correct size

=== Results ===
Passed: 23
Failed: 0

ALL TESTS PASSED
(base) jasmehta@Jass-MacBook-Air-874 test_local % mc ls local/test-ntuple/ 
[2026-05-21 14:29:20 IST]     0B STANDARD test-put-empty.bin
[2026-05-21 14:29:20 IST] 256KiB STANDARD test-put-large.bin
[2026-05-21 14:29:20 IST]    24B STANDARD test-put-overwrite.bin
[2026-05-21 14:29:20 IST]    16B STANDARD test-put-same-conn.bin
[2026-05-21 14:29:20 IST]    39B STANDARD test-put-small.bin
[2026-05-21 14:29:20 IST]    26B STANDARD test-put-verify-content.bin

@JasMehta08 JasMehta08 requested a review from dpiparo as a code owner May 21, 2026 13:06
@jblomer jblomer self-assigned this May 21, 2026
@jblomer jblomer requested review from jblomer and removed request for dpiparo May 21, 2026 14:39
@jblomer jblomer assigned JasMehta08 and unassigned jblomer May 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants