-
Notifications
You must be signed in to change notification settings - Fork 22
Description
Problem
In shock-server/node/fs.go, QueueUpload(node.Id) is called inside SetFile() (line 51) before the node has been saved to MongoDB. The upload worker (processUpload) immediately calls Load(nodeId) from the database, which may return stale data or fail if the node hasn't been persisted yet.
The call chain is:
SetFile()sets file metadata on the in-memory node structQueueUpload()enqueues the node ID to the upload channel- Later, the caller (e.g.,
node.Update()) callsSave()to persist to MongoDB - The upload worker may dequeue and call
Load()before step 3 completes
Practical Risk
The risk is low in practice because:
- The upload queue is a buffered channel, and
Save()happens very shortly afterSetFile() - The upload worker processes items sequentially
- Under normal load, the timing window is small
However, under heavy concurrent uploads, the race could manifest as failed uploads or stale file metadata being used for S3 uploads.
Suggested Fixes (in order of preference)
-
Move
QueueUploadto afterSave(): The simplest fix — move the auto-upload trigger fromSetFile()to the caller (Update()orCreateNodeUpload()) afterSave()succeeds. -
Add a small retry/backoff in the upload worker: If
Load()returns stale data (mismatched checksum/size), retry after a brief delay. -
Pass the node snapshot directly: Instead of re-loading from DB, pass the in-memory node state to the upload worker.
References
- PR Add cache-to-S3 upload, Chi router, test infrastructure, and build fixes #402 review comment by Copilot
- File:
shock-server/node/fs.go:49-52 - Upload worker:
shock-server/node/upload.go