Skip to content

Fix QueueUpload race condition: upload queued before node is persisted #404

@wilke

Description

@wilke

Problem

In shock-server/node/fs.go, QueueUpload(node.Id) is called inside SetFile() (line 51) before the node has been saved to MongoDB. The upload worker (processUpload) immediately calls Load(nodeId) from the database, which may return stale data or fail if the node hasn't been persisted yet.

The call chain is:

  1. SetFile() sets file metadata on the in-memory node struct
  2. QueueUpload() enqueues the node ID to the upload channel
  3. Later, the caller (e.g., node.Update()) calls Save() to persist to MongoDB
  4. The upload worker may dequeue and call Load() before step 3 completes

Practical Risk

The risk is low in practice because:

  • The upload queue is a buffered channel, and Save() happens very shortly after SetFile()
  • The upload worker processes items sequentially
  • Under normal load, the timing window is small

However, under heavy concurrent uploads, the race could manifest as failed uploads or stale file metadata being used for S3 uploads.

Suggested Fixes (in order of preference)

  1. Move QueueUpload to after Save(): The simplest fix — move the auto-upload trigger from SetFile() to the caller (Update() or CreateNodeUpload()) after Save() succeeds.

  2. Add a small retry/backoff in the upload worker: If Load() returns stale data (mismatched checksum/size), retry after a brief delay.

  3. Pass the node snapshot directly: Instead of re-loading from DB, pass the in-memory node state to the upload worker.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions