Feature/6 cloud native tps services

[vidyuthdev]

Add Service 2 (Beam Model), async Geometry jobs, and DICOM upload
This PR ships two major features and one refactor, building on the Geometry Service foundation already on main. End-to-end validated against an anonymized clinical patient from TCIA's Lung CT Segmentation Challenge dataset.
Summary

Service 2 — Beam Model Service (new): turns a built geometry + a beam configuration into a deliverable proton/photon beam model, with full async + caching parity to Service 1.
Service 1 — DICOM upload + async jobs (new): clients can now POST a ZIP of DICOM files and receive an upload_id to use in geometry builds; geometry builds now dispatch via Celery with a polling endpoint.
_helpers refactor: BDL and MCsquare calibration loading now delegate to the new pluggable ProtonMachineModel, removing duplication across the proton workflows.

What's new
Service 2 — Beam Model Service
The next service in the planning pipeline. Takes a geometry_id + a BeamSetSpec + DeliveryParams + a Modality and produces a BeamModelResult plus a pickled OpenTPS plan artifact.

Models — Modality, BeamSetSpec, DeliveryParams, BeamModelBuildRequest with modality-filtered compute_cache_key(), FluenceElementSet, BeamModelResult, BeamModelJobStatus.
Pluggable machine model — ProtonMachineModel / PhotonMachineModel with lazy BDL + MCsquare calibration loaders. get_machine_model(modality, id) factory.
Modality builders — proton_spots.generate_proton_spots() drives OpenTPS ProtonPlanDesign.buildPlan(); photon_beamlets.generate_photon_beamlets() computes a beam's-eye-view beamlet grid for each PlanPhotonBeam.
Orchestrator — BeamModelService.build() with 6-stage progress callback (queued → loading_geometry → loading_machine_model → building_beams → computing_elements → persisting → done).
Persistence — atomic on-disk store under {artifact_dir}/beam_models/, content-addressable by SHA-256 of the modality-filtered request.
Async path — Celery task + beam_model_jobs table + Alembic migration c2f8d2a4b5e3.
REST surface — POST /beam-model/build (200 cache hit / 202 dispatch), GET /beam-model/jobs/{id}, GET /beam-model/{id}, GET /beam-model/{id}/artifact (streams pickled plan), DELETE /beam-model/{id}.

Same architectural patterns as Service 1, so reading one teaches you the other.
Service 1 — DICOM upload endpoint
Removes the dependency on the bundled OpenTPS data root for production use. Clients can now feed real DICOM directly.

POST /api/v1/uploads/dicom — accepts a single multipart file (ZIP), extracts safely under {upload_dir}/{upload_id}/ (zip-slip protected), walks the directory and returns CT-slice + RTSTRUCT counts via pydicom modality detection.
GET /uploads/{id}, DELETE /uploads/{id} for inspection and cleanup.
PatientRef.upload_id field — when set, GeometryService._load resolves to the upload directory and disk-loads from there. PACS and data-root paths remain as fallbacks. Cache key includes upload_id so two builds against the same upload dedup.
Settings — new RADIARCH_UPLOAD_DIR env var, defaults to {artifact_dir}/uploads.

Service 1 — async geometry jobs

Celery task build_geometry_job with progress callbacks mirrored to Postgres-backed geometry_jobs table.
GET /geometry/jobs/{id} for polling state, progress, stage, and final geometry_id.
Alembic migration b1e7c1f3a4f2.
API contract: POST /geometry/build returns 200 + full result on cache hit, 202 + job_id on cache miss.

Refactor — _helpers.py
setup_calibration() and load_bdl() are now thin shims delegating to ProtonMachineModel.from_default(). Public signatures unchanged — proton_basic, proton_robust, proton_optimized, photon_ccc workflows continue to work without modification. Single source of truth for machine calibration data.
Tests
271 passing (259 previously + 12 new for uploads), 4 skipped (existing slow-marked OpenTPS integration tests).

tests/test_api_uploads.py (10 tests) — happy path, error paths, geometry build with upload_id, stale upload_id handling, PatientRef validation
tests/test_api_beam_model.py (12 tests) — async dispatch, cache hit, jobs endpoint, retrieval, delete
tests/services/test_beam_model_* (50+ tests) — model invariants, machine model factory, persistence atomicity, modality builders, service orchestrator

Run with:
bashpytest tests/ -v
Live validation on clinical data
Validated end-to-end against LCTSC-Test-S1-101 from TCIA (anonymized clinical lung CT, 130 slices, 5 OARs):

frame_of_reference_uid correctly populated and linked CT ↔ RTSTRUCT
structure_index = {'Esophagus': 1, 'Heart': 2, 'Lung_L': 3, 'Lung_R': 4, 'SpinalCord': 5}
Real clinical grid (0.977, 0.977, 3.0) mm at (512, 512, 130)
First build: 7.2 s (full pipeline) — second build: 0.3 ms (cache hit, 21,655× speedup)

See demo/README_DICOM.md for the TCIA download walkthrough.
How to demo
bashsource src/.venv/bin/activate

Service 1 (geometry) on real DICOM

python demo/show_geometry.py --upload ~/lctsc_patient.zip --show

Service 2 (beam model) — both proton and photon paths

python demo/show_beam_model.py
Migrations
Two new Alembic revisions: b1e7c1f3a4f2 (geometry_jobs) and c2f8d2a4b5e3 (beam_model_jobs). Run alembic upgrade head after pulling.
Files
38 files changed, 5,412 insertions(+), 99 deletions(-)