feat: LSST RSP integration — TAP catalog queries, SIA v2 cutouts, and RubinLensDataset#166
Open
nikhilchhokar wants to merge 3 commits intoML4SCI:mainfrom
Open
feat: LSST RSP integration — TAP catalog queries, SIA v2 cutouts, and RubinLensDataset#166nikhilchhokar wants to merge 3 commits intoML4SCI:mainfrom
nikhilchhokar wants to merge 3 commits intoML4SCI:mainfrom
Conversation
added 3 commits
January 23, 2026 06:18
- Add SuperResolutionAutoencoder (U-Net architecture) - Implement perceptual loss (reconstruction + gradient) - Add PSNR and SSIM evaluation metrics - Support real DeepLense Model I/II/III data loading - Include comprehensive training script with CLI args - Add full documentation Addresses DEEPLENSE2 Task ML4SCI#1: unsupervised SR on simulated images. Builds on PR ML4SCI#109 (baseline infrastructure). Tested on Windows 11, PyTorch 2.10, Python 3.13.
Adds rsp_pipeline/ as a complementary approach to the existing Butler-based RIPPLe pipeline. Targets the public Rubin Science Platform web endpoints — no local LSST stack required. Components: - RubinTAPClient: ADQL cone-search on DP0.2/DP1 catalog via pyvo TAP - RubinSIAClient: SIA v2 cutout retrieval per band (g/r/i) - Normaliser: asinh/minmax/zscore (asinh default for lens arc preservation) - CutoutExtractor: resize + multi-band stacking to (C, 64, 64) float32 - RubinLensDataset: drop-in PyTorch Dataset compatible with LensDataset (ML4SCI#151) - build_deeplense_transforms: rotation-aware torchvision augmentation pipeline - YAML config, I/O helpers, 21 offline unit tests, end-to-end notebook Complements RIPPLe (Butler/USDF) by covering the TAP+SIA access path for researchers using data.lsst.cloud without a local stack.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a modular pipeline connecting the Rubin Science Platform (RSP)
public web APIs to DeepLense workflows, enabling lens candidate
retrieval and preprocessing from LSST data without requiring a
local LSST stack installation.
Motivation
The Rubin Observatory will produce ~10M strong lensing candidates.
DeepLense has no direct interface to RSP's TAP/SIA APIs. This PR
fills that gap so existing classification, SR, and lens-finding
training loops work on real LSST data with zero modification.
Components added
RubinTAPClient— ADQL cone-search on DP0.2/DP1 object catalogRubinSIAClient— SIA v2 cutout retrieval per band (g/r/i)Normaliser— asinh/minmax/zscore pixel normalisationCutoutExtractor— resize + multi-band stacking → (C, 64, 64)RubinLensDataset— drop-in PyTorch Dataset for RSP data,compatible with the refactored LensDataset interface (Refactor: LensDataset and WrapperDataset for Cleaner Data Handling #151)
build_deeplense_transforms— torchvision augmentation pipelineTesting
21 offline unit tests — no RSP token or network required:
Relation to prior work
Complements GSoC 2025 RIPPLe (@kartikmandar) which uses the
Butler API for local stack installations. This PR targets the
public RSP web endpoints, covering researchers without USDF access.
Related PRs