Port virtual_staining examples into applications/cytoland#409
Open
edyoshikun wants to merge 19 commits into
Open
Port virtual_staining examples into applications/cytoland#409edyoshikun wants to merge 19 commits into
edyoshikun wants to merge 19 commits into
Conversation
Tutorial-style scripts under applications/*/examples use jupytext-style percent cells, markdown docstrings, and long URL references that trip D205/D400/D100/D103 and E501. Treat them like notebooks and tests.
Bring back the four VSCyto inference demos (VSCyto2D, VSCyto3D, VSNeuromast, and TTA-augmented) plus plot.py helper from examples/virtual_staining/VS_model_inference on main. Imports are updated to the modular package layout: - viscy.data.hcs -> viscy_data.hcs - viscy.transforms -> viscy_transforms - viscy.trainer -> viscy_utils.trainer - viscy.translation.engine -> cytoland.engine - viscy.translation.predict_writer -> viscy_utils.callbacks
Bring back the DL@MBL 2024 image-translation exercise (solution.py, README, setup.sh, prepare-exercise.sh) from examples/virtual_staining/dlmbl_exercise on main. Notebooks are not ported; regenerate from solution.py via jupytext if needed. Imports are updated to the modular package layout: - viscy.data.hcs -> viscy_data.hcs - viscy.transforms -> viscy_transforms - viscy.trainer -> viscy_utils.trainer - viscy.translation.engine.VSUNet -> cytoland.engine.VSUNet - viscy.translation.engine.MixedLoss -> viscy_utils.losses.MixedLoss - viscy.translation.evaluation_metrics.mean_average_precision -> viscy_utils.evaluation.metrics.mean_average_precision setup.sh now installs applications/cytoland[metrics] editable instead of the legacy top-level viscy[metrics,visual,examples]>=0.2 wheel, plus cellpose and torchview as extra tutorial dependencies.
Tutorial-style scripts routinely import late inside % cells (E402) and reference notebook builtins like get_ipython without importing them (F821). Extending the per-file-ignores so the vcp tutorial scripts lint cleanly alongside the existing D and E501 exemptions.
Bring back the Virtual Cell Platform tutorials (quick_start.py, hek293t.py, neuromast.py, README.md) from examples/virtual_staining/vcp_tutorials on main. Notebooks are not ported; regenerate from .py via jupytext if needed. quick_start.py imports are updated to the modular package layout: - viscy.data.hcs -> viscy_data.hcs - viscy.transforms -> viscy_transforms - viscy.trainer -> viscy_utils.trainer - viscy.translation.engine.FcmaeUNet -> cytoland.engine.FcmaeUNet - viscy.translation.predict_writer -> viscy_utils.callbacks hek293t.py and neuromast.py have no viscy Python imports (they demonstrate the viscy preprocess / viscy predict CLIs), so no code changes are required. The commented-out pip install "viscy[...]" hints are left as-is for historical reference.
Add a Tutorials and demos section listing VS_model_inference, vcp_tutorials, dlmbl_exercise, and configs, so the newly ported examples are discoverable from the cytoland landing page.
Bring back the phase-contrast virtual staining tutorial (solution.py, README, setup.sh, prepare-exercise.sh) from examples/virtual_staining/phase_contrast on main. Notebooks are not ported; regenerate from solution.py via jupytext if needed. Imports are updated to the modular package layout: - viscy.data.hcs -> viscy_data.hcs - viscy.transforms -> viscy_transforms - viscy.trainer -> viscy_utils.trainer - viscy.translation.engine.VSUNet -> cytoland.engine.VSUNet setup.sh now installs applications/cytoland[metrics] editable instead of the legacy top-level viscy[metrics,visual,examples]>=0.2 wheel.
Add a concise Lightning primer before Part 1 (three-object mental model: LightningDataModule, LightningModule, Trainer; what trainer.fit replaces) so learners unfamiliar with Lightning can follow the rest of the exercise. Expand inline explanations at the three points where Lightning concepts first appear in code: - HCSDataModule section: describe the DataModule role and the exact dict shape (source, target, index) yielded to training_step. - VSUNet instantiation: frame the class as the LightningModule that bundles network, loss, and per-batch logic; gloss lr, schedule, freeze_encoder, log_batches_per_epoch. - Trainer constructors: annotate fast_dev_run, accelerator, devices, precision="16-mixed", max_epochs, log_every_n_steps, and TensorBoardLogger; spell out what trainer.fit does internally so learners see it replacing a hand-written training loop. Markdown only, no code changes.
Six concise additions for learners new to deep learning and Lightning: 1. Typos: fluoresecence -> fluorescence, componets -> components, Person Correlation -> Pearson Correlation. 2. Tighten the OME-Zarr / HCS section: one-paragraph primer on the row/col/field/level/T/C/Z/Y/X hierarchy before open_ome_zarr is called. 3. Add an augmentation motivation table mapping each MONAI transform to the real-world microscope variation it simulates. 4. Add a pre-model markdown cell explaining the UNeXt2 config (encoder_blocks, dims, decoder_conv_blocks, stem_kernel_size, in_stack_depth) in U-Net terms. 5. Same cell explains MixedLoss (L1 + MS-SSIM tradeoff) and the WarmupCosine LR schedule. 6. Part 2 opener distinguishes regression vs segmentation metric families; fill in the Task 2.1 TODO with real Pearson/SSIM definitions tied to the image-translation setting. Markdown only, no code changes.
Issue dl-janelia/image_translation#16: - Fix the nested duplicate for-loop after Task 1.1 that shadows the loop variable (single loop now). - Replace the broken viscy.transforms source link with authoritative MONAI docs links for RandAffined and RandGaussianNoised. - Add explicit path + public download URL for the VSCyto2D pretrained checkpoint next to Task 2.2 so students don't guess where it went. - Switch spatial dimension labels from (D, H, W) to (Z, Y, X) in Part 3 Gaussian-blur tasks to match the convention used everywhere else in the exercise. - Note MicroSSIM as a microscopy-appropriate SSIM variant in the metrics primer. - Warn students to restart the kernel before re-running training to release GPU memory (CUDA OOM prevention).
Rewrite setup.sh to use uv instead of conda so students can provision the exercise without a working conda install. The script now: - Installs uv via the official installer if missing. - Creates a Python 3.11 venv at .venv/ inside the exercise folder. - Installs cytoland editable from the monorepo plus the tutorial extras (cellpose, torchview, jupyter, ipykernel, ipywidgets, jupytext, nbformat, nbconvert) into that venv. - Registers the venv as a Jupyter kernel named 06_image_translation so VSCode and JupyterLab both surface it by default. - Downloads the training / test OME-Zarr stores and the VSCyto2D pretrained checkpoint into ~/data/06_image_translation/. Script is bash strict-mode (set -euo pipefail) and resolves the monorepo root relative to the script path so it works regardless of the caller's cwd. README updated to match the new flow and document the VSCode / Jupyter kernel selection.
Classic skimage SSIM assumes natural-image dynamic range and scores collapse into a narrow band on sparse, dim, noisy fluorescence microscopy predictions — so the metric can barely rank good vs bad outputs. microSSIM (Ashesh et al. 2024, arXiv:2408.08747) subtracts background and fits a per-image rescale before running SSIM, which restores sensitivity over the intensity range microscopy predictions actually live in. - Replace skimage.metrics.structural_similarity with microssim.micro_structural_similarity at all 6 call sites (two metrics blocks: single-model and phase2fluor vs pretrained). - Drop the now-unused from skimage import metrics import. - Rename DataFrame columns SSIM_nuc / SSIM_mem to microSSIM_nuc / microSSIM_mem so plots and saved CSVs name the metric correctly. - Rewrite the Part 2 Task 2.1 metric definition to explain WHY microSSIM instead of SSIM for microscopy. - Add microssim to setup.sh install list and to the README install summary.
setup_TA.sh stages data + checkpoint to a shared DATA_ROOT (no env). setup_student.sh creates the per-user venv + kernel and skips the download when DATA_ROOT already has the data.
setup_student.sh now defaults to python 3.13 and installs cytoland + viscy from PyPI when run outside a VisCy monorepo clone. When the monorepo is detected (and root pyproject is the viscy umbrella), it falls back to the existing editable workspace install. README updated to match.
- Replace stale torch.no_grad() with torch.inference_mode() in the prediction-visualization block to match the rest of the file. - Renumber the first 'Task 1.5' (model instantiation) to 'Task 1.4'. The exercise was missing a Task 1.4, leaving two Task 1.5s. - Update the source-code reference under that task to point at VSUNet and the fcmae network (the Unet2D link was obsolete).
Three small wording fixes ported from the DL@Janelia version of this exercise where it had clearer phrasing: - Task 1.1 hint: 'what are your options' -> 'what your options are'. - Task 2.4: add the missing alert heading and move the section header outside the alert box (was structurally mismatched). - Task 2.4 question: typo fix 'How do yout model' -> 'How does your model'.
Adds the Task 2.5 section that walks students through the inverse translation: predicting QPI from nuclei + membrane fluorescence using a pretrained model and Test-Time Augmentation. Inserted between Task 2.4 and Part 3. setup_TA.sh and setup_student.sh now also stage the fluor2phase_step668.ckpt checkpoint (previously commented out in TA, absent in student) since Task 2.5 loads it. The hardcoded /mnt/efs/... paths from the original Janelia version are replaced with the top_dir-based DATA_ROOT layout used elsewhere in this exercise.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR ports virtual staining tutorials/demos into applications/cytoland/examples, updates modular imports, adds setup guidance, and relaxes linting for example-style scripts.
Changes:
- Adds Cytoland example/tutorial trees for model inference, VCP tutorials, phase contrast, and DL course exercise.
- Updates documentation to point users to the new examples.
- Adds Ruff per-file ignores for example scripts.
Reviewed changes
Copilot reviewed 19 out of 20 changed files in this pull request and generated 39 comments.
Show a summary per file
| File | Description |
|---|---|
pyproject.toml |
Adds Ruff ignores for examples. |
applications/cytoland/README.md |
Adds tutorials/demos section. |
applications/cytoland/examples/VS_model_inference/plot.py |
Adds plotting helper. |
applications/cytoland/examples/VS_model_inference/demo_vsneuromast.py |
Adds VSNeuromast inference demo. |
applications/cytoland/examples/VS_model_inference/demo_vscyto3d.py |
Adds VSCyto3D inference demo. |
applications/cytoland/examples/VS_model_inference/demo_vscyto2d.py |
Adds VSCyto2D inference demo. |
applications/cytoland/examples/VS_model_inference/demo_vscyto_w_ttas.py |
Adds TTA/sliding-window inference demo. |
applications/cytoland/examples/vcp_tutorials/README.md |
Adds VCP tutorial index. |
applications/cytoland/examples/vcp_tutorials/quick_start.py |
Adds Cytoland quick-start tutorial. |
applications/cytoland/examples/vcp_tutorials/neuromast.py |
Adds neuromast CLI/analysis tutorial. |
applications/cytoland/examples/vcp_tutorials/hek293t.py |
Adds HEK293T CLI/analysis tutorial. |
applications/cytoland/examples/phase_contrast/solution.py |
Adds phase contrast virtual staining demo. |
applications/cytoland/examples/phase_contrast/setup.sh |
Adds phase contrast setup/download script. |
applications/cytoland/examples/phase_contrast/README.md |
Adds phase contrast usage instructions. |
applications/cytoland/examples/phase_contrast/prepare-exercise.sh |
Adds notebook generation helper. |
applications/cytoland/examples/dl-course-exercise/solution.py |
Adds DL course exercise script. |
applications/cytoland/examples/dl-course-exercise/setup_TA.sh |
Adds TA data staging script. |
applications/cytoland/examples/dl-course-exercise/setup_student.sh |
Adds student venv/data setup script. |
applications/cytoland/examples/dl-course-exercise/README.md |
Adds DL course exercise instructions. |
applications/cytoland/examples/dl-course-exercise/prepare-exercise.sh |
Adds exercise/solution notebook generation helper. |
Comments suppressed due to low confidence (5)
applications/cytoland/examples/VS_model_inference/demo_vsneuromast.py:87
HCSDataModuledoes not accept anarchitecturekeyword in the current modular API (packages/viscy-data/src/viscy_data/hcs.py:104-132), so this example raisesTypeErrorbefore prediction starts. Remove this argument and keep the architecture selection on the model only.
architecture="UNeXt2",
applications/cytoland/examples/VS_model_inference/demo_vscyto3d.py:90
HCSDataModuledoes not accept anarchitecturekeyword in the current modular API (packages/viscy-data/src/viscy_data/hcs.py:104-132), so this example raisesTypeErrorbefore prediction starts. Remove this argument and keep the architecture selection on the model only.
architecture="UNeXt2",
applications/cytoland/examples/VS_model_inference/demo_vscyto2d.py:82
HCSDataModuledoes not accept anarchitecturekeyword in the current modular API (packages/viscy-data/src/viscy_data/hcs.py:104-132), so this example raisesTypeErrorbefore prediction starts. Remove this argument and keep the architecture selection on the model only.
architecture="fcmae",
applications/cytoland/examples/vcp_tutorials/quick_start.py:91
- This setup command still installs the retired monolithic
viscy[metrics,visual]package, but the tutorial importscytoland.engine,viscy_data,viscy_transforms, andviscy_utils. A user who follows the notebook setup from a clean environment can end up without the modular packages needed by the code below; update the install instruction to install cytoland/the modular workspace instead.
# !pip install "viscy[metrics,visual]==0.4.0a3"
applications/cytoland/examples/dl-course-exercise/README.md:127
- The README says to run cells top to bottom, but the raw
solution.pyincludes task cells with placeholders that are only stripped byprepare-exercise.shwhen generatingsolution.ipynb. Following these steps directly onsolution.pywill fail before reaching the solution cells.
--display-name "Python (06_image_translation)"
References
</details>
---
💡 <a href="/mehta-lab/VisCy/new/modular-viscy-staging?filename=.github/instructions/*.instructions.md" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Add Copilot custom instructions</a> for smarter, more guided reviews. <a href="https://docs.github.com/en/copilot/customizing-copilot/adding-repository-custom-instructions-for-github-copilot" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Learn how to get started</a>.
| data_module = HCSDataModule( | ||
| data_path=input_data_path, | ||
| source_channel=phase_channel_name, | ||
| target_channel=["Membrane", "Nuclei"], |
| data_module = HCSDataModule( | ||
| data_path=input_data_path, | ||
| source_channel=phase_channel_name, | ||
| target_channel=["Membrane", "Nuclei"], |
| data_module = HCSDataModule( | ||
| data_path=input_data_path, | ||
| source_channel=phase_channel_name, | ||
| target_channel=["Membrane", "Nuclei"], |
| # Name of the input phase channel | ||
| source_channel="Phase3D", | ||
| # Desired name of the output channels | ||
| target_channel=["Membrane", "Nuclei"], |
| # %% [markdown] | ||
| """ | ||
| # Prerequisites | ||
| Python>=3.11 |
|
|
||
| # Get the Phase Contrast channel | ||
| c_idx = dataset.channel_names.index(source_channel_name) | ||
| phase_image = image[0:1, c_idx : c_idx + 1, Z // 2 - 3 : Z // 2 + 3, y_slice, x_slice] |
Comment on lines
+52
to
+53
| Regenerate paired `.ipynb` notebooks with `jupytext --to ipynb solution.py` | ||
| if you prefer the notebook UI. |
Comment on lines
+3
to
+10
| This directory contains tutorial notebooks for the Virtual Cell Platform, | ||
| available in both Python scripts and Jupyter notebooks. | ||
|
|
||
| - [Quick Start](quick_start.ipynb): | ||
| get started with model inference in Python with a A549 cell dataset. | ||
| - [CLI inference and visualization](hek293t.ipynb): | ||
| run inference from CLI on a HEK293T cell dataset and visualize the results. | ||
| - [Virtual staining _in vivo_](neuromast.ipynb): |
Comment on lines
+1295
to
+1300
| pretrained_phase2fluor = VSUNet.load_from_checkpoint( | ||
| pretrained_model_ckpt, | ||
| architecture="UNeXt2_2D", | ||
| model_config=phase2fluor_config, | ||
| accelerator="gpu", | ||
| ) |
| |--------|----------------------| | ||
| | [`examples/VS_model_inference/`](./examples/VS_model_inference/) | Python API inference demos for VSCyto2D, VSCyto3D, VSNeuromast, and TTA-augmented sliding-window prediction | | ||
| | [`examples/vcp_tutorials/`](./examples/vcp_tutorials/) | Virtual Cell Platform quick-start and organism-specific walkthroughs (HEK293T, neuromast) | | ||
| | [`examples/dl-course-exercise/`](./examples/dl-course-exercise/) | Image-translation course exercise (training from scratch + evaluation) — used at DL@MBL and DL@Janelia | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
examples/virtual_staining/tree frommainintoapplications/cytoland/examples/on top ofmodular-viscy-staging, covering four tutorial groups: VS_model_inference, dlmbl_exercise, vcp_tutorials, phase_contrast.viscy.*import to the modular package layout (viscy_data,viscy_transforms,viscy_utils,cytoland.engine,viscy_utils.callbacks.HCSPredictionWriter,viscy_utils.losses.MixedLoss,viscy_utils.evaluation.metrics.mean_average_precision).setup.shin dlmbl_exercise and phase_contrast topip install -e "applications/cytoland[metrics]"instead of the retired top-levelviscy[metrics,visual,examples]>=0.2wheel.**/examples/**to[tool.ruff] per-file-ignoresforD,E402,E501,F821so jupytext percent-cell scripts lint cleanly without forcing docstrings, top-of-file imports, short lines, or importing notebook builtins.applications/cytoland/README.mdpointing at the new tree.Notebooks (
.ipynb) are intentionally not ported — regenerate fromsolution.py/*.pywithjupytext --to ipynbwhen needed.Import migration
viscy.data.hcsviscy_data.hcsviscy.transformsviscy_transformsviscy.trainerviscy_utils.trainerviscy.translation.engine.{VSUNet, FcmaeUNet, AugmentedPredictionVSUNet}cytoland.engine.{...}viscy.translation.engine.MixedLossviscy_utils.losses.MixedLossviscy.translation.predict_writer.HCSPredictionWriterviscy_utils.callbacks.HCSPredictionWriterviscy.translation.evaluation_metrics.mean_average_precisionviscy_utils.evaluation.metrics.mean_average_precisionTest plan
python -c "import ast; ast.parse(open(f).read())"passes for all ported.pyfilesruff check+ruff formatpass via pre-commit on every commituv run --package cytoland python -c "from cytoland.engine import ...; from viscy_data.hcs import ...; ..."resolves every ported symbolpublic.czbiohub.org/comp.micro/viscyand a GPU; not part of this PR)