Skip to content

feat: MIL meta arch#5

Open
xrusnack wants to merge 24 commits intomasterfrom
feature/meta-arch/mil
Open

feat: MIL meta arch#5
xrusnack wants to merge 24 commits intomasterfrom
feature/meta-arch/mil

Conversation

@xrusnack
Copy link
Copy Markdown
Member

@xrusnack xrusnack commented Mar 17, 2026

Summary by CodeRabbit

Release Notes

  • New Features
    • Added crop-level experiment training configuration with support for multiple-instance learning, masking, and batch processing
    • Introduced nuclei MIL model architecture providing end-to-end training, validation, testing, and prediction pipelines with performance metrics

@xrusnack xrusnack requested review from matejpekar and vejtek March 17, 2026 00:24
@xrusnack xrusnack self-assigned this Mar 17, 2026
@xrusnack xrusnack requested a review from a team March 17, 2026 00:24
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 17, 2026

Warning

Rate limit exceeded

@xrusnack has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 42 minutes and 31 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 42 minutes and 31 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6e5fbcd3-62e1-4538-b387-565ee2dbd908

📥 Commits

Reviewing files that changed from the base of the PR and between e2c0421 and 932559e.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (4)
  • configs/experiment/modeling/training/crop_level/prostate_cancer_mmci_tl.yaml
  • configs/experiment/modeling/training/crop_level/radboud.yaml
  • nuclei_graph/nuclei_mil_meta_arch.py
  • pyproject.toml
📝 Walkthrough

Walkthrough

This PR introduces a new Multiple Instance Learning (MIL) framework for nuclei graph analysis. It adds experiment and model configuration files, along with a new LightningModule that implements training, validation, testing, and prediction workflows with graph and nuclei-level metric tracking.

Changes

Cohort / File(s) Summary
Configuration Files
configs/experiment/modeling/training/crop_level.yaml, configs/model/meta_archs/nuclei_mil.yaml
New experiment configuration with training defaults, data sampling, and model architecture reference; new model architecture configuration targeting NucleiMILMetaArch class.
Package Initialization
nuclei_graph/__init__.py
Added __all__ = [] to explicitly define empty module export list.
MIL Training Framework
nuclei_graph/nuclei_mil_meta_arch.py
New LightningModule implementing multiple-instance learning wrapper with training/validation/test/predict steps, graph and nuclei-level metric tracking, custom optimizer parameter grouping, and learning rate scheduling with warmup and cosine annealing.

Sequence Diagram

sequenceDiagram
    participant Batch as Data Batch
    participant Forward as forward()
    participant Net as Wrapped Network
    participant Loss as Loss Computation
    participant Metrics as Metric Updates
    participant Epoch as Epoch Callbacks

    Batch->>Forward: Supply batch data
    activate Forward
    Forward->>Forward: Extract block_mask
    alt Not Training
        Forward->>Forward: Apply mask_mixed_blocks
    end
    Forward->>Net: Forward pass
    Net-->>Forward: Return logits & outputs
    deactivate Forward

    alt Training Step
        Forward->>Loss: Compute BCEWithLogits<br/>on graph logits/targets
        Loss-->>Forward: Training loss
        Forward->>Metrics: Log train/graph/loss
    else Validation/Test Step
        Forward->>Loss: Compute graph loss
        Loss-->>Metrics: Update graph metrics
        Forward->>Loss: Compute nuclei loss<br/>(if supervised)
        Loss-->>Metrics: Update nuclei metrics
        Metrics-->>Epoch: Accumulate losses
    end

    Epoch->>Metrics: on_validation/test_epoch_end()
    activate Epoch
    Metrics->>Metrics: Compute metric collections
    Metrics->>Metrics: Reset accumulators
    alt Validation
        Metrics->>Metrics: Track best graph loss
        Metrics-->>Epoch: Log best/... metrics
    end
    deactivate Epoch
Loading

Estimated Code Review Effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 A new MIL framework hops into place,
With metrics and loss cascading through space,
Graph logits and nuclei dance in the light,
Optimizers configured—training's just right!
Thump thump—forward pass, loss flows clean! 🌱

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: MIL meta arch' directly and clearly describes the main change: adding a Multiple Instance Learning meta architecture component.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/meta-arch/mil

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist
Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a foundational Multiple Instance Learning (MIL) meta-architecture within the PyTorch Lightning framework. The primary goal is to enable robust training and evaluation of models for nuclei graph analysis, providing a structured approach to handle both graph-level and individual nuclei-level predictions. The changes facilitate the integration of self-attention transformer models and ensure comprehensive performance monitoring throughout the model lifecycle.

Highlights

  • New MIL Meta-Architecture: Introduced a new Multiple Instance Learning (MIL) meta-architecture, NucleiMILMetaArch, implemented as a PyTorch Lightning module, designed for nuclei graph analysis.
  • Comprehensive Metric Tracking: Implemented extensive metric tracking for both graph-level and nuclei-level predictions, including Precision, Recall, AUROC, and AUPRC, across training, validation, and testing phases.
  • Advanced Optimizer and Scheduler: Configured an AdamW optimizer with a sophisticated learning rate schedule featuring a linear warmup followed by cosine annealing, and differentiated weight decay for parameters.
  • New Configuration Files: Added new YAML configuration files to define the nuclei_mil meta-architecture and set up a specific training experiment for a crop-level self-attention transformer model.
Changelog
  • configs/experiment/modeling/training/crop_level.yaml
    • Added a new configuration file for a crop-level self-attention transformer training experiment.
  • configs/model/meta_archs/nuclei_mil.yaml
    • Added a new configuration file to define the nuclei_mil meta-architecture.
  • nuclei_graph/init.py
    • Initialized the __all__ variable.
  • nuclei_graph/nuclei_mil_meta_arch.py
    • Implemented the NucleiMILMetaArch class, a PyTorch Lightning module for MIL.
Activity
  • No activity has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new meta-architecture for Multiple Instance Learning, NucleiMILMetaArch, along with its configuration. The implementation is well-structured, but there are opportunities to improve maintainability by reducing code duplication in the validation_step/test_step and on_validation_epoch_end/on_test_epoch_end methods. I've also suggested a cleaner and more robust way to separate optimizer parameters.

Comment thread nuclei_graph/nuclei_mil_meta_arch.py Outdated
Comment thread nuclei_graph/nuclei_mil_meta_arch.py
Comment thread nuclei_graph/nuclei_mil_meta_arch.py Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
nuclei_graph/nuclei_mil_meta_arch.py (1)

46-50: Unused metric collections.

predict_graph_metrics and predict_nuclei_metrics are instantiated but never used — predict_step returns raw outputs without updating these metrics.

Consider removing them to avoid unnecessary memory allocation, or implement metric updates in predict_step if prediction-time metrics are intended.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@nuclei_graph/nuclei_mil_meta_arch.py` around lines 46 - 50,
predict_graph_metrics and predict_nuclei_metrics are created but never updated
in predict_step; either remove their creation or update predict_step to record
prediction-time metrics. Locate the metric initializations
(predict_graph_metrics, predict_nuclei_metrics) and either delete those lines
and any related references, or modify predict_step to compute and call the
appropriate update/compute methods on predict_graph_metrics and
predict_nuclei_metrics (matching how val/test metrics are updated) so prediction
outputs are logged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@nuclei_graph/nuclei_mil_meta_arch.py`:
- Line 60: Fix the typo in the inline comment in nuclei_mil_meta_arch.py where
"pediction" is written; update the comment text to read "prediction" (this is
the comment near the mixed blocks handling in the nuclei_mil_meta_arch module).

---

Nitpick comments:
In `@nuclei_graph/nuclei_mil_meta_arch.py`:
- Around line 46-50: predict_graph_metrics and predict_nuclei_metrics are
created but never updated in predict_step; either remove their creation or
update predict_step to record prediction-time metrics. Locate the metric
initializations (predict_graph_metrics, predict_nuclei_metrics) and either
delete those lines and any related references, or modify predict_step to compute
and call the appropriate update/compute methods on predict_graph_metrics and
predict_nuclei_metrics (matching how val/test metrics are updated) so prediction
outputs are logged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 40b9e3f8-39b8-4758-877e-11417be23865

📥 Commits

Reviewing files that changed from the base of the PR and between a377343 and e2c0421.

📒 Files selected for processing (4)
  • configs/experiment/modeling/training/crop_level.yaml
  • configs/model/meta_archs/nuclei_mil.yaml
  • nuclei_graph/__init__.py
  • nuclei_graph/nuclei_mil_meta_arch.py

Comment thread nuclei_graph/nuclei_mil_meta_arch.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant