fix: compute micro_auroc correctly and fix CPU-bound label tensor conversion#165
Open
PuneetKumar1790 wants to merge 1 commit intoML4SCI:mainfrom
Open
Conversation
…conversion Bug 1: micro_auroc was initialized as an empty list and never computed. np.mean([]) always returned nan, corrupting W&B experiment logs. Renamed to weighted_auroc and computed using auroc_fn with average='weighted' (torchmetrics multiclass AUROC does not support average='micro'). Bug 2: labels.type(torch.LongTensor) always creates a CPU tensor regardless of current device. In train.py it was immediately moved back via .to(device), wasting an allocation. In eval.py it was never moved to device at all. Fixed by using labels.to(device, dtype=torch.long). Added tests/test_eval_and_train.py with 5 test cases verifying both fixes.
212507f to
eb055f5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #164
Summary
This PR fixes two bugs in the Classification Transformers sub-project (
DeepLense_Classification_Transformers_Archil_Srivastava):Bug 1:
micro_aurocalways returnsNaNmicro_aurocwas initialized as an empty list and never populated.np.mean([])always returnednan, corrupting W&B experiment logs.Fix: Compute
micro_aurocusingauroc_fn(logits, y, num_classes=NUM_CLASSES, average="weighted").Bug 2: CPU-bound label tensor conversion
labels.type(torch.LongTensor)always creates a CPU tensor, regardless of the current device. Intrain.pyit was immediately moved back with.to(device), wasting an allocation. Ineval.pyit was never moved to device at all.Fix: Use
labels.to(device, dtype=torch.long)which converts dtype and moves to device in one step.Files Changed
eval.py— Fixed micro_auroc computation and label device placementtrain.py— Fixed label device placementtests/test_eval_and_train.py— New verification script with 5 test casesVerification
All 5 tests pass locally (CPU, Python 3.10, PyTorch 2.10.0, torchmetrics 1.8.2):
Note on diff size: The +291 additions come mostly from the new verification test script (
tests/test_eval_and_train.py) with setup, before/after comparisons, and integration tests. The actual fixes ineval.pyandtrain.pyare small (~15–20 lines changed total).