Skip to content

Remove outdated evaluate code and just use trainer.validate.#1327

Merged
jveitchmichaelis merged 1 commit intomainfrom
standardize_main_evalaute
Mar 3, 2026
Merged

Remove outdated evaluate code and just use trainer.validate.#1327
jveitchmichaelis merged 1 commit intomainfrom
standardize_main_evalaute

Conversation

@bw4sz
Copy link
Copy Markdown
Collaborator

@bw4sz bw4sz commented Feb 26, 2026

This PR

  • Removes __evaluate__ and calculate_empty_frame accuracy
  • uses trainer.evaluate in main.evaluate()
  • Keeps the output structure that user is used to
  • stashes the .results object in the trainer.

closes #1316

and simplifies the code.

Thanks to @jveitchmichaelis for his idea here.

@bw4sz bw4sz added the API This tag is used for small improvements to the readability and usability of the python API. label Feb 26, 2026
@bw4sz
Copy link
Copy Markdown
Collaborator Author

bw4sz commented Mar 2, 2026

Okay, @jveitchmichaelis I think this ready to review. I removed predictions arg from the evaluate CLI, it was a weird edge pattern that didn't match the rest of the workflow.

@bw4sz
Copy link
Copy Markdown
Collaborator Author

bw4sz commented Mar 2, 2026

I reverted the .toml, it was specific to my laptop, let's make that a separate PR.

Comment thread src/deepforest/scripts/evaluate.py
Copy link
Copy Markdown
Collaborator

@jveitchmichaelis jveitchmichaelis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good aside from that docstring + needs squash.

I do like the idea of being able to run eval on saved predictions, but I agree there's probably a better place to do it (and it feels awkward here if we standardize on using the trainer).

sure the trainer stashes the entire results dictionary, not just logs
the individual pieces to keep user behavior the same
@bw4sz bw4sz force-pushed the standardize_main_evalaute branch from 0fd3808 to 69a71b8 Compare March 3, 2026 01:54
@bw4sz
Copy link
Copy Markdown
Collaborator Author

bw4sz commented Mar 3, 2026

rebased.

Copy link
Copy Markdown
Collaborator

@jveitchmichaelis jveitchmichaelis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@jveitchmichaelis jveitchmichaelis merged commit 0d9ae15 into main Mar 3, 2026
8 checks passed
@jveitchmichaelis jveitchmichaelis deleted the standardize_main_evalaute branch March 3, 2026 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API This tag is used for small improvements to the readability and usability of the python API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GPU (ddp) error for evaluate_empty_frames

2 participants