[ENH] V1 → V2 API Migration - evaluation measures by EmanAbdelhaleem · Pull Request #1603 · openml/openml-python

EmanAbdelhaleem · 2026-01-07T09:01:52Z

Fixes #1593
Depends on #1576
Related to #1575

Details

This PR implements EvaluationMeasures resource, and refactor its existing functions

codecov-commenter · 2026-01-07T09:24:09Z

Codecov Report

❌ Patch coverage is 53.55191% with 340 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.97%. Comparing base (d421b9e) to head (90f0223).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
openml/_api/clients/http.py	23.36%	164 Missing ⚠️
openml/_api/resources/base/versions.py	24.71%	67 Missing ⚠️
openml/_api/setup/backend.py	64.04%	32 Missing ⚠️
openml/_api/resources/base/fallback.py	26.31%	28 Missing ⚠️
openml/_api/resources/evaluation_measure.py	31.81%	15 Missing ⚠️
openml/_api/setup/_utils.py	56.00%	11 Missing ⚠️
openml/testing.py	55.55%	8 Missing ⚠️
openml/_api/setup/builder.py	81.08%	7 Missing ⚠️
openml/_api/clients/utils.py	55.55%	4 Missing ⚠️
openml/_api/resources/base/base.py	86.36%	3 Missing ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1603      +/-   ##
==========================================
+ Coverage   52.04%   52.97%   +0.92%     
==========================================
  Files          36       64      +28     
  Lines        4333     5082     +749     
==========================================
+ Hits         2255     2692     +437     
- Misses       2078     2390     +312

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

geetu040

This looks great. The implementation is clean and nicely decoupled. The next step should be adding tests, but let's wait until we have a clearer roadmap for that.
Also, please update the PR description to follow this format:

Fixes #1593
Depends on #1576
Related to #1575

EmanAbdelhaleem · 2026-02-03T20:29:29Z

@geetu040 Ready for review

source: openml#1606 (comment)

- removing this since it was not part of the sdk previously - some tests fail because of the timeout in stacked PRs - this option can easily be added if needed in future

geetu040

this looks good, some code is duplicated, let's wait for the design discussion

tests/test_api/test_evaluation_measure.py

identified while debugging openml#1616 (comment)

geetu040

please sync with base PR and update with these comments #1576 (comment)

geetu040 and others added 4 commits December 30, 2025 09:11

set up folder structure and base code

0159f47

Merge branch 'main' into migration

58e9175

Merge branch 'main' into migration

bdd65ff

fix pre-commit

52ef379

geetu040 mentioned this pull request Jan 7, 2026

[ENH] V1 → V2 API Migration #1575

Open

25 tasks

geetu040 reviewed Jan 7, 2026

View reviewed changes

geetu040 and others added 6 commits January 7, 2026 22:14

refactor

5dfcbce

implement cache_dir

2acbe99

refactor

af99880

Merge branch 'main' into pr/1576

74ab366

undo changes in tasks/functions.py

4c75e16

Merge branch 'main' into migration

5762185

geetu040 assigned EmanAbdelhaleem Jan 19, 2026

geetu040 added 12 commits January 21, 2026 10:47

Merge branch 'main' into migration

7e9bc1f

add tests directory

c603383

use enum for delay method

ff6a8b0

implement cache

f01898f

refactor clients

5c4511e

fix import in resources/base.py

43276d2

refactor and add exception handling

1206f69

refactor resources/base/

4948e99

implement delete

a354167

implement publish and minor refactoring

1fe7e3e

implement tag/untag

54a3151

implement fallback

2b6fe65

EmanAbdelhaleem force-pushed the evaluation-measures-mig branch from f0c57de to e091d57 Compare January 28, 2026 03:28

geetu040 added 3 commits January 28, 2026 13:50

add test_http.py

fa53f8d

add uses_test_server marker

2b2db96

implement reset_cache

c9617f9

EmanAbdelhaleem force-pushed the evaluation-measures-mig branch from 6236112 to a19cb35 Compare February 3, 2026 19:46

update client confit in test_evaluation_measure

9e5db43

geetu040 added 3 commits February 4, 2026 13:57

fix unbound variables: "code", "message"

85c1113

source: openml#1606 (comment)

use requests.Session()

39bf86a

remove "timeout_seconds" entirely

7b66677

- removing this since it was not part of the sdk previously - some tests fail because of the timeout in stacked PRs - this option can easily be added if needed in future

EmanAbdelhaleem mentioned this pull request Feb 4, 2026

[ENH] V1 → V2 API Migration - studies #1610

Open

geetu040 added 4 commits February 4, 2026 22:42

update/refactor tests

d2224c4

remove unused current_api_version from TestAPIBase

9608c36

make TestAPIBase inherit TestBase

f6bc7f7

nits: test classes

baa3a38

geetu040 reviewed Feb 5, 2026

View reviewed changes

tests/test_api/test_evaluation_measure.py Outdated Show resolved Hide resolved

geetu040 and others added 17 commits February 7, 2026 00:03

minor fix in _sync_api_config

52b93fe

identified while debugging openml#1616 (comment)

chore: rerun CI

ec9477f

Merge branch 'pr-1576' into evaluation-measures-mig

3efd5cc

updated tests with the new design

a909fc7

remove duplicates in _api/resources/__init__.py

10d134a

implement HTTPClient.download and add tests

935f0f4

add docstrings

9514df8

update minio

53bee94

make delay functions static

33b4ca0

rename: retry_raise_e -> exception

a6b9a45

use context-manager for requests.Session

f924b32

remove "assert response is not None"

541b0f2

verify checksum before caching

acb173f

update tests

3e8d1f0

minor fix in ResourceV1API.untag

f83bdb5

update tests

658a696

Merge branch 'pr-1576' into evaluation-measures-mig

90f0223

geetu040 suggested changes Feb 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ENH] V1 → V2 API Migration - evaluation measures#1603

[ENH] V1 → V2 API Migration - evaluation measures#1603
EmanAbdelhaleem wants to merge 94 commits intoopenml:mainfrom
EmanAbdelhaleem:evaluation-measures-mig

EmanAbdelhaleem commented Jan 7, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Jan 7, 2026 •

edited

Loading

Uh oh!

geetu040 left a comment

Uh oh!

EmanAbdelhaleem commented Feb 3, 2026

Uh oh!

geetu040 left a comment

Uh oh!

Uh oh!

geetu040 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

EmanAbdelhaleem commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Uh oh!

codecov-commenter commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

geetu040 left a comment

Choose a reason for hiding this comment

Uh oh!

EmanAbdelhaleem commented Feb 3, 2026

Uh oh!

geetu040 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

geetu040 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

EmanAbdelhaleem commented Jan 7, 2026 •

edited

Loading

codecov-commenter commented Jan 7, 2026 •

edited

Loading