Skip to content

fix leak in ms_passes_big_int_constraints#1017

Merged
Siyet merged 1 commit intojcrist:mainfrom
provinzkraut:fix-bigint-constraint-leak
Apr 12, 2026
Merged

fix leak in ms_passes_big_int_constraints#1017
Siyet merged 1 commit intojcrist:mainfrom
provinzkraut:fix-bigint-constraint-leak

Conversation

@provinzkraut
Copy link
Copy Markdown
Contributor

While checking something else I noticed and oddity in the refcounts; Turns out, there's a missing Py_DECREF in ms_passes_big_int_constraints.

I've attached the original question as a comment, as I'm yet unsure about it.

Comment thread src/msgspec/_core.c
@@ -10140,6 +10140,7 @@ ms_passes_big_int_constraints(PyObject *obj, TypeNode *type, PathNode *path) {
Py_DECREF(base);
if (remainder == NULL) return false;
long iremainder = PyLong_AsLong(remainder);
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per the docs, we should check for PyErr_Occurred() here to disambiguate a -1, which is also returned in case of an error.

This might be fine, if the intent is that we want to treat any error as failing the constraint check, in which case there's no need to disambiguate between the two.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically yes, but it's harmless here: if PyLong_AsLong fails with overflow, the remainder doesn't fit in a long, which means it's certainly not zero, so the constraint check correctly fails. In practice, overflow is impossible on Linux 64-bit (long == int64_t), and on Windows it would require multiple_of > 2^31.

For comparison, in line 9131 disambiguation is needed because -1 is a valid value there. Here -1 as a remainder also means "not a multiple", so both cases (error and real -1) lead to the same branch.

@Siyet
Copy link
Copy Markdown
Collaborator

Siyet commented Apr 12, 2026

Verified on a Linux x86_64 VPS (Python 3.12, Ubuntu) with a targeted test: Annotated[int, Meta(multiple_of=100000)] decoding big ints (> 2^100) where the remainder (50000) falls outside CPython's small int cache.

Without the fix (5000 iterations): tracemalloc reports ~160KB net memory growth (~5700 leaked PyLong objects, ~28 bytes each). The leak is real and proportional to the number of calls.

With the fix: ~700 bytes net growth (noise). Leak is gone.

Also confirmed:

  • Big int that IS a multiple passes validation correctly
  • Big int that is NOT a multiple fails with ValidationError
  • Negative big int with C remainder -1 fails correctly
  • No dangling exception after successful decode

Copy link
Copy Markdown
Collaborator

@Siyet Siyet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed the leak with tracemalloc on a Linux VPS. The fix is correct, clean one-liner. Thanks Janek!

@Siyet Siyet merged commit 88ce26d into jcrist:main Apr 12, 2026
22 checks passed
@Siyet Siyet mentioned this pull request Apr 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants