libmem: Add unit tests for zones

[ozhuraki]

No description provided.

[askervin]

In reality, not all zones are of equal size, and in unreality NodeCapacity() could have a bug that would assume all sizes to be equal.

Using different capacities (and why not different types, movabilities, different closeCPUs set sizes on different nodes as well), these tests could cover cases that are more difficult with e2e tests where we have to actually simulate such platforms.

Unit tests for libmem could actually stress test the allocator very thoroughly, which I would find quite useful. I mean using a fuzzer that would keep track on the expected state of allocators and zones, too. @ozhuraki, would you be interested in going to that direction? @klihub, do you think it would make sense?

[ozhuraki]

@askervin

would you be interested in going to that direction?

Thanks. Yes, I would be interested. I added a very basic one, FuzzZoneUsageAndFree().

I would like to experiment a bit, i.e. to make it more developer-friendly (preferably to parameterize it, add a bit of ASCII visualization, and add record/replay for error cases and document). I could cut it off and add it separately, WDYT?

[klihub]

I would like to experiment a bit, i.e. to make it more developer-friendly (preferably to parameterize it, add a bit of ASCII visualization, and add record/replay for error cases and document). I could cut it off and add it separately, WDYT?

@askervin @ozhuraki Since we are talking about fuzzing, I think what would make sense is to have randomized but reproducible fuzzing tests. IOW, write the test so that it can take an externally provided test seed (for instance from an environment variable), and generate (and print) a random one if none is given.

Additionally, it would make sense to write actual tests so that we have tests both with just sequences allocation from 0 till maxed out mem, and sequences of both allocations and releases, with allocations fuzzed wrt. both container type (IOW relative priority) and allocation size, and releases fuzzed wrt. which existing allocation we release, instead of these being fixed/hardcoded.

Also IMO either some capping of max. allocation size or other way of enforcing minimum number of steps till memory is exhausted could make sense, so that with a randomly generated seed we don't degenerate the fuzzing test sequence in reality to just a few steps which already exhaust all the memory, which is then not an interesting case for fuzzing.

With such a setup we should be able to achieve good fuzzing coverage over time and have reproducibility for any bugs we might trigger and uncover.

[askervin]

FuzzZoneUsageAndFree().

Cool!

For more complex fuzzing, what I had in mind was a little bit of a research if we could apply a stateful fuzzer like:

https://github.com/askervin/gofmbt

The idea is that the fuzzer understands which allocations are valid, which are invalid. It's able to test them both and cover them in different scenarios (that is states) and orders (that is different subsequences of inputs).

This fuzzer is documented in

https://github.com/askervin/gofmbt/blob/main/gofmbt/doc.go

and there is an example that tests a music player: it knows when the player is paused or playing, and which song in the playlist it is playing, and generates inputs based on that, aiming to cover more and more untested combinations:

https://github.com/askervin/gofmbt/blob/main/examples/player/main.go

Take your time to familiarize yourself with it, tell me if it looks something that could be useful in this scenario, and let's decide whether or not to go with it. I haven't tested myself how well copilot picks up the idea and supports model creation...

[askervin]

@klihub, I'm optimistic that above requirements (all very sensible and definitely needed) could be satisfied with this approach. Should there be new features needed, which I doubt, they'd be easily accepted into the project.

[ozhuraki]

@klihub @askervin Thanks for the input! Let me check what practical approaches (including gofmbt) would make sense on the allocator first, and then let's think where else such might help. I will cut the fuzzying off for now, re-work and add it separately.

[klihub]

@klihub, I'm optimistic that above requirements (all very sensible and definitely needed) could be satisfied with this approach. Should there be new features needed, which I doubt, they'd be easily accepted into the project.

@askervin I am totally fine with using gofmbt for more intelligent fuzzing.