Memory usage impact of `DerivedComponentMap`

[malte0811]

While analyzing memory usage in All The Mods 10 for FerriteCore (total heap: 4.5 GB), I noticed that owolibs DerivedComponentMap feature increases memory usage by about 350 MB. I think the root cause is that the logic is applied to all stacks (about 2.75M instances in memory), rather than just those for items that implement a nontrivial deriveStackComponents (i.e. a very small fraction).

The additional memory usage has two separate causes:

• Calling PatchedDataComponentMap::applyPatch/ComponentMapImpl::setChanges on a newly initialized component map will call ensureMapOwnership, even if the argument is an empty map as it is here. This causes the singleton returned by Reference2ObjectMaps.emptyMap to be replaced with a new Reference2ObjectArrayMap, which uses 72 extra bytes per stack. Since nearly all stacks in memory have empty patches, this means an extra memory usage of 72 * 2.75M = 198 MB. I implemented a more general fix for memory usage from these empty maps in the latest release for FerriteCore, but I think it would still be good if this was fixed on your end as well.
• Even when this is resolved, the feature results in noticeably increased memory usage. This usage is simply from the presence of two new object for each ItemStack instance, one DerivedComponentMap (24 bytes shallow) and the PatchedDataComponentMap/ComponentMapImpl stored in it (32 bytes shallow). This works out to 56 * 2.75M = 154 MB of additional memory usage.

[Noaaan]

Out of curiosity, since I do not have a ton of experience with using profilers: how did you achieve 2.75 million instances of Item Stacks in memory? That sounds like a worst-case scenario of some kind

[malte0811]

That seems to be a fairly normal amount for large packs, but I agree that it "seems too big" at first glance. Most of them are "owned" by JEIs recipe infrastructure, I haven't investigated further than that (not sure when I'll find time for it). Maybe a lot of eagerly resolved tag contents, at least at one point all recipes were stored that way client-side.

My test setup was to launch ATM10 and join a newly generated singleplayer world. Then I took a Java heap dump (essentially a file containing every active Java object with its contents) and analyzed it using Eclipse's Memory Analyzer Tool. You can also get the 200 MB and 150 MB numbers directly through that, but I think computing them "from scratch" has less potential for measurement mistakes.