Fix AsdfFile pickling for instances without file descriptors#2038
Fix AsdfFile pickling for instances without file descriptors#2038sydduckworth wants to merge 14 commits into
AsdfFile pickling for instances without file descriptors#2038Conversation
|
@braingram I've reverted the public API for |
|
I think we'd benefit from including a wider set of possible Testing this PR if import asdf, pickle, numpy as np
af = asdf.AsdfFile({"arr": np.arange(42)})
pickle.loads(pickle.dumps(af)) # passes
af.validate()
pickle.loads(pickle.dumps(af)) # failsThe same error appears if array storage type is set on "arr" prior to pickling. af.set_array_storage(af["arr"], "inline")
pickle.loads(pickle.dumps(af)) # failsAs part of this we should address what operations cause pickling to fail and why? What are the "typical" AsdfFile instances we want to support (probably at least a tree created from in-memory objects but also one read from a file and read then modified)? I think we shouldn't claim pickling support until we support whatever we decide are the "typical" cases. |
|
Is there a different changelog entry you would prefer for a PR that specifically fixes the two bugs addressed here but that does not fully enable pickling? |
Description
This PR fixes two bugs that prevented
AsdfFileinstances from being pickled, which partially addresses #1782.The result is that now asdf files can be pickled if they don't contain a file descriptor, which basically means files that were created from an in-memory dict.
AsdfObjectinstances were failing to deserialize because of their conflictingdictandUserDictbase classes. Added a manual__reduce__override which resolves the problem.ValidatorManagerto not return local callables defined inside a method, which can't be pickled.ValidatorManagernow returns a newJsonSchemaValidatorscallable classValidatorManagerimplementation._tests/test_asdf.pyto verify the limited pickling that is now supported.