Cache ignores variable reassignment between processes

[ccmcintyre-kingsds]

When a value is synced in either direction, Bifrost caches it by key and value, using xxhash. This combination will subsequently not be transmitted between the processes again; this is to save on redundant shared memory operations which have previously been significant, especially in Notebook environments (where we sync all variables in top-level scope by default when using %%node cell magic).

However, if a previously-cached variable is changed, that change in value will not reliably translate across processes. This is especially true when reassigning a variable to a value it has previously held, such as when a variable v changes from value x to y and back to x; the v:x mapping, being previously cached, will be ignored by Bifrost, leaving v equal to y despite the user's intentions (see linked issue which follows). Further existing example code for reproductions and potential solutions for the issue can be made quickly available to the assignee as desired.

Relevant issues:

• Relates to Python and JavaScript Variable Translation Errors in Colab #49 , in which several initial examples of syncing issues from this caching behaviour are demonstrated by Jonah1234567.

Relevant files:

• bifrost/py_storage.py
• bifrost/main.js

Suggested solutions:

• Investigate caching values instead of value:key pairs (most work; potentially high gains; most room for error)
• Turn off the cache or force it to sync cached variables (least work; potentially high loss in notebook performance)
• Remove individual elements from the cache whenever the associated variable is reassigned in either process (best value for effort)