Xet not working on Kaggle TPUs

[windmaple]

Filed this bug but I'm guessing it's a Xet issue instead of dataset issue under the hood.

I'm trying to load a dataset on Kaggle TPU image. This does not work as you can see in here. The error is:

2025-10-14T03:12:33.133556Z ERROR  Python exception updating progress:, error: PyErr { type: <class 'LookupError'>, value: LookupError(<ContextVar name='shell_parent' at 0x7ce1aa0b1ee0>), traceback: Some(<traceback object at 0x7ce0b6599600>) }, caller: "src/progress_update.rs:313"
    at /home/runner/work/xet-core/xet-core/error_printer/src/lib.rs:28

And the code is just stuck there.

[jsulz]

Thanks for the report, @windmaple!

I was able to replicate this on a TPU image as well (other accelerators - GPU T4 x2 and GPU P100 worked just fine). We'll look into what's going on, but in the meantime, so you're not blocked, you can uninstall hf-xet (!pip uninstall hf-xet) and the download will fall back to a more traditional HTTP download path.

[windmaple]

Thanks Jared! I was able to move a bit further after uninstalling hf-xet. But I hit another error NonMatchingSplitsSizesError right after. See the updated notebook.

[jsulz]

Hm, did you do a reset of the notebook and start over?

Here's the Kaggle notbook where I was debugging this in (also on a TPU v5e-8) and am able to make it through a download successfully.

[windmaple]

You are absolutely right. For some reason it wasn't working for me; now it works! Thanks again!

[windmaple]

Instead of uninstalling and restarting, we found that os.environ["HF_HUB_DISABLE_XET"] = "1" also works.

[jsulz]

Ha! I was going to suggest that, but in my testing it was not sticking and was trying to figure out why. Glad it worked for you!

[rajatarya]

@windmaple : Been a long time - we have completely revamped the progress reporting in hf-xet in the last few months - can you try to reproduce this error now to confirm if it still exists?

[windmaple]

I have upgraded to the latest datasets lib and it still hangs.

https://www.kaggle.com/code/windmaple/hf-datasets-issue

[seanses]

I noticed the below error message. This aligns with our thinking that updating progress to Python side via a GIL callback is dangerous. And our existing Xet Session work to replace it with a pull based progress update pattern is promising.

2026-03-25T00:52:36.440236Z ERROR Python exception updating progress:, error: PyErr { type: <class 'LookupError'>, value: LookupError(<ContextVar name='shell_parent' at 0x7be9615c3f60>), traceback: Some(<traceback object at 0x7be886586b80>) }, caller: "src/progress_update.rs:313"
at /home/runner/work/xet-core/xet-core/error_printer/src/lib.rs:28