torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 72.00 MiB.

[madcrou]

Hello ! After updates and problems with installing Onetrainer stopped loading the model. Previously, on my 8 gb I was able to do this. Perhaps the developer added some new parameters that I don’t know about? Help please!

Windows 11 pro 10.0.22621 22621
Zephyrus GX701GX_GX701GX
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz 2.21 GHz
NVIDIA GeForce RTX 2080 with MaxQ Design
Python 3.12.9 64-bit

[13.03.2026 19:01] numbra: activating venv C:\OneTrainer\venv
Using Python "C:\OneTrainer\venv\Scripts\python.exe" -X utf8
HF_HUB_DISABLE_XET=1

NOTE: Xet disabled, to enable it set as 0 before launch
Checking Python version...
Python 3.12.9

Starting UI...
Exception in thread Thread-2 (_readerthread):
Traceback (most recent call last):
File "C:\Users\Numbra\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1075, in _bootstrap_inner
self.run()
File "C:\Users\Numbra\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Numbra\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 1601, in _readerthread
buffer.append(fh.read())
^^^^^^^^^
File "", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xad in position 7: invalid start byte
No backup found, continuing without backup...
C:\OneTrainer\venv\Lib\site-packages\tensorboard\default.py:30: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
import pkg_resources
TensorFlow installation not found - running with reduced feature set.
Fetching 9 files: 100%|██████████████████████████████████████████████████████████████████████████| 9/9 [00:00<?, ?it/s]
Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
TensorBoard 2.20.0 at http://localhost:6006/ (Press CTRL+C to quit)
Exception in thread Reloader:
Traceback (most recent call last):
File "C:\Users\Numbra\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1075, in _bootstrap_inner
self.run()
File "C:\Users\Numbra\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\data_ingester.py", line 108, in _reload
self._multiplexer.Reload()
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\plugin_event_multiplexer.py", line 263, in Reload
Worker()
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\plugin_event_multiplexer.py", line 241, in Worker
accumulator.Reload()
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\plugin_event_accumulator.py", line 202, in Reload
for event in self._generator.Load():
^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\directory_watcher.py", line 88, in Load
for event in self._LoadInternal():
^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\directory_watcher.py", line 118, in _LoadInternal
for event in self._loader.Load():
^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\event_file_loader.py", line 270, in Load
for event in super().Load():
^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\event_file_loader.py", line 244, in Load
for record in super().Load():
^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\event_file_loader.py", line 178, in Load
yield next(self._iterator)
^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\backend\event_processing\event_file_loader.py", line 109, in next
self._reader.GetNext()
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\compat\tensorflow_stub\pywrap_tensorflow.py", line 207, in GetNext
header_str = self._read(8)
^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\compat\tensorflow_stub\pywrap_tensorflow.py", line 273, in _read
new_data = self.file_handle.read(n)
[13.03.2026 19:01] numbra: ^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\compat\tensorflow_stub\io\gfile.py", line 736, in read
(self.buff, self.continuation_token) = self.fs.read(
^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\tensorboard\compat\tensorflow_stub\io\gfile.py", line 141, in read
data = f.read(size)
^^^^^^^^^^^^
MemoryError
Traceback (most recent call last):
File "C:\OneTrainer\modules\ui\TrainUI.py", line 712, in __training_thread_function
trainer.start()
File "C:\OneTrainer\modules\trainer\GenericTrainer.py", line 140, in start
self.model_setup.setup_train_device(self.model, self.config)
File "C:\OneTrainer\modules\modelSetup\FluxLoRASetup.py", line 151, in setup_train_device
model.transformer_to(self.train_device)
File "C:\OneTrainer\modules\model\FluxModel.py", line 162, in transformer_to
self.transformer.to(device=device)
File "C:\OneTrainer\venv\src\diffusers\src\diffusers\models\modeling_utils.py", line 1451, in to
return super().to(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 1371, in to
return self._apply(convert)
^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 930, in _apply
module._apply(fn)
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 930, in _apply
module._apply(fn)
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 930, in _apply
module._apply(fn)
[Previous line repeated 3 more times]
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 957, in _apply
param_applied = fn(param)
^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 1357, in convert
return t.to(
^^^^^
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 72.00 MiB. GPU 0 has a total capacity of 8.00 GiB of which 1024.00 KiB is free. Of the allocated memory 6.91 GiB is allocated by PyTorch, and 17.11 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Для продолжения нажмите любую клавишу . . .

[madcrou]

When I changed the main load parameters, I get this

activating venv C:\OneTrainer\venv
Using Python "C:\OneTrainer\venv\Scripts\python.exe" -X utf8
HF_HUB_DISABLE_XET=1

NOTE: Xet disabled, to enable it set as 0 before launch
Checking Python version...
Python 3.12.9

Starting UI...
Exception in thread Thread-2 (_readerthread):
Traceback (most recent call last):
File "C:\Users\Numbra\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1075, in _bootstrap_inner
self.run()
File "C:\Users\Numbra\AppData\Local\Programs\Python\Python312\Lib\threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Numbra\AppData\Local\Programs\Python\Python312\Lib\subprocess.py", line 1601, in _readerthread
buffer.append(fh.read())
^^^^^^^^^
File "", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xad in position 7: invalid start byte
Clearing cache directory workspace-cache/run! You can disable this if you want to continue using the same cache.
Fetching 9 files: 100%|██████████████████████████████████████████████████████████████████████████| 9/9 [00:00<?, ?it/s]
loss_type=None was set in the config but it is unrecognized. Using the default loss: ForCausalLMLoss.
Quantizing model weights: 100%|██████████████████████████████████████████████████████| 463/463 [00:13<00:00, 33.67it/s]
Quantizing model weights: 100%|████████████████████████████████████████████████████| 1279/1279 [01:31<00:00, 13.91it/s]
Selected layers: 504
Deselected layers: 0
Note: Enable Debug mode to see the full list of layer names
enumerating sample paths: 100%|██████████████████████████████████████████████████████████████████| 1/1 [00:00<?, ?it/s]
caching: 100%|█████████████████████████████████████████████████████████████████████████| 67/67 [00:16<00:00, 4.00it/s]
caching: 100%|█████████████████████████████████████████████████████████████████████████| 67/67 [00:24<00:00, 2.68it/s]
caching: 100%|█████████████████████████████████████████████████████████████████████████| 67/67 [00:24<00:00, 2.79it/s] C:\OneTrainer\venv\Lib\site-packages\torch\nn\functional.py:2920: UserWarning: Mismatch dtype between input and weight: input dtype = float, weight dtype = struct c10::Half, Cannot dispatch to fused implementation. (Triggered internally at C:\actions-runner_work\pytorch\pytorch\pytorch\aten\src\ATen\native\layer_norm.cpp:347.)
return torch.rms_norm(input, normalized_shape, weight, eps)

[13.03.2026 21:43] numbra: step: 0%| | 0/16 [06:00<?, ?it/s]
epoch: 0%| | 0/100 [07:10<?, ?it/s]
Traceback (most recent call last):
File "C:\OneTrainer\modules\ui\TrainUI.py", line 717, in __training_thread_function
trainer.train()
File "C:\OneTrainer\modules\trainer\GenericTrainer.py", line 747, in train
model_output_data = self.model_setup.predict(self.model, batch, self.config, train_progress)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\modules\modelSetup\BaseFluxSetup.py", line 294, in predict
packed_predicted_flow = model.transformer(
^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\src\diffusers\src\diffusers\utils\peft_utils.py", line 315, in wrapper
result = forward_fn(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\src\diffusers\src\diffusers\models\transformers\transformer_flux.py", line 758, in forward
encoder_hidden_states, hidden_states = block(
^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\modules\util\checkpointing_util.py", line 135, in forward
return torch.utils.checkpoint.checkpoint(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch_compile.py", line 53, in inner
return disable_fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch_dynamo\eval_frame.py", line 1044, in _fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\utils\checkpoint.py", line 496, in checkpoint
return CheckpointFunction.apply(function, preserve, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\autograd\function.py", line 581, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\utils\checkpoint.py", line 262, in forward
outputs = run_function(*args)
^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\modules\util\checkpointing_util.py", line 119, in __checkpointing_forward
output = self.orig_forward(*args) if self.checkpoint is None else self.checkpoint(*args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\src\diffusers\src\diffusers\models\transformers\transformer_flux.py", line 390, in forward
mlp_hidden_states = self.act_mlp(self.proj_mlp(norm_hidden_states))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\venv\Lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\OneTrainer\modules\module\LoRAModule.py", line 329, in forward
[13.03.2026 21:43] numbra: return self.orig_forward(x) + ld * (self.alpha / self.rank)
~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 448.00 MiB. GPU 0 has a total capacity of 8.00 GiB of which 0 bytes is free. Of the allocated memory 13.00 GiB is allocated by PyTorch, and 1.42 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

[madcrou]

This helped me: Train Data Type = Float 32 / Gradient CheckPointing = CPU_OFFLOADED / ETA = OFF. In the optimizer: Fused Bask Pass On / Fused Off