Hello, when training the model, despite specifying multiple GPUs, it seems that the model is only training on a single GPU. Is there a way to train the model in parallel across multiple GPUs?
I've noticed the "local_rank" parameter, and I'm not sure if modifying this parameter can enable parallel training. Could you provide further guidance on how to use this parameter? Thank you.
Hello, when training the model, despite specifying multiple GPUs, it seems that the model is only training on a single GPU. Is there a way to train the model in parallel across multiple GPUs?
I've noticed the "local_rank" parameter, and I'm not sure if modifying this parameter can enable parallel training. Could you provide further guidance on how to use this parameter? Thank you.