Skip to content

Comments

Fix: Install correct CUDA toolkit during build#2088

Open
chamalgomes wants to merge 3 commits intoabetlen:mainfrom
chamalgomes:main
Open

Fix: Install correct CUDA toolkit during build#2088
chamalgomes wants to merge 3 commits intoabetlen:mainfrom
chamalgomes:main

Conversation

@chamalgomes
Copy link

@chamalgomes chamalgomes commented Nov 12, 2025

This ensures that correct requested cuda-toolkit is installed via mamba during the cuda build process.

Fixes : #2089

TheBigEye added a commit to TheBigEye/guanaco-py that referenced this pull request Jan 24, 2026
@anr2me
Copy link

anr2me commented Feb 17, 2026

You will also need to set cuda-version's version, so the other cuda-* packages will have a matching version, otherwise you will get mixed versions where some of them use the latest cuda (which can be a problem when using the wheel on platforms that doesn't have the latest driver like Colab/Kaggle with T4 GPU), for example:

$cudaVersion = $env:CUDAVER
$cudaParts = $cudaVersion -split '\.'
$cudaMajorMinor = "$($cudaParts[0]).$($cudaParts[1])"
...
...
mamba install -y "cuda-version==$cudaMajorMinor.*" cuda-toolkit cuda-cudart-dev cuda-cccl cuda-nvvm cuda-nvcc "conda-forge::gcc==13.*"

However, this will only works up to cuda 13.0 on Windows, trying to build with cuda-toolkit 13.1 on Windows will get a lot of errors (suspected to be related to deprecated older cuda capability targets), but it can be built successfully on Linux with cuda-toolkit 13.1 (not sure whether it can run properly or not on old GPU architectures).

I also have issue in building wheel for cuda 12.9, which can only be solved by using cuda toolkit 13.x (due to CCCL macro issue), so i decided not to build wheel for cuda 12.9, since it will need cudart library from cuda 13 (which doesn't works well without the latest driver, like in Colab/Kaggle with T4 GPU).

There are also installation directory difference between cuda 13.x and older cuda version, which can be solved easily on Linux with env vars, but quite complicated on Windows (i ended creating many symlinks).
For example:

            if (-not (Test-Path -Path "$env:CONDA_PREFIX\\bin")) {
              New-Item -ItemType SymbolicLink -Path "$env:CONDA_PREFIX\\bin" -Target "$env:CONDA_PREFIX\\Library\\bin"
            } else { echo "Warning: bin directory already existed! Not creating symlink."}
            if (-not (Test-Path -Path "$env:CONDA_PREFIX\\nvvm")) {
              New-Item -ItemType SymbolicLink -Path "$env:CONDA_PREFIX\\nvvm" -Target "$env:CONDA_PREFIX\\Library\\nvvm"
            } else { echo "Warning: nvvm directory already existed! Not creating symlink."}
            if (-not (Test-Path -Path "$env:CONDA_PREFIX\\lib")) {
              New-Item -ItemType SymbolicLink -Path "$env:CONDA_PREFIX\\lib" -Target "$env:CONDA_PREFIX\\Library\\lib" # -Force
            } else { 
              echo "Warning: lib directory already existed! Not creating symlink."
              if (-not (Test-Path -Path "$env:CONDA_PREFIX\\lib\\x64")) {
                if ([int]$cudaMajor -ge 13) {
                  New-Item -ItemType SymbolicLink -Path "$env:CONDA_PREFIX\\lib\\x64" -Target "$env:CONDA_PREFIX\\Library\\lib\\x64"
                } else {
                  New-Item -ItemType SymbolicLink -Path "$env:CONDA_PREFIX\\lib\\x64" -Target "$env:CONDA_PREFIX\\Library\\lib"
                }
              } else { echo "Warning: lib\x64 directory already existed! Not creating symlink."}
            }
            if (-not (Test-Path -Path "$env:CONDA_PREFIX\\lib64")) {
              New-Item -ItemType SymbolicLink -Path "$env:CONDA_PREFIX\\lib64" -Target "$env:CONDA_PREFIX\\Library\\lib" # -Force
            } else { echo "Warning: lib64 directory already existed! Not creating symlink."}
            if (-not (Test-Path -Path "$env:CONDA_PREFIX\\include")) {
              New-Item -ItemType SymbolicLink -Path "$env:CONDA_PREFIX\\include" -Target "$env:CONDA_PREFIX\\Library\\include"
            } else { echo "Warning: include directory already existed! Not creating symlink."}

Additionally, there was also an issue with gcc 14+ (i forgot what the issue was 🤔 seems to build successfully on my last workflow without cuda 13.1).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: CUDA build workflow not installing correct CUDA version

3 participants