CUDA Version Management

Managing CUDA versions

Problem

There are many libraries that only support specific versions of CUDA. Downgrading/upgrading CUDA can sometimes be tricky (especially downgrading). It’s often desirable to manage CUDA versions per project (instead of globally), without having to reach for Docker.

Solution

You can use conda to manage your CUDA versions! This allows you to isolate specific CUDA versions to specific environments rather than managing CUDA versions globally.

Note

I’m using mamba which has faster solvers than conda. Refer to the docs for installation instructions.

Let’s say I want to downgrade to CUDA 11.7 in its own conda environment. First, I will create a new environment named cuda11-7 with the following command:

mamba create -n cuda11-7 python=3.8
mamba activate cuda11-7

Before I downgrade, we can check our CUDA version with the following command:

> nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

As you can see, I have CUDA version 11.8 but I want to downgrade to 11.7. We can downgrade CUDA by using cuda-toolkit:

mamba install -c "nvidia/label/cuda-11.7.1" cuda-toolkit

This will take several minutes to complete. Next, recheck your CUDA version:

> nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Jun__8_16:49:14_PDT_2022
Cuda compilation tools, release 11.7, V11.7.99
Build cuda_11.7.r11.7/compiler.31442593_0

Next, you need to install the correct version of PyTorch for your CUDA version. It is crucial to install the right version of PyTorch that matches your CUDA version exactly. For example, if you want to install PyTorch with CUDA 11.7, you can use the following command:

mamba install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

You can find PyTorch installation instructions on this page.

Viola! You have downgraded your CUDA version successfully. Note that this version of CUDA is isolated to this specific environment.

To make sure that everything is working correctly, make sure you can import torch and check the CUDA version from within Python:

> python -c "import torch; print(torch.version.cuda)"
11.7

Additional Resources

  • Why does nvcc --version sometimes report a different CUDA version than nvidia-smi? See this answer on Stack Overflow.
  • Twitter discussion on this topic.