Most GPU applications like TensorFlow, PyTorch or Blender need the NVIDIA CUDA toolkit in order to perform GPU computations.


Checking for CUDA


You can check if a system already has the NVIDIA CUDA toolkit installed and which version by running:

nvcc --version

The following page contains all the information on installing CUDA: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html.

Before Installing

  • Please note that a regular CUDA installation will also ask to (re-)install the NVIDIA GPU driver. 

  • If your instance already has NVIDIA GPU driver installed, reinstalling the drivers during the CUDA installation is discouraged since modifying the drivers often causes issues.
  • You can check if you have the driver installed by running nvidia-smi. Which should report all your instance GPUs.

Installing CUDA


Choose the CUDA toolkit version that you would like to install from the Nvidia Developer Page and follow the instructions, e.g. to download CUDA 12.6 run:

1) wget https://developer.download.nvidia.com/compute/cuda/12.6.2/local_installers/cuda_12.6.2_560.35.03_linux.run
2) sudo sh cuda_12.6.2_560.35.03_linux.run --silent --toolkit

The --silent flag suppresses all output to the command line and the --toolkit flag makes sure that only the toolkit (not the drivers) are installed. Run the command without these flags if you do want to install the drivers. 


Make sure to only select a driver for installation, if you have not installed one already as reinstallation can lead to problems. To finish up two variables need to be added to the bottom of your .bashrc file:

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}$ 
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

For changes to take effect you need to execute these commands outside of .bashrc or by restarting your shell. 


Finally you can check whether everything worked out by calling:

nvcc --version