While LLM (Large Language Model) is getting more and more popular, lots of people asks me how to setup home lab to train, fine tune, or inference LLM models. We will publish a few articles on how to setup different GPU backed single lab step by step.
In this article, we will work through the installation on how to setup Nvidia GeForce RTX3080 in Ubuntu 22.04. You could refer to below article on how to install Ubuntu 22.04.
Note: We are using a testing server for this article. My local server name is rogserver and local username is roguser. Please replace those to your local hostname and local username.
Overview
Before we start installation, let’s first go through what we are going to install here:
- Nvidia Driver — This is the hardware driver from Nvidia. Find out your graphic card model before the installation.
- CUDA Toolkit — The CUDA Toolkit is provided by Nvidia. It contains libraries, compilers, development tools and samples, CUDA runtime, etc. in a GPU-accelerated environment.