Obviously, as you can see from my blog, I have a bunch of high end GPUs for my AI work, the GPU I use on my daily driver PC on the other hand is a complete joke (Nvidia rtx 1650) with 4GB of ram… Not exactly a GPU you would use for anything remotely demanding
But running whisper on my local machine is very convinient, the audio files are already there, no need to login to any remote machines and the like, so i will be installing a small version of whisper here, and let us see how this ancient GPU does
1- I already have Python 3.12.7 installed, if you don’t, then “sudo apt install python3 python3-pip”
python3 -m venv whisper-envAnd activate it
source whisper-env/bin/activate
Now, before you procede, if you want your “HuggingFace” directory on a different drive or something (Where the models actually live), you should start by adding the following line to ~/.bashrc or whatever your system uses, also remember to either run (source ~/.bashrc) or to close and open your terminal again for the changes to take effect
export HF_HOME=/mnt/bigdrive/huggingface
Now, let us go ahead and install faster-whisper
pip install faster-whisperAlso, make sure PyTorch with GPU support is available:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
Test GPU availability:
python3 -c "import torch; print(torch.cuda.is_available())"Now, I thought tiny would be the correct size that suits my GPU, but it turned out “mini”base” works just fine !
faster-whisper sample.wav --model-size base --compute-type float16How do we know if we are hitting the GPU limits ?
watch -n 1 nvidia-smi