Build gentle /w CUDA enabled kaldi

Gentle: Robust yet lenient forced-aligner built on Kaldi. A tool for aligning speech with text.

We’re using it to help us align captions with video, the most problem is, it’s too slow to meet our requirement. After doing some research, we found that Kaldi can speed up by using NVIDIA CUDA technology.

Here are the steps we used to build Gentle with CUDA.

We’re using Ubuntu 16.04, logged in with root.

Install dependencies

apt update && apt install -y python build-essential gfortran

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/0.dependencies.svg

Install NVIDIA driver

1. Download from Nvidia website [here]

NVIDIA DRIVERS Tesla Driver for Ubuntu 16.04

2. Install the driver

dpkg -i nvidia-diag-driver-local-repo-ubuntu1604–396.37_1.0–1_amd64.deb
apt-get update && apt-get install cuda-drivers
reboot

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/1.install.driver.svg

3. Check driver installed correctly

lsmod | grep nvidia
nvidia_uvm            786432  0
nvidia_drm             40960  2
nvidia_modeset       1089536  2 nvidia_drm
drm_kms_helper        172032  1 nvidia_drm
drm                   401408  5 nvidia_drm,drm_kms_helper
nvidia              14024704  79 nvidia_modeset,nvidia_uvm
ipmi_msghandler        53248  2 nvidia,ipmi_devintf

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/2.check.driver.svg

Install CUDA Toolkit

1. After driver installed, you MUST reboot first.
2. Download toolkit from Nvidia website [here]

CUDA Toolkit 9.2 Download

3. Install them

dpkg -i cuda-repo-ubuntu1604-9-2-local_9.2.148-1_amd64.deb
apt-key add /var/cuda-repo-<version>/7fa2af80.pub
apt-get update && apt-get install cuda

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/3.install.cuda.svg

4. Check Toolkit

nvidia-smi

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/4.check.cuda.svg

Build gentle from source

1. Clone gentle from GitHub

lowerquality/gentle

git clone https://github.com/lowerquality/gentle

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/5.git.clone.svg

2. Init submodule first

cd gentle
git submodule init
git submodule update

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/6.git.init.subm.svg

3. Patch Makefile

vim ext/kaldi/tools/Makefile
:%s/openfst\.cs\.nyu\.edu/www\.openfst\.org/g
:wq

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/7.patch.makefile.svg

4. Patch install_kaldi.sh

vim ext/install_kaldi.sh
:%s/--use-cuda=no/--use-cuda=yes --cudatk-dir=\/usr\/local\/cuda/g
:wq

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/8.patch.config.script.svg

5. Patch k3.cc

vim ext/k3.cc
-> comment line 99
//cu_device.SetVerbose(true);
:wq

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/9.patch.k3.cc.svg

6. Build gentle

./install.sh

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/10.build.gentle.svg

7. Demo gentle /w CUDA

python3 server.py

Terminal Recording
https://cdn.rawgit.com/merik-chen/termtosvg-gentle-w-cuda/fb825d94/11.demo.svg