I was recently working with ffmpeg and NVIDIA T4 GPUs on GKE for a encoding pipeline. To get started with GPUs on GKE, the NVIDIA drivers need to be installed on the nodes. After installing, ffmpeg should be able to access NVIDIA GPU capabilities like nvenc, nvdec , yadif_cuda, etc…


A recent task had me taking a look at alternative JSON libraries for the purpose of performance. One of them was python-rapidjson which offered support for SIMD.

Compiling

To get python-rapidjson to compile with SIMD, we need to define one of the SIMD macros, either RAPIDJSON_SSE2, RAPIDJSON_SSE42, or RAPIDJSON_NEON.

The chosen flag would then need to be passed to pip during install via CFLAGS. Depending on the flag, you would have to pass some addition options.

  • SSE2: CFLAGS="-DRAPIDJSON_SSE2=1"
  • SSE4.2: CFLAGS="-DRAPIDJSON_SSE42=1 -msse4.2"

One-liner to re-install the currently installed version as the SIMD version.

CFLAGS="-DRAPIDJSON_SSE42=1 -msse4.2" pip -v install -force-reinstall -no-binary python-rapidjson $(pip freeze | grep python-rapidjson)

Many popular apps offer live stream as a feature integration like Twitch and Youtube Live. With the ever increasing users drawn towards video based services, as seen with the trending race to top Youtube subscribed, the technology behind video delivery is equally increasing in importance. …


To use XFS with Persistent Volumes, the host node needs to have the command xfs_mkfile available so disks can be created and formatted. The problem comes when needing to do this on GKE where there are 2 OSs available, ubuntu which has xfsprogs installed by default and cos which does…


I’ve recently needed to revisit some of our deployments which were created in the earlier days of GKE where some useful features were not available. One component revisited was the disabling the kernel setting Transparent Huge Pages (THP) recommended for mongo and redis.

The solution at the time was to…


Without creating custom images

tl;dr Use empty volume, initContainers, and subPath to copy and mount kubectl.

The Why

I needed access to the kubernetes API from within a pod so that the pod can self label itself.

For example, I am currently working with redis and redis-sentinel. When sentinel triggers a reconfigure script, I want the…


There are some situations where you just need to run a simple single command cronjob. This is where the alpineDocker image comes in very handy. It comes with a simple yet flexible cron package via busybox.

Single Line

For very simple stuff, a single command using the default alpine image.

Mounted Volume

For more complex crontabs, mount a crontab file!

Compiled Docker Image

Compiling your own image with an entrypoint makes it cleaner to run.


Update Sept 6 2017: As of Aug 8 2017, GKE has released the ubuntu image to replace container-vm. This image has xfsprogs pre-installed. Use this instead.

We had a mongo replicaset cluster setup in Google Container Engine that was using the default ext filesystem when creating PersistentVolumes. …


To start streaming camera video, we simply combine raspivid and ffmpeg by piping one into another.

Prerequisites

  • raspivid
  • ffmpeg
raspivid -t 0 -w 1280 -h 720 -fps 25 -g 75 -fl -o - | ffmpeg -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 -i pipe:0 -c:v copy -c:a aac -strict experimental -f flv -f flv…

Allan Lei

The Adventures of Me

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store