Raising shared memory limit of a Kubernetes container

Anuj Arora
Dive into ML/AI
Published in
2 min readMar 8, 2021

--

While using Pytorch’s (v1.4.0) Dataloader with multiple workers (num_workers > 0), I encountered the following error,

Bus error. It is possible that dataloader's workers are out of shared memory. Please try to raise your shared memory limit.

With this started my couple of hours long struggle for increasing the shared memory size. Now, if one is running a docker container with docker run command, this issue can be handled by inserting following command line argument.

--shm-size=desired_memo…

--

--