Saliya EkanayakeChatGPT Can Write Parallel Code and Even Use OpenMPThis is mind-blowing to say the least.Dec 15, 20221Dec 15, 20221
Saliya EkanayakeI've been always thinking why we call toss of a coin or roll of a dice random.Mar 6, 2021Mar 6, 2021
Saliya EkanayakeMegatronLM: Model Parallelism Using Multiple Tensors in PyTorchAn interesting article and its code repo came from Nvidia on their MegatronLM model. You can read more about it at…Aug 13, 2019Aug 13, 2019
Saliya EkanayakeChoosing a Specific Collective Algorithm Implementation in OpenMPIUnless you are profiling the performance of specific collective routine, this is not something an everyday MPI programmer would have to do…Jun 19, 20191Jun 19, 20191
Saliya EkanayakeCombBLAS Essentials — Part 1CombBLAS is a distributed graph library that has some neat and efficient sparse matrix operations. This blog serves as a refresher on how…May 21, 2019May 21, 2019
Saliya EkanayakePrecision, Reall, and Accuracy in Binary ClassificationWriting this so I wouldn’t have to look it up again in another blog. Hopefully, you’ll find it useful too.May 4, 2019May 4, 2019
Saliya EkanayakeGPipe and Pipeline Parallelism in Neural NetworksIn a previous article, I mentioned how one could incorporate pipeline parallelism to improve the training of a neural network. Well, now…Mar 5, 2019Mar 5, 2019
Saliya EkanayakeModel Parallelism in Deep Learning is NOT What You ThinkI’ve some layers in GPU-0 and others in GPU-1. Is this model parallelism?Nov 10, 20185Nov 10, 20185
Saliya EkanayakePyTorch Distributed with MPITL;DR: mpirun -np 2 python pytorch_distributed.pyAug 24, 20182Aug 24, 20182