PyTorch 1.9 — Towards Distributed Training and Scientific Computing

PyTorch Overview| Explanation | implemented | Distributed Training

So what does the newest release of PyTorch, i.e, 1.9 have to offer? Facebook’s PyTorch team has vastly ramped up its support for accelerated implementations in the domain of distributed training and scientific computing. This release is composed of over 3,400 commits since version 1.8, made by 398 contributors. Let’s go over the new improvements:-

• Improved support towards scientific computing which includes the likes of torch.linalg, torch.special.
• Major improvements to Autograd support over 98% of operators providing the functionality to calculate complex gradients and optimize real-valued loss functions with complex variables.
• TorchElastic has now been added to Pytorch Core for gracefully handle scaling events.
• Pytorch RPC now supports large scale distributed training with GPU support
• New APIs to optimize performance and packaging for model inference deployment
• Support for Distributed training, GPU utilization and SM efficiency in the PyTorch Profiler

Well, there is a lot to cover here so strap in. Before we dive in, please note that some of…