Deep Learning with PyTorch Lightning
Deep Learning is what humanizes machines. Deep Learning makes it possible for machines to see (through vision models), to listen (through voice devices like Alexa), to talk (through chat-bots), to write (through generative models like auto-complete or Q&A) and even be an artist by trying to paint (through GAN or Style Transfer models).
PyTorch Lightning lets researchers build their own Deep Learning models quickly & easily without having to worry about the complexities. This book will help you maximize productivity for Deep Learning projects while ensuring full flexibility from model formulation to implementation.
PyTorch has been trending upwards since 2019 and has been slowly replacing TensorFlow. In trend in the research community has been as clear as daylight about which is the most preferred platform with majority of papers published in PyTorch.
The book provides a hands-on approach for implementing PyTorch Lightning DL models and associated methodologies that will have you up and running and productive in no time. You’ll learn how to configure PyTorch Lightning on a cloud platform, understand the architectural components, and explore how they are configured to build various industry solutions. Next, you’ll build a neural network architecture and deploy an application from scratch and see how you can expand it based on your specific needs, beyond what the framework can provide. The book also demonstrates how to implement capabilities to build and train various models like Convolutional Neural Nets (CNN), Natural Language Processing (NLP), Time Series, Self-Supervised Learning, Semi-Supervised Learning, Generative Adversarial Network (GAN) using PyTorch Lightning.
Who this book is for
If you have always been curious about DL but don’t know where to start or feel intimidated by the complexities of large neural networks, then this book is ideal for you! It will make DL feel like a cakewalk!
This Deep Learning with PyTorch Lightning book is primarily for citizen data scientists who are beginning their DL journey and need a practical guide to ace their game! It is also beneficial to expert data scientists making the transition from other frameworks to PyTorch Lightning. This book will also appeal to DL researchers who are just getting started with coding for DL models using PyTorch Lightning by serving as a ready reckoner.
What this book covers
Chapter 1, PyTorch Lightning Adventure, will start with a brief history of Deep Learning and why PyTorch is the most preferred framework of the community today. We will also see what PyTorch Lightning is, how it is built, and how it differs from PyTorch. We will cover the module structure of PyTorch Lightning and how it makes research more feasible with users putting less effort into engineering and more into modelling.
Chapter 2, Getting off the Ground with the First Deep Learning Model, focuses on how to get started with building models using PyTorch Lightning. As examples, we will build multiple models, ranging from a very simple Multilayer Perceptron (MLP) to a real-life image recognition model using CNN. It shows how to build a image recognition model to identify cancer tissue cells from the normal ones.
A 3 layer CNN network will help us in image classification.
Chapter 3, Transfer Learning Using Pre-Trained Models, mainly focuses on how to customize the models built using pre-trained architecture to achieve great results without large training budgets or time for different datasets. We will walk you through the steps of customizing models using pre-trained models for images and NLP.
We will use ResNet-50 architecture to show how we can get better accuracy on previous cancer detection dataset.
We will then further see how we can do Text Classification using Tranformers to help fact-checking of public health claims.
Chapter 4, Ready-to-Cook Models from Flash, mainly focuses on PyTorch Lightning Flash — a state-of-the-art model architecture library. It includes most of the common algorithms or frameworks out of the box, thereby improving the productivity of data scientists by a huge margin for benchmarking & experimentation. This chapter will share various Flash models for video (video classification) and audio (automatic speech recognition) tasks.
We will see Slow and Slow-Fast architecture for video classification.
for Speech Recognition by Steffen Schnei
Followed by wav2vec speech architecture for Automatic Speech Recognition to recognize Scottish language.
Chapter 5, Time Series Models, mainly focuses on the working of time series models, with PyTorch implementation, along with step-by-step working and in-detail examples from basic to advanced time series techniques, such as (Recurrent Neural Networks) RNN and Long Short Term Memory (LSTM) models, along with real-world use cases.
Chapter 6, Deep Generative Models, focuses on the step-by-step, in-detail working and implementation of generative types of DL models, such as Generative Adversarial Networks (GANs), where it will be used to generate non-existent images.
We will see how to generate fake food images and fake butterfly images using the various GAN architectures.
Chapter 7, Semi-Supervised Learning, mainly focuses on how semi-supervised models work, and how they can be implemented with PyTorch Lightning. We will also cover, in detail, working examples and implementation from basic to advanced semi-supervised models using PyTorch Lightning to handle label propagation and image caption generation using a combination of CNNs and RNNs.
We will see how to automatically generate captions for images by using a combination of CNN and RNN architectures.
Chapter 8, Self-Supervised Learning, mainly focuses on how self-supervised models can be implemented with PyTorch Lightning. We will also cover working examples of contrastive learning and some techniques like SimCLR architecture.
We will learn the image representation for unlabeled dataset in self-supervised manner and then apply the learned representation to contrast any given image from dis-similar images.
Chapter 9, Deploying and Scoring Models, will cover, in detail, techniques and ways to deploy a DL model natively as well as in inter-operable formats such as ONNX. It will also cover, in detail, techniques and ways to perform model scoring on massive volumes of data.
Chapter 10, Scaling and Managing Training, will take a nuanced view of the challenges of training a model at scale and managing the training. It will describe some of the common pitfalls and tips and tricks on how to avoid them. It will also describe how to set up your experiments, how to make model training resilient and how to use the hardware to improve training efficiency, among other things.
