Bing AI blogs about AI

Bing AI has a new feature that composes blog posts. To test it out, I asked it to write about the upcoming release of PyTorch, the open source Python library originally built by Meta/Facebook that is used to program most AI applications. The newest version of PyTorch, 2.0, was announced in December 2022 is coming out spring/summer 2023. The developers are promising that it will be up to twice as fast and easier to use. To read about it from a human (and more accurate) perspective, here’s a Medium post: https://medium.com/@MaziBoustani/pytorch-2-0-release-explained-b5f167b86819 And here’s the overview: https://pytorch.org/get-started/pytorch-2.0/#overview

PyTorch 2.0 is still very much in the development stage. Now is great time to learn and experiment with PyTorch 2.0, as it will probably the standard in 1–2 years, but it’s still too early to use for production. For instance, someone successfully added malware to the code between Christmas and New Years 2022. https://pytorch.org/blog/compromised-nightly-dependency/

I chose this subject for my Bing AI test because there’s not going to be any knowledge in the training data, other than information about PyTorch. Thus, Bing AI is relying on its search engine to compile the information in the post. The result is a combination of correct information about changes from PyTorch 1 to 2, things that are true about both PyTorch 1 and 2, and advice that is the exact opposite of correct: recommending PyTorch 1 approaches that will be replaced (or “deprecated” in programming parlance) for PyTorch 2.

Although I had to factcheck this article extensively, Bing ultimately did save me time, because it provided me with a structure for the article. PyTorch 2.0 will have so many changes that it’s difficult to wrap your head around all of them, let alone finding a coherent story about the changes. With a structure in hand, the remaining task (factchecking) is rather straight forward. This particular experience is actually the opposite of the conventional wisdom: AI has done the “creative” portion of structuring the article, and I have done the non-creative portion: factchecking. The remainder of this post is from Bing AI, with my commentary in bold italics.

The most exciting portions of PyTorch 2.0

PyTorch is one of the most popular frameworks for deep learning, offering a flexible and intuitive way to build, train and deploy neural networks. PyTorch 2.0 is the latest major release of PyTorch, which brings many new features and improvements to the core library as well as its ecosystem.

In this blog post, we will highlight some of the most exciting portions of PyTorch 2.0 that make it faster, more powerful and more user-friendly than ever before. So far, so good, if a little boilerplate

Compiler-level optimizations

One of the biggest changes in PyTorch 2.0 is the introduction of a new compiler infrastructure that enables faster performance and support for dynamic shapes and distributed training. The compiler leverages TorchScript, a subset of Python that can be statically analyzed and optimized by PyTorch. It’s actually the opposite: TorchScript is being replaced by the superior TorchDynamo

The compiler works by transforming TorchScript code into an intermediate representation (IR) that can be further optimized by applying various passes such as constant folding, dead code elimination, loop fusion and vectorization. The optimized IR can then be lowered to different backends such as CPU, GPU or TPU. Mostly right, as TorchDynamo will improve the TorchScript’s processes. But Tensor Processing Units (TPUs) won’t be supported right away. I asked the developers about support for TPUs and Apple M1/M2 chips, and they said that’s on the backlog.

The compiler also supports dynamic shapes, which means that it can handle tensors whose dimensions are not known at compile time. This allows for more flexibility and expressiveness in defining neural network architectures that can adapt to different input sizes or batch sizes at runtime. Accurate: this is a major area of improvement in PyTorch 2.0

Another benefit of the compiler is that it enables distributed training across multiple devices or nodes. The compiler can automatically partition the computation graph into subgraphs that can be executed in parallel on different devices or nodes. This reduces communication overhead and improves scalability for large-scale models. Also accurate

Eager-mode development

PyTorch 2.0 maintains the same eager-mode development experience that PyTorch users love. Eager mode allows users to write Python code that executes immediately without any compilation or graph construction steps. This makes debugging easier and enables interactive experimentation with different models and hyperparameters. Yes!

Eager mode also supports autograd, which automatically computes gradients for any tensor operation without requiring users to manually define backward functions. Autograd makes it easy to implement custom layers, loss functions and optimization algorithms using standard Python control flow constructs such as loops, conditionals and exceptions. This is accurate, and the area I’m most excited about.

Eager mode also integrates seamlessly with TorchScript, allowing users to annotate parts of their code with `@torch.jit.script` or `torch.jit.trace` decorators to enable compilation for performance or deployment purposes. Users can also mix eager mode code with TorchScript code within the same program using `torch.jit.interface` or `torch.jit.ignore` annotations. Exact opposite: torch.compile is replacing torch.jit.script

Ecosystem enhancements

PyTorch 2.0 also brings many enhancements to its ecosystem of libraries and tools that extend its functionality beyond core deep learning tasks. Some of these enhancements include: All of the below examples exist now, and PyTorch 2.0 will not specifically improve any of them, other than Profiler

• PyTorch Lightning: A high-level framework that simplifies writing complex PyTorch code by providing abstractions for common components such as data loading, model definition, training loop, logging and checkpointing.
• PyTorch Profiler: A tool that helps users diagnose performance bottlenecks in their PyTorch code by providing detailed information about CPU/GPU/TPU utilization, memory consumption, operator execution time and tensor shape inference. Again, no TPU support
• PyTorch Hub: A repository of pre-trained models from various domains such as computer vision, natural language processing, audio processing and generative modeling that can be easily loaded and used with a few lines of code.
• PyTorch Mobile: A library that enables deploying PyTorch models on mobile devices such as Android and iOS using native APIs such as Java/Kotlin/Swift/Objective-C/C++/C#[6]. Bing hallucinates a reference
• PyTorch Serve: A framework that simplifies serving PyTorch models in production environments using RESTful APIs or gRPC protocols with features such as model versioning, load balancing, health monitoring and logging[7]. Bing added reference numbers but didn’t provide endnotes

Conclusion

PyTorch 2.0 is a major milestone for the PyTorch community that delivers significant improvements in performance, flexibility and usability for deep learning practitioners. Whether you are new to PyTorch or an experienced user, you will find something exciting and useful Bing just ends mid-sentence, which is probably for the best, given that it’s using a lot of words to say not much. I had to extensively factcheck this article, but the silver lining was that I learned a lot about what PyTorch 2.0 does and doesn’t do. I’ll cover that in a future — more accurate and more engaging — post.