Q&A: Hear from PyTorch Community Leaders Ahead of Ecosystem Day

Author: Team PyTorch

The first PyTorch Ecosystem Day, a virtual event designed for the PyTorch ecosystem and industry communities to showcase their work and discover new opportunities to collaborate, kicks off next week on Wednesday, April 21st Pacific Time. Attendees have the opportunity to engage in discussions on new developments, trends, challenges, and best practices with PyTorch through poster sessions, keynotes, and a unique networking opportunity hosted through Gather.Town.

Ahead of the event, our team sat down with a few of the speakers to learn more about their work on PyTorch, trends they are seeing in the ever-growing community, and more!

This interview has been edited for length and clarity.

Q: Piotr, tell us about your journey to PyTorch and how you use the platform in your work at NVIDIA.

A: My journey to PyTorch started in 2017 when I was switching back and forth between different frameworks and felt I hadn’t found the right one yet. When I used PyTorch for the very first time, I immediately felt that it just “fits” into my way of thinking and the projects I was working on as a research assistant.

To learn more about PyTorch, I signed up in the forum and thought I could learn more about it by reading questions and answers from the experts. Since my background is only partly in Machine Learning (Biomedical Engineering and Information Technology), I understood the initial difficulties new users might face when starting with a new deep learning framework, so I decided to try to help a bit in the forum and at the same time learn more about PyTorch myself. From then on, I also started to contribute to the framework directly, became a moderator in the forum, was invited to the first PyTorch DevCon, and finally joined NVIDIA as a PyTorch developer.

Q: There might be some newer PyTorch contributors or users in the Ecosystem Day audience. Piotr, what advice would you give them as they start using PyTorch more?

A: A good way to learn more about PyTorch and all its features is to get your hands dirty with a project you are passionate about that can be enhanced using Machine Learning / AI. Are you a musician? Great! What about trying to apply style transfer to your songs or working on a smartphone app for automatic musical notation? Start with a basic use case and use the entire PyTorch ecosystem to deploy the model on your smartphone or as a web service. While you are working on the project, you will automatically learn more about the framework and probably encounter some difficulties. In case you get stuck, we are happy to help you out in the forum with any questions, so don’t be shy in asking for advice, as I’ve also recently forgotten to zero out the gradients in the training loop.

Q: PyTorch 1.8 released in early March, and contained more than 3,000 commits since the release prior. Joe, can you tell us the key takeaways from 1.8 that developers should be aware of?

A: Yes, the PyTorch 1.8/1.8.1 releases were not only big in terms of code commits but also brought really powerful new features to the PyTorch ecosystem. I would expect that these releases continue to grow in scope as new features are developed, new platforms are supported, and features get hardened over time. We are really proud of these releases and what the community has come together to deliver is amazing.

Q: Joe, what feature were you personally most excited about from the 1.8 release?

A: There are so many I could talk about! Overall, over 40 major features were reviewed and considered for the release. Some of the major features to checkout are:

1) Torch.profiler is now available and is going to change the way users are able to debug and profile on accelerators like GPUs at scale and have the ability to visualize in TensorBoard and VSCode;

2) Continued additions to support more and more numpy compatible APIs — torch.fft, torch.linalg as PyTorch continues to drive into more and more usage across the sciences; and

3) New tools and APIs for larger and larger scale training — this includes support for pipeline parallelism, gradient compression and other improvements to the RPC framework.

Q: PyTorch adoption continues to rise globally. Ritchie, can you share trends you’re seeing in Asia Pacific with regards to AI and the role of PyTorch?

A: There is a growing impetus to leverage AI for the smart automation of many tasks in a sizable number of firms from start-ups to major corporations in Asia Pacific. This is one of the low-hanging fruits where many firms can find success in implementing AI without a high failure rate. Another major trend is the use of computer vision deep learning algorithms for the majority of image or video data, where the algorithms are mature enough in research for rapid production deployment applied in many use cases.

PyTorch’s advantage is in accelerating the lifecycle of rapid prototyping to production deployment within a single framework. This prevents porting of code between the different phases in the AI product development cycle which adds up to a sizable technical burden that slows down rapid innovation required to thrive in a competitive scene in Asia Pacific. I envision PyTorch maturing their production features so that major corporations in Asia Pacific are comfortable with adopting the framework on an end-to-end basis.

Q: Ritchie, what’s the most interesting use case of AI you’re seeing in the Asian Pacific region right now?

A: Beyond the corporate and start-up use cases I will highlight in my presentation at Ecosystem Day, an interesting development is in the growing use of AI in social enterprises. An example of promising developments is the use of computer vision deep learning algorithms to aid the visually impaired community. It is interesting to see how AI has moved beyond research and private firms to tackle projects being led by social enterprises.

Q: Closing with a question for the group. What are you most looking forward to at Ecosystem Day next week?

A:

• Piotr: Besides all the great presentations and posters presented at the Ecosystem Day, I’m mostly looking forward to meeting many community members, which might have already interacted with me in the virtual world. It’s always enjoyable to have a chat and to “see” the people behind the posts even if it’s also virtual this time.
• Joe: For me, it is catching up with everyone I haven’t seen live in so long. This is such a diverse, talented and enthusiastic community of AI leaders focused on a broad range of areas. So in some ways it is like getting a heart beat for the world of AI. During the PyTorch Developer Day last fall, I (virtually) ran into so many people I hadn’t talked to in such a long time and Gather.Town is just such a fun way to serendipitously bump into people and see what they are up to.
• Ritchie: I am looking forward to seeing everyone’s posters, which will showcase the range of applied deep learning projects. The gradual move to production use cases in PyTorch is where we can finally see PyTorch maturing into a reliable, end-to-end research to production framework, which is very exciting.

Interested in attending PyTorch Ecosystem Day? Visit pytorchecosystemday.fbreg.com to register. We look forward to seeing you there!

Speaker bios

Piotr Bialecki, Technical Lead of The PyTorch Team at NVIDIA:
Piotr Bialecki is the Technical Lead of The PyTorch Team @ NVIDIA, where he supports internal as well as external users in using PyTorch on NVIDIA GPUs. Before that he was applying PyTorch to research projects until he decided to work on the framework directly. When Piotr isn’t checking the PyTorch discussion board, you can find him in a rehearsal room playing drums.

Joe Spisak, Product Lead at Facebook AI
Joe is the product lead for PyTorch. His work spans collaborations with internal Facebook teams as well as working with the AI developer community to bring scalable tools to help push the state of the art forward.

Ritchie Ng, CEO & Executive Director at Hessian Matrix
Ritchie leads applied AI research and live systematic trading with monthly multi-billion dollar notional size at Hessian Matrix, an AI systematic global hedge fund based in Singapore. He is also an NVIDIA Deep Learning Institute instructor leading all deep learning workshops in NUS, Singapore and conducting workshops across Southeast Asia.