SyncedReview
Published in

SyncedReview

Google Trains a 540B Parameter Language Model With Pathways, Achieving ‘Breakthrough Performance’

In March, Google introduced Pathways, a large-scale orchestration layer for accelerators that employs a novel asynchronous distributed dataflow design to enable highly efficient training across multiple TPU Pods. Now, scarcely two weeks later, a Google Research team has put Pathways to the test, using it to train Pathways…

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store