Published in


Google Researchers Explore the Limits of Large-Scale Model Pretraining

Studies have shown that scaling up powerful pretrained models and their training data sizes significantly improves performance, and that these performance improvements can transfer to downstream tasks, even in few-shot settings. But is there a limit to the performance improvements attainable via such model size and training data scale-ups?




We produce professional, authoritative, and thought-provoking content relating to artificial intelligence, machine intelligence, emerging technologies and industrial insights.

Recommended from Medium

Baidu at CES: Developing AI at “China Speed”

PwC-Powered Code Tab Added to arXiv ML Papers

Learning to Think Like a Smart Machine

10. Newser: AI spots things that cows may be hiding

AlphaGo: The Ultimate Go Master

Vancouver Named NeurIPS 2019 & 2020 Host as Visa Issues Continue to Plague the AI Conference

The “Uncanny Valley” Effect and Interaction Fidelity

Augmented Training Scheme Fixes CNN Texture Bias

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store


AI Technology & Industry Review — | Newsletter: | Share My Research | Twitter: @Synced_Global

More from Medium

Why Is Cross Entropy Equal to KL-Divergence?

UC Berkeley, Waymo & Google’s Block-NeRF Neural Scene Representation Method Renders an Entire San…

Meta AI Published Data2Vec. A New Milestone for Learning Across Modalities

What is Deepmind’s retrieval-based transformer (RETRO) & how does it work?