ML Resources — August 25

Charles Frye has published this excellent piece on the PyTorch trace viewer, that you can use to explore PyTorch under the hood and better understand how to optimize PyTorch code.

Stanford’s CRFM has released 10 GPT-2 models initialized with different seeds, as well as a codebase for facilitating large-scale language model training.

Wu et. al. introduce ‘SmartBird’, a transformer variant that addresses the quadratic complexity in self-attention by learning important token pairs to attend to using a sparse attention mechanism.

Wu et. al also published ‘Fastformer’, an efficient transformer variant that uses additive attention to summarize the query matrix into a global query vector.

skweak is a great NLP library that is used for weak supervision. This library is built on top of spacy and allows you to specify labeling functions and heuristics for auto-labeling your data.

Aggregate Intellect

Aggregate Intellect is a Global Marketplace where ML Developers Connect, Collaborate, and Build. Connect with peers & experts at https://ai.science or Join our Slack Community.

• Check out the user generated Recipes that provide step by step, and bite sized guides on how to do various tasks
• Join our ML Product Challenges to build AI-based products for a chance to win cash prizes
• Connect with peers & experts through the ML Discussion Groups or Expert Office Hours