Scaling Law

Studies on various machine learning models have shown that applying neural scaling laws — increasing compute, model size, and pretraining data — can reduce errors and significantly improve model performance. …

State-of-the-art AI models have ballooned to billions of parameters in recent years. Although the machine learning (ML) community has shown keen interest in the scaling properties of transformer-based models, there has been relatively little research on scaling effects with regard to the inductive biases imposed by different model architectures. In…

Why Decentralized Platforms Will Eat Centralized Platforms — Blockchain technology has the potential to truly revolutionize network computing and associated markets such as finance, supply chain, social etc. One of its main attractions has been its promise of bringing more virtuous and trustworthy governance as a result of more decentralized control of the underlying systems. Decentralized control has…