Meta Releases LLama 3.1, the Biggest Open Source AI Model Yet
Earlier today, Meta announced that open source is leading the way. Meta unveiled Llama 3.1, stating it’s their most capable collection of models yet.
Meta’s release today includes the highly anticipated 405B. These models deliver enhanced reasoning capabilities, an upgraded 128K token context window, and improved support for eight languages, among other advancements. Llama 3.1 405B rivals leading closed-source models with next-gen capabilities across a range of tasks, including general knowledge, math, steerability, tool usage, and multilingual translation. The models are available to download now directly from Meta or Hugging Face.
The ecosystem is also set to go with over 25 partners rolling out their latest models with today’s release — including AWS, NVIDIA, Databricks, Groq, Dell, Azure, and Google Cloud. The model was trained on over 15 trillion tokens over several months, and required more than 16K NVIDIA H100 GPUs, marking it the first Llama model ever to be trained at this large of a scale.
Meta also used the 405B parameter model to improve the post-training quality of their smaller-sized models. With Llama 3.1, Meta evaluated performance on over 150 benchmark datasets across a wide range of languages, in addition to ample human evaluations in realistic scenarios. Their results show that the 405B competes with leading closed-source models like GPT-4, Claude 2, and Gemini Ultra across a wide range of tasks. Meta’s upgraded Llama 3.1 8B & 70B models are best-in-class, and of similar size while offering an improved balance of helpfulness and safety compared to their peers. Their smaller models support the same 128K token context window, enhanced reasoning, multilinguality, and state-of-the-art tool use to enable advanced use cases. Additionally, Meta updated their license to enable developers to use the outputs from Llama models including 405b to improve other models.
Meta is looking forward to how this will accelerate new advancements in the field through synthetic data generation & model distillation workflows, which are capabilities that have not yet been achieved at this large of scale in open source technologies. Mark Zuckerberg shared in an open letter this morning: “We believe that open source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn’t concentrated in the hands of a small few, and that the technology can be deployed more evenly and safely across society.”
With this groundbreaking announcement, Meta continues to forge ahead the journey for open-source AI to become the industry standard and push the space forwards.
Read the full article and more on our website https://machinelearningartificialintelligence.com!
