Weekly AI and NLP News — June 17th 2024

OpenAI is doubling revenues, NVIDIA owns 98% of revenue share in data center GPU market, and Mistral $640M funding

Image by DALLE 3

Here are your weekly articles, guides, and news about NLP and AI chosen for you by NLPlanet!

😎 News From The Web

• OpenAI Doubles Annualized Revenue to $3.4 Billion. OpenAI’s projected revenue for 2024 is $3.4 billion, up from $1.6 billion in 2023, with CEO Sam Altman citing $3.2 billion from core AI products/services and $200 million from partnerships such as with Microsoft Azure. The company’s valuation is at $86 billion as it continues to advance in the AI industry.
• Microsoft’s Nadella Is Building an AI Empire. OpenAI Was Just the First Step. Microsoft CEO Satya Nadella is enhancing the company’s AI capabilities by acquiring AI assets worldwide, cultivating proprietary AI technologies, and possibly positioning Microsoft as a competitor to OpenAI. This expansion includes investing in AI startups and recruiting industry experts.
• Nvidia shipped 3.76M data center GPUs in 2023 — dominates business with 98% revenue share. In 2023, Nvidia consolidated its position in the data center GPU market with a 98% share by distributing 3.76 million units and achieved a remarkable 126% revenue increase since 2020, reaching $60.9 billion, even amidst U.S. export restrictions and manufacturing hurdles.
• Paris-based AI startup Mistral AI raises $640M. Mistral AI, a Paris-based AI startup with founders from Meta and DeepMind, secured $640M in a Series B round led by General Catalyst, reaching a $6B valuation, and focuses on creating cutting-edge AI technologies, balancing open-source and proprietary offerings.
• Apple’s WWDC24 highlights. Apple’s 2024 WWDC highlighted the introduction of Apple Intelligence, a new personal intelligence system leveraging generative models and personal context integration across its ecosystem, alongside significant updates to iOS 18, iPadOS 18, macOS Sequoia, watchOS 11, tvOS 18, and visionOS 2.
• Luma Dream Machine. The Luma Dream Machine by Lumalabs is an AI model designed for synthesizing high-quality, realistic videos from text and images, leveraging a transformer-based method optimized for video content.
• Musk wants to ban Apple for cosying up to OpenAI. Elon Musk has expressed intent to prohibit Apple devices in his firms in response to Apple’s announcement of deploying OpenAI’s ChatGPT in their OS, due to security apprehensions.
• Claude’s Character. The article examines “character training”, focusing on imbuing the Claude 3 model with attributes like curiosity and open-mindedness in addition to harm avoidance. It describes a training strategy that seeks to harmonize AI’s interactive capabilities with ethical norms by flexibly aligning AI behavior with specific traits.

📚 Guides From The Web

• Introducing Apple’s On-Device and Server Foundation Models. At the 2024 WWDC, Apple introduced “Apple Intelligence” in iOS 18, iPadOS 18, and macOS Sequoia, with state-of-the-art on-device and server-based AI generative models (~3 billion parameters) focused on enhancing user experience while emphasizing privacy and operational efficiency.
• Apple’s AI Strategy in a Nutshell. Apple showcased its AI strategy at WWDC 2024, focusing on vertical integration through in-house on-device AI models and proprietary data centers powered by Apple silicon. Emphasizing privacy, this strategy aims to enhance market stance and user trust while minimizing reliance on third-party chipmakers.
• Top Important LLMs Papers for the Week from 03/06 to 09/06. This article summarizes the latest research on LLMs from early June 2024, highlighting progress in benchmarking, training, quantization, and alignment, with a focus on uncertainty quantification, speech generation, multi-agent systems, and robust multi-task language understanding.
• Rotary Positional Embedding(RoPE): Motivation and Implementation. The article delves into Rotary Positional Embedding (RoPE) used in transformer models. Unlike traditional absolute sinusoidal embeddings, RoPE leverages vector rotations to improve recognition of long-range dependencies in data.

🔬 Interesting Papers and Repositories

• The Prompt Report: A Systematic Survey of Prompting Techniques. The “Prompt Report” provides a comprehensive analysis of prompting methods in Generative AI, introducing a taxonomy and a unified set of terms with 33 vocabulary entries for prompts. It details 58 techniques for text-based systems and 40 for non-text modalities to standardize understanding in this emerging domain.
• Depth Anything V2. Depth Anything V2 improves monocular depth estimation using synthetic images and a larger teacher model, along with pseudo-labeled real images for better generalization. It offers significantly faster and more accurate results, with model sizes varying between 25M and 1.3B parameters.
• Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling. Samba is a novel language model architecture that merges Mamba’s selective State Space Model with Sliding Window Attention to enable efficient long-sequence compression and precise memory recall. With a sizable 3.8 billion parameter scale, Samba outperforms existing language models in handling unlimited context.
• Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation. LlamaGen is a novel image generation approach that utilizes autoregressive models featuring an efficient tokenizer and class-conditional models for producing text-aligned images with high fidelity.
• When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models. This study presents advancements in autoregressive LLMs through the combination of linear attention mechanisms and speculative decoding, resulting in notable efficiency gains, including reduced perplexity and up to a 2x increase in generation speed.

Thank you for reading! If you want to learn more about NLP, remember to follow NLPlanet. You can find us on LinkedIn, Twitter, Medium, and our Discord server!