Weekly AI and NLP News — June 10th 2024

Apple may be partnering with OpenAI, NVIDIA more valuable than Apple, and OpenAI reboots its robotics team

Image by DALLE 3

Here are your weekly articles, guides, and news about NLP and AI chosen for you by NLPlanet!

😎 News From The Web

• Apple’s Reported ChatGPT Deal Could Crown OpenAI as King of the Valley. Apple is anticipated to partner with OpenAI to incorporate ChatGPT into the iPhone’s operating system, which could be announced at the next WWDC. This integration, which could revolutionize AI interaction on iPhones, might see ChatGPT enhance Siri or launch as a separate application, signaling Apple’s pivot towards external AI expertise.
• Nvidia is now more valuable than Apple at $3.01 trillion. Nvidia has achieved a market capitalization of $3.01 trillion, propelled by the artificial intelligence surge, overtaking Apple to become the world’s second most valuable company.
• Apple Keeps It Simple, Will Call Its AI ‘Apple Intelligence’. Apple is set to unveil “Apple Intelligence,” an AI solution with chatbot capabilities akin to ChatGPT, at WWDC on June 10. This will be included in upcoming iOS, iPadOS, and macOS updates and is designed for offline operation, marking a partnership with OpenAI and improvements to Siri.
• AMD unveils new AI chips to compete with Nvidia. AMD is challenging Nvidia’s leadership in AI with upcoming releases: the MI325X in 2024, and the MI350/MI400 series in 2025–2026, promising notable performance boosts to satisfy increasing AI demands.
• OpenAI Is Rebooting Its Robotics Team. OpenAI is reinstating its robotics division, focusing on creating AI models for robotic applications in collaboration with external robotics companies. This marks a strategic pivot from producing in-house hardware to empowering humanoid robots through partnerships, as evidenced by investments in entities like Figure AI. The team expansion is underway through active recruitment.
• Nvidia and Salesforce may double down on AI startup Cohere in $450 million round. Generative AI startup Cohere has secured a $450 million funding round led by Nvidia and Salesforce, alongside new backers such as Cisco and PSP Investments, boosting its valuation to $5 billion from its prior $2.2 billion mark. The company also disclosed an annualized revenue of $35 million.
• Stability AI releases a sound generator. Stability AI has launched “Stable Audio Open,” an AI model that generates sound from text descriptions using royalty-free samples, geared towards non-commercial use.

📚 Guides From The Web

• Extracting Concepts from GPT-4. Researchers have employed sparse autoencoders to break down GPT-4’s neural network into 16 million human-interpretable features, allowing for enhanced comprehension of AI processes. However, fully deciphering these features continues to pose a challenge, restricting the effectiveness of existing autoencoders.
• Uncensor any LLM with abliteration.
• KL is All You Need. The author highlights the importance of Kullback-Leibler divergence as a fundamental objective in machine learning, crucial for measuring differences between probability distributions and optimizing models across diverse methods in the field.
• AI-Powered Tools Transforming Task Management and Scheduling. The article highlights AI advancements in productivity platforms such as Motion, Reclaim AI, Clockwise, ClickUp, Taskade, and Asana, detailing their use of machine learning to improve task management, scheduling, and overall workflow optimization.
• What We Learned from a Year of Building with LLMs (Part II). The article discusses the complexities of developing applications with LLMs, highlighting the necessity for high-quality data, careful management of model outputs, and strategies for effectively integrating and maintaining LLM versions. It underscores the critical roles of early designer engagement, assembling a skilled team, and cultivating an innovative work environment to navigate the unique operational challenges in LLM-based product development.

🔬 Interesting Papers and Repositories

• Seed-TTS: A Family of High-Quality Versatile Speech Generation Models. Seed-TTS encompasses advanced autoregressive and non-autoregressive text-to-speech models capable of generating human-like speech with emotional variability, speaker similarity, and naturalness, also showcasing proficiency in end-to-end speech generation and editing through a diffusion-based architecture.
• Hello Qwen2. The Qwen2 series is an advancement over the Qwen1.5, introducing five enhanced AI models with new features such as support for 27 additional languages and improved coding and mathematics functions. The standout Qwen2–72B offers superior safety and can comprehend lengthy contexts of up to 128K tokens. These models are available on Hugging Face and ModelScope.
• Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality. This article presents an analysis of the structured relationship between Transformers and state-space models (SSMs) using matrix analysis, introducing a theoretical framework that connects the two. It also introduces an improved architecture, Mamba-2, building on its predecessor Mamba by being significantly faster (2–8 times) and maintaining comparable performance in language modeling tasks.
• LLM Merging Competition: Building LLMs Efficiently through Merging. The article introduces a competition that challenges participants to integrate multiple fine-tuned LLMs to improve their performance and adaptability to novel tasks. Competitors will utilize pre-trained expert models with up to 8 billion parameters from the Hugging Face Model Hub, which are available under research-friendly licenses. The goal of the competition is to minimize the costs and challenges of training LLMs from the ground up by utilizing existing models.
• Diffusion On Syntax Trees For Program Synthesis. The paper presents an approach to program synthesis using neural diffusion models that iteratively refine code through edits on syntax trees, ensuring syntactic correctness and addressing the limitations of token-based code generation without output feedback in existing large language models.

Thank you for reading! If you want to learn more about NLP, remember to follow NLPlanet. You can find us on LinkedIn, Twitter, Medium, and our Discord server!