Weekly AI News — July 15th 2024

Microsoft gives up seat on OpenAI board, Figma retracted its AI tool, and new Stable Assistant features

Solarpunk village — Image by DALLE 3

Here are your weekly articles, guides, and news about NLP and AI chosen for you by NLPlanet!

😎 News From The Web

• Microsoft gives up observer seat on OpenAI board. Microsoft has stepped down from its observer seat on OpenAI’s board, reflecting confidence in OpenAI’s trajectory under CEO Sam Altman. This move streamlines Microsoft’s relationship with OpenAI, also possibly addressing antitrust issues. OpenAI will not offer future observer roles, preferring direct partnership interactions, as with Microsoft and Apple.
• Figma pulls AI tool after criticism that it ripped off Apple’s design. Figma retracted its AI tool, Make Designs, after accusations of replicating Apple’s iOS weather app interfaces. The swift rollout was recognized as flawed by CEO Dylan Field, and CDT Kris Rasmussen noted the use of third-party AI models, not internally developed by Figma, indicating possible training concerns with AI models potentially sourced from entities like OpenAI or Amazon.
• OpenAI unveils five-level AI scale, aims to reach level 2 soon. OpenAI’s five-stage AGI progression scale indicates a near-approach to Level 2 “Reasoner,” demonstrating human-like problem-solving. Level 2, characterized by advanced logic and reasoning, is anticipated to be attainable within the next 1.5 years.
• Stability AI Releases Stable Assistant Features. Stability AI has enhanced its Stable Assistant with new capabilities from Stable Diffusion 3, featuring “Search & Replace” for object swapping in images, alongside existing functions for image editing, upscaling, and video generation.
• Licence updates for Stability AI. Stability AI has revised its licensing to the “Stegree AI Community License,” which offers more generous terms for individuals and small businesses, including free usage below certain revenue thresholds and no restrictions for non-commercial and small business use, while also tackling quality issues with the SD3 Medium model.

📚 Guides From The Web

• FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision. FlashAttention-3 enhances Transformer model efficiency by optimizing GPU utilization, particularly for attention mechanisms. Leveraging Hopper GPU’s Tensor Cores and Tensor Memory Accelerator, it achieves up to 75% utilization and 1.2 PFLOPS in FP8, boosting speed by 1.6–2x and enabling more effective training of large language models with reduced memory requirements.
• The jobs for which I have been using AI as a solopreneur. The author examines the application of AI models to streamline coding, UI development, search functionality, documentation, and business communication, highlighting tools like Cursor for coding tasks and Claude for optimizing email interactions.
• The AI summer. The article discusses the discrepancy between the high expectations for AI and its slower actual uptake in business and consumer sectors, emphasizing challenges like protracted enterprise sales cycles, conservative CIO approaches, and misconceptions about AI as a plug-and-play solution. It also touches on the issues related to overinvestment fueled by market hype and competitive dynamics, despite significant user growth for solutions such as ChatGPT.
• Train a Llama model from scratch. The article provides a step-by-step guide to training a Llama language model using the Transformers library, including code snippets for each stage, from library installation and tokenizer setup to model training and uploading the final model to the Hugging Face Hub.
• Agent Dev & The Case for The Engineer’s Creative Process. The article discusses how developing intelligent agents through machine learning has evolved into a more artful practice, urging engineers to embrace creativity and a relational mindset due to the non-linear and unpredictable aspects of the development process.

🔬 Interesting Papers and Repositories

• Distilling System 2 into System 1. This article examines the integration of System 2’s intricate reasoning methods (such as Chain-of- Thought) into the faster System 1 processes in LLMs. By employing self- supervised learning, the authors have improved System 1 performance and lowered computation costs by embedding System 2’s reasoning capabilities into System 1, suggesting a more efficient approach to handling complex reasoning in AI.
• Harnessing Discrete Representations For Continual Reinforcement Learning. The article presents findings that discrete, vector-based categorical representations in reinforcement learning (RL) agents lead to more efficient world modeling and improved policy learning. Empirical evidence from various RL scenarios, including continual learning contexts, indicates that such representations allow for faster adaptation and better performance.
• MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?. MJ-Bench is a new benchmark designed for evaluating multimodal reward models used to provide feedback on text-to-image generation technologies, such as DALLE-3 and Stable Diffusion. It tests models on criteria such as alignment, safety, image quality, and bias. Notably, the benchmark found that closed-source VLMs like GPT-4o excel in providing effective feedback. MJ-Bench relies on a comprehensive preference dataset to fine-tune these feedback mechanisms, with its results being accessible on Huggingface.
• AriGraph: Learning Knowledge Graph World Models with Episodic Memory for LLM Agents. AriGraph is a novel approach that enhances LLM agents by incorporating a structured memory graph, improving their decision-making and planning in environments such as TextWorld. It enables efficient associative retrieval from episodic and semantic memory, proving superior in complex tasks relevant to autonomy in practical domains like cooking, cleaning, and puzzles.
• SylphAI-Inc/LightRAG: The Lightning Library for LLM Applications.. LightRAG is a modular library akin to PyTorch for building LLM applications like chatbots and code generation, featuring a Retriever-Agent-Generator pipeline that’s customizable for various use cases. Its transparent and modifiable codebase is designed to foster trust and ease of adaptation.

Thank you for reading! If you want to learn more about NLP, remember to follow NLPlanet. You can find us on LinkedIn, Twitter, Medium, and our Discord server!