Memory Leak — #31
VC Astasia Myers’ perspectives on AI, cloud infrastructure, developer tools, open source, and security. Sign up here.
🚀 Products
Meta Releases Code Generation Model Code Llama 70B, Nearing GPT-3.5 Performance
Code Llama 70B is Meta’s new code generation AI model. Previous versions of Code Llama included models of varying sizes, from 7 up to 34 billion. According to HumanEval, Code Llama 70B scores higher than Code Llama 34B, at 65.2 vs. 51.8; but still lower than GPT-4, which reigns with a score of 85.4. As a further comparison, GPT-3.5 scores at 72.3. Similar results are given by the MBPP benchmark.
Why does this matter? Meta’s metaverse is dead. Long live Meta AI. Over the past year, we’ve seen Meta dramatically shift to focusing on AI. Working on AI is not new for Meta. In 2016 Meta open sourced PyTorch, a framework to enable the implementation and training of Deep Learning models in a simple and efficient way. The company has gone back to its roots. In February 2023, they released Llama, a foundational, 65 billion parameter large language model. Llama 2 came out in July 2023, and Code Llama 70B builds on this work, but focuses on code tasks.
Multi-Database Support in DuckDB
DuckDB can attach MySQL, Postgres, and SQLite databases in addition to databases stored in its own format. This allows data to be read into DuckDB and moved between these systems in a convenient manner.
Why does this matter? In modern data analysis, data must often be combined from a wide variety of different sources. Data might sit in CSV files on your machine, in Parquet files in a data lake, or in an operational database.
Previously DuckDB could read and write data to different files. DuckDB now has a pluggable storage and transactional layer, which allows new storage back-ends to be created by DuckDB extensions. These storage back-ends can support all database operations in the same way that DuckDB supports them, including inserting data and even modifying schemas. DuckDB’s MySQL, Postgres, and SQLite extensions allow DuckDB to connect to those systems and operate on them in the same way that it operates on its own native storage engine.
Oasis Security Leaves Stealth With $40M to Lock down the Wild West of Non-human Identity Management
Oasis tackles non-human identity, which covers not just how two apps may interact together by way of an authentication, but also how two machines or any processes might work in tandem in an organization.
Why does this matter? Identity is the new perimeter. There are about 50x more nonhuman identities than human ones. Nonhuman identities have been persistently hard to discover, resolve, and manage. With AI agents growing in popularity, nonhuman identities will have more presence and power so will be increasingly critical to manage.
📰 Content
Meta to Deploy In-House Custom Chips This Year to Power AI Drive
This year Meta plans to deploy a new version of a custom chip aimed at supporting its AI to its data centers. The chip is the second generation of an in-house silicon line Meta announced last year. A Meta spokesperson said it would work in coordination with the hundreds of thousands of off-the-shelf graphics processing units (GPUs), the go-to chips for AI.
Why does this matter? Nvidia sells GPUs, the dominate computing platform for AI. With the rise of GenAI over the past two years, Nvidia experienced significant need for its products and has struggled to keep up with demand. Social media jokes that there are “GPU Rich” and “GPU Poor” companies. Meta leveraging its own chips can help it reduce dependence on Nvidia. As a testament to the need for AI chips, OpenAI CEO Sam Altman appears to be engaged in discussions with key Middle Eastern investors and the Taiwanese chip giant TSMC to launch a new chip venture to design and build semiconductors for accelerating AI workloads. A major cost and limitation for running AI models is having enough chips to handle the computations behind bots like ChatGPT or DALL-E that answer prompts and generate images.
The Cost Crisis in Observability Tooling
High costs are baked into the observability 1.0 model. Each pillar, metrics, logs, and traces has a price because teams collect and store data for every single use case. Observability bills may grow at a rate +3x faster than traffic growth. Observability 2.0 doesn’t have three pillars. It has a single source of truth. Observability 2.0 tools are built on top of arbitrarily-wide structured log events, also known as spans. From these wide, context-rich structured log events you can derive the other data types (metrics, logs, or traces).
Why does this matter? When speaking with engineering teams we consistently hear complain about the cost of their observability stack. It is hypothesized that Coinbase paid Datadog $65 million in 2021. Often observability tools cost 25–30% of infrastructure spend. We are excited about products that help offset these costs.
Apple Says It’ll Show Its GenAI Efforts ‘Later This Year’
Expect Apple to reveal what it’s been working on in this buzzy slice of artificial intelligence “later this year”, per CEO Tim Cook. There was no more steer on when exactly Apple will pull back the curtain on its AI efforts.
Why does this matter? While Apple’s Vision Pro VR/AR headset demonstrated its efforts to create the metaverse, it hasn’t appeared to put as much emphasis on GenAI, unlike Meta. With deep experience in NLP from Siri, Apple has already familiarized users with the power of AI. It will be interesting to see the release of their upcoming products.
💼 Jobs
⭐️DragonflyDB — React Tech Lead — Dragonfly Cloud
⭐️Chroma — Member of Technical Staff
⭐️Speakeasy — Product Engineer
Views expressed in posts and other content linked on this website or posted to social media and other platforms are my own and are not the views of Felicis Ventures Management Company, LLC. The posts do not constitute investment, legal, tax, or other advice and do not constitute an offer to invest in any security.
