Memory Leak — #36
VC Astasia Myers’ perspectives on AI, cloud infrastructure, developer tools, open source, and security. Sign up here.
🚀 Products
Foundry offers GPU instances, at scale, with competitive price-performance metrics. As a result of their novel structural and technical approach, they are often able to provide computing power at an order of magnitude lower costs than our users could access otherwise.
Why does this matter? GPUs are the bedrock of AI. Foundry focuses on the core economic and technical challenges in AI including access and utilization. One area that Foundry stands out is its ability to map workloads to devices that deliver maximum ROI per unit of compute spend. This functionality is part of the core magic of their product.
Meet DBOS: A Database Alternative to Kubernetes
DBOS runs operating system services on top of a high-performance distributed database. All state, logs, and other system data are stored in SQL-accessible tables. The first commercial service built around this architecture is DBOS Cloud, a transactional Functions as a Service (FaaS) platform, available for developers in this initial launch. The project was founded by Dr. Stonebraker, along with Apache Spark creator (and Databricks co-founder and CTO, Matei Zaharia), and a joint team of MIT and Stanford computer scientists.
Why does this matter? All state and scheduling information is often tracked in a PostgreSQL database that can have poor performance. The central idea behind the DBOS project comes from a simple idea: Keeping track of the operating system state should be a database problem. The creators found that building an OS on top of a database would offer the ability to roll back to a state before a vulnerability has been exploited and help with debugging.
GritQL is a declarative query language for searching and modifying source code. It’s designed to make simple searches easy, but not limit the complexity of transformations.
Why does this matter? Grit helps with code migrations and dependency upgrades. From this experience they learned that any complex migration ends up being a full code mode program. Codemod frameworks are language-specific, so if you’re hopping between multiple languages — or trying to migrate a shared API — you have to learn different frameworks. GritQL helps teams work across languages with one tool.
📰 Content
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
Quiet-STaR is a generalization of STaR in which LMs learn to generate rationales at each token to explain future text, improving their predictions. It addresses key challenges including 1) the computational cost of generating continuations, 2) the fact that the LM does not initially know how to generate or use internal thoughts, and 3) the need to predict beyond individual next tokens. To resolve these, they propose a tokenwise parallel sampling algorithm, using learnable tokens indicating a thought’s start and end, and an extended teacher-forcing technique. Encouragingly, generated rationales disproportionately help model difficult-to-predict tokens and improve the LM’s ability to directly answer difficult questions.
Why does this matter? AI researchers are mirroring the behaviors of humans in models. Quiet-STaR enables AI to have more reasoning. AI builds intuition and learns to infer unstated rationales in arbitrary text. This is the first work explicitly training LMs to reason generally from text, rather than on curated reasoning tasks or collections of reasoning tasks.
How People Are Really Using GenAI
Looking through thousands of comments on sites such as Reddit and Quora, the author’s team found that the use of this technology is as wide-ranging as the problems we encounter in our lives. The 100 categories they identified can be divided into six top-level themes, which give an immediate sense of what generative AI is being used for: Technical Assistance & Troubleshooting (23%), Content Creation & Editing (22%), Personal & Professional Support (17%), Learning & Education (15%), Creativity & Recreation (13%), Research, Analysis & Decision Making (10%).
Why does this matter? Even amongst the world’s billion knowledge workers, just 10% use ChatGPT (which enjoys 60% market share) regularly. It is interesting to see that technical assistance e.g. GitHub Copilot and content creation e.g. RunwayML are the top two categories. The first use case benefits from deterministic outputs, while the second benefits from non-deterministic outputs. We expect research, analysis, and decision making to dramatically increase when AI agents move to production.
You Should Be Playing with GPTs at Work
Lenny Rachitsky crowdsourced 20 examples of how people are using GPTs at work.
Why does this matter? Most of the examples reflected opportunities where non-deterministic outputs are helpful. These are the use cases where GenAI shines. For example, “dream up copy experiment ideas” and “refine new product copy.” The use cases also centered around text versus visual content, a reflection of which has the most opportunity today.
💼 Jobs
⭐️DragonflyDB — React Tech Lead — Dragonfly Cloud
⭐️Chroma — Member of Technical Staff
⭐️Speakeasy — Product Engineer
Views expressed in posts and other content linked on this website or posted to social media and other platforms are my own and are not the views of Felicis Ventures Management Company, LLC. The posts do not constitute investment, legal, tax, or other advice and do not constitute an offer to invest in any security.
