The Journey to Radically Improving Global Computation
This article is for ambitious AI researchers interested in the application of deep reinforcement learning (DRL).
As a founder, one of the most enjoyable exercises is mapping out the broad-scale effects our success could have on the world. It’s particularly exciting at Astrus because the ripple effects of significantly improving global computation are immense. My co-founder and I often imagine a world with 10 or 100x computation — a goal within Astrus’s reach.
At Astrus, we are leveraging AI to enable superhuman-level chip design — starting with a design agent that can create superhuman-level analog layouts. Analog layout is a manual and time consuming step in the chip design process. It stands as a major bottleneck, limiting chip performance and delaying time to market. Remarkably, analog design can represent up to half of the design costs for most cutting-edge microchips, amounting to millions and, in some cases, hundreds of millions of dollars per chip.
The path to realizing Astrus’s mission largely hinges on creating a top-tier, dedicated AI research team. Beginning with the launch of our latest role, the Founding Research Scientist. In this article I explain why my cofounder, Zeyi, regards this as “The Dream DRL Role.”
In the earliest days of Astrus, Zeyi and I would spend countless hours on the whiteboard, diving into how DRL could be applied to solve the analog layout problem. Our brainstorming sessions were intense: whiteboards covered in diagrams of transistors, neural networks, search trees, physics equations, and the principles behind AlphaGo’s success; those moments were nothing short of magical.
I am extraordinarily grateful to work alongside Zeyi, an expert in the AlphaGo line of research. His graduate studies at the University of Alberta placed him at the heart of AI innovation. The very lab responsible for the research foundations for DeepMind’s AlphaGo. In fact, both DeepMind’s lead scientists, David Silver and Aja Huang, completed their grad studies in this same research lab.
Diving into this technology with Zeyi has been nothing short of profound for me. It has deepened my appreciation for the nuances of intelligence, human cognition, and the potential of DRL to surpass human capabilities. I understand why Vinod Khosla, our lead investor, described AlphaGo’s victory over Lee Sedol as a religious experience.
Yet, for all its promise, widespread adoption and application of DRL remains surprisingly limited. Its influence still primarily within academic circles.
This begs the question: Why isn’t DRL seeing more industry application?
Well, building a DRL environment is extremely challenging.
The strength of a DRL system lies in the integrity of its environment; flaws in this foundation can propagate issues. Robotics, particularly the “sim-to-real problem”, underscores the difficulty of constructing accurate environments. Environment creation is fundamentally an engineering endeavor, necessitating the application of practical, real-world models.
In the academic sphere, researchers often gravitate towards simple environments, like board games (Go, chess, Poker, etc.) for their studies. Even within these seemingly straightforward settings, crafting an RL-compatible environment poses its challenges. By adapting Atari games for RL studies, Michael Bowling’s research group has played a crucial role in enhancing AI research capabilities globally. His efforts have enabled researchers to concentrate on advancing the field, unencumbered by the engineering intricacies that typically accompany application-focused projects.
This is why at Astrus, we invest heavily in creating an environment to model analog layout.
Our efforts have been geared towards abstracting away domain expertise by focusing on the axiomatic and measurable objectives of analog layout. By doing so, we’ve reimagined the analog layout challenge as a simplified game. This simplification enables both our AI design agent and researchers to zero in on the quintessential dilemma of analog layout: amidst a bewildering array of potential placements, determining the optimal location for a transistor.
Designing an accurate environment is made possible because of the chip design industry’s need to perform extensive software validation before manufacturing. With the high costs associated with chip manufacturing, ensuring a design’s functionality before it hits manufacturing is critical. Generally speaking, if your chip passes the software validation, it will work once manufactured. This reliance on models that are deeply grounded in physics and manufacturing principles, provides a solid foundation for Astrus’s environment.
As the Founding Research Scientist, you’ll be stepping into a role that benefits from our solid infrastructure, ready to make strides from the outset. You will be supported by our world-class team of founding engineers, focused on creating a performant environment and scalable cloud infrastructure.
Yet, our sights are set far beyond these initial achievements.
What truly excites us at Astrus is the potential to extend this success into a pioneering role in the AI landscape. We envision Astrus as one of the most impactful AGIs in the future network of AGIs.
We are building towards Computational AGI — a master designer and architect of computational systems.
By leveraging its deep grasp of generalized design principles, the system facilitates designs that surpass human reach. Extending from precise transistor adjustments to architecting new computational paradigms.
It may sound like a tall order, but here is why Astrus is uniquely positioned to achieve this goal.
Multidisciplinary Approach
Astrus naturally sits within the intersection of software, AI, and microchip design. As you conduct research to solve the analog layout problem, you will also learn about semiconductor design. Gaining a holistic understanding of computation, from defining a neural net in Python, all the way to the structure of the transistor. This is a powerful combination of skills, allowing you to understand algorithms from a holistic angle. Our team will have a deep understanding of the hardware, cloud scaling, and the AI algorithms.
Leveraging Computation with Learning and Search
Learning from Professor Sutton’s The Bitter Lesson, there are two categories of algorithms that excel at leveraging such computing power, namely learning and search. We use a combination of algorithms that adhere to such principles, such as deep reinforcement learning, monte-carlo tree search, and real-time path planning. This enables our system performance to scale with the availability of computing power.
Learning from Scratch, Rooted in Physics
We believe in the potential of algorithms learning tabula rasa based on first principles. Central to our strategy is the development of a deep reinforcement learning algorithm that learns how to design microchips. The main learning signals are based on fundamental properties of microchips, which are governed by the laws of physics. Our system will create designs liberated from the constraints of traditional human capabilities, relying on neither human generated data nor human guided practices.
Serious Compute Infrastructure
Not to beat a dead transistor, but did I mention Astrus cares about computation? We believe that the science and engineering of AI are inseparable. To support our learning system, we will need to build the most scalable and performant compute infrastructure. Already, we have hired incredible founding members with strong engineering and cloud infrastructure backgrounds. In a few years, Astrus will be performing the largest scale of deep reinforcement learning training in the world to improve the exact hardware that enables such training.
Deep Pockets
Successfully automating analog layout is not just a technological milestone, but a highly lucrative and profitable venture. We have also been fortunate enough to partner with incredible funds like Khosla Ventures, Drive Capital, 1517 Fund, and many more — all eager to finance and support our ambitious mission. This allows us to channel our future profits and venture funding into further research and development.
At Astrus, we’re not just dreaming of a future where AI transforms computation; we’re actively building it. And the key to this future? A Founding Research Scientist who’s ready to explore the depths of DRL and use it to unlock unprecedented computational power. From automating analog layout to spearheading our drive towards Computational AGI, the potential for impact is vast. It’s an unparalleled opportunity to lead groundbreaking research to radically improve global computation.
— The DRL dream role —
Are you ready?
