AI Top-of-Mind for 9.12.24 — AMD v Nvidia
Today: AMD v Nvidia, big data renewable energy initiatives, Reflection 70B, designing neural networks, and predicting the future
Top-of-mind is hardware, and is Nvidia’s Cuda moat sustainable? Although there are many startups developing GPUs and the like, only AMD is in striking distance albeit with 1/10 the revenue. ‘The Information’reports on Oracle’s viewpoint:
· While AMD’s latest flagship server chips, launched in 2023, may not be as powerful as the latest Nvidia flagship — the H100, launched in 2022 — executives at Oracle and Microsoft feel they have little choice but to support AMD to promote competition.
· As Nvidia runs away with the server chip market, competition is needed to drive down hardware costs, said Batta. Doing so will keep AI inference costs under control for businesses that want to use LLMs to make their own AI apps or want to buy such apps from software vendors, he said. The cost to rent Nvidia chips for inference has been dropping, but its next flagship AI chips might change that.
· As Oracle looks to capture more of the market for powering AI model training, co-founder Larry Ellison said Monday that it is designing a data center with a capacity of more than one gigawatt, enough to power a city like San Francisco. Batta declined to provide details on that data center or when it might be operational.
Thinking of the gigawatt data center mentioned above, an update on how we’ll power all of this. ‘CB Insights’ latest research offers a look into how Amazon, Google, Microsoft, and Nvidia are tackling the problem:
- Amazon is working to decarbonize its transportation and fulfillment center operations, with a focus on hydrogen tech.
- Google is pioneering new models for clean energy procurement as it works to boost the sustainability of its data center network.
- Microsoft is focusing on renewable energy sources — like solar and fusion — and carbon capture technologies to meet the growing energy demands of its AI-driven operations.
- Nvidia is enhancing data center energy efficiency and investing in the development of a green and reliable power grid.
On the model front, ‘Cogni Down Under’ dives into ‘Reflection 70B’ and its architecture designed to prevent hallucinations. Based on Meta’s Llama 3.1 70B Instruct, the author describes its advantages:
At the heart of Reflection 70B is a technique called Reflection-Tuning. It’s not just a clever name; it’s a fundamental shift in how AI processes information. Here’s how it works:
1. Step-by-Step Reasoning: Reflection 70B breaks down its thought process into distinct steps, much like showing your work in a math problem. It’s the AI equivalent of “let me walk you through my thinking.”
2. Error Detection: As it’s thinking, the model uses special tokens to flag potential errors or inconsistencies in its reasoning. It’s like having a little AI editor sitting on its shoulder, constantly asking, “Are you sure about that?”
3. Self-Correction: If it spots a mistake, Reflection 70B doesn’t just plow ahead — it stops, reconsiders, and corrects itself. It’s the digital equivalent of catching yourself mid-sentence and saying, “Actually, let me rephrase that.”
If you want to learn more about how to create neural networks, Sean Jude Lyons writing in ‘Towards AI’ describes how to create a simple program to predict future outcomes based on past data. For example, based on the data below, how much would a 5-bedroom home cost? Good reading!
Also on predictions, and to close, who will win the next election, and can AI tell us the outcome? Ignacio de Gregorio describes the approach:
In simple terms, FiveThirtyNine (that’s the name, inspired by Nate Silver’s username on X) is a system that includes the following:
· A Large Language Model (LLM) (in their case, it’s GPT-4o)
· A search engine tool that allows the model to search the Internet and retrieve data
· A carefully crafted set of two prompts, one for the search, and one for the actual prediction, with no specific fine-tuning on the model.
He looks at how the model creates the necessary prompts and instructions, with the final result:
