AI Top-of-Mind for 6.7.24 — Shovels
Top of mind… Who is really making money with AI? Rohan Balkondekar offers an analysis, citing surveys from ‘a16z,’ the VC firm, and others, and includes data I’ve not yet seen. Two of the more interesting charts on Gen AI spend and views of open source.
To add some perspective, I was driving up 880 yesterday and got into a discussion as to why Dell has so many AI ads on billboards, just as we were passing Supermicro’s headquarters. Two companies that are more-or-less flying under the AI radar but are doing quite well providing the server infrastructure if you look at their share prices. Something to be said for shovels!
Continuing coverage of Anthropic’s research, this time a deeper dive by Ignacio de Gregorio where he sets out to further explain LLMs and what the research has uncovered. A few snippets from his post:
· For example, instead of learning every sentence by heart, humans learn about grammar and syntax, aka how words are written and how they commonly follow each other, to extrapolate that knowledge into new sentences without requiring rote memorization.
· Humans know ‘I banana eat’ is not a correct sentence without having to memorize every single way those three words follow each other.
· This is precisely what LLMs do also, but we can’t really explain why or how, which is the point of mechanistic interpretability.
And…
· Luckily, in October 2023, Anthropic made a huge discovery: while neurons are unequivocally polysemantic, certain combinations of them are monosemantic.
· In layman’s terms, although the outcome of a model was unpredictable based solely on one neuron’s behavior, whenever a set of neurons fired together, the outcome was always the same.
But he does offer the following warning. If an LLM can be ‘clamped’ as Anthropic describes, it could be altered to provide pre-determined outputs. This goes beyond simple training bias and could be a real concern for the future.
Turning to the Stanford AI Index Report, we’ll now look at education. From the report:
· In 2019, 13% of new AI faculty in the United States and Canada were from industry. By 2021, this figure had declined to 11%, and in 2022, it further dropped to 7%. This trend indicates a progressively lower migration of high-level AI talent from industry into academia.
· In 2022, 201,000 AP CS exams were administered. Since 2007, the number of students taking these exams has increased more than tenfold. However, recent evidence indicates that students in larger high schools and those in suburban areas are more likely to have access to CS courses.
Evidence of the shift of AI from the theoretical to the practical:
Looking state-by-state, some of the numbers are surprising, counter to what we sometimes think about certain regions.
You’ve probably read about Google’s ill-fated ‘AI Overview’ and issues with its Gen AI generated summaries. AL Anany offers his thoughts, and in particular, the problem with traditional search:
Here’s what might happen if you googled “I need a logo.”
- At the top, you’ll most likely get a bunch of sponsored ads (which may or may not be relevant to what you want) — And that’s the main source of Google’s $305 billion in ad revenue.
- Below the ads, you’ll get a page that’s so well known for “designing logos” that it’s become the go-to website for this task — e.g., Canva. It’s optimized to organically come at the top of search because it’s actually good…
In contrast, tools like ChatGPT offer real guidance, posing an existential threat to Google.
On the same line of thought, the week wouldn’t be complete with yet another dig at Humane’s AI Pin, this time from the ‘NY Times.’
· Humane’s founders, Bethany Bongiorno and Imran Chaudhri, were right. In April, reviewers brutally panned the new $699 product, which Humane had marketed for a year with ads and at glitzy events like Paris Fashion Week. The Ai Pin was “totally broken” and had “glaring flaws,” some reviewers said. One declared it “the worst product I’ve ever reviewed.”
· About a week after the reviews came out, Humane started talking to HP, the computer and printer company, about selling itself for more than $1 billion, three people with knowledge of the conversations said. Other potential buyers have emerged, though talks have been casual and no formal sales process has begun.
· Businesses are interested in the device, she added. Within 48 hours of its launch, more than 1,000 companies — including in retail, medicine and education — reached out to discuss potentially working together or building software for the pin, Ms. Bongiorno said.
This last paragraph got me thinking as to how devices like this could be useful for specific industry verticals. In the same way that Magic Leap has found a second life in the enterprise, the same could be said for these types of devices.
Finally, from the website of one of my former yoga teachers, a podcast to help navigate the new world:
· Our REAL Eyes Realize podcast episode with Chiprian (“Chip”) Rarau is one where we honestly talk about the intersection of Artificial Intelligence (AI) and Humanity, from a spiritual and philosophical perspective. We investigate the way the mind works, how letting go is not giving up, and how to adapt to the arrival of artificial intelligence with mindfulness.
· This podcast episode is a primer on investigating AI and how we navigate the possibilities and risks present as it evolves.
