The Market Digest
Week of July 21
By Joana W and Tejaswini Kumar — Edition 74 (Edition 1 for public consumption)
The Headlines
If you don’t want to read the whole thing.
🧠 AI News
- “OpenAI’s New GPT-4o Mini Boosts Chatbot Security with New Instruction Hierarchy“
- “Anthropic releases Claude app for Android”
- Allegedly: “Apple, Nvidia, Anthropic Used Thousands of Swiped YouTube Videos to Train AI”
- “Meta releases the biggest and best open-source AI model yet”
- “How’s AI self-regulation going?”
👓 Headset News
- “The Samsung XR Headset, rumored for release in 2024, is generating significant anticipation”
- Rumor: “Meta In Talks to Acquire Stake in Ray-Ban Owner EssilorLuxottica”
- Rumor: “Meta Reportedly tells its VR/AR teams to cut spending by a fifth”
- “Meta Quest App Now Called ‘Meta Horizon’ in Preparation for Third-party OS Licensing”
- Rumor: Reports say “Meta Quest 4 Headset Launching in 2026”
- “Quest 3’s Mixed Reality Dynamic Occlusion Is Now Higher Quality With Better Performance“
- “HTC Vive launches 3D streaming platform”
🤖 AR Tech News
- HaptX Gloves G1 now shipping!
- “Future Apple Vision Pro update could let users create their own gestures“
- Meta Smartglass Patents: Mechanism to adjust Inter-Axial Lens Distance, Thermal Hinge System, Smartgloves
Editors’ notes:
AI news: Security and Privacy — and implications on integration with AR/VR devices.
Security is now crucial for LLMs, and responsible AI is key for accountability. Microsoft’s Responsible AI Impact Assessment template offers a strong framework for this. Apple’s WWDC 2024, Google I/O, and ChatGPT’s Spring update highlighted private cloud, on-device processing, and smaller LLMs to improve security and trust.
AI-integrated AR/VR devices, like Meta Ray-Bans with ChatGPT, are trending and selling more, showing that AI integration is vital for success in consumer markets. Additionally, being open-source may give LLaMA an edge over other LLMs by enhancing transparency, customization, and community contributions.
Headset News: Samsung and Meta Drive AR/VR Innovation.
Highlighting rapid innovation and competition in the AR/VR market, Samsung’s XR Headset is rumored for a 2024 release. Meta plans to launch the Meta Quest 4 and AR glasses in 2026. To focus on AR glasses, following strong Meta Ray-Ban gen 2 sales, Meta is cutting AR/VR spending by 20%. Meta is also opening the Quest OS to third-party OEMs.
Key strategies to excel in the AR/VR space include open-sourcing software, reducing costs, and improving features. This includes improved security, especially with AI integrations, and advances in hand/body tracking for realistic 3D interactions, and more.
AR Tech News: HaptX Gloves G1 and Meta’s Innovations.
The HaptX Gloves G1, now shipping, offer unparalleled touch feedback for VR and robotics with hundreds of air-powered actuators and up to 40 lb of resistive force per hand.
Meta’s recent patent filings for tactile smart gloves, inter-axial lens distance adjustments, and thermal hinge systems highlight their push in AR/VR innovation. Apple’s Vision Pro update may soon allow custom gestures, enhancing virtual interaction.
Tracking patents will be a useful way to predict the direction AR companies are taking to enhance their products.
A quick announcement
If you like this newsletter, please reshare for visibility.
Tejaswini and Joana are no longer at Magic Leap due to company restructuring.
From Joana: Two years ago, I started this newsletter as a fun way to share valuable insights. It quickly gained traction, with readers from C-Suite to hardware finding it helpful. I hope you do too.
As I explore new opportunities in product marketing, I’ll continue sharing valuable content. If you’re hiring or in tech, let’s connect! Here’s my LinkedIn profile.
From Tejaswini: I’m an experienced data scientist turned product manager, and I’m eager to explore opportunities in Product Management and Technical Program Management space. If you’re a hiring manager, in AI and emerging tech space, or have creative ideas to collaborate on, let’s connect on Linkedin!
Now back to regular programming.
The Deep Dive
A summary of each headline
AI News 🧠
“OpenAI’s New GPT-4o Mini Boosts Chatbot Security with New Instruction Hierarchy” (Win Buzzer, Open AI, Microsoft Azure Blog)
GPT-4o Mini: The Most Affordable and High-Performing Small AI Model:
- Affordable and Efficient: GPT-4o mini is priced at 15 cents per million input tokens and 60 cents per million output tokens, making it significantly cheaper than previous models.
- Superior Performance: It scores 82% on MMLU and excels in reasoning, math, coding, and multimodal tasks, outperforming other small models.
- Enhanced Safety: Built-in safety measures and new techniques improve the model’s reliability and resistance to misuse, ensuring safer application at scale.
Note: Additionally, there’s news that Azure OpenAI Service now supports GPT-4o mini (Windows Report)
“Anthropic releases Claude app for Android” (Tech Crunch)
The Android app will include “free access to Anthropic’s best AI model, Claude 3.5 Sonnet, alongside upgraded plans through Anthropic’s Pro and Team subscriptions. Users will be able to sync their conversations with Claude across devices, and upload photos or files to the app for..,analysis.“
“The app allows enterprise customers to access their Claude accounts on mobile.”
Note: Companies like Google, OpenAI, and Anthropic are releasing mobile apps with free access to basic models and subscriptions for more advanced models. This increased availability may lead consumers to use these LLMs more frequently, becoming more proficient yet potentially dependent on them.
“Apple, Nvidia, Anthropic Used Thousands of Swiped YouTube Videos to Train AI” (Wired)
Proof News investigation found, allegedly:
“Subtitles from 173K YouTube videos, siphoned from more than 48K channels, were used by Silicon Valley heavyweights, including Anthropic, Nvidia, Apple, and Salesforce.”“YouTube Subtitles [dataset], contains video transcripts from educational channels like Khan Academy, MIT, and Harvard.” Also: The Wall Street Journal, NPR, and the BBC.”
Additionally, they allegedly found that “More than 180K books, including those written by Margaret Atwood, Michael Pollan, and Zadie Smith, had been lifted. Many authors have since sued AI companies for unauthorized use… and alleged copyright violations.”
Note: According to MIT, many news sites have started blocking companies from scraping their content for AI training.
“An MIT study found that 5% of all data, and 25% of data from the highest-quality sources, has been restricted.”
It won’t be long before more media types and outlets block their content from scrapping.
“Microsoft Copilot can now summarize longer Word documents”
(Tech Loy, Copilot Blog)
“We are significantly increasing the size of the documents you can summarize using Copilot” up to 80K words which is 4x Copilot’s previous limit.
The feature is available “through the Copilot Pro subscription, priced at $20 per month per user.”
“Meta releases the biggest and best open-source AI model yet” (The Verge, Chief Data Scientist)
“Llama 3.1 outperforms OpenAI and other rivals on certain benchmarks. Now, Mark Zuckerberg expects Meta’s AI assistant to surpass ChatGPT’s usage…”
“Meta isn’t saying much about the data it used to train Llama 3.1
Critics say it’s a tactic to delay the inevitable onslaught of copyright lawsuits.”
“Meta AI is also coming to the Quest headset in the coming weeks, replacing its voice command interface. …Use Meta AI on the Quest to identify and learn about what you’re looking at” while using passthrough.
Note: Does being open-source give LLaMA an edge over other LLMs like OpenAI’s ChatGPT and Google’s Gemini?
“OpenAI reassigns top AI safety executive Aleksandr Madry to role focused on AI reasoning” (CNBC)
OpenAI reassigned Aleksander Madry, one of OpenAI’s top safety executives, to a job focused on AI reasoning.
Madry was OpenAI’s head of preparedness, “tasked with tracking, evaluating, forecasting, and helping protect against catastrophic risks related to frontier AI models”.
How’s AI self-regulation going? (MIT Technology Review)
We’re including regulation since it’s a hot topic.
“Right now, the US is relying on the tech sector’s goodwill to protect its consumers from harm” — voluntarily.
However, there is a “requirement that developers share safety test results for new AI models with the US government if the tests show that the technology could pose a risk to national security.”
Headset News 👓
Lots of Meta News this week.
The Samsung XR Headset: Rumored Specs (XR Today)
“A Samsung XR headset is coming” (est. 2024)
“Powered by a Qualcomm chip (probably the Snapdragon XR2+ Gen 2 chip).”
Sony screens, rumored: Size: 1.3” || Resolution: 3,840 x 3,552 px || Refresh rate: 90 frames/s || Brightness: ≤1K nits.
There’s speculation that the developer edition will launch in 2024 and the consumer edition for 2025.
It’s also speculated that the headset may work with the Samsung Ring since it allows for gestures such as pinch.
“Meta Reportedly tells its VR/AR teams to cut spending by a fifth” (Tech Story quoting The Information)
AR/VR hardware teams ”have been asked to axe their spending by 20% between now and 2026.”
This was after ‘Essilux’ touted great sales of the Meta Rayban gen 2 and Meta separated both AR and VR teams so that it could focus more on AR glasses.
Rumor: “Meta In Talks to Acquire Stake in Ray-Ban Owner”
(XR Today, The Verge)
“Meta is planning to spend billions to buy roughly 5% of EssilorLuxottica, the €88 billion… [company] it has collaborated on two generations of Ray-Ban smart glasses.”
“Meta Quest App Now Called ‘Meta Horizon’ in Preparation for Third-party OS Licensing” (RoadtoVR)
Meta “plans to open up the Quest operating system to third-party OEMs, including Asus, Lenovo, and Xbox. When, how and what those devices will look like, we just can’t say — although we’re hoping to learn more at Meta’s Connect 2024” in September.
Reports say “Meta Quest 4 Headset Launching in 2026” (XRToday, Android Central)
“The Information, …spoke to several people who have worked on Meta’s hardware projects:
Meta’s first AR glasses are scheduled for a 2026 launch (codename: Hypernova).
Also planned for 2026 is Meta Quest 4 in standard and premium versions (codenamed Pismo Low and Pismo High).
A high-end Quest to compete with Apple Vision Pro (codename: La Jolla) is scheduled for release in 2027.
A cheaper Meta Quest 3, called the Quest 3S, is also planned for release this fall.”
There’s also rumors regarding its glasses:
“Meta teased its new AR glasses back in February and no one realized it” When asked about it, Zuckerberg said, “will be ready to share more later this year.“
Image from Zuckerberg, showcasing several headsets on the right side. And below, a much thicker pair looking like smartglasses.
“Quest 3’s Mixed Reality Dynamic Occlusion Is Now Higher Quality With Better Performance” (UploadVR)
“Meta has slightly improved the visual quality of the Depth API and significantly optimized its performance.“
“The Depth API now has an option to exclude your tracked hands from the depth map so that they can be masked out using the hand tracking mesh.”
“Apple Vision Pro has dynamic occlusion only for your hands and arms…You’ll see peculiarities like objects you’re holding appearing behind virtual objects in AR and being invisible in VR.”
Image from Meta showcasing occlusion with an animated character and a table
Image from Meta’s video.
Note: Advances where the hand/body tracking can be excluded from the depth map are key for 3d content interactions to be more realistic.
There’s more news on Hand Tracking and Gesture advances; check out the patent section.
“HTC Vive launches 3D streaming platform”
(Viveverse for Business, Develop 3D)
“The service will make Polygon Streaming technology available outside of Viverse for the first time. It will combine server-side processing with client-side rendering, reducing bandwidth and power requirements.
Polygon Streaming is intended for use in industries including media and entertainment, engineering, industrial design and game development.”
AR Tech News 🤖
Several news on input methods with hands this week.
HaptX Gloves G1 now shipping!
(Haptx newsletter shared by JD Leonard. Here’s a video and their product page)
The gloves, initially announced over a year ago, and now shipping, are geared towards training and focusing on industrial, medical, and defense.
“With hundreds of air-powered actuators, HaptX Gloves G1 is the most realistic system for touch feedback for VR and robotics.”
“Our flexible, integrated, Force Feedback Mechanism applies up to 40 lb of resistive force per hand, so you feel the size, weight, and shape of virtual objects.”
Their website also boasts “Submillimeter motion tracking”
Image of Haptx, which is designed to provide realistic tactile feedback. Image from ChannelXR, a channel reseller.
Note: A 4 pack is priced at $50–53K via their enterprise channel.
To accommodate various workers — orgs would likely need a 4 pack. Plus headsets. It would be interesting to understand more on the impact on use cases requiring tactile feedback and its scores/metrics.
Interestingly, Meta filed a patent for tactile smartgloves recently. See further below.
“Future Apple Vision Pro update could let users create their own gestures” (Apple Insider)
“Imagine being on a Zoom call and being able to mute yourself the way a radio DJ does by making a slicing gesture. Or perhaps make the age-old hand signal for “phone me”” to “send your contact details to whoever you’re looking at.
That’s the implication of Apple’s newly-granted patent… It’s a method of assigning any gesture to any Apple Vision Pro function.”
Meta Smartglass Patents: Mechanism to adjust Inter-Axial Lens Distance and Thermal Hinge System, Smartgloves (Patently Apple, Patently Apple)
Mechanism to adjust Inter-Axial Lens Distance (IAD)
(FIG. 1) — “a mechanism to adjust inter-axial lens distance (IAD) between eyepieces of smartglasses”
FIGS. 2A and 4D — “a threaded rod mechanism to adjust inter-axial distance (IAD) in VR/AR headsets/smartglasses.”
In Re: IAD. I originally thought this might be to address interpupillary distance SKU issues. I was curious, so I read the patent. It says adjusting the mechanism to adjust IAD distance aims to solve drop test problems — where misalignments and readjustments are an issue.
Thermal Hinge System
(FIG. 4) “The method employs a thermal hinge system configured to transfer or spread thermal energy, and optionally electrical energy, through a mechanical articulation or hinge.”
In re: Hinges, hinges are tricky in AR. In a headset you just put on the “band” of sorts, but on glasses where you have to fold the legs, hinges can impact everything from frame stability, pressure distribution, electric components, optics, etc.
Smartgloves that could enhance games & more
“A set of SMI sensors and an associated processing system may be incorporated into a glove that may sense the surface qualities of objects while the glove’s user navigates an AR or VR environment.”
