Machine Learning for the Metaverse. Why Meta’s AI Lab is so random.
Having gone over a lot of Machine Learning Research from Meta’s/Facebook AI Lab, I noticed a pattern
As someone deeply involved in Machine Learning Research, I like to keep track of the major players in the domain. Among the biggest players is Meta’s AI Lab (the link is still https://ai.facebook.com/ interestingly). Using the large amounts of computational power available to them, the team is able to pursue projects that would not be possible for most organizations. While this is a huge concern for AI openness, replication, and safety, the great results achieved by their teams can’t be denied.
As I was going through their papers, I noticed a very interesting trend. In this article, I will cover the trend, and how it ties into Meta’s Metaverse Aspirations. With VR being such new technology (and the idea of a Metaverse being new), people haven’t really figured out how it could play into the economy. Looking at Meta’s strategy is a good hint into why Mark Zuckerberg and Meta are pouring Billions of Dollars into it.
Some context
Taken individually, a lot of the papers published by the Meta AI Research Team seemed very random. For example, in 2020, Facebook AI came up with a machine learning agent that could translate code from one programming language to another. Rather than using rules, the code instead uses deep learning to achieve unprecedented results. Shared in their post, Deep learning to translate between programming languages, the AI agent uses self-supervised learning, deep noising, and lots of Github data to work.
This work is certainly impressive, but it has very little to do with Meta’s Children Drawing Animator (Using AI to Animate Children’s Drawings)or their extremely powerful few shot learner (Harmful content can evolve quickly. Our new AI system adapts to tackle it.). As a slight tangent, all these, and much more were covered on my YouTube channel. Check it out to never miss a thing and learn about ML.
All of these projects are truly impressive, but they seem to have no rhyme or reason behind them. One is self-supervised learning, another is a classification problem of natural language processing, and the third is a regression-like implementation in Computer Vision and art. So why is a for-profit company like Meta dedicating so many resources into so many areas of Machine Learning Research? Contributing to human knowledge (they open source a lot)? The researchers are bored? Clout? Do they have too much money?
Turns out there is a method to Zuck’s madness. To spot it, we need to take a birds-eye view.
A Birds Eye View at the Social Media Landscape
Twitter, Facebook, YouTube, TikTok, Reddit, and other social media platforms (including Medium) make money through people’s attention. Different platforms focus on different aspects and maximize in that niche. However, with such intense competition, things are bound to get very ugly, with serious price gouging by competitors. This is terrible for the bottom line.
The leadership at Facebook recognized this. This is why they decided to pick a new fight, instead of pouring resources to take over a saturated battle-ground. And thus, Facebook became Meta.
In his talk, Introducing Meta, Mark Zuckerberg talked about how Technology was meant to allow people “To connect with anyone, to teleport anywhere”. The Metaverse is supposed to be a series of universes, where one would switch between office worker, gamer, and real-estate tycoon based on their requirements. Much of the metaverse will probably play out similar to an open-world MMORPG to allow people to interact with each other. How does this tie into Meta’s Machine Learning Research?
Contextualizing Meta’s AI Research
Think of how big and complex the world is. How much things vary across different regions and groups. For the Metaverse to deliver on its promise, it would need to be able to handle that (and so much more) in a sustainable manner. It sounds quite overwhelming, but the mad lads at Meta are attempting to do it. And they’ve done very well, so far.
Think of all the languages spoken in the world. Think of all the people, you will never be able to communicate with because you don’t speak a common language. Remember the struggle of articulating yourself in a non-native language? For the Metaverse to work, the language barrier would have to be solved in real-time. Enter, Teaching AI to translate 100s of spoken and written languages in real-time. Or Pseudo labeling: Speech recognition using multilingual unlabeled data
Meta AI is announcing a long-term effort to build language and MT tools that will include most of the world’s languages. This includes two new projects. The first is No Language Left Behind, where we are building a new advanced AI model that can learn from languages with fewer examples to train from, and we will use it to enable expert-quality translations in hundreds of languages, ranging from Asturian to Luganda to Urdu. The second is Universal Speech Translator, where we are designing novel approaches to translating from speech in one language to another in real time so we can support languages without a standard writing system as well as those that are both written and spoken.
If people are to use the Metaverse for leisure, the ability to express themselves is crucial. Enter the animation agent mentioned earlier. Want something for a Zoom meeting? Creating better virtual backdrops for video calling, remote presence, and AR has you covered.
Once the Metaverse takes off, the world will be uploading trillions of uncurated images up there. These images will be messy, with tons of variation in content and quality. The AI needs to go through them, and identify important aspects. What do you know, SEER 10B: Better, fairer computer vision through self-supervised learning on diverse datasets is another Facebook publication.
The authors of the paper had this to say:
In particular, advancing computer vision is an important part of building the Metaverse. For example, to build AR glasses that can guide you to your misplaced keys or show you how to make a favorite recipe, we will need machines that understand the visual world as people do. They will need to work well in kitchens not just in Kansas and Kyoto but also in Kuala Lumpur, Kinshasa, and myriad other places around the world. This means recognizing all the different variations of everyday objects like house keys or stoves or spices. SEER breaks new ground in achieving this robust performance.
What about having a positive experience in the Metaverse? No matter how feasible the Metaverse is, people won’t use it if there is a chance of harassment, bullying, or other harmful interactions. That is where we can combine the various research elements done, from the FewShot Learner to the Language Translators and Computer Vision models. Throw-in their work in Deepfake Detection (Creating a dataset and a challenge for deepfakes)and their efforts to push new learning paradigms (Yann LeCun on a vision to make AI systems learn and reason like animals and humans) to help their AI agents catch any malicious behavior.
Closing
The reason it looks like Meta research focuses on everything under the sun is that they quite literally have to do that. Mark Zuckerberg and Meta are trying to drag our lives into the Metaverse. To be successful they have to replicate the many systems that we take for granted.
The Metaverse is definitely coming. Companies have started to develop ways to monetize the transition into the Metaverse. It’s important that we learn to do the same. As the world goes virtual, learning how to code will be crucial for a good job with tons of pay. I created Coding Interviews Made Simple using new techniques discovered through tutoring multiple people into top tech firms. The newsletter is designed to help you succeed, saving you from hours wasted on the Leetcode grind.
To help me write better articles and understand you fill out this survey (anonymous). It will take 3 minutes at most and allow me to improve the quality of my work.
Feel free to reach out if you have any interesting jobs/projects/ideas for me as well. Always happy to hear you out.
For monetary support of my work following are my Venmo and Paypal. Any amount is appreciated and helps a lot. Donations unlock exclusive content such as paper analysis, special code, consultations, and specific coaching:
Venmo: https://account.venmo.com/u/FNU-Devansh
Paypal: paypal.me/ISeeThings
Reach out to me
Use the links below to check out my other content, learn more about tutoring, or just to say hi. Also, check out the free Robinhood referral link. We both get a free stock (you don’t have to put any money), and there is no risk to you. So not using it is just losing free money.
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
If you’re preparing for coding/technical interviews: https://codinginterviewsmadesimple.substack.com/
Get a free stock on Robinhood: https://join.robinhood.com/fnud75
