An Unexpected Party
As we wrapped up at GDC 2024 a few weeks ago, I started receiving ever more requests from people wanting to know the background of Meshcapade, how we got started and my thoughts and vision around it. So I decided I’d bring everyone along on the journey of Meshcapade with a blog series to start sharing the stories about the past, our vision, and my thoughts as we move forward.
For the uninitiated, Meshcapade is a tech startup building foundation models to enable 3D digital humans for AI, gaming, VR, fashion & healthcare. Meshcapade was founded by Talha Zaman (CTO), Michael Black (Chief Scientist) and me, Naureen Mahmood (CEO), in the quaint little, fairytale town of Tübingen in 2018. Now, we’re a stellar team of 26 (and growing), spread all across the globe. In our quest for helping computers understand the world, we’ve spent years building a whole suite of features that help machines perceive how humans look, move, talk, interact. The latest features we were super excited to showcase at our first ever GDC expo this year were text-to-motion and markerless mocap. See our promo for GDC here.
Before getting into the hows and whys — let me start off with something I’ve been thinking about a lot lately. In the movies, especially in sci-fi, there’s often a futuristic vision of having machines that interact with us on an almost human level. Why, yes! In some distant future, of course we’ll have robots that will talk to us and have thoughts and personalities. Even the disembodied thinking machines of our sci-fi worlds, Jarvis from the Avengers or HAL from 2001: Space Odyssey — it seems perfectly logical for them to exist in a somewhat distant future.
But for a machine that can do that, there are so many modalities of “human-ness” that need to be learned, and pretty much all of them have a lot to do with Perception. The ability to understand the world through vision, touch, sound. Even just to interact with each other, we humans rely on our senses to see and to hear each other, to understand our motions, behaviors, expressions, emotions, gestures and our words.
When we started Meshcapade, what drove us all was an ambition, a dream, to solve these perception problems. At our core, we are computer vision scientists. But that enthusiasm very easily expanded into computer graphics, machine learning and AI — all the tools available to us to solve a century old problem of helping machines understand and interact with the world around them!
When I first came to Germany 11 years ago, it was to join the Perceiving Systems Department at the Max Planck Institute for Intelligent Systems (MPI-IS) in Tübingen, led by Michael Black. My best friend in Pakistan, a brilliant humanities major, asked me what I’ll be doing there. I told her I’d be working on computer vision. She said, “What’s that?”. I started telling her about how to detect different kinds of features in images, I excitedly told her how one can track corners and edges in a photo or videos, how to use these to perceive motion and so on. Of course, I was making no sense. She wasn’t asking how, she was asking what I would do. When she still looked puzzled after my spiel, I had to think for a bit.
Then, I sighed and told her what it was really all about, “I just want to teach robots to see!” That answer gave her immense joy. Since that day, every time she mentioned me to someone, that’s what she told them :)
So, that’s precisely what I did at Max Planck — with many, many amazing people. And now with Meshcapade, we’re getting to bring all the groundbreaking research we’ve worked on to real users everywhere.
And yes, our tech is indeed used to train robots to see. And so much more too!
Michael, who is the brains behind many adventures in computer vision, was also the mastermind behind Meshcapade. It was a crazy idea to me at the time he suggested it. There was no doubt in my mind that there should be a startup that takes over the fantastic work we’d all been doing at Max Planck, to continue it, to make it bigger and better so that everyone, from big tech to the indie creators could use it.
The vision was clear. The potential was clear. There was no doubt in my mind about how important it was. I just never imagined myself taking the reins. I knew very few examples of women founders / CEOs, and zero examples of women founders who looked like me, wore a headscarf. But between Michael and Talha, the two convinced me. Their main argument was, if not you, who else?
So we dove in. I was the jack of all trades, but I can also be the brute force implementor with a slight inability to understand the word impossible. That’s essentially how we managed to launch one of the very first AI startups in the tiny town of Tübingen in Germany (and now Tübingen even has a whole AI startup ecosystem, CyberValley). Michael became our Chief Scientist and builder of grand ideas. Talha Zaman, the meticulous software engineer and avid gamer (it’s frustrating how much better he is at every game), was the missing piece to bring it all together from big ideas and dirty research code to real, elegantly designed software where all the moving parts flow seamlessly together.
I think Talha could write a book on how badly researchers write code. I might get my own chapter in it.
