AI Conversations from the Field
I had the pleasure of working with Shylaja Ramachandra at Facebook building *8, prior to the launch of the smart speaker, Portal product. I was extremely impressed with her knowledge, and leadership on the smart camera feature. The Portal smart camera is an AI-enabled camera that recognizes individuals, and can direct focus while users move around the focal lens. Think movie quality video calling experience with hardly any user effort!
Transcript of our Conversation:
Karen: Okay, great. So just to start, I want to thank you for speaking with me today! I am really curious to understand what interests you about AI? Talk to me about your thoughts on what you like about this space, and what you think is particularly interesting.
Shylaja: Yeah. Thank you for setting this call up so we can chat!
So, to answer your question, when I was working at Yahoo — actually more than 10 years ago — I kind of fell into AI/machine learning by accident. I think that was my first exposure to it. It was very fascinating to see how many different ways we could show search results to users, and also other features, such as, using algorithms for sponsored search listings and presenting this to users. This is all basically machine learning, because people type in something, and your goal is to show the most relevant results. It was fascinating to see that you could crunch through so much data, and be able to come up with these machine learning algorithms. So, that was my first real exposure to AI.
Shylaja: Then later on, when I went to Netflix, and it was all about looking at AI in a different way. So this was about making sure that we were showing users more useful movies, and shows, based on so many different dimensions, based on time of day, based on what day of the week it was, because people tend to watch cartoons, and really light-hearted things on a Saturday morning. So why would you show, Mindhunter, some heavy duty stuff on a Saturday morning? It just feels jarring to even see that kind of stuff on a Saturday morning, when you’re ready to watch something while you’re eating your cereal.
There’s no way as engineers, we could ever sit down, and write code which would satisfy this for 100 million users. Instead, this is where really the power of AI begins to shine.
The second part to that is hardware, and just the capability of what we can compute has been increasing exponentially. When I went to school, all of this was theory. It was like these things that you could only imagine, that we couldn’t really program it. We couldn’t do much about it except like studies and theory.
Shylaja: Now that hardware has accelerated, and also the costs have come down at the same time, suddenly all of this is possible. So it’s so fascinating to see how before, what used to take a week, we’re doing in real-time, in terms of results, and showing the perfect recommendations, and all that stuff.
Karen: Yeah, that’s a great point. It definitely requires an environment that allows for compute and processing.
Shylaja: Yeah.
Karen: The hardware is a huge factor.
Shylaja: Yeah.
Karen: We talked about your work at Yahoo and Netflix, which is fascinating, but I met you at Facebook, and I would love to hear about some of the work that you did there, and how it relates to AI.
Shylaja: Yeah, so with Facebook, it was actually the first time that I was working on a different facet of AI. Where before, it was classified as machine learning, where it’s more statistical machine learning — the “if-else” approach, for the most part. In Facebook, the product used more deep neural networks, where now these are more like your multi-dimensional matrix computations, which really have been enabled by the sheer amount of inexpensive compute power.
This now enables image processing, and video processing in real-time, which was almost unheard of 10 years ago, and then being able to make predictions based on that. This is what my experience at Facebook was, where you’re looking at real-time videos and trying to predict who’s in the scene, and what they’re going to do next. And then we’re trying to do something based on that information, which in this case, was making sure that we’re capturing the right person,and framing the right people, alnd capturing all details to create a cinematic video call experience.
So Facebook was really a perfect example of where we took deep learning, for the first time, and productized it. In this way, for the end user, it was almost art, where we could have very well produced a movie in the end. Like if you did a reality show, it would have been the most beautiful-looking reality show because, without a camera man, it would have just followed, zoomed, panned, and captured all the right things at the right time.
Karen: That’s amazing. I love that reference. That’s so good.
Shylaja: Yeah, so it was such a beautiful combination of science and art which came together, and in a way that wasn’t this very sci-fi look and feel, where you’re putting on some magic glasses or anything. You didn’t have to do anything. It just happened, and I think it was very fulfilling to be able to do it. I think one of the references, Jason, one of my colleagues would say, (Jason Harrison) would say was, “Grandma can use this, and she doesn’t have to do a thing.”
Karen: Yeah, exactly. I know, it’s one of my favorite products that I’ve seen using what you were saying, this deep learning, to really productize the experience. I think the end result speaks for itself. It is just a beautiful example of AI in a product that Grandma can use. So yeah, that’s really amazing. And as you said, it definitely is a cinematic experience and that’s wonderful.
So along those lines, I know sometimes it can be very tricky to implement a lot of these algorithms into a product and have it work 100% of the time, and that kind of thing. Where do you think there’s room for improvement?
Shylaja: So from a purely technical standpoint, it’s always going to be an evolving field from existing products that are out there, or minor extensions of products that are already out there. Whether it’s with computer vision, a lot of photography now on smartphones is all computer vision, where they’re analyzing the images in real-time, and they’re addressing blurring, and those are things which feel natural, and of course, there’s the really more obvious that there’s computer vision going on. There’s the overlaying filters, giving you the bunny ears, and all that kind of stuff, which will evolve.
It’s always going to be an evolving form, where they’re going to try and get accuracy close to 100%, but something, somewhere is always going to fail. That’s just the nature of AI right now, and anybody who thinks they’re going to solve that 100%, we’re not there yet. And if someone claims the experience is 100% accurate, I would take it with a grain of salt, because something’s not right about that statement.
So really, for all the products that I’ve worked on, in every company where I was involved, the big thing with anything with respect to machine learning and AI was, “This is going to fail somewhere for someone, and what should we do when it fails?” That was always part of the plan. It’s like, “It is going to fail. Do not even presume. Don’t even think this is going to work 100% all the time.” Unlike traditional programming, where you know there are only five ways this can deliver an outcome, with AI, there are a million ways. No, like hundreds of millions of ways. So it’s going to fail for somebody somewhere, and it still needs to be a graceful product at the end of the day. All projects that I’ve worked on have always been built with that assumption.
In terms of evolving in the future, as new use cases come up, I think it would be interesting to see how close AI gets to predictions, and being right 100% of the time. I think if it gets to 99%, for example, in self-driving cars, it’s going to be interesting. I think that’s going to be the next big frontier in terms of really pushing AI, in terms of saying, “How close do we need to be to 100% accuracy?” There are already strong arguments from companies, like Tesla, looking at the viewpoint that, “Well, humans cause accidents, so why shouldn’t AI take over?” Well, that’s one way of looking at it. That’s from a purely technical standpoint. But AI, from a human and ethical standpoint, is very interesting. I think this is one of the things I specifically focused on at Facebook, because here we were trying to build a product that just needed to work for everyone. And there’s no standard defined per se.
Shylaja: There’s been a lot of conversation recently, more for ads and stuff, but from a computer vision standpoint where it needs to work for everyone. Your skin color shouldn’t differentiate you from having a good experience on different lighting conditions. Like somebody with fair skin under super white lights, the computer cannot see you. That’s the truth, because you cannot see… Light is light, and it’s going to do what it’s… And similarly for a person with darker skin on dark conditions, the computer cannot… The image is basically one blank screen for the computer. But still, our technology needs to be evolved enough to not distinguish, to be able to produce the same effect, same product for everybody. And if you don’t think we can, then maybe we shouldn’t ship these products. And I think that’s the conversation to be had in terms of AI evolving, going forward.
Shylaja: So now we have AI coming into more similar experiences like this. And I really do hope in terms of… I look at it as, primarily, as men will build the… Like it’s still a male-dominated industry for the most part, for many reasons. But as he goes out looking into applications which are primarily driven by women, with, for example, the try-on-the-makeup app, they’re still very driven towards the demographic that it’s built by, which is primarily white men. That’s the truth. However terrible that sounds. So it’s literally like they’re testing on fairer skin. And so the whole application is built, its AI is trained on data that is things that people are familiar with. And I think that’s going to be the big shift, where it’s no longer about things that you’re familiar with, but you just need to look at a corpus, which is just like the matrix that your problem needs to solve for. I think that’s a big mind shift that needs to occur.
Shylaja: So for example, when I look at the try-on-the-makeup apps, which are great because I don’t even need to go out. I mean, I wanted to shop at home, just give me everything. And it’s like, “Well, it just needs to work for my skin tone.” And so I can clearly see the difference, where that’s a big improvement and evolution that really needs to happen before AI is there.
Shylaja: And the same dimension then goes towards… Skin tone is one, but this also then evolves into sizes of people, because children are different than adults. Like their proportions are completely different than adults. So we train on body proportions and face proportions of adults, and in children, for example, the head and body proportions are the same thing. Imagine your head is almost the size of your body, because that’s how babies are long, and they grow like that till about two or three. And after that your head is like less than one third the size of your body. So for a computer, it needs to know all these things. Like we don’t really internalize this as human beings, but that’s why what works for adults will not work for… As people get smaller in size.
Shylaja: So it needs to work for everyone, when you’re building general products, or we need to be able to say, “No, this only works for certain types of people. Everybody else, you go figure out your stuff.” You don’t think that’s the best thing, but at least it needs to be called out as such.
Karen: That’s really, really great feedback. The last thing I wanted to hit on with you is, what articles, blogs, podcasts or books do you recommend? What kind of things are you reading, or what organizations do you recommend joining or following?
Shylaja: At a high level,there is a great newsletter that MIT puts out for AI, which is very research-driven, coming from their labs. This is a great source because it’s instructive, and they aggregate information from across several universities, so it’s a great way to look at what academia is doing with respect to AI, on all the different levels that are available.
There are a couple of great books that have come out recently, especially coming from Kai-Fu Li, who used to work at Microsoft, and then moved back to China, because the truth is China is really dominating the field. We shouldn’t even dispute that at this point. China has taken a very different approach to AI, compared to the rest of the world, and they’ve doubled down on it big time.
Shylaja: Kai-Fu Lee’s book, “AI Super Powers: China, Silicon Valley, and the New World Order”, is very interesting because it’s highlighting how China is looking at problems, which the US and the rest of the world, are not really heavily focused on. For example, they’re looking at how do you use AI for an aging population, in terms of being able to predict wellness and needs. Potentially using AI in terms of robots to help you around the house, and those kinds of things, that will carry over to help everybody else.. So it’s been interesting to see, from their perspective, what they’re working towards.
Karen: Yeah, definitely. I know I have Kai-Fu Lee’s book, and I saw him on CBS 60 Minutes, and you’re right. There’s is a lot of data, a lot of brute force number-crunching, in terms of AI in China. I’m curious to see what the differences are in terms of what AI is developed in the US versus in China, versus in Africa. What technologies will bubble up to the forefront? Is the US going to play a more creative role, or is it like our facial recognition landscape, it’s very different here than in China.
Shylaja: Agreed, so a whole bunch of stuff there, and also there’s academia versus industry perspectives. If you want to understand the underlying technology and what this is about, Andrew Ng’s course on Coursera is really a good place to wrap your head around things.
Shylaja: And then there’s also a bunch of research, which is funded especially by Disney. Of course, there are the usual players like Facebook, Amazon, Apple, and Google, who put out papers on what they’re doing, and how they’re building new products. I particularly find Disney interesting because they actually have a lot of AI already in their theme parks. They’re actually way ahead of a lot of the companies, and they’re doing fascinating stuff because they’re also using a lot of the technology in their movies, in their 3D animation. They actually fund a lot of research and universities, but they are also already using these AI tools. I actually follow their YouTube channel where they put up a lot of their research.
Karen: Oh, that’s cool. I actually didn’t know that. I’ve heard of the Coursera stuff, and I actually hadn’t even really thought about Disney theme parks as an AI reference point. I’m so glad that I asked you that question.
Shylaja: Yeah, they do very cool stuff.
Karen: I think we covered a lot of ground, but speaking of unique products and AI already in the field, what are your thoughts on diversity in AI?
Shylaja: I think in terms of diversity within AI, it’s one of the things where it’s a bigger problem in the sense that it’s going to take time to fix, but the shotgun fix is to make sure that, at a bare minimum, data is functionally and accurately representative.
Even if the pipeline for getting more diverse experts into the field is going to take longer, actually making sure that there is representation in the data is extremely important. There’s no reason why that can’t be fixed in the short term. In most cases, this is just brute code data collection, which should be done, and there’s no excuse for not doing that when creating algorithms for products.
Karen: Exactly. Well, I know you and I will talk more, hopefully soon, and you can give me some more updates on all the fun stuff you’re working on.
You have this amazing perspective. And so I’m really, really grateful to be able to share this with everybody.
In closing, a big thank you to Shylaja for agreeing to discuss with us her experience as a woman working in tech and on AI products.
Shylaja’s experiences are unique, and highly valued, and I am grateful for her time, and her willingness to share her perspectives.
Articles/Books/Resources mentioned:
https://www.amazon.com/AI-Superpowers-China-Silicon-Valley/dp/132854639X
https://www.coursera.org/courses?query=machine%20learning%20andrew%20ng
