My Data Science Journey — Getting started, leadership, and predictions for 2022.
Happy New Year, folks! This is an unabridged version of the interview I gave Jean Lee at Exaltiude last month (December 2021). Wanted to share it here for the off-chance a reader would like to know about my journey.
Q. What is Data Science and how is it different from AI?
Great question! Artificial Intelligence is the general field of using computers to automate some decision-making activity, assist humans, or personalize their experience — all, at scale. Instead of writing custom programs, AI software uses processes similar to how humans think — for example: using rules, understanding a natural language request using grammar, learning from labeled examples, and predicting on some visual task, etc. to provide these solutions.
Data Science is one discipline of AI (and maybe even overlapping into other Computer Science disciplines) where the solution is discovered using the data. That is, instead of encoding human reasoning into heuristics and rules, the data scientist derives insights from user behavior, software usage, and collective data captured by the software and solves one part of the problem helping the overall success metrics.
Let me give an example. When Uber matches you (the rider) with a driver (the ride) it uses heuristics, maybe some rules, and even some real-time data-driven matching (based on who’s available, routes, traffic, etc.). All of these are techniques under the umbrella of AI. But then, let’s say a data scientist looking at the app logs notices that some of the passenger pick-ups are taking an awfully large time. Upon investigation, she discovers that these are actually bus stops and no-stopping areas. Then, using Machine Learning on the data collected, she identifies the “optimal” pick-up spot near high-traffic areas and works with the engineering team to add these to the app — saving users some unnecessary frustration of locating their pick-up spot. That is Data Science.
Q. How did you get into Data Science?
After completing my master’s at USC, I was a researcher in AI in the automotive domain at one of the subsidiaries of Toyota. While working there was rewarding in many ways, one source of personal frustration for me was the time it took for my work to get it inside the car. Since the company’s role in the larger organization was to develop early-stage technologies (rightly so), I decided that I needed to leave and try to figure out for myself what I can do to bring my AI solutions to actual products. And so, in 2017, I took a sabbatical. Deep Learning was an emerging technology at that point. I “leveled up” my skills in Machine Learning and Deep Learning using multiple online courses and landed a job at Figure Eight (now, Appen) in 2018.
I had learned from the research role that I was passionate about using Machine Learning to create novel solutions for user problems and improve user experience. In Figure Eight, I worked as an ML Engineer with other data scientists, engineering, and product teams to create ML solutions to data annotation problems for various types of data. As I became a manager and then a director, my focus changed from writing code to connecting the dots to find data-driven solutions to business problems. It’s been an exciting journey!
Q. What made you switch from an individual contributor to a leader?
Well, it wasn’t a switch in a single day. Becoming a leader is also not something you can decide on unilaterally. It comes about by working with others. One of the most exciting projects I worked on for Toyota was to develop an in-car assistance system. I wasn’t a manager then, and barely even the tech-lead on one aspect of the project. Personally, I loved designing and creating systems with smaller intelligent services that worked together to bring out a great user experience. I knew that I couldn’t do this myself. So I read a lot, learned new technologies, and proposed solutions to the team to build this proactive assistant in the car. Working together with the team, I realized that I can build that larger system only by leading people towards it with my ideas (which, by the way, often need iterations) and well-formed arguments.
Figure Eight was where I really honed my soft-leadership skills to position myself to build these larger solutions. While the company was transitioning after the Appen acquisition, I worked closely with engineers, the product team, the customer success team, and other data scientists to create ML solutions in different scrum teams as an ML tech-lead. While doing so, by communicating what UX the ML solutions could achieve, absorbing feedback, aligning my ideas with others, and taking end-to-end ownership of getting those solutions built and released — I was able to position myself as a person that people looked up to. My peers and the management also liked my “Get Shit Done” attitude and that contributed to me becoming a manager and eventually a director. The support from peers, other mentors and leaders, and especially people reporting to me has been instrumental in making this happen.
Q. What advice would you give to someone who is just starting their Data Science journey?
First of all, welcome to this fascinating world! I promise you that the more you dig into Data Science, the more you’ll discover and you’ll have a great time if you like learning new technology. It’s also been a very sought after role this past year, and so I’m sure having one of the highest paying jobs doesn’t hurt either ;)
The best advice I can give to someone starting in this field is to align your work with outcomes. Most often, data scientists get bogged down with cleaning data, training models with the latest technique from the latest conference, or hyperparameter tuning to squeeze out that extra half of a percent accuracy, and lose sight of what the business needs are. Instead, I’d recommend that you regularly talk to the stakeholders, the customer success/sales teams, the product owners, etc. Working closely with them, you’ll make sure that you’ll deliver value with your work.
I love the “Lean Startup” methodology, and you should too. Instead of getting a product requirements document and going away for months to work on a model only to come back and find out that the business needs have changed, take a more iterative approach. Heck, even try to sit in on the customer interviews, so that you can gain insights on what their pain points are, and then use your data science skills (which you may even level up in your free time) to suggest solutions, which are not obvious unless you use data, that deliver customer value. (Pro-tip: This is also the best way to get your work noticed by management and get rewarded.)
Q. Looking forward, what are the newest trends in Data Science to look out for in 2022?
From what I am aware of now, it looks like 2022 is going to be a year with data science on steroids. As annotated data become more accessible thanks to data labeling companies like Appen, Scale, Snorkel, and others, the trove of data you can generate insights and solutions from will only get larger. Technologies like self-supervised learning, Vision Transformers, federated learning, an ever-increasing repertoire of MLOps tools and frameworks, and hardware acceleration will make it much faster to build solutions.
That being said, we are likely to see a switch from the “gold rush” mentality to a more responsible approach to AI — you will more likely be spending time on data-centric AI to deliver business value instead of EDA, model training, and hyper-parameter tuning; users/product owners will demand the usage of explainable AI techniques to make sense of model predictions; companies will demand to see real value from the large funding that Data Science teams have enjoyed the last couple of years, making work more outcome-focused (a caveat: expect to downsize if data science teams are not generating business impact, or worse, negatively affecting it); and lastly but most importantly — as more decisions become more automated the negative effects of bias from blindly using data are also going to get larger and you will need to also become champions of creating ethical AI solutions.
Data Science in 2022 is going to be epic! Good luck!
