Founder Spotlight: Bobidi
By Ryan Mendoza
Welcome to our next Scrum Ventures “Founder Spotlight” with Jeong-Suh Choi, one of the founders of Los Gatos, California-based Bobidi, a platform aimed at helping companies validate their AI systems by exposing them to the global community.
With the adoption of AI on the rise, it is only fitting that we catch up with this up-and-coming startup that helps companies make sense of AI. In the rush to build, test and deploy AI systems, businesses often lack the resources and time to fully validate their systems and ensure they’re bug-free — this is where Bobidi comes in.
In this interview, Choi gives an update on the company’s growth and provides insight into how AI has changed and what we should watch out for as more companies enter this space.
For those who aren’t familiar with your company, can you give a brief overview of your company and what you do?
Bobidi is an AI model test platform with apps on both Android and iOS. We have built a global bug bounty platform that helps companies validate AI models to make them much more efficient. We do this by securely connecting customer AI models to our global bug bounty community via an API. This community then tests customer models, figures out if there are loopholes, and uncovers biases. This process reduces the time needed for validation from months to a couple of weeks if not days.
The idea is to let end users test AI systems and biases — that is, the edge cases where the systems perform poorly — to reduce the time needed for validation.
What kind of momentum has the company experienced thus far?
Since being founded in June 2021, we have experienced growth in a few different phases. The first six months were really about testing our prototype including understanding different model types like computer vision and speech recognition. The next phase was about raising the capital needed to expand our business. Once we secured funding from Meta, Scrum Ventures, Y Combinator, We Ventures, Hyundai Motor Group, and others, we were able to start building our team which led to securing our first major customer — Naver — one of the largest technology companies in Korea. I think one of the main keys to our success is that we built a phenomenal, cohesive team in Korea — enabling our first project with Naver to take off. This project also allowed us to hone in on our first focus area — speech recognition.
Can you tell us a little bit more about your current focus area?
Right now we are focused on AI speech models. Our group of testers will test automated speech models and then mark the errors, including the time of the errors. This in turn creates metadata and helps AI companies determine their next steps. This is important as many AI companies have yet to learn how their AI will perform before its release. Today it’s critical for companies to find loopholes and problems before products are released and Bobidi helps them do this.
What are some of the lessons you have learned so far?
Whenever I think of lessons learned, I always go back to some advice I heard from Y Combinator’s Dalton Caldwell and Michael Seibel — that if you want to be a startup founder, you need to expect to be punched a few times. Over the past couple of years, I have had to remember that setbacks are “normal.” In my experience being a founder is like being a boxer — getting punched in the face happens all the time because that’s the game. You just have to learn to go with it. Every day there is a new problem to solve. You have to expect that things will go wrong and quickly adapt.
I would also say that you should be careful not to lower your standards, even when times are tough — especially in terms of hiring. Don’t hire anyone that is in the gray area. Just know that you WILL find the right person. Having the wrong person is worse than not having someone at all.
What are you most excited about in the year to come?
On a macro level — generative AI is something I am super excited about. And from a micro level, I am excited for Bobidi to find its role in the hype of generative AI throughout a full year. For example, almost all generative AI will need quality human feedback (e.g. Reinforcement Learning from Human Feedback) to continuously improve and Bobidi is best suited to help those companies with a high-quality community that fits their needs. It will also be helpful for customers to reduce the risk of misinformation. Our mission is to help customers improve their model really quickly and we are already starting to see this prove out.
Taking a step back though, it’s interesting to look at generative AI and where that is going. I just heard a VC say that for 70 years — the entire AI industry has wondered whether this can work and now we are seeing glimpses of it working. But now what does this mean?
I think it means that AI is moving from being just a toy to something that actually has incremental business value. There will always be debates about problems that AI can cause but it can truly create value. For example, a Japanese anime can be created in minutes vs hours.
But what’s really exciting is the entire ecosystem has started to respond to the technology. It’s not just the tech community, AI has now hit the mainstream.
What else are you excited about? What are some of the benefits you are seeing of AI?
We’re really excited about the value loop that AI is starting to create. The growth of AI can inevitably cause issues with things like CRM but with Bobidi’s bug bounty community, we can solve these problems. For example, if I call a customer call center in the US and that person doesn’t recognize my Korean accent, I can just hang up and call again and probably get someone who better understands me. In the world of everything going to AI though, if I call a call center and the AI doesn’t understand, there is nothing I can do. My hands are tied if the system doesn’t understand me. One of the ways that a company like Bobidi can help solve this problem is to use our community to hone in on these types of issues so that the system can understand and correctly translate any kind of speech. The more we can test the more we can allow AI to do its job correctly.
Why is a bug bounty testing community beneficial for companies deploying AI?
The best people to test the model are potential users themselves. And because we have this amazing community we can run A/B tests to understand the incremental value. Up till now, startups did not have this luxury but now can provide this.
Tell me a little more about your community of testers. We just saw that you launched weekly challenges for users, how is that going?
To test models, Bobidi connects the customer’s models with the people in the community. As people attempt to find loopholes in the system, customers like Naver get an analysis that includes patterns of false negatives and positives and the metadata associated with them (e.g., the number of edge cases). Generative AI companies can also get quality human feedback that they can use to quickly improve the model (e.g. Reinforcement Learning from Human Feedback).
The amount of money people can make through Bobidi — $10 to $20 per hour — is substantially above the minimum wage in many regions around the world.
In fact, it has become beneficial to many people from around the world. One funny story that I just heard about is that a person working on Bobidi from the Philippines said he is going to name his baby Bobidi.
What keeps you up at night? What are some of the biggest challenges?
It’s really a combination of half excitement and half worry. Generally, I am excited about the magnitude of the shift that has happened in the AI industry.
As an early-stage company though it’s sometimes overwhelming because there are so many different strategies we can pick. This market is constantly evolving. I am constantly thinking about this. We have a solid vision and mission but the strategy can always change. How do we balance this? How can Bobidi stay focused while determining the next steps?
Do you have any advice on raising funding?
I would tell founders to forget about valuation and fancy decks — make something people want and show investors that it works. Understand that innovation is in the spectrum of iteration. The ordinary days aggregate into something innovative. It does not happen overnight. It’s important to have a strong rationale for what you need the money for. While more is merrier, it could be better to have a little bit more money than before given today’s economic environment. Have a little cushion so you can survive first and evolve long-term.
Why did you choose to work with Scrum Ventures?
Scrum has been really awesome to work with. The team has helped us set up meetings with customers in Japan and they are very proactive in helping us find and connect with the right people. They are a great partner and they pay attention to the problems we are facing and intervene with solutions when needed.
