A Thousand Days of Edtech, Part 2 of X
Lessons from an Ex-Bookseller in Edtech
Hello! If you’re coming in from elsewhere, you may want to…Read Part 1 here!
Going into a tech company after retail is a bizarre experience. I remember asking where I had to clock in when I started on my first day at Udemy. (I was laughed at, though not unkindly. It was more of a “silly Robbin, that’s not how it works” kind of way.) I majored in psychology and Japanese in college with a minor in education, so I had no background in anything remotely useful when it came to helping a business. I did, however, have a background in classroom education and in selling reading material. If you read part 1, you’ll know that the first lesson I learned was that I should never stop learning. I didn’t need a my job to tell me that, but I took this to heart and dove in whenever my boss asked if I knew how to do something.
Here are some of the three-letter terms I learned on the job, which at first left me feeling scared stiff because I thought it was some kind of secret code:
- KPI: key performance indicator (or, the “how are you going to show me that your project/change/solution actually did anything” variable)
- ROI: return on investment (or, “did that KPI make/save us any money?”)
- SQL: structured query language — what is being referred to when someone asks for a query built by your friendly backend and/or data team, depending on how big the company is (or, “everyone seems to know how to do this data code thing to measure KPIs for ROIs so I guess I should learn it too”)
Lesson 2: Scrappy Isn’t Crappy When You’ve Got Numbers
Udemy also valued the quality of being scrappy. I didn’t really understand what it meant to be scrappy until I started this job. I had assumed that I would just be there to answer tickets and then go home, but everyone was encouraged to share a potential solution if they had an idea. One of the internal channels I learned about was Udemy Imagines, a place where colleagues could present said potential solutions. Every once in a while, these suggestions would make a lot of sense and then it would just happen all of a sudden with the help of engineers or the product team. It was fascinating. It was strange. How did this even happen? But then it started to make sense: there were a lot of things that looked polished to me on the outside, but were actually the result of very resourceful people doing clever things with data to prove a point.
I learned that being scrappy could happen in a couple of ways.
- “I’ll do it myself because I know everyone else is busy”: Depending on your skillset, this could go really well and could get picked up by a product team. Many mini-projects were born this way, which is great, but isn’t so useful if you’re not sure what you’re measuring. There were a few individual projects that got off the ground only to fail three months later. I’d learn that this isn’t so bad, either, because even a failed project is a learning to not do that thing again. The idea is that you don’t need a whole team to implement your solution.
- “I’ll get the data and show the appropriate team why this change should happen”: In a data-driven company, this is clearly the best route. It requires a lot more time if you don’t know what SQL is or how to collect data or what it even means. I was in that position, so I made sure to sit down and take time to understand how to write queries. In this situation, you may be in a good spot, because you’ve built a case like a lawyer ready to fight, and you don’t need to be the one to build it.
- “I’ll wait for a hackathon”: More on this later, but a lot of folks in the company would look forward to hackathon because it was a time where we could set aside our normal work in order to scrappily solve a problem. It’s a mixture of the two items I just mentioned, and it’s awesome because our hackathons promoted cross-team projects.
“WTF is SQL?” (and Other Questions)
One of the first things they asked me to learn was SQL. I’d never heard of it before, but I’d taught myself HTML and CSS in middle school, and if I could do that then, I knew I could do it now. It helped that I now worked for a company that specialized in education. I took full advantage of the free online courses in the topic and learned about the wonderful world of data. I sat down with the Excel experts, who were able to explain what queries did in basic terminology. It’s something I’d like to dedicate more time to, but I’m happy to say that I can manage all right on my own if I needed to find some information to do a quick comparative analysis.
When I started out, I learned how to write queries that would help make my job on support more efficient. The queries were simple, but they helped immensely whenever our team needed to do some detective work. It helped answered questions like, “Who used this coupon code?” “Did the customer delete their account?” and other things the support team leads would need to Sherlock out for our remote teams.
Eventually, these queries would be adapted for use on tools like Chartio so that our remote teams could run them without needing to learn the intricacies of SQL. In this sense, we had to look at how many tickets our team could reasonably answer in a work day. Having these queries increased productivity: back in 2014, we were expected to answer at least 25 tickets. With these queries, our agents didn’t have to keep asking the SF team questions that required data lookup, and they were able to knock out 40 tickets in a workday. I’d gotten to a point where I was so efficient I could do 100 in a day. It was a little insane. (Once I got to the point where I was dreaming about answering tickets, my boss said I should really consider taking a vacation.)
In edtech, it’s critical to understand and remember why people are sending in support tickets in the first place. Students just want to learn. Instructors just want to teach. There’s something blocking them from doing that. Data will be the sword you need to charge into battle for your customers.
Understanding the Process of Small Changes
Being on a support team can be frustrating, because all we would deal with is angry customers because something wasn’t working the way it should.
I had come to understand that there were certain bugs that were important enough to be prioritized, and others that we couldn’t work on simply because there were other issues that were more pressing. At some point, I’d started using tags within the ticketing system to start logging feature requests from our users.
The patterns became easily recognizable. Each week there would be our normal slew of tickets; in addition, there were also a lot of students who wanted certain features from the site. I didn’t know what to do with this information. I’d gone to one of the PMs and told them, “This is something students are asking for…a lot. Is there some way we can fix this?”
“How many students are asking for this feature?”
Answering with this question became the norm. This was frustrating at first, but after learning more about product management and UX design, it made sense. Since I was on the front line of defense on the support team, I had the brunt of all the negative comments and feature requests. But when it came down to it, these tickets were less than 1% of the daily active user number. My bias was clear, and I only had a very small part of the picture in my viewpoint.
I’d been lobbying for a certain feature since I’d started at Udemy, and it finally became a reality after I’d brought the problem to the right people who wanted to do something about it. It took a long time because it takes time to collect enough data. My grasp on data still isn’t as robust as I’d like it to be, but I’ve made sure to always keep track of ticket patterns in a spreadsheet just in case something is worth looking into. This way, whenever that question came up, I could give them the number of requests, how often they came in (daily, weekly, monthly), and the potential impact it could have on our users.
Keep in mind that this was just for feature requests, which has traditionally been a very small part of our tickets — if our team found a severe bug, that generally got fixed right away. We’re the canaries in the mines.
I’m a huge proponent of at least learning the basics of data. You could have the luxury of having a database already built out, and if you don’t, there’s nothing wrong with making your own spreadsheet to keep track (and hopefully you’ll have someone to help turn that into a tool for you). Combining quantitative numbers with qualitative anecdotes (i.e., a quote from a customer talking about how problematic the missing feature is in a ticket) is a very, very good start when you want to make a change.