Beyond the Buzzwords: Jeff Rubenstein on the Real Impact of AI in Education.
What do a passion for philosophy, a stint in software engineering, and a deep-rooted empathy for educators have in common? For Jeff Rubenstein, these elements have been the driving forces behind a career that has not only spanned decades but has also shaped the EdTech industry in profound ways. Now serving as the Chief Product Officer at Doowii, Inc, Jeff opens up about his journey from the lecture halls of academia to the cutting-edge of educational technology. He reflects on the pivotal moments, from the early days of online learning to the game-changing developments in AI and data integration. Jeff’s story is one of continuous learning, unyielding curiosity, and a commitment to improving the educational experience for students and teachers alike. Join us as we explore Jeff’s unique perspective on the future of EdTech and the lessons he’s learned along the way.
Can we start by you sharing who you are, a bit of yourself, your background?
My name is Jeff Rubenstein. I’ve been in EdTech for about 30 years. I started as a software engineer when I was in high school, a long time ago. I went to college thinking I was going to study computer science and physics. But once I got there, I did a complete change of direction. I was in the bookstore, looking at all the books for courses I wasn’t taking, and it hit me. There are moments in your life when things take unexpected turns. I came across a book in the philosophy section by Immanuel Kant, titled A Prolegomena to Any Future Metaphysics. I thought to myself, “I don’t understand a single word in the title. I need to figure this out.” So, I dropped computer science and switched to philosophy. I ended up spending the next 10 years pursuing a PhD in philosophy.
After a long and wonderful decade of studying and living abroad, I asked myself, “Am I really going to be a professor as a career?” I realized that life as a professor, especially in the U.S., can be challenging. It often involves taking one-year jobs in places you might not want to live. I decided that I wasn’t serious enough about becoming a professor of philosophy, and I asked myself, “What should I do now?” Part of my studies had involved pedagogy — I was reading Vygotsky, Dewey, and others — so I thought, “Maybe I’ll go back to the U.S. and get a job in software, just as EdTech was emerging.” I really just fell into it. It was the early days. We were building some of the first online courses, with a little bit of real-time audio, though we weren’t quite at video yet. This was the mid-’90s. I fell into EdTech by accident, thanks to my academic background and some understanding of coding and the early Internet. The rest is history — that’s how I got into EdTech.
My background in philosophy and pedagogy certainly set me on a path where I knew what I wanted to do because, to me — and I think to many people in EdTech — it’s a mission. I felt driven to use my skills to make a difference in the world, and I realized that maybe I could do that in EdTech. That’s why, once I recognized that this could be a career path, I decided to make my career in EdTech.
What do you think is the greatest achievement of the EdTech industry?
I have two. The first is more an achievement for educators, though EdTech played a role — surviving COVID. The entire industry, from K-12 to higher ed, adapted quickly under challenging circumstances, which was no easy feat. Despite ongoing concerns, especially around mental health, the fact that education continued during such a crisis is an incredible collective achievement.
As for EdTech itself, one standout achievement is the LTI standard. Today, integrating LMS tools is seamless, but it wasn’t always that way. Back in 2007 or 2008, I was working at Wimba, which was later acquired by Blackboard. We faced a significant challenge with multiple LMS versions and tools that didn’t easily interface. Each update required extensive testing across various versions, leading to a complex and costly process. Schools often delayed upgrades to avoid compatibility issues, creating a mess that could have spiraled out of control.
The community, under IMS Global (now 1EdTech), came together to create the LTI standard, which solved this problem. Now, as long as both the LMS and tools are LTI compliant, upgrades and integrations are straightforward. This was a game-changer for the industry. LTI has continued to evolve, with the latest version, LTI 1.3, offering even more capabilities, including better communication with LMS and third-party databases. I encourage everyone to explore these standards and get involved in their development because the collective effort of the community has been crucial to our progress.
You mention in your LinkedIn profile that you are an amateur EdTech historian. The spirit of education aligns closely with the spirit of open-source or free software. Are you really idealistic, or have you seen open-source fail at something during your years in EdTech?
That’s a good question. There are several meanings of “open,” and there are endless debates about what “open” means. I’ll try not to dive too deeply into those debates right now. But I will say that there are a couple of big definitions of “open” that have been part of the debate over the years. One interpretation of “open” is “open as in free.” I think that’s become less important over time because people have realized there’s always a cost. Even if you have full rights to the software, hosting, servicing, and supporting it still have a cost. In the world of cloud computing, it’s now easy to buy just the services you need to run the software you need, and you can move it to a different cloud if you want or take it down if you don’t need it anymore. So, the alignment of cost and value has become the norm in many cases. In most cases now, you only pay for what you use, so the question of whether it’s free is less of an issue today.
The other side of it is the “free” meaning that you aren’t locked in terms of your data or contract. I think that issue has also been largely resolved. It’s rare to see any value offering where you don’t have access to your data or where it’s hidden in a black box somewhere. Those companies don’t last in education. So, in a way, the open philosophy has won. No one will buy a product anymore unless they own their data.
Those principles have influenced the industry so much that the characteristics once unique to open-source offerings are now mainstream. In my lifetime, I’ve seen companies go out of business because they tried to lock up schools’ data and extort money from them. The schools finally said, “Okay, goodbye, you’re done.” And those companies didn’t last long.
What’s your vision for a future where AI is mainstream or has achieved widespread adoption and is part of everyday life in education?
AI, I think, is a challenging term because it’s really an enabling technology. It’s like electricity. When electricity first came out, people probably talked about things being “electric,” and that felt meaningful just because it was electric. But in reality, an electric toaster to grill your bread, an electric car to move you somewhere, and an electric light so you can see at night have almost nothing to do with each other, other than the fact that they’re powered by electricity. I feel like AI is in that same boat. It’s an enabling technology that can power lots of things, so it doesn’t make much sense to talk about AI in general. It makes more sense to talk about AI in specific contexts where it can add value.
When I talk to educators, I mention that most people in education have heard of Bloom’s Two Sigma problem. Benjamin Bloom was a researcher in the latter part of the last century. Among other things, he conducted a study comparing students who had a normal classroom experience with those who had a private tutor. He also explored mastery-based learning, but I’ll set that aside for now. The study showed that the average achievement of students with a private tutor was two standard deviations higher than those without — a difference known as the Two Sigma problem. The question in education is how to help all students achieve better outcomes like those with a private tutor, even though we can’t afford to give every student one. That’s where AI can help.
AI can help in two ways. First, it can provide an experience for students that’s more like having a private tutor. While it may never be as good as a one-on-one tutor, it can be better than a class of 30. AI can offer students an agent that answers their questions outside of class hours or understands their needs better, offering tailored activities or explanations.
Second, AI can take over some of the unnecessary work from teachers, making them more efficient so they can spend more time with students. In K-12 education, teachers only have so many hours in a day, and a lot of it is spent on tasks that don’t require such a high level of training, like administrative work. AI can handle some of these tasks, such as lesson preparation. For example, if a teacher has students reading at different Lexile levels or who are English language learners, AI can create material that’s more aligned with those specific needs. AI is quite good at text leveling — taking a passage at one Lexile level and generating versions at different levels. In an ideal system, the AI could know the average Lexile level of a class and generate passages at the appropriate levels with just a push of a button. This is personalization at scale, making the learning experience more tailored to each student.
AI can also assist with other tasks, like creating reading passages based on students’ interests. For example, if a student likes dogs, football, and cake, AI could generate a passage at their Lexile level about those topics, making the lesson more engaging. This kind of personalization, which often doesn’t get done at all, can be handled easily by AI.
Beyond the teacher and student side, AI can also benefit administrators. While Bloom’s Two Sigma problem is focused on students, I’d postulate (without research) a “Rubenstein’s corollary” — the idea that this concept applies to almost any activity involving humans. For example, administrators often handle tasks that don’t require much intelligence. The more they can offload these activities to AI, like communicating with parents, the more efficient they can be. If the system knows that certain parents speak Spanish, for example, every document sent to them should be automatically translated into Spanish.
When calculators and computers became part of our lives, arithmetic skills became less valuable. Are there any skills at risk with AI, such as language learning?
That’s a very good question. I think it’s important for people to learn the basics of reasoning, whether in mathematics, science, or other disciplines. These fundamental skills shouldn’t go away. Beyond that, there’s a lot of grunt work that we could do without. It’s interesting you brought up the calculator debate, which was raging at the end of the last century. But you know, my father-in-law learned to use a slide rule, and back then, no one thought, “We shouldn’t use slide rules” because they understood that you first learn to calculate, and then you use a machine to make it faster and more practical. The same logic applies today. People should learn the basics, but then use tools to build on those skills.
Let’s start by discussing your current company and role. What can you tell us about Doowii?
I’m now the Chief Product Officer for Doowii, and our mission at Doowii is to make educational data accessible, visible, and easy to query, even without the need for data scientists. The idea is to enable ordinary users — administrators, and soon teachers and student success agents — to simply query their data in natural language. Data itself is a challenge, and many schools struggle with data issues, although they are making progress. However, even those who have their data organized often still require a team of data scientists to make sense of it, generate reports, and help schools understand it. Some of this data gets put into dashboards, but these dashboards are often in tools that are not user-friendly for the average person to generate their own reports or understand how to do so.
Our goal is to create a world where the average user can say, “Tell me which of my students are failing in this subject,” or “Tell me which students are likely to fail based on their current performance,” or “Do I have a gender disparity? Are more women than men failing these courses?” — all in natural language. Beyond that, the tool itself might suggest additional avenues of inquiry. For instance, if you ask a question about gender differences, it might suggest looking into gender and class or gender and language, highlighting other potential areas of interest. The aim is to democratize access to insights, making it possible for everyone to use data effectively without relying on data scientists.
What’s the major challenge in data integration that you see today with institutions? You mentioned some are not in a very good place with that, while others are in a very good place. What makes them different?
There are a couple of challenges. The first is that higher education has developed in data silos, largely due to the evolution of the SIS (Student Information System) over the past 60 to 70 years. Now, of course, these systems are being moved to the cloud, sometimes somewhat painfully. Then you have other systems that have emerged, such as various marketing systems and CRMs for recruiting, application systems, LMS (Learning Management System) and its toolsets, and more. Each of these systems often has its own database because they are server-side technologies. Then you may have other systems like career systems, alumni systems, and all sorts of others. Each of these not only holds data but often thinks about its data separately. So the first challenge is making it possible to get this data into something that is at least understandable centrally, even if it’s not a big data lake. In the case of the LMS and its tools, this is getting easier, in part because of OneEdTech, and there are some standards there that are helping to centralize that data. But then you have all the other sources. Traditionally, it’s been very expensive and time-consuming, requiring massive database projects to try and centralize all this data, even partially, and then apply some business intelligence tools.
I think this is getting easier as people realize that it doesn’t have to be a massive big bang solution, especially now that many of these technologies have moved to the cloud. Not all of them have, and some SIS implementations are still heavy. But as these systems move to the cloud, it becomes much easier to just take the data you need. We’re now reaching a point where AI can also play a role because AI can start to understand databases and how they’re structured, eliminating the need for one schema to rule them all.
This is another way AI can help: it can enhance our ability to manage and use this data. I think as we move forward, CIOs and other institutional leaders will realize that they don’t need to embark on a three-year, $10 million project to start mining their data in a way that delivers business value. Given the current budget constraints and declining enrollment, it’s a tall order to invest $10 million and three years before seeing any results. But I’m optimistic that we are in a new world now. It should be iterative, starting with some of the data — maybe learning data and admissions data — and asking, “Are these students likely to be successful?” or “What additional support do they need to be successful?” For example, do we need to resource the writing lab better, or do we need more language support for certain demographics to ensure their success? You don’t want to admit a lot of students only to see them drop out later. I believe there’s a lot of business value that can be unlocked relatively easily, and that’s certainly the direction Doowii is heading as a product.
The biggest challenge, I think, is that leaders right now may have data backing up their intuitions, but only within a narrow slice of the business. I believe admissions officers should care about student support, but they have no access to that information right now. If we truly want institutions to flourish — and we all do, because we all want students to succeed — a more holistic view of the entire student journey is crucial. In business, this is how customer experience is approached. No business would let its marketing team have isolated data, separate from its product team and customer support team. Instead, they think about the entire journey, aiming to create a positive, integrated experience. That’s something higher ed hasn’t really been able to do in the past but can now.
What is realistic in terms of time to value? What’s realistic with a solution like Doowii from the moment you start to getting the first results?
Really quick. We don’t see it as necessarily trying to get every data point on day one. I think the big bang approach has been declining with the rise of cloud technology over time. You can start loading in some data and see business value right away. We’re talking weeks, not even months.
Can you share an anecdote about something an educational institution discovered that they were not aware of or capable of knowing before working with you?
There are some schools that are doing parts of this really well. In fact, some colleagues of mine — it’s been interesting to watch the evolution — because over the years, with a certain number of person-months of work, you could get a small result 10 years ago, then a bigger result five years ago, and now you can get some interesting results in very little time at all. One thing that might be obvious to people, but now there’s actual data to back it up, is that one of the best indicators that a student is going to fail is simply engagement — how much of the work they’ve done by a certain point in the term. If you haven’t completed a quarter of the work by halfway through, you’re probably going to fail, or you have a higher chance of failing because you can’t wait until the night before and cram an entire semester’s worth of material. In the old days, we couldn’t get that real-time data; we had to wait until the term was over. I recall an EDUCAUSE study around 2008 that showed a correlation between the number of clicks and grades, but again, it was way after the fact. Now, we can actually measure how much work a student has done week by week throughout the term. If you’re falling behind and not completing a percentage of the work that’s appropriate for the time period, that’s the best indicator you’re likely not going to have a good experience. It seems obvious, but we’re now at a point where we can monitor this easily.
What’s important to know and to master for someone that’s starting or that is looking for a career in product?
This is just my personal opinion, but I believe there are two critical things that are closely related. First, passion for what you do is essential. You really need to love what you’re doing. I encourage everyone who loves EdTech to join the industry because I think it’s a great field. People who want to work in EdTech generally find their way into it. This passion facilitates the second critical element: empathy. You really need to understand the motivations of all the users. In EdTech, this is challenging because you have many stakeholders — students, teachers, student success advisors or counselors, instructional designers or learning engineers, administrators, marketing and recruiting personnel, and more. Each has different goals and challenges. Empathy is crucial because you need to understand not just their goals in using the product but also what makes their lives easier or harder, their struggles, and how they relate to others. A teacher doesn’t just have a goal; they have motivations regarding their students, pressures, and resource limitations. Empathy helps you understand these dynamics.
I believe empathy, fueled by passion, is what allows you to add value to the process. That’s what the product function is about — finding value. It’s like saying, “If I do this, this person’s needs are met, or their life or job becomes easier.” That’s where you find value. I’ll share a funny anecdote: many of us have the impulse to bring value to the world. One way I do this is by always carrying band-aids in my wallet. It doesn’t happen often, but every three or four months, someone gets a cut and is bleeding — they’re about to ruin their clothes. I hand them a band-aid, which costs me almost nothing, but I might save someone $80 because they don’t bleed on their shirt. That’s $80 of value added to the world at no cost to me. In product management, you can do the same — find ways to add value through a bit of code or an interface that helps someone learn, teach, or assist a university. The more value you bring, the better you’ll be at product management.
Everyone who has asked me for advice, I’ve told them to put themselves out there. Maybe the first job isn’t the perfect fit, but EdTech is a self-selective community. People are in EdTech because they want to be in EdTech. If you have the passion for it, you’ll find your way. I also want to put in another plug for OneEdTech because they do great work. It’s an important group of people, and if your organization isn’t involved yet, I’d encourage you to join and help move the whole community forward. Progress may be slow and measured, but 15 to 20 years later, I can tell you it’s a different world.
What would be practical advice on actually keeping in touch with that variety of stakeholders in education?
Mostly, ask for their time. Whenever I’m in a product role, I try to be as close to the users as possible. I want to be in their daily lives — visit their schools, talk to them, ask about their problems, and how I can help fix them. I try to really put myself in their place, which is where empathy comes in.
It’s also about experiencing what they do. I make it a point to always be learning something — whether it’s a language, martial art, musical instrument, or whatever — because when it comes to EdTech products, the more you put yourself in the place of the learner or teacher, the more you understand what it’s like to be in their position. You can then identify what’s not a good experience and think about how to fix it.
What are the most valuable EdTech events that In the market today in our space?
Certainly, the big conferences are a must. You should attend EDUCAUSE or ISTE You can get a floor pass for either free or just a few dollars, walk the floor, see what the exhibitors are doing, and talk to them. I always encourage anyone interested in entering the EdTech space to do this because it gives you a sense of what companies are out there. There’s quite a wide range — you have the back-end technology companies like SIS vendors, the content players (basically the publishers and publishing products), the tool environment around the LMS, analytics providers, and more. It’s a good idea to understand where you want to work. You might prefer working in the database and back-end world, or in curriculum for the publishing world, where you have math, reading programs, or platforms for delivering subjects like math, reading, or physics. You might feel more comfortable in a startup environment or a big company. It’s really valuable to see what’s out there and what you like. That’s the advice I always give to people who ask me how to get started in EdTech.
The EdTech Mentor is a 27zero publication. Let’s start a transformative EdTech marketing conversation today.
Read more!, Growth experts insight you.
The EdTech Mentor Interview Series.
Conversations with the product, marketing, and startup leaders shaping the EdTech space.
