What’s Next In UX: Episode 1
Automation, autonomy and UXR with John Deere
“This is not your grandfather’s tractor.” — Tom Carpenter
Tom Carpenter leads the UX Research Team within John Deere’s Intelligent Solutions Group. He joined the Global UX team at Ipsos recently to discuss automation, autonomy and trust — how to grow it, how to measure it, and how to recalibrate trust when, inevitably, your automation systems fail.
While Ipsos UX has offices and researchers across the country (and all over the world — 90+ countries!), here in the NYC office, I do a lot of work with the big tech companies and startups of Silicon Valley. And, while it’s not news that technology is impacting every industry, I’ve never considered the impact that it’s having on the farming industry. As you’ll learn from this episode, the farming industry may be at the forefront of developing trust in automated, autonomous systems, and driverless vehicles. While many fear this technology will displace workers, John Deere is using it to do just the opposite by turning operators into supervisors — effectively promoting them. Wow.
As with any AI systems, whether you’re relying on a bot to arrange your travel or to reliably answer your search query, trust is key. Tom confirms. His team focuses on trust but he stresses an important point:
It’s important to take a step back and remember what problem we’re trying to solve. We exist to help feed a growing world. That means every seed that gets put in the ground has to have its best chance of reaching its full potential.
That’s one big mission statement! As a researcher, I immediately thought about how this impacts customer centricity at John Deere — namely who are their customers? A few groups emerged in my mind — the end customers, aka “a growing world”, the farmers who buy machines from John Deere, and the people who actually operate them. How does Tom and his team prioritize needs across these user groups? And how do they think about tackling such a big problem to solve?
It seems feeding a growing world = very high-tech + very skilled operators. And while John Deere is working on the high-tech part, it’s the skilled operators that present the challenge. In fact, one of the biggest customer pain points is finding enough people to do these jobs year in and year out. Farming is seasonal, which means part-time labor and long hours. Access to highly skilled labor throughout the year is inconsistent.
For Tom and his team, automation and autonomy is the solution to this problem. But this type of solution requires increased levels of trust. To build trust, we must learn how to measure it.
Measuring trust
- Initial use before — measures how operators think the automation is going to work. Tom’s team uses the TOAST or trust of automated systems test.
- Initial use after — Tom says many operators have a heathy dose of skepticism. They need to see it to believe it. After the initial use, Tom’s team measures and ends up seeing a spike in trust scores.
- Extended use — Crucially the team measures attitudes after they've had the machine for a few weeks. Have scored plateaued, have they gone up or down?
But measuring trust isn’t enough. It’s important to balance these subjective measures with behavioral measures — how shifts in trust impact what operators are doing. Balancing some of the subjective measures (like trust) with behavioral measures — what operators are doing — is important. For this reason, Tom says his team measures a lot of other things out there with customers in the field. He’s talking about an actual field.
Auto-track for example makes the turn for you. It’s all automated. At first, customers got to the end of the field and grabbed the wheel. Once they are comfortable, they take on an active monitoring task. Over time they start to do other things. They’ve established trust. They expect the machine to perform well if not better.
Tom’s team uses eye tracking technology to study operators’ behavior. Operators used to look outwards 80% of the time and look down (doing data entry and higher level cognitive tasks) 20% of the time while in the tractor. Now, that’s flipped on its head. The operators are empowered to get more done with the assistance of automation — transitioning from “machine operator” to “machine supervisor”.
But what happens when the machine fails? What happens to trust? How long does it take to recalibrate and rebuild? These are the questions Tom and his team are currently asking.
Even more questions arise when operators start to make the transition to supervisors. Automation components can help operators be more efficient — i.e., they’re working on other tasks while supervising — but full autonomy means something else entirely. How do we let supervisors know their machine is doing a good job? Do they have to have eyes on their machine all the time? What if the supervisor is supervising not one machine but a fleet of different machines? The questions start to get really interesting as we progress further down the automation journey.
Of course we were keen to understand how the team designs these displays to maximize situational awareness given operators may be multitasking. What are the design decisions involved in effectively signaling critical information to the user when there’s information overload? The Global UX team at Ipsos partners with clients to answer these questions and more.
“Deere” as Tom refers to it, is an old company. Building on brand trust in a 185-year-old company is a lot to live up to. Tom’s advice is at the heart of what is so exciting about research, especially within automation and autonomy:
Fall in love with the problem. Be intentional and design with sincerity. It shines through. The rest takes care of itself. Automation autonomy is new for everyone. We don’t know the answers to everything yet.
