The unseen dangers of AI and personalization in UX

Artificial intelligence and personalization have changed the game for user experience. Much of the acceleration of this trend could be attributed to the rise of the aptly named iPhone, which convinced us that our identities could be tied to our smartphones. And our phones these days do know a great deal about us; apps can utilize that information to customize an experience for you, wherever and whoever you are.

It’s hard to deny the benefits. Smarter apps can save us time, the most precious resource of all, and free up our mental capacity to focus on what’s important. Personalization can validate the needs of marginalized people and maximize every individual’s capacity to be more productive at every turn.

On the flip side, computational advancements of this scale have raised our expectations for tailor-made experiences, and reduced our patience for ones that aren’t made “just for me.” As a creative director, I’m beginning to notice this shift in business and user expectations. A product has to do its best to meet the needs of a broad sample of people, and often that means we end up with something that is not ideal for some, but best for all. I’d like to believe that at some point, every customer was aware of this and kept their expectations down to earth. But things have changed. “But, what about me?” was at one point the anthem of the most selfish among us, but I wonder if this design trend is making it the anthem of us all.

This is a trend that magnifies the individual at a time when we are more divided and less empathetic than ever. What ideas can we as UX designers bring to the table to increase our collective capacity for empathy, humility, and awareness of others? In what ways can we use AI and personalization to bring about more altruistic outcomes?