Could an AI Agent Become One of Your Coworkers?
Two MIT researchers, using a custom experimental platform, find human-AI teams with the right “personality pairing” can outperform humans.
By Peter Krass
What’s the most effective way for AI and humans to work together?
Personality pairing, finds a new paper by two MIT researchers.
In a large experiment, the researchers explored how collaboration between humans and AI agents can reshape teamwork, productivity and performance. The researchers randomly assigned personality types to AI agents, put them on teams with human workers, and then gave them a task to complete together. They also randomly paired human-only teams to complete the same task.
The results showed that individual humans, when assigned to work with AI agents, enjoyed a 60% productivity boost over those working without AI help. And when AI agents with certain personality traits were paired with humans, workflow and output were both enhanced.
The two researchers are Harang Ju, a postdoctoral associate at MIT Sloan and the MIT Initiative on the Digital Economy (IDE); and Sinan Aral, an MIT professor and Director of the IDE. They conducted the experiment last October, and their work — involving 1,800 human-AI and human-human teams — appears to be unique. So does their use of “agentic” AI, agents that can do the same tasks as human teammates.
As part of this IDE-sponsored project, researcher Ju developed an online experimentation platform known as MindMeld. The platform enables real-time collaboration between humans and AI agents. It also allows pairings of humans and AI to be randomized. Further, MindMeld lets AI agents perform most of the same actions that humans can do online, including sending chat messages, editing copy, selecting images and submitting ads.
“By recording every keystroke on real tasks — including chats, sentiment and edits — we have a high-fidelity set of rich data about the differences between human-human and human-AI teams,” Aral says.
Filling Gaps
The project was driven by the researchers’ desire to fill three important gaps in our understanding of AI’s impact on productivity and collaboration:
· Now that AI agents are multimodal and can take actions independently, what is their proper role in collaboration?
· When humans collaborate with AI, how do work processes and collaboration patterns change?
· Can human-AI collaboration be improved with model engineering? If so, when is the best time to do this? And how do various strategies compare?
To answer these and other related questions, the researchers selected 2,310 human participants, all from the United States and stratified by gender and ethnicity. Then the researchers randomly assigned these participants to either human-human or human-AI teams. To avoid bias, the participants were not told which team they’d been assigned to.
All the teams were given a common task: produce ads for a think tank. Completing the task took about 40 minutes, after which participants also completed a 35-item post-task survey that included questions about their personality types. Each participant was paid a nominal fee of $9, and two participants were also later awarded $100 each for producing the best-performing ads.
People in the human-AI teams were paired with an AI agent based on a version of GPT-4o. This Generative AI model, supplied by OpenAI, can process and generate text, images and audio. (The “o” in the name is short for “omni,” or “all.”)
Personality pairing
In one intriguing aspect of the experiment, the researchers randomly assigned temporary personality types to the AI agents. They did this expecting that certain AI traits, when paired correctly, would complement human personalities and enhance their collaboration.
For this task, the researchers relied on what are known as the Big Five personality traits: openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism/emotional instability. The underlying idea is that an individual’s personality can be expressed by their unique mix of these five traits.
Next, the researchers randomly assigned various levels of each Big Five trait to the AI agents. They trained the AI agents using open-source prompts that generate a detailed description of individuals with various blends of traits.
Also needed was a Big Five traits profile of the human participants. So each person in the experiment was given a 10-point survey that measured their personality traits. This survey, which people completed before doing the task, allowed the researchers to essentially mix and match various human and AI personality types.
The Day After
Once the lab portion of the experiment was completed, the researchers conducted a field evaluation. They first obtained quality ratings of the ads created by the experimental teams from both the human participants and AI agents. The ratings evaluated the overall quality of ads, image quality and estimated click-through rates. The ad copy was also evaluated by both humans and AI for clarity, concision, grammar and other qualities.
Next, ads created by the experimental teams were posted on the X social media site, where they generated nearly 5 million impressions. After the test period, the researchers collected data on the ads’ performance, including click-through rates, view-through rates, view durations and cost-per-click metrics.
The researchers had a large number of interactions to analyze. The teams collectively generated some 180,000 messages, 63,000 image edits, 1.9 million copyedits, 10,000 AI-generated images, and over 11,000 ads.
The researchers also analyzed the experiment’s logs for communication, collaboration and workflows. Here are some of the performance outcomes they found:
· In the human-AI teams, communication increased by 137%.
· When collaborating with AI agents, humans focused 23% more on text and image content generation messaging, and 23% less on direct editing.
· Humans assigned to work with AI agents sent 23% fewer social messages than did those in the human-human teams.
· Within the set time period, individuals in the human-AI teams sent 60% to 73% more ads than did their counterparts in the human-human teams.
· When conscientious humans were paired with open AI agents, image quality was improved. But when extroverted humans were paired with conscientious AI agents, text and image quality both dropped; that also lowered the number of clicks these ads ultimately received.
In the future, Aral says, “AI will be able to infer your personality, then adjust itself.” He believes the potential for human-AI personality pairing to boost productivity and performance alike is great. Depending on your personality type, that could be welcome news.
Read Harang Ju and Sinan Aral’s full research paper, Collaborating with AI Agents: Field Experiments in Teamwork, Productivity and Performance.
Peter Krass is a contributing writer and editor with the MIT IDE.
