Like stealing cookies from a robot: Lessons from a pilot study
Austin L. Toombs, Ahreum Lee, Zhuang Guo, Abbee Westbrook, and Yuqing Wu
Our lab (www.c-cilab.com) recently ran a quick pilot study as a proof of concept for a larger study we’re hoping to run about social robots. Specifically, we’re interested in ways we can subtly adapt existing robots to perform more as social agents.
The goal of this pilot study was to understand the feasibility of using the delivery robots we have on Purdue’s campus (Starship Robots: https://www.starship.xyz/) in some creative ways to get people to meet each other for impromptu socializing. We had hoped by doing this that we’d:
- Start to build up a sense for how these robots could function in some potentially awkward social situations, such as by performing social agent roles in a tacit way like implicitly introducing people just by being present;
- Experiment with methods for engaging in lightweight modifications of existing “disruptive” infrastructure to create moments for deeper relational — rather than transactional — interactions;
- To see how participants might react to the idea that a robot (or robot-facilitated delivery company, to be more accurate) would generously provide them with free cookies for no specific reason.
For this first, quick portion of the study, our plan was to order cookies to be delivered to a public location on campus and to see how participants would react. Would they share? Would they discuss the robot and/or delivery company with each other? Would any of them stick around? If the robot was late, would they wait for it?
While thinking through this pilot study, we briefly considered what would happen if someone stole all of the cookies we were planning on having our participants share. WE SHOULD HAVE THOUGHT MORE DEEPLY ABOUT THIS POSSIBILITY, because that’s (almost) exactly what happened.
Timeline of events
Our original plan
Our plan, as loosely structured as it was for this pilot study, was as follows:
- Distribute flyers announcing some kind of event where people might show up and be given some food.
a) Ensure that the event foregrounds the food, and not necessarily social participation. If we made it sound like we’d expect people to talk to each other, and then they did talk to each other, then that wasn’t going to be very interesting to us.
b) Ensure that the flyers are noticeable, but also ambiguous. Who is running this event? Is Starship in charge? Is this someone else’s event? The risk here is that people might find this too sketchy. That’s an okay risk for a pilot study, but not for a real, full-blown study, so we’ll have to think about that strategy moving forward. - Select participants for the pilot.
a) Our plan was to use the survey distributed through the flyer to help us select participants. Originally we were going to limit participation to 10, just in case this became a popular event. For typical events of this nature we would not have expected much interest, but because Starship is still relatively new to our campus and there is still some interest among students, this could have been enticing enough to become popular.
b) Selected participants would be messaged about the precise location.
c) Those not selected would be put on a waitlist in case other dropped out. - Order cookies through the delivery service, to be delivered at the meeting location around when we said the event would take place.
a) It is not possible to pin this delivery time down. Starship provides estimates, but those estimates are often fairly fuzzy estimates.
b) Our hope was that if the robot was early, we could make it wait. We pre-piloted this the week before and were able to comfortably make the robot wait for five minutes. In hindsight, we should have pushed this as far as it would have let us.
c) If the robot was late, our main concern was that participants would leave too soon, being unwilling to wait for a robot that may or may not show up. - To account for this trust issue, we built into our plan that we would text participants when the robot was on its way.
- Then we’d observe what took place. One student, Abbee, took on the role of “participant observer” in that she would pretend to be someone who saw the flyer, rather than someone helping to run the study. The rest of us would stand back and congregate in an inconspicuous location in order to watch and be able to unlock the Starship robot from a distance.
Tuesday, November 5th, 2019
After creating a Qualtrics survey with a response limit of 10 phone numbers, we distributed 15 flyers in several prominent places on campus.
These flyers led potential participants to the survey
We monitored the survey responses afterward.
Wednesday, November 6th, 2019
11am. Austin (Dr. Toombs) texted each participant (though they were not yet told who was texting them):
Hi! This is an update about the robot cookie party that you signed up for. The robot cookie party will take place on the Hello Walk on the Purdue Memorial Mall at 2 pm today! We will text you with more updates when the cookies are on the way. If you can no longer participate, please let us know. See you soon!
https://goo.gl/maps/Xpd3hjBfAhAjEPwHA
1:30pm. We placed the order for the cookies. The original estimate was that they would take 40 minutes to be delivered. At 1:34pm, we sent:
“The robots are coming! The robot x cookie party is on schedule, with cookies arriving at the above location in 26 minutes.”
Shortly after placing the order, the estimated arrival time dramatically decreased and we had to rethink our strategy. We waited nearby (approximately 20 meters away) so that we could intervene if people started to congregate and were unsure of what to do. We also used this location to take notes.
When the robot arrived 15 minutes early, we asked Abbee to stand at the meeting spot and wait by the robot and we sent the following message to the remaining participants:
The robot was early! It’s waiting in the center of the Hello Walk.
Our new strategy was to wait as long as the robot would let us and unlock the robot at the last minute with (hopefully) some of our participants in place.
1:45 pm. Abbee waited for other participants near the robot
At this time, we began to be pressured by Starship to pick up our order. This is understandable, as the robots are often in high demand. Before knowing how long the robot would wait, Austin received an automated call informing him that the robot would only wait 4 more minutes.
1:52 pm. One participant (not the cookie thief) texted Austin, “Can I open it?” to which he replied, “Yes” and “Please do!”.
At this point, Austin instructed Abbee to wait for 30 seconds and, if nobody arrived, to go ahead and open the robot. Austin then unlocked the robot remotely.
1:55 pm. P1 walks up to Abbee and the robot and asks, “Did you order something?” Abbee replied, “No, I saw a flyer about free cookies.” P1 then took the bag of all of the cookies and began to walk away. He then turned around, returned the bag of cookies to the robot, and said to Abbee, “Wait, let me take a video of me taking them out.” He then recorded himself taking all of the cookies (presumably to post on social media later), and quickly left.
From our position (perhaps too far away?), the rest of us thought he was taking a bag with a single cookie in it.
Within a minute, P2 walks by and asks Abbee “Did he take all of them?” Abbee told her that he did, and P2 kept walking.
1:57 pm. Austin sent the following text:
Thanks for participating! I hope you were able to share some cookies with some new friends at 2 pm!
One participant, possibly P2, replied, “I wish I had some cookies to share!” Austin asked “Oh no! What happened?” but did not receive a reply.
2:14pm. Austin sent the following debrief message, to account for the fact that we were unable to debrief any participants in person.
Hi! Sorry for the extra messages. We wanted to let you know that this was a quick pilot study that we conducted to see how these starship robots might be used as a way to get people to talk to each other. If you would like to discuss anything with us about this pilot study, please let us know. Otherwise, thanks for participating (and sorry that many of you were not able to get cookies).
Later that afternoon, at 4:28pm, we received a text from P1!
“What shop were those cookies ordered from?”
“They were ordered from Port Cafe. I hope they were good! I have not tried them before so it was just a guess.”
Are you sure those cookies were from Port Cafe?
Yes, that’s where my receipt says they were from. Why? Was everything okay?
I’m just curious. Yes, I liked the cookies. They were awesome
I’m glad that you enjoyed them.
Lessons Learned
Late robots might be better than early robots, especially if they’re busy.
The main issue we had with this pilot study was with the timing. If the participants would have had to wait a few minutes (our original intention), then more of them would have been around the meeting location and the cookie thieving might not have occurred. We wanted the participants to have to wait around 5 to 10 minutes for the service so that they could have time to speculate about the flyer, about the event, about the robot, about the cookies, or about anything.
Unfortunately (?), the delivery bot was faster than estimated. We tried to make it stick around as long as possible, and there were initially no clear indications about how long the robot would wait for us to take the cookies out of it. But we eventually received a phone call informing us that it would only wait 4 more minutes, which would have still been 6 minutes before our event was planned to start. At this point we instructed our participant observer to wait 30 seconds before opening the robot. It was within this time window that our cookies were taken!
Being present isn’t enough to stop a cookie thief
Abbee was told to act as though she was not “in the know,” with the assumption that we’d then debrief everyone afterward while discussing their reactions to the pilot study. The cookie thief left so quickly that we were not able to debrief them and they are still at large (you have been warned). If we had considered cookie thieving more seriously ahead of time, Austin would have let Abbee know that “breaking character” would be totally appropriate to prevent the thief from getting all the way away, and that that would be a great opportunity to talk about what his assumptions were about the event!
Other things we’d like to try
We think making sure the robot was closer to on time, or maybe purposefully late, would have dramatically impacted the outcome of this event.
We are also curious about how the event would have unfolded if Abbee had not been there, though we think the cookie thief would have then been even more likely to take place. However, without someone there, it would be interesting to see how they begin to congregate. We’d need to know this before knowing how feasible this kind of impromptu, subtle, tacit, robot-influenced socializing activity could be.
Special Thanks
To members of the C-CI Lab and to Colin Gray for consulting.
