Sherlock Holmes and the Immersive Storytelling Adventure

As the “Sherlocks” were animatedly discussing exactly which theory of time travel was at work in the case they were trying to solve, my nerd-self did a little skip of joy: this was just the kind of debate my colleague and I had envisioned while designing the User Journey for this particular iteration of Sherlock Holmes & the Internet of Things.

The participants of this immersive storytelling game were tasked with coming up with the best possible solution to a complex case revolving around CERN’s Large Hadron Collider in Geneva, time travel and wormholes (the title of the event was, as a matter of fact, “Sherlock Holmes: A Trip Through The Wormhole”).

The occasion was Droidcon Italy 2016, a networking event for developers and companies working within the Android ecosystem, organized by Synesthesia.

Okay, let’s back up a moment now here. So, what exactly is an immersive storytelling game and what is Sherlock Holmes & the Internet of Things (aka #SherlockIoT for brevity’s sake and hashtag-related purposes)?

What is SherlockIoT?

Sherlock Holmes & the Internet of Things is an ongoing prototype developed and run by the Digital Storytelling Lab at Columbia University School of the Arts that explores new forms and functions of story, launched by its director, Lance Weiler.

It involves a massive online and offline collaboration across different countries, and its main objective is to build a massive connected crime scene with the aid of smart objects.

Several posters for Sherlock Holmes & the IoT events around the world

If Escape rooms and Alternate Reality Games might spring immediately to mind, allow me to swiftly tweak that mental image: SherlockIoT is actually much more, for it is conceived as an open R&D space that experiments with shifts in authorship and ownership of stories and usually asks players to actively participate in the creation of the storyline, instead of merely “solving” a case. The fact that the game requires a creative effort on the participants’ side, combined with the technological input provided by the smart objects and the development of the story across various media platforms, contributes towards the immersiveness of the experience.

Various “meetups” have sprung up all across the globe — in France, Italy, Brazil, Australia, just to mention a few — each experimenting their own version of the prototype.

A Versatile Prototype

In Italy, Turin and Rome are experimenting with SherlockIoT, with events organized respectively by Crossmedia Factory and In4mal.

In our experience with SherlockIoT, my colleague Sabrina Giacardi and I (cofounders of Crossmedia Factory) particularly appreciated the more or less blank canvas at our disposal on which we could sketch out the outline of a story — with more or less detail, depending on the type of event — and subsequently granting the participants the agency to “complete the picture” to their own liking, negotiating it with their fellow teammates.

“Sherlocks” investigating in Piazza Castello (Turin, Italy), defying the bitter cold to solve the case

The detective story narrative provided a strong backbone on which we could build, each time, a storyline to suit the particular environment which hosted our event. The prototype thus proved itself to be highly adaptable, and the participants were asked to solve, in turn, radically different cases:

· a series of crimes connected to an obscure cult, masterminded by an all-seeing, all-knowing serial killer (Torino Film Festival, November 2015)

· a time-bending mystery revolving around wormholes, paradoxes and artificial intelligence (Droidcon Italy, April 2016)

· a steampunk-infused, historically-based scenario centring around the 1884 Turin World Fair (Torino Mini Maker Faire, May 2016).

In Rome, In4mal, a network of interaction designers and storytellers, came up with a narrative which was based on Doyle’s short story The Adventure of Dancing Men and helped us develop the storyline of the event at the Torino Mini Maker Faire; in Paris, France, Learn Do Share Paris used Doyle’s short story The Adventure of the Six Napoleons as their framework.

SherlockIoT lends itself therefore to a variety of approaches, constantly sparking new ideas as the organizers of the various meetups compare notes on their experiences, sharing the outcome of this ongoing experimentation.

A Trip through the Wormhole

On the occasion of Droidcon Italy 2016, we decided to go with a sci-fi scenario for our backstory, partly inspired by the Android Logo (a little robot) and partly because we thought it would fit in nicely with this kind of high tech-centred event.

The narrative consisted in the time-warping adventure I previously mentioned, and we used Virtual Reality and Augmented Reality to create clues, along with more traditional, hard-copy material.

Thanks to UDOO, a family of open source single board computers made by the Aidilab team, we also had Artificial Intelligence, in the form of a cute little robot named “Mario” which worked via voice recognition technology. The participants interacted with the robot in order to gain information pertaining to the case.

Players interacted with Mario the Robot and used Augmented and Virtual Reality to find clues

The Case of the Hot Air Balloon

At the Torino Mini Maker Faire, we designed a story which was partly based on the history of the location which hosted the event, Toolbox, a former foundry designed by famed civil engineer Giovanni Porcheddu, and on the industrial revolution Turin was undergoing during the late 19th century.

We also drew inspiration from the whole “digital artisan” movement which a lot of makers subscribe to, and wove steampunk undertones into the story, evoking marvellous machines and alternate history scenarios; the title of the event was “Sherlock Holmes and the Case of the Hot Air Balloon”.

In this particular iteration of the project, we not only used VR and AR, but — for the very first time — we got our IoT (Internet of things) in our SherlockIoT! We managed to pull this off thanks to the students at ITS per le tecnologie dell’informazione e della comunicazione, who were attending a workshop organized by Officine Arduino.

They came up with a “secret knock box”, a sort of treasure chest which could only be opened with a certain sequence of knocks, and a punch card-reading box, which performed a sequence of actions depending on the punch card inserted.

As for the steampunk side of the story, we also collaborated with the steampunk collective Vaporosamente, swiftly finding out that Doyle’s oeuvre is vastly cited in this subculture.

Players used smart objects, Augmented and Virtual Reality to solve the case

Looking through the Magnifying Glass

I started off by talking about the expected — the way we envisioned the public would interact with our experience; however, what maybe is even more exciting in an immersive storytelling game, is the unexpected.

The narrative for the Sherlock Holmes and the Case of the Hot Air Balloon event was quite complex: nevertheless, three kids aged between 13 and 14 proved to be on a par with the other, adult participants, when it came to solving puzzles and processing vast quantities of information in a short period of time.

This gave us an insight on the potential educational and cultural impact this kind of event could have on young people, and on how it could be used as a powerful learning device.
Kids were really quick on the uptake and required minimum supervision during the game

Moreover, moving onto a wider public, this kind of experience could also be valuable to improve users’ problem-solving skills, aid team-building efforts and help promote a country’s cultural heritage.

A further instance of things taking an unexpected turn took place during the Sherlock Holmes event organized on the occasion of the 33° Torino Film Festival: as soon as the players on the team at the National Cinema Museum realized that they were being spied on via webcam by a team of storytellers on the other side of town, they immediately tried to contact someone on a third team to inform them of their discovery. It was an interaction we hadn’t anticipated, and it gave us an insight into how this kind of experience could help people to develop a collaborative attitude towards one another.

Put on your Deerstalker and join in

If you’ve made it till here, you may be interested in following the progress of this experimental project, or in taking part in one of the local meetups and events.

Here are a couple of useful links to this avail:

Official site —

International Facebook Page —

Italian Facebook Page —

Learn Do Share France —

If you’re interested in actively collaborating towards the next edition of the experiment, you can enroll in this years’ FREE open Massive Online/Offline Collaboration (MOOC) open to participants who are interested, willing and committed to being active collaborators. Please find more info here: