Prototyping beats guessing: A process that can remove doubt from even your largest scale experiment
What a Philly Tech Week presentation taught the Lab about its own processes framework.
When we conduct experiments in the Lab, I often wonder how others approach the iterative process and how they explain it to others. Recently Faye Teng, the lab’s UX designer, and I attended a talk during Philly Tech Week, a 10-year-old innovation festival in Philadelphia hosted by local online publisher Technically Media, that gave us a chance to learn from other local companies who experiment and prototype, and compare our process to theirs. The presentation was by Bluecadet, a local experience design agency, and focused on their five-year, $15 million dollar expansion and renovation project with Philadelphia’s Independence Visitor Center. The site opened in 2001 and welcomes 2.5 million visitors each year.
Most notably in their presentation, From Architecture to AI: Experiential Planning at the Visitor Center, Bluecadet managing director Sara Pasch and project leader Pete Hall emphasized that one of the most important insights that came from their on-site interviews with tourists was that “visitors expect spaces to be as dynamic as the web.” This idea intrigued us since some of our experiments attempt to deliver relevant local news based on your location.
This observation by the Bluecadet team transformed what started as a project to build a two-sided digital information display visible to both staff and visitors into a more immersive experience. That included interior architectural changes as well as a 42’ concierge wall powered by machine learning where information adapts to light, distance and motion. Faye and I took notes and distilled them into a few observations that resonated with us about how to successfully approach experimentation. We hope other innovators and teams tasked with experimentation will find them useful as they continue to explore new ideas for local news products.
Prototyping is important (because “prototyping beats guessing”). Bluecadet built a full-scale production model of the interactive concierge wall in their studio to see how the large-panel displays, Microsoft Kinect cameras, and other elements worked while developing the project rather than approximating them until the final build. They believed the concept needed to be built at full scale to be fully explored or evaluated.
Weeks later, members of their project team were still working on-site to calibrate the display screen color levels after the wall’s Memorial Day weekend debut.
Be willing to embrace the larger question. The original scope of Bluecadet’s project was to create a two-sided information wall to enable visitors and staff to see the same information, which they thought would make it easier for staff to assist visitors. They also believed the wall could be dynamic and draw visitors to the information desks. Bluecadet’s user testing and in-person interviews revealed a more complex set of problems though: there was a lack of clear purpose for the originally proposed wall. They also noticed that elements of the existing physical space lacked definition. The development of the wall wouldn’t address one of the larger issues, the flow of visitors through the building. As a result, the team took a step back and focused on refining a plan for the ideal visitor flow through the space and reconfiguring the interior. Instead of waiting to have everything finished and set up to test them together, each new feature, including the digital signage, the touch wall, and the concierge wall, was tested and updated individually throughout the entire design and development process. Eventually, the team recommended moving the public restrooms, expanding the gift store and installing better signage to direct people to help desks.
Rely on others with additional expertise when you can: The Bluecadet team reached out to architects and developers when they needed expertise outside their skillset. For example, the team spoke to an architecture firm to help understand the construction needed to make changes to the hallway, restrooms and gift shop. They also worked with outside software engineers to bring their information design for the touch wall to life.
It doesn’t have to be complete: Bluecadet and the Independence Visitor Center both shared a desire to make the new display walls accessible to as many people as possible. The images on the digital displays could adapt to many heights and abilities, so adults, children, and people with disabilities had access. However, the images and text were English only. The team’s hope is that visitors who need more help will engage with staff trained in several languages.
We left the session even more comfortable in our belief that prototyping beats guessing as we continue to carry out experiments.
We felt more confident in our approach to building full-scale prototypes of our experiments and testing them first. All of our experiments in the lab have been full working drafts first. The GroundSource platform enables us to send working drafts of our text messages to ourselves and interested parties before launching an experiment, and TestFlight allows us to share early versions of apps with collaborators and testers before sharing them with outside users.
We found Bluecadet’s practice of reaching to other subject matter experts familiar. In the lab, we’ve reached out by email, phone or video chat to others who can provide insight and expertise we don’t have on hand to help us avoid common pitfalls and be smarter along the way. In particular, we’ve reached out to people with experience sending postcards in the mail to drive participation and other newsrooms that have experimented with texting election coverage to their audience. It’s important to realize when and how to seek assistance.
Throughout an experiment’s design and development process, we also separate pieces of the experiment or app features and test them separately, like Bluecadet. It could be a paper prototype, a digital wireframe or a coded interactive prototype — but we’re constantly conducting user testing to gather impressions and usability tests to make sure our experiments are easy to understand.
We also often find ourselves wanting to continue to add features and information along the way to make the best experimental products possible. This is when we regularly remind ourselves that we’re conducting an experiment and our goal is to see if our hypothesis is correct — not to necessarily build something for long-term use.
It didn’t hurt that both Bluecadet and the Independence Visitor Center’s effort to better equip Philadelphia’s tourism industry and our ongoing research are similar in their desire to support the information needs of people located in and around Philadelphia.
Those working in the journalism field should take advantage of opportunities to attend non-industry conferences and other similar types of events, including local showcases like Philly Tech Week, as a way to learn things that could apply to their work and open their minds to new ideas. It can also serve as a valuable reminder of wider best practices for their own work.
NOTE: Faye Teng contributed to this post.
The Lenfest Local Lab is a multidisciplinary product and user experience innovation team located in Philadelphia supported by The Lenfest Institute for Journalism.
The Lenfest Institute for Journalism is a non-profit organization whose mission is to develop and support sustainable business models for great local journalism. The Institute was founded in 2016 by entrepreneur H.F. (Gerry) Lenfest with the goal of helping transform the news industry in the digital age to ensure high-quality local journalism remains a cornerstone of democracy.
