BCI Demo — Emotional and Cognitive Analysis at the Urban Scale

Fred Simard - CEO @ RE-AK
7 min readAug 22, 2023

--

Sainte-Catherine Street is the main of downtown Montreal. It’s a bustling street where spontaneity thrives. Aligned with our aim to capture the emotional and cognitive identity of Montreal, Canada, we conducted an urban exploration activity during which we traversed a segment of this street. We now present our preliminary findings in the form of a micro-report.

THE EXPERIENCE

The activity took place on Friday, June 30, 2023. Six participants (limited by the available equipment) signed up through our Meetup page, and we gathered at Dorchester Square around 7:15 PM. It took about ten minutes to set up the participants with the equipment, and the activity lasted around 1 hour and 10 minutes.

In the spirit of the city’s spontaneity, we didn’t have a specific plan besides walking from Dorchester Square to Place des Arts. As luck would have it, Montreal had a surprise for us, as the Montreal International Jazz Festival was in full swing at Place des Arts. Along the way, we also took a break at ‘L’Anneau’, an artistic installation that consists of a large ring suspended between two buildings.

Fig 1. Path Diagram of our Experience

OUR HYPOTHESES

This is our first urban exploration activity, strictly speaking, but based on our previous experiences, we have the following hypotheses:

  • Whenever we move around in a public place, passersby approach us to discuss our equipment and satisfy their curiosity. These social interactions will likely bring a sense of joy to our participants.
  • Navigating through a crowd requires alertness and reacting to people’s movements. We predict that moving through the crowd will lead to a heightened level of physical awareness, a decrease in engagement, and potentially some frustration.
  • In general, we hypothesize that the level of engagement will be lower compared to an activity involving audiovisual entertainment, such as video games or a trip to the cinema.

Within the scope of this micro-report, we won’t be able to assess the third hypothesis. However, we commit to revisiting it as part of our larger study Mindscape-Emotions, which gathers and compares a variety of entertainment experiences.

Furthermore, it’s important to note that the small number of participants limits the scope of our findings. This dataset will ultimately be combined with other urban exploration activities to form a more representative set.

OUR RESULTS

The heatmap (Fig 2, A) of our participants’ positions in space accurately reflects their journey. It’s relatively easy to distinguish the segments where we were moving and where we took breaks, as the latter are broader and shaded in light yellow.

The experiential map (Fig 2, B) provides a visualization of participants’ states of mind. Engagement and physiological arousal are consistently present, but their relationship varies. Joyful segments are noticeable (the mind states are defined in Figure 4).

We’ve divided the experience into columns to simplify the analysis. With a few exceptions, the group’s movements progress from left to right.

Figure 2. Heat maps of movements (A) and experiential map of mind states (B). Each column (I to V) represents a segment of the experience discussed in the text. The color code for the mind state map is defined in Figure 4.
Figure 3. Mind states isolated by our algorithm. From left to right, top to bottom: Orange, state dominated by physiological arousal. Purple, state dominated by cognitive engagement. Yellow, blend of arousal, cognitive engagement, and joy. Cyan, blend of arousal and engagement. Gray (2x), State below the median, neutral. Only the emotions of joy, surprise, and anger were evaluated, but the algorithm did not capture the latter two, hence they were not experienced.

EXPERIENTIAL MINDSCAPE

In Column I, the denser cluster of points (Fig 2, A-I) is located at Dorchester Square, representing the beginning of the experience capture. During this moment, we took some group photos (Fig. 4).

On the experiential map, there’s a presence of nearly all the mind states. There’s engagement, with or without arousal, as well as traces of joy (Fig 2, B-I). There are also signs of neutral expression, suggesting there might have been some waiting during the setup of the experience.

Next, we observe a segment dominated by physiological arousal (Fig 2, B-I, in orange) as our group moves towards the official starting point of the urban journey, Saint-Catherine Street. We frequently notice a dominance of arousal at the beginning of experiences. Our interpretation is that this state indicates people are not fully engaged yet, probably due to some excitement, making them more distracted than engaged.

This mind state gradually transitions into a blend of engagement and arousal (B-I, in cyan) as our group immerses into the urban exploration activity. There are a few points of neutral expression, indicating the experience has not yet reached a high level of intensity.

In Column II, we find our turn towards ‘L’Anneau’. The segment where we are moving is dominated by engagement (B-II, in purple), while our pause at ‘L’Anneau’ shows a mix of engagement, arousal, with some traces of joy (Fig 2, B-II, cyan, yellow, purple). During this break, we took a moment to admire ‘L’Anneau’ and its surroundings. We also engaged in conversation with a group of curious tourists regarding our equipment.

Column III isolates a segment where we are on the move. This segment is almost exclusively a mix of engagement and arousal (Fig 2. B-III, in yellow).

Column IV includes a break, during which we measure joy, arousal, and engagement (Fig 2. B-IV, in purple, yellow, and cyan).

At this point, another group of tourists approached us with curiosity during a red light, and an engaging conversation took place. This social interaction lasted quite a while, as other passersby joined the discussion and passed on the word to new onlookers.

Following this conversation, we moved towards the Quartier des Spectacles, where the Montreal International Jazz Festival was taking place. The entry to the site is marked by a prevalence of engagement.

Column V indicates a strong presence of joy (Fig 2. B-V, in yellow). This is the moment when our group moved through the crowd while a jazz performance was in full swing. Needless to say, the atmosphere was highly festive.

Joy continues to be predominant afterward, but the prevailing state of mind shifts towards engagement and arousal (Fig 2. B-V, in cyan).

Figure 4. Our group photo. Naturally, this smile was captured by the camera. Even though it’s a smile posed for the shot, I believe it’s evident that it wasn’t all that forced ;)

EMO)MENTS

This video montage depicts moments from the experience captured in a point-of-view (POV) perspective, corresponding to intense moments of feeling. The concept is still under development, but all sequences were extracted automatically by our algorithm.

DISCUSSION

This experience is limited to 6 participants, which restricts the scope of the results. However, it still provides a valuable opportunity to formulate hypotheses based on our observations.

Excluding the preparatory phase at Dorchester Square, the initial stages of the experience are characterized by physiological arousal. This pattern is frequently observed in our data at the start of experiences. Our hypothesis is that people initially have a distracted mindset and haven’t yet fully engaged cognitively in the experience. We often refer to this segment as the “onboarding” phase. For instance, Figure 6 illustrates the interplay between arousal and engagement from another experiment conducted by our company.

Figure 6. Example of the arousal-engagement dynamics during an audiovisual entertainment activity.

Unlike the example in Figure 6, however, arousal remains consistently present throughout our urban exploration activity, except for a few instances dominated by engagement.

This leads to our second observation: we notice an alternating pattern between the “arousal + engagement” and “engagement” mind states. This seems to reflect an indicator of the urban ambiance. Our hypothesis is that calmer segments induce engagement, while livelier segments induce both arousal and engagement.

It’s surprising for us to see engagement being so prevalent, as our other experiences suggest that an increase in arousal is often associated with a decrease in engagement. It’s likely that urban navigation leads to this blend because, on one hand, the urban environment is full of surprises and unexpected stimuli (arousal), and on the other hand, one needs to be focused to avoid collisions with obstacles (engagement). We’re still in the early stages of researching the human experience in an urban context, and we’ll see if these hypotheses hold as we gather more data.

The various points of social interaction brought about joy. This is a result we anticipated, as we observed in other contexts. What’s more surprising is that joy dominated our experience at the Montreal International Jazz Festival.

We’re not suggesting that the festival should have been a source of frustration, but navigating through a dense crowd could have potentially caused some friction. As it turns out, we were wrong; our results support the idea that organizing festive events directly impacts a city’s experiential landscape, creating pockets of pleasure that passersby can enjoy spontaneously, even in challenging navigation conditions.

CONCLUSION

This micro-report is written following our urban exploration activity. While the scope of the results is limited, we’ve been able to delve into our hypotheses regarding the experiential landscape of downtown Montreal.

There’s still much work ahead of us to fully capture the emotional and cognitive identity of our beautiful city, but this project is a brushstroke in the realization of our masterpiece. We’re excited to see how far this project will take us.

The data collected will be integrated into our collection of Mindscape-Emotions experiences. If you’d like to explore our other findings:

Scan to access to our white book: Mindscape-Émotions

AKNOWLEDGEMENT

This project is an initiative of RE-AK technologies, experts in emotional and cognitive analysis applied to entertainment and marketing.

We extend warm gratitude to Daniel Fournier Stratégies d’Affaires, a firm specializing in customer experience and a partner of the Mindscape-Emotions project, for actively participating in this data collection activity.

We also express our thanks to the main partner of the Mindscape-Emotions project: SOM, the leading marketing research firm in Québec.

The French version of this article is available here.

--

--