A toolkit to optimise our memory of past experiences

Luke Sturgeon
Apr 30, 2015 · 16 min read

This paper outlines an approach to the study of self-perception and self-reflection in performance, through a research tool that arose from a collaborative project between the author and a dance artist. Original ideas and questions were used to interrogate our own process and methods during the project, that suggested new tools for design research and approaches to self-reflection. This paper starts by introducing the unique qualities of intimate-performance as a research context and addressing relevant research themes and their relationship to this context. Next it discusses the creative process that was undertaken and how practise-based iterative investigation shaped the project. Finally, it focuses on the toolkit that manifested itself as a way to initiate and capture self-reflection.


We began to realise that if we wanted to change the situation we first had to change ourselves. And to change ourselves effectively we first had to change our perceptions (Stephen R. Covey, The 7 Habits of Highly Effective People)

1. Introduction

Reflextion is a tool that was created by the author in 2013 as part of a Master level thesis project completed at the Copenhagen Institute of Interaction Design (CIID). The project addresses several fundamental questions about modern society and our attitude toward self-perception, technology, information and data interpretation.

The project was developed as a research method to discover how reflexivity affects our interpretation of a performance and our self-perception. Inspired by wearable methods of tracking and biometric quantification in order to optimise interpretation of self-reflection. This project speculates how biometric personal data may be collected in order to optimise our memory of past experiences.

The author proposes that the use of the final toolkit goes beyond the experiments and research conducted during the process, and raises questions about our attitudes towards the design and adoption of new software tools and devices for reflection and personal optimisation. It is not the author’s opinion that people must change their attitudes, behaviours or perceptions, but that people should be given the opportunity to be presented with enough information that they become aware of their own decision-making and opinions towards change. Awareness and knowledge is required before change can take place, as awareness can lead to discussion that can provoke meaningful change (if necessary) through critical reflection and dialogue.

2. Definition of theories

Much work has been done within the theories and concepts that are present within this paper. The author acknowledges the work of researchers, artists and designer who have explored and helped to define these theories. For clarity and to avoid confusion or debate around implied meanings, the exact definition and intention for each theory — as encountered during the research — is described below.

2.1 Quantified self

In recent years the Quantified Self movement has changed the way society adapts to new technology, our familiarity with the interpretation of data and introduced self-tracking as an everyday activity. Automated data entry and wearable sensors have been introduced to many aspects of our personal lives for the purpose of data collection, analysis and self-knowledge through self-tracking.

The acceptance and familiarity of wearable technology and data visualisation was considered a benefit for the purpose of this research project. Without formal training in statistics, mathematics or programming people are willing and able to view visualisations of data created through convenient and simple visualisation tools. Time spent explaining and familiarising participants with body-mounted sensors and data visualisation software was significantly lower than expected. The familiarity and interest in personal data tracking enabled each experiment to be initiated quickly and created an instant dialogue with the participant. Participants also displayed keen interest in the recorded data in order to understand and interpret their own data.

This paper defines two categories of motivation for individuals to practise self-tracking. The first is health and wellness, where an individual may be affected or concerned about a medical condition whereby the monitoring of their everyday activity and their own body may assist in the diagnosis, treatment or reassurance of the individual. The second motivation is self-improvement and the optimisation of personal actions and activities for personal satisfaction. This can be achieved in the form of individual or social comparison where the individual tracks their progress towards a goal that they have set for themselves.

The quantified self movement raises questions about the way we conduct ourselves, capture, observe and interpret our own actions and our willingness to share that information socially for others to interpret and compare against their own data. This paper suggests an approach to research that could allow us to investigate the following questions: If we allow ourselves to be influenced by others and by the interpretation of our own personal data in order to change our behaviours and actions, who is really in control of our actions? If we focus on the optimisation of our own actions what is the threshold between success and failure and what are the personal implications of this? Instead of reflecting upon our actions in order to optimise our future actions, could we reflect upon our perceptions in order to optimise our self-perception?

2.2 Research context

For the research to be effective in the discovery of how reflexivity might affect our interpretation of a performance and self-reflection, a suitable research context was required that would allow for rapid prototyping and fast iteration. A situation whereby the necessary interventions could exist within a controlled environment without affecting the abilities of the performer or compromising the integrity of the performance itself. As the focus of the project was on self-reflection within performance this dismissed sport, exercise, acting and improvisation whereby the actor would take on the role of a fictional character and questions would be raised about the validity of any form of self-reflection. Would the actor or the character, be the focus of reflection and interpretation?

Lap-dancing was the method of performance that was selected for the project. This extreme case example of intimate performance provided an ideal situation whereby two people (the dancer and the spectator) could share an intimate experience. The natural conditions of a lap-dance provided a controlled environment where an isolated space could be used to perform a dance for approximately 3 minutes in duration (the duration of a single song). The dancer would rely on a collection of rehearsed movements and the focus of both participants would be upon the spectacle of the performance. Whereby it is intended that the spectator should be focused on the dancer and their performance, while the dancer should respond to visual indicators and the spectator’s body language. This feedback process involved the interpretation of subtle body language and assumptions in order to shape the performance as perceived by the dancer.

2.3 Self-perception

This paper defines the concepts of the ‘self’ and ‘self-perception’ as five separate layers. Each layer can be used to investigate and compare self-reference, when an individual interprets their own behaviours rationally, in the same way they attempt to explain other’s behaviours.

2.4 Reflexivity

Reflexivity is a concept within social theory that refers to a circular relationship between cause and effect that can occur when an individual attempts to examine themselves. When an individual is observing themselves within a situation the act of self-reference and reflection affects the individual and changes they way they behave within the situation under observation.

The reflexive relationship was a key focus throughout the research, as the act of self-reflection was intended to provoke the participant to critique their own expectations and preconceptions whilst simultaneously comparing their recorded behaviours, interpretations and memories of performance.

2.5 Observation

The author acknowledges that the inclusion of an observer within the intimate performance context, did influence the behaviour of all participants and therefore influence the results from the experiments. This phenomena is commonly referred to the observer’s paradox and was a consideration during the experiments.

Due to the uncertainty and unknown results that would materialise from the experiments the decision was made to acknowledge the role of the observer within the experiment, along with the additional wearable sensor and video recording equipment. It was acknowledged that, as the participant was aware of the experiment and that they were under observation and would be asked to reflect upon their own actions — their own behaviour during the experiment cannot be argued as original and totally ‘natural’ in the research context.

However it was discovered during the research that despite all efforts to conceal any methods of surveillance, a lap-dance would most frequently take place inside a nightclub, where CCTV surveillance would (as required by UK law) record the activity of every dancer and patron within the club. The surveillance footage is also monitored live by a ‘bouncer’, who is responsible for intervening if a performance appears to break the laws and rules of the nightclub. Therefore, in this project the bouncer and the observer share similar roles within a lap-dance, as controllers of the performance situation. Professionally this role is intentionally hidden from the spectator’s view to suspend the disbelief of the spectator, so they might suspend judgment concerning the situation they are currently experiencing.

“There’s never really 2 people involved in a lapdance. Not really. There’s the dancer, the customer and the bouncer. You try to pretend it’s just the two of you but the bouncer is always there to monitor. For these experiments your sort of the bouncer.” (Dancer during reflection interview)

It must be noted that the reflection process defined during the research may be a new experience for the participants. All recorded information and footage are initial responses to stimuli over the duration of four weeks. Therefore the individual’s response may change in subsequent experiments and over increased amounts of time, as the participants become more familiar with the tools and process.

2.6 Galvanic skin response

The human sympathetic nervous system affects the body’s sweat glands and if the sympathetic nervous system is highly stimulated, sweat gland activity will also increase, which increases the skin’s electrical conductance. This way skin conductance can be used as an indication of emotional and sympathetic response and psychological stimulation.

A galvanic skin response sensor consists of an electronic circuit which is split by two metal contacts that — when touched at the same time — will complete the circuit and allow a small electronic current to flow. If a person’s fingers are pressed against the metal contacts the amount of electrical current that flows through the circuit will depend on the person’s skin conductivity. With the correct sequence of resistors and an op-amp (operational amplifier) to amplify the voltage the electrical current can be used to determine and track a person’s skin conductivity. Based on the existing research behind electrodermal activity these signals can be expected to fluctuate and demonstrate a person’s reactions to emotional stimuli.

The electric signal that is recorded by the galvanic skin response sensor is not accurate enough to determine an individuals emotional state or state of mind. But when the data is visualised (in the form of a line graph) the behaviour of the line does indicate that the individual’s body has responded in some way to emotional stimuli. During this research no figures were shown on the graph but the data line was used as a prop for interpretation and discussion.

A benefit for the research, of lap-dance performance is physical restrictions enforced upon the spectator, who was asked to remain stationary during the performance. The restriction provided a controlled environment for the galvanic skin response sensor which could otherwise be severely affected by changing finger pressure and subtle movements in the wearer’s body.

3. Research process

The research process involved extremely close collaboration between the author and the dance artist. Weekly performance sessions were defined so that questions could be prioritised and appropriate time could be taken to research and prepare suitable experiments and tools for performance. During each weekly performance the outcome was analysed, and the results were critiqued to validate the project process and progress.

Rapid prototyping and an iterative and experimental approach to design-research was required. It was acknowledged that repeating the experiment iterations with the same participants would influence their behaviour and feedback during the investigation. But it was accepted by the author that the benefits of consistent participants lowered the threshold toward self-reflection and allowed the participants to become more comfortable with the examination process, which would potentially improve the feedback and response from participants.

The next part of this paper documents the structure for each collaborative performance session and the methods used for each experiment. The objective, method, and conclusion for each experiment are described, and the insight was used to determine the following experiment structure.

The term ‘participant’ is used to describe any individual currently under observation during the performance and examinations. During the research one female dance artist, one male spectator, one male and one female external observers and twenty male and female volunteers participated in the experiments described below.

3.1 Sensor data

Objective: Can we capture and read signals between participants during a performance?

Methods: Using a variety of participants to calibrate wearable sensor and recording software, experiments were carried out to synchronise, audio, video, data and narration tracks for analysis and comparison.

Conclusion: Participants were interested in the interpretation of their own data and curious about differences between their own assumptions, expectations and the reality. Participants also expressed an interest in the comparison between their data and other participant data.

Insight: Explaining the purpose of the experiment clearly and allowing the participant to preview the recorded data can lead to unexpected and valuable conversations.

3.2 Stories

Objective: Can an individuals personal data provoke unexpected stories during self-reference and interpretation?

Method: Using a galvanic skin response sensor and video recording equipment, several lap-dance performances were captured between the same dancer and spectator. After the performance each participant was isolated and asked to examine the recorded footage. Participants were asked to narrate their own thoughts during the performance and during the examination.

Conclusion: A correlation was identified between the behaviour of the sensor data, the activities during the performance, and the participant’s own description of the performance.

Insight: When presented with previously unseen and unfamiliar information a participant’s memory of an event is compared to the actual event, the possible reasons for any differences can be speculated by the individual, leading to new stories and narratives.

3.3 Interpretation

Objective: What is the difference in emotional response between an original experience, reviewing an experience, and watching another’s experience?

Method: Using a galvanic skin response sensor and video recording camera and a head-mounted video camera, several lap-dance performances were captured between the same dancer and spectator. These performances were recorded in a nightclub to improve the realism of the video footage and attempt to suspend the disbelief of external observers who also examined the edited video as part of the experiment.

Conclusion: The examination process can trigger emotional response within the dancer and spectator, during and after a performance. But other participants described feelings of interest and disinterest towards the content, they explained that because they were not part of the original performance it did not provoke the same emotions described by the dancer and spectator. Other participant focus was upon the performance, the situation and trying to view and analyse the expression and body language of the dancer and performer.

Insight: The examination focus should be towards the dancer and spectator, to provide suitable emotional stimuli and provoke reflection and discuss self-perception.

3.4 Self-reflection

Objective: Can we define a process that allows the participant to analyse and interpret their own responses towards themselves?

Method: Recorded video and audio footage of the previous lap-dance performances from different angles, and galvanic skin response sensor data are synchronised and loaded in to custom examination software written in C++ using openFrameworks. A galvanic skin response sensor is connected to the participant before the examination begins and the sensor data is recorded by the examination software. All video and audio footage is available to the participant, but the participant’s view is limited to a single piece of footage and they must use the computer keyboard to toggle between different footage.

When the examination is over and the complete recording has been replayed, the participant is shown their actions during the examination including sensor data, choice of video footage, the moments they switch between different footage sources. This information is replied in real-time and the participant is asked to describe their interpretation of the way they reacted to the footage and speculate upon their reasons for switching between footage.

Conclusion: The process of capturing a participants initial response to a recorded performance in order to replay the responses, can provoke reflection and self-criticism. Within the examination context, the participant is free to respond naturally to the recorded performance.

Insight: The cognitive load required for a participant to immediately self-reflect and describe their emotional response will distract the participant, affecting their natural response to the recordings. By recording the participant’s response during the examination for the purpose of self-reflection and replaying the recorded response, the participant is more comfortable self-reflecting and providing feedback to the observer.

4. Toolkit

The discovery process has been defined as five steps and demonstrated within the case study above.

This paper will now define each step within the process as the designed approached to the study of self-perception and self-reflection in performance.

4.1 Calibration

The observer must set up the galvanic skin response sensor and audio/video recording equipment appropriately according to the nature of the performance type and the environment for recording. All sensors, audio and video recording equipment must be tested and synchronised during a short performance rehearsal to ensure successful recording and to familiarise all participants with the affect the recording equipment may have on the performance.

4.2 Performance

The performance takes place naturally for as long as is necessary. The role of performer, spectator and observer are defined and expectations are set for each participant. It is the goal of the author that the recording process should be as transparent as possible and avoid any influence on the performance.

4.3 Recording

All recording equipment and output is monitored during the performance to ensure the calibration of the recording equipment was successful.

4.4 Examination

The recorded footage and sensor data is synchronised and loaded in to the custom examination software. A galvanic skin response sensor is connected to the participant before the examination begins. All video and audio footage is available to the participant, but the participant’s view is limited to a single piece of footage and they must use the computer keyboard to toggle between different footage sources.

4.5 Reflection

When the examination is over and the complete recording has been replayed, the participant is shown their actions during the examination including sensor data, choice of video footage, the moments they switch between different footage sources. This information is replied in real-time and the participant is asked to describe their interpretation of their own behaviours when they reacted to the footage, and speculate upon their reasons for their behaviour.

These behaviours can include the participant’s choice of footage, the moments when they change between footage, the speed of change, the reasons for change, the galvanic skin response data, the footage being view by the subject and the amount of time spent viewing a single footage source.

5. Conclusion

Through the experiments in this research a correlation between the actions of the participants, the emotions they describe, and the responses that are measured in the galvanic skin response sensor can be made.

It is important to state that the sensor data itself does not provide meaning and cannot be used to determine how the participate might be ‘feeling’ during the performance or examination or why they chose to behave and respond to the recordings in a specific way. Without any context the information is abstracted and any statements of meaning or sentiment can be inferred. Likewise without any context any statements of meaning or sentiment cannot be proved are pure speculation.

This research does not claim to show the participant how they ‘felt’ or suggest to the participant that certain peaks or drops in the data correspond to a specific level of ‘happiness’, ‘arousal’ or emotion. But through the experiments this research does explain the way the sensor is expected to work, and that the reading can indicate a change in the human body possibly due to emotional and psychological response to a situation. This statement allows the participant to interpret the data in their own way, whilst observing their behaviours towards video footage of themselves. It gives the participant an opportunity to compare the differences between their memory of their behaviours during the examination, and their recorded behaviours during the examination. This can reduce the threshold towards self-reference and allow the participant to speculate upon the reasons for their own behaviours, using their own perceptions and interpretation as the criteria for self-analysis.

Through practise-based research and experimentation the approach, process, and tools defined in this paper can successfully provide a situation for a dance artist to reflect upon their behaviour before, during and after a controlled performance and examination. However this situation has severe implications and repercussions for the participant. A moment is created whereby the participant begins to question and critically reflect upon their self-perception and perception of others, how they see themselves and the possible reasons for their expected and unexpected responses.

We must be aware that this self-reflection does not begin or end with the controlled performances and experimentations in this research. The participant is able to recall any and all previous memories of their actions and decisions, from any point in their life, in order to compare and reflect upon past actions — based on new opinions they might conclude during the reflection process. There is also the possibility that the future actions of the individual may be affected by their participation in the research.

It is not the author’s opinion or the purpose of the research to state that people must change their attitudes, behaviours or perceptions. It is the belief that people should have the opportunity to be presented with enough information that they become aware of their own decision-making processes and opinions towards change. Awareness and knowledge is required before change can take place, as awareness can lead to discussion that can provoke meaningful change (if necessary) through critical reflection and dialogue.


Acknowledgements

The research is only possible due to the close collaboration of Sarah Homewood, a dance artist who’s experience in performance and flexible approach to an unknown process and methods allowed the freedom necessary to explore the notion of self-perception within intimate performance.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store