Data Collection Toolkit for Behavior Observational Research
Seminar III | 2017 Fall MDes
I developed this design research toolkit for Advanced Interaction and Service Design Concepts, a course taught by Dr. Dan Lockton at Carnegie Mellon University’s School of Design
Data Collection Toolkit for Observational Research
Where it comes from
Why for Observation?
Observational methods involve a researcher gathering data through watching users’ behavior and event, which is one of the major user-centered research tools to understand an overall user experience. Although it sounds easy in theory, I found some difficulties in my practice in observing customers’ shopping behavior for my thesis project. First, due to the limited time and documenting methods it can be difficult to record data in a field research. Besides, researchers can lost their objectivity since they could selectively record information instead of documenting everything they see. Thus, I tend to create a framework of data documenting for observational research which is aimed to guide the research planning, simplify the data collection process and make research results to be more objective.
Theoretical Background
The theory of measuring behavior was generated in the fields of behavioral biology, which showed great prominence in conducting sample study and interpret data. According to the theory of measuring behavior, there are two decisions should be made before recording behavior: sampling rules and recording rules.
Consider the feasibility in research practice, I screened the four original sampling rules and refined them to be: focal sampling and behavior sampling. For recording rules, I just keep the two methods introduced in the theory — continuous recording and time sampling (including instantaneous sampling and one-zero sampling) methods.
How it works
The toolkit suggests a six-step design process and will be used to facilitate the first four steps. In design practice, this process can be repetitively used s to narrow down the design scope or test assumption for one project as a circle :
- Ask a question: this question is the goal of the observational research. The researcher can start with a broad question and narrow it down through using the toolkit. Alternatively, with enough knowledge, the researcher may start with a specific question aimed to study a particular issue.
- Preliminary observation: before jumping into the formal research, an informal observation should be implemented. On this stage, the researcher can get a sense of the direction of the study. The collected raw data can guide further research decision-making.
- Choose subjects: In this step, the researcher can choose between two main design subjects based on the study demands. These two subjects represent two different ways of observation: a. focal sampling — the researcher only observe one individual for a certain period of time. b. behavior sampling — the researcher will focus on one behavior while observing a group of subjects. Each occurrence and relevant individual will be documented.
- Observe and document: the researcher uses the following cards to record the observation data.
- Analyze data: after analyzing the collected information and generate conclusion and assumption for further testing. With a new design question, the researcher can use this tool to test or narrow down his/her learnings from previous research.
Pilot test
I used this toolkit as a part of the design research for my thesis project — improve the user experience of grocery shopping in a physical store. I recruited some CMU students as participants, and used both focal sampling and behavior sampling methods while observing their behaviors in local supermarket.
Learnings
a. Things worked
Most pages were confirmed as pretty useful for observational research. Listing the main behaviors before observation helped me to focus on the key touchpoint during the research and made the study more productive. The participant profile was really helpful in providing key information while the research has to be documented anonymously. Behavior mapping was a little bit time-consuming, but it showed the relationship between the research subject and the physical contest in a fairly direct way. The behavior timeline interesting, which is hard to document. However, it has potential in generating quantitative research results in later analysis stage.
b. Things didn’t work
While doing behavior sampling, it was really hard to record every participant’s profile. I wish there was a easier way to document this data. Besides, I found that the behavior timeline did not only fit to behavior sampling method, but also could be useful while doing focal sampling data recording.
Next Step
First, as keeping on working on my thesis project, I need to do more observational research in next semester. Thus, I will optimize this design tool based on the learnings from pilot test and apply it in later user testing stage.
In the data recording process, I found handwriting could be problematic. Mobile phones could be a more useful and convenient tool in this scenario. If possible, I want to explore the possibility to make a digital version for this toolkit.
