KIOKU
Connecting remote families through serendipitous moments
These following are the research phases we used for this project. Our topic was food when we started off, we were asking questions around mindful consumerism, then narrowed in on family bonding through remote cooking, for remote families, but through the stages we realized that food or cooking was only an instance of what remote families bond over and so decided to concentrate on….
TARGET USER
Remote families
For remote family bonding, We realized that the real challenge lies in the current communication methods, and we will tell you more about that….
INSIGHTS
Insight #1
Families living together, they share and experience everyday activities together and have shared environment and objects and non-verbal communication other than verbal communication to depend on. Whereas, remote families rely only on verbal communication and there is no shared context. So you could be having dinner, and share a picture of what you are eating with your remote family while they are at work.
Insight #2
Connected to the previous insight, when families are together you know what each other is eating, drinking, what they are watching, how they feel — these smaller things, small but important things. But when communication becomes lesser in case of remote families, a lot of everyday things have to be left out. So those spontaneous interactions are missed out on.
Also we heard instances of how on both sides, something happens, and then you think that — when I talk to them I’ll tell them about this and you forget.
Team four eyes here made a great point in one of the previous presentations about capturing pictures with the intention of sharing, but ending up not sharing them, and you end up with thousands of pictures in your device which you don’t go back to.
Insight #3
Like it or not, when you are staying away from your family, you have to plan and schedule when you are going to call, when is it going to be a video call, maybe on a sunday while making breakfast or while washing the dishes with you trying to multitask. This becomes tougher when there is a significant time difference between the countries.
On a different note about the current communication methods, it has been all about instant messaging, instant responses. Somewhere, we are questioning if there is a need for everything to be instant.
We took a looked at current communication services and plotted them on this 2X2. The range for the horizontal axis is from conversation-centric to experience centric. While the vertical range involves how active or passive methods are. By active, we mean the communication is happening in real-time while passive involves leaving some type of messages. So a letter would be conversation-centric and passive a phone call is conversation-centric but active skype leans towards experience-centric since you can see the other person and they can show you what they are seeing.
This helped us realize that there is a void in the upper right corner of the matrix. We saw an opportunity area within the passive, experience-centric space.
These are two members of a remote family, they are separated by distance and time. Kioku lives inside of a prevalent mixed reality device in the future, which we have depicted as glasses.
One member captures moments, which are things they see or experience. Kioku then holds onto these moments and present them to other family members when they are in the right context. By right context, we mean a context similar to where the original person captured the moment.
This can happen both ways. Each family member can capture moments for others as well as experience moments that the others have captured.
They are also able to save the moments that other family members have captured to a personal library and revisit anytime.
Kioku leverages the current machine learning technologies for context recognition. When a family member has captured a moment, kioku will analyze that moment and generate a set of categorical tags that can define the context. When another family member is using kioku somewhere else, the system will constantly scan the physical environment and detect if any stored tag combination matches with the surroundings. If there is a match, the related moment will be retrieved from the cloud and ready for the family member to experience.
Kioku also embraces mixed reality as the new communication interface. We noticed that when information are presented in current devices, the devices themselves rarely have any relationship to the content of the information. Kioku uses the physical environment as the communication platform, and present augmented information in a context that relates to both the sender and the receiver. Through this approach, kioku aims to deliver a passive and experience-centric communication method that connects remote families in a mixed reality world.
As mentioned earlier, kioku can generate a set of categorical tags that defines the context. Three major categories are identified here: environment, objects and people.
For the environment category, it will be further broken down to several subcategories, such as GPS-defined location, indoor or outdoor condition, specific season, or certain time of the day. For objects, the system will generate tags that recognize landmarks, brands, animal and plant species, and even object movement patterns. As for people, the tagging subcategories include gender, age,, etc. A context will be defined as a combination of all of these tags and the moment is presented only if multiple criteria is satisfied.
Here we have the father and the daughter using the kioku system as a family. The father can capture a 10-second moment by tapping on the glasses temple. Kioku will record and process the moment to determine its context, and then store it in the cloud server.
The point to note here is how easy it is to capture. Kioku enables communication without active or focussed participation. The dad can capture anything at any time with a simple tap, and the moment is automatically saved.
On the daughter’s side, kioku is always looking for the right context. Once there is a match, the moment will become visible in the environment through the mixed reality device. The moment’s initial form of presence is a virtual sphere that is located at a reasonable distance and height from the daughter.
The daughter can then gaze at the moment and double tap the device to open and experience the moment.
Then the daughter is able to experience her parents’ precious moment through her father’s eyes. The immersive moment will fill up her field of view, and the edges will gradually merge with the physical environment. We were mindful of the fact that people will be using this in public and we also wanted the moment to be in the receiver’s physical context.
Kioku knows if a moment has been watched or not and will act accordingly. The moment won’t be shown again once it is watched. After she is done watching, the daughter can save it in her private library by swiping inwards on her glasses. If she doesn’t want to save it, she can just leave and the moment will disappear.
Later on, she can bring up her personal library by swiping outward on her glasses, anytime she misses her family. This would be a collection of all the memories the daughter has saved from her dad, mom, maybe grandparents, or anyone who has been identified as a family member.
And the daughter can play and experience any moment she wants, with the same interactions she previously used. Instead of active searching and sorting, the daughter will see older or less viewed moments closer to her and newer or frequently viewed moments further away. We thought this would gently guide the user to experience and remember different moments.
Throughout the semester, we met and listened to many families living away from each other. Being remote from our own families and considering how nomadic the current era is, we hope that our design would ease the burden and fatigue of current communication as well as enable families to feel closer to each other through the magic of mixed reality.
