Brain reading — are we there yet?
Everyday brain recordings
Some years ago I gave an interview for IDG Connect about the future of brain tech and forecasted that by 2020 there will be existing everyday brain wearables on the market. Today, I think this was quite an accurate assumption. This blog is about the technology state-of-the-art and how close we are to the day when recordings are a part of our everyday.
In another blog, I wrote about the benefits of brain wearables in everyday life. In this one, I will focus on the technical challenges that we have to solve to make my predictions come true.
We are experiencing challenges on two fronts:
- How to encourage and inspire people to embrace brain recording wearables and make them part of their everyday. Developers today struggle to provide instant gratification to the users of brain recording gadgets. After the initial “wow” effect that brain reading has on consumers, the subsequent question is always… sooo… I record brain data and then what?
- Development of the sensors that will precisely measure the freakingly weak brain signals (on the order of a few microvolts — this is more than a hundred times weaker than for example ECG). Also, the brain is protected by the skull, and then a few layers of tissue and muscles, and then skin, with even hair on top of all this (a more detailed description of the human brain can be found in this great blog post by Tim Urban). Electrical signals generated by the brain are reflected on the skin layer, but the hair makes it difficult for our sensors (electrodes) to approach and detect it. In many cases, what is detected by the sensors are not brain signals, just (physiological or artificial) noise, that we cannot do much with — shit in, shit out.
The first of these is kind of a chicken-and-egg problem. On the one hand, no one will record their brain because just maybe, one day, 10 years from now, it will predict a potential stroke. On the other hand, if people do not take recordings continuously, we will not be able to design algorithms for more serious (medical) purposes.
The second challenge to solve is to identify the real need consumers have that can be instantly solved with brain recordings. Some good and potentially groundbreaking concepts, however, emerge, e.g. Kokoon, Mindset, and the one we are working on at mBrainTrain. There are still some challenges behind all these. Some of the possible applications are discussed in a separate blog.
This post focuses on the technical development needed to make consumer EEG possible in the data acquisition sense in the first place.
The bottleneck in the consumer brain recording industry is acquiring high-quality data under real-life circumstances. Some of the factors that influence are: motion artifacts (due to moving electrodes with respect to the skin, and moving cables), muscle artifacts (as signals from neck muscles are a couple of orders of magnitude stronger than the brain signals, or chewing artifacts), good contacts and adherence to the skin, etc. A good scientific overview of these issues is provided in this paper by Vojkan Mihajlović.
For us, in mBrainTrain, bringing EEG to everyday life was the vision that has driven us from the very beginning. To do that, it was important for us to take this development step-by-step and to do it properly. Our first step was to develop a high-end scientific EEG system that can easily be used outside the experimental labs and coupled to mobile phones. Based on the expertise with these systems and close collaboration with the scientists, we then develop the consumer-graded products.
Smartphone compatible systems with gel-based electrodes
This first system we developed (SMARTING) is fully customized for brain scientists, not consumers, but it was important to master the technology. Scientists can now quickly set up complex experiments and walk them out to the natural surroundings. They can easily synchronize it with other systems for multi-sensor studies (like EEG-Eyetracking studies, e.g. in retail, EEG-VR/AR studies, etc.).
The first systems we used (and we still use) exploit liquid gel for making good contact between the head and the electrode. This is still the most robust solution, that accounts for most of the above-mentioned artifacts and makes the preprocessing steps significantly easier. If one needs to observe the smallest changes in brain signals (such as single-trial ERP changes) this solution is still irreplaceable and he should definitely go for it. It also, under all circumstances, provides the cleanest signal with minimal noise.
Saline-solution based electrodes
The next step is going for less obtrusive caps, that are easy to put on, that do not leave dirt on hair (like gel that needs to be washed after each EEG recording) but still have good contact. We have tried many different solutions, some of them were polymer-based, or Ag/AgCl tips comb electrodes from different producers. Finally, we believe that the best solution currently available is sponge electrodes sunk in the saline solution. These electrodes provide good contact and are reasonably susceptible to movements (still not as good as liquid gel electrodes). They are easy to put on (less than 1 min) and to put off, and the only thing left in the hair is water, so the subject can easily walk away after the study/experiment.
However, these caps require more careful preprocessing compared to the gel-based systems, especially in the conditions where the subject is moving during the experiment.
The systems above provide full head coverage brain recordings, streamed directly via Bluetooth to a PC or Android, and can be readily used to set up any cognitive psychology experiment (behavioral studies, consumer insights, neuroergonomic studies, etc.) in the lab, or outdoor in a free environment. The scientists get raw, noise-free data, ready for processing in real-time or offline. This was the step we were really happy with and were ready to move forward…
The next step towards making an EEG recorder a consumer product is to make it invisible. One should have in mind that making something a consumer product, means creating new habits with the users. However, it is highly unlikely that people will create a new habit of wearing something weird on their heads and this is one of the main reasons EEG is not yet popular as a consumer product. Rather than creating a new habit, providers should try blending brain recordings into the already existing habits of the users (just like Kokoon and Mindset are trying to do). However, robust technology for this is still missing.
Around-the-ear electrodes
Our first step towards “invisible EEG recordings” was using the electrodes that record brain data from the restricted area around the ear (although there are some concepts for “in-ear EEG” developed by other laboratories and producers). The concept is highly exploited by the Stefan Debener laboratory in Oldenburg, Germany.
With these the system in the picture, the scientists from the Debener laboratory performed a number of experiments describing the performance of such a system — Measuring resting-state EEG (Debener et al., (2015)), P300 ERP (Debener et al., (2015)), decoding attended speaker with speech envelope tracking (Mirkovic et al., (2015), Mirkovic et al., (2016)). They also noted some preliminary interictal spikes in epilepsy patients, as well as K-complexes, sleep spindles, and slow-wave EEG in sleep studies.
Although almost invisible, the system is based on the Ag/AgCl electrodes printed on the flexible foil and still needs electrolyte gel to provide good contact and be susceptible to movements. Still, it is an exciting direction to further exploit and we may write about it more in the following texts.
How close are we?
It is still uncertain if the brain wearables will be around by 2020, but we are definitely getting close to it, especially taking into account some recent developments. There are a couple of things left to be figured out. The key question remains— what is the “killer app” and what such a system is going to do for a user before it is able to detect stroke and treat Parkinson's disease… What is the user experience user wants to see? What are the signal processing algorithms that are needed to clean the data from noise? What algorithms should be developed to provide such an exciting experience?
In the meantime, we keep our eye on all the technological novelties that can accelerate the development of everyday brain wearables, we keep on exploring all the interesting applications of brain recordings in real life, and support scientists in their pioneering work. And you? You keep an eye on what we do and contact us with any comments, ideas, and recommendations.
