TinyEats: The chip that knows how many chips you ate

Automated Food Diary

Monitoring food intake and eating habits are important for managing and understanding obesity, diabetes and eating disorders. Because self-reporting is unreliable, many wearable devices, coupled with neural network classifiers, have been proposed to automatically monitor and record individuals’ dietary habits.

The challenge is that these devices store or transmit raw data for offline processing. This is a power-consumptive approach that requires a bulky battery or frequent charging, which intrudes on the user’s normal daily activities and is thus prone to poor user adherence and acceptance.

One solution is to use specialized neural networks that are easily embedded on the wearable device. However, that approach requires a power-consumptive analog-to-digital converter (ADC). It also forces the microcontroller unit (MCU) to unnecessarily spend power processing irrelevant data.

Analog Neural Network

Analog neural networks are one way to eliminate the ADC and also to minimize the microcontroller’s processing of irrelevant data. Unfortunately, the state-of-the-art analog LSTMs (a type of neural network that is well-suited for embedded processing of streaming data) are implemented with operational amplifiers (opamps), current/voltage
converters, Hadamard multiplications and internal ADCs and
digital-to-analog converters (DACs). These components represent a significant amount of overhead cost in terms of power consumption, which diminishes the benefits of an analog LSTM.

Block diagram of proposed eating detection system. From the contact microphone output, the ZCR and RMS blocks extract features based on zero-crossing rate and root-mean-square. The analog neural network (labelled ‘AFUA’) processes these features and produces a one-hot encoded output that predicts the presence or absence of a chewing event. The microcontroller (‘µC’) merges and filters the individual chewing events into whole eating episodes. The analog signal processing chain up to the AFUA block consumes 1.8 µW of power. The microcontroller is active only 9 % of the time, during which it consumes 180 µW of power

We tackled this problem with the Adaptive Filter Unit for Analog (AFUA), which is a a novel analog integrated circuit LSTM for embedded eating event detection that eliminates the need for a power-consumptive ADC. Unlike previous analog LSTM implementations, our solution contains no internal DACs, ADCs, operational amplifiers or Hadamard multiplications. Our novel approach is based on a current-mode adaptive filter, and it eliminates over 90% of the power requirements of a more conventional solution.

The AFUA governing equations are:

AFUA governing equations. Here, [·]j is the j’th element of the vector. Also, x is the input, hj is the hidden state and h˜ j is the candidate state. The variable τ is the nominal time constant, while zj controls the state update rate. Wz, Uz, W, U are learnable weight matrices, while bz, b are learnable bias vectors.

where the activation function is given by:

a saturating nonlinear function that is easily implemented in analog circuitry by taking advantage of the unidirectional nature of a transistor’s drain-source current.

AFUA Evaluation

Dataset

We asked 20 volunteers to eat a variety of foods — including carrots, protein bars, crackers, canned fruit, instant food, and yogurt — for at least 2 minutes per food type. The volunteers also performed non-eating activities like talking, coughing, laughing, drinking water, sniffling, and deep breathing. During these activities, a contact microphone was secured behind the ear with a headband, to measure any acoustic signals present at the tip of the mastoid bone.

Training

For training, the AFUA neural network was implemented in Python, using a custom layer defined by the discretized system of Eqn. (18). Chip-specific parameters were extracted for each neuron and incorporated into the custom layers. The AFUA network was trained and validated on the laboratory data (train/valid/test split: 68/12/20) using the TensorFlow Keras v2.0 package. Training was performed with the ADAM optimizer and a weighted binary cross-entropy loss function to learn full-precision weights. Python training was followed by a quantization step that converted the full-precision weights to signed 3-bit values (0, ±1, ±2, ±3).

The AFUA was implemented, fabricated and tested as an integrated circuit in a standard 0.18 µm mixed-signal CMOS process with a 1.8 V power supply. The network was then evaluated on the test dataset.

Results

The figure below shows the AFUA chip’s typical response to input data. The input currents Ix1, Ix0 represent the ZCR-RMS and ZCR-ZCR features extracted from the contact microphone signal.

Chip measurement response to different input patterns
(Ix1, Ix0) taken from the test dataset. Ix0 is the output of the cascade of
an RMS block and ZCR block. Ix1 is the output of the cascade of two ZCR
blocks. The circuit’s class prediction is encoded as output currents (Ih1, Ih0).

Inputting a stream of (Ix1, Ix0) patterns produces output currents (Ih1, Ih0), which represent the hidden states of the AFUA neural network. According to our encoding scheme, (Ih1, Ih0) = (2Iunit, 0) means that the circuit classified the input as chewing, while (Ih1, Ih0) = (0, 2Iunit) corresponds to a prediction of not chewing. But the presence of noise and circuit non-ideality produces some ambiguity in the encoding: some AFUA output patterns can be interpreted as either chewing or not chewing, depending on the choice of threshold used to distinguish between 0 A and 2Iunit. Varying this threshold current produces a receiver operating characteristic curve (ROC), as shown in the figure below.

Receiver operating characteristic curve (ROC) from AFUA chip measurements. These results were produced from repeated AFUA chip measurement responses to 1.6 hours of previously-unseen test data. Circuit noise produces slightly different performance from one measurement to another, with the area under ROC (AUROC) ranging from 0.95 to 0.99 (average AUROC=0.97). The highlighted point corresponds to a sensitivity of 0.91 and a specificity of 0.96.

The highlighted point on the ROC is a representative operating point, where the classifier produced a sensitivity of 0.91 and a specificity of 0.96. This corresponds to a false alarm rate of (1−specificity) = 0.039.

Power Analysis

In this section, we consider the power consumption implications of using the AFUA neural network in a complete eating event detection system. To process a 500 Hz signal, the ZCR and RMS feature extraction blocks consume a total of 0.68 μW. Also, the AFUA network consumes 1.1 μW, assuming Iunit = 10 nA. Finally, a microcontroller from the MSP430x series (Texas Instruments Inc., Dallas, TX) running at 1 MHz consumes 180 μW when active and 0.72 μW when in standby mode.

The feature extraction and AFUA circuitry are always on, while the microcontroller remains in standby mode until a potential chewing event is detected. The fraction of time the microcontroller is in the active mode depends on how often the user eats, as well as the sensitivity and specificity of the AFUA network. Assuming the user spends 6% of the day eating, then, using the classifier operating point highlighted in ROC plot, the fraction of time that the microcontroller is active is

So, the microcontroller consumes an average of 180 μW × 0.09 + 0.72 μW × (1 − 0.09) = 16.9 μW. As the figure below shows, the average power consumption of the complete AFUA-based eating detection system is 18.8 μW.

Power consumption of eating detection system. The feature extraction
and AFUA circuitry continuously consume 1.8 μW of power. The microcontroller
is active for 9% of the time, during which it consumes 180 μW
of power. For the remaining 91% of the time, the microcontoller consumes
0.72 μW while in standby mode. On average (red dashed line), the whole
system consumes an estimated 18.8 μW.

Summary

The AFUA — an adaptive filter unit for analog long short-term memory — is a vital component of a low power eating event detection system. Measurement results of the AFUA implemented in a 0.18 μm CMOS integrated circuit technology showed that it can identify chewing events at a 24-second time resolution with a recall of 91% and an F1-score of 94%, while consuming 1.1 μW of power.

The AFUA precludes the need for an analog-to-digital converter, and it also prevents a downstream microcontroller from unnecessarily processing irrelevant data. If a signal processing system were built around the AFUA for detecting eating episodes (that is, meals and snacks), then the whole system would consume less than 20 μW of power. This opens up the possibility of unobtrusive, battery-less wearable devices that can be used for long-term monitoring of dietary habits.

Interested in sources? Check out our original research paper.

Interested in tea? This article pairs well with Oolong.