EQ-Radio: Emotion Recognition using Wireless Signals

 

Computer Science & Artificial Intelligence Laboratory, MIT

 

5.46 GHz to 7.25 GHz electromagnetic waves reflected from chest record its displacement due to heart beat and respiration (radar FMCW). These indices can be correlated with emotional states.

(Note that a Wi-Fi router emits 2.4GHz waves)

 

http://news.mit.edu/2016/detecting-emotions-with-wireless-signals-0920

 

“Just by knowing how people breathe and how their hearts beat in different emotional states, we can look at a random person’s heartbeat and reliably detect their emotions,” says Zhao.

 

Excerpts from paper “Emotion Recognition using Wireless Signals” mentioned at this link http://eqradio.csail.mit.edu/

 

“EQ-Radio operates on RF reflections off the human body. To capture such reflections, EQ-Radio uses a radar technique called Frequency Modulated Carrier Waves (FMCW) [5]. “

 

“We reproduced a state-of-the-art FMCW (Freradio designed by past work on wireless vital sign monitoring [7]. The device generates a signal that sweeps from 5.46 GHz to 7.25 GHz every 4 milliseconds, transmitting sub-mW power.”

 

“(…) The radio transmits a low power signal and measures its reflection time. It separates RF reflections from different objects/bodies into buckets based on their reflection time. It then eliminates reflections from static objects which do not change across time and zooms in on human reflections. It focuses on time periods when the person is quasi-static. It then looks at the phase of the RF wave which is related to the traveled distance as follows [58]: φ(t) = [2π d(t)]/ λ , where φ(t) is the phase of the signal, λ is the wavelength, d(t) is the traveled distance, and t is the time variable. The variations in the phase correspond to the compound displacement caused by chest expansion and contraction due to breathing, and body vibration due to heartbeats.

“Recall that a person’s emotions are correlated with small variations in her/his heartbeat intervals; hence, to recognize emotions, EQ-Radio needs to extract these intervals from the RF phase signal described above.”

 

“After EQ-Radio recovers individual heartbeats from RF reflections, it uses the heartbeat sequence along with the breathing signal to recognize the person’s emotions.” 

 

 

Measuring heart beat from a distance of 1m and respiratory rate from 1.5m with a Doppler Radar Transceiver (chip)

 

Use of microwaves identical to those emitted by routers (2.4GHz)

 

“Non-contact Measurement Of Heart And Respiration Rates With A Single Chip Microwave Doppler Radar”

 

Dissertation submitted to the Dept of Electrical Engineering of Stanford University

 

Amy Diane Droitcour 2006 (https://www.linkedin.com/in/amydroitcour)

 

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.7516&rep=rep1&type=pdf

 

Comment based on reading notes: Contactless detection and monitoring of heart rate and respiratory rate can be performed by Microwave Doppler Radar. This technique can be used in patient care in hospitals or at home (cf. sleep apnea) and in applications such as measuring energy expenditure. It is possible to measure chest wall motion through clothing or bedding in order to extract in real-time the heart beat and the respiration rate. This dissertation adapts methods associated with bulky equipment to create a chip, a Doppler radar transceiver, that functions as a heart rate and respiration rate sensor/monitor (transceiver: transmitter and receiver).

 

What is the principle of the RADAR (RAdio Detection And Ranging)? Let us assume that we have an automatic tennis ball machine that sends balls onto a wall with a velocity of 1 meter per second. If the ball comes back in 2 seconds, what is the range/the distance the wall was found at? It is 1 meter; it took 2 seconds to do two identical travels (to the wall and back).

 

Similarly we can send an electromagnetic wave like the one that is emitted from our router. This is a 2.4GHz wave that falls into the category of microwaves (its wavelength is 12cm). Doppler radar motion detection systems typically transmit a continuous wave (CW) electromagnetic signal, (sometimes frequency modulated) that falls on a target, is reflected by it and is captured by a receiver.

 

What happens if the wall is moving? For instance towards us? If we send one ball every 2 seconds and the wall was still we would get 1 ball every 2 seconds. If the wall was moving towards us, we could get the first ball after 2 seconds and the second ball in less than that as it would bounce on a wall that was closer to us. So we would get the balls back more frequently. This shift in frequency is termed Doppler shift and it allows us to calculate the velocity of the moving object.

 

The function of respiration is linked to a moving chest/thoracic wall!

How much does the chest wall move during respiration? From 4mm to 12mm (peak-to-peak).

How much does the chest wall move during heart beat? Approximatively 0.5mm.

It must be mentioned the chest wall just moves back and forth and does not mark a net displacement in space. This is a special case that falls into the category of periodic motions and the net velocity is zero. In this case, it is said that the phase of the reflected signal is modulated by the time varying position of the chest. Also, it is said that heart and respiration information is encoded in phase modulation of 0.1Hz to 10Hz.

 

How do you demodulate the signal that is reflected off the chest? Experts say that this depends on whether the peak-to-peak motion is greater than the wavelength used. For 2.4GHz, the wavelength is 12cm and this is much greater than the chest motion. Experts say that at this case, in order to demodulate the signal you multiply the signal with an unmodulated signal from the same source.

 

This dissertation could detect accurately the heart beat from a distance of 1m and the respiratory rate from a distance of 1.5m. This is based on measurements from 22 patients using standard techniques as control.

 

 

Key words: Doppler-radar cardiorespiratory (cardiopulmonary) monitor

 

 

 

Figure: Image from Amy Diane Droitcour's thesis at above link

 

 

 

Figure: Amy Diane Droitcour is co-author of book “Doppler Radar Physiological Sensing” http://eu.wiley.com/WileyCDA/WileyTitle/productCd-1118024028.html

 

@infobook tweet

@infobook tweet (book)

 

 

 

 

 

 

 

 

Radar Detection of heart and respiration rate of athletes at ranges exceeding 10m (Greneker et al).

“At 100m the limit was moving background clutter, not the system sensitivity”.

 

Use of hearbeat as a biometric identifier for security applications

 

Excerpt from Amy Diane Droitcour’s thesis (2006):

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.7516&rep=rep1&type=pdf

 

p.23 “An automatic clutter-cancellation circuit was developed by Chuang, et al., facilitating measurement of the heart and respiration signatures through seven layer of brick [52] and through ten feet of rubble [51]. Heart and respiration rates of athletes were successfully detected at ranges exceeding 10m by Greneker et al [54]; at 100m, the limit was moving background clutter, not the system sensitivity. A quadrature receiver was used to avoid phase-demodulation null points by J. Seals, et al. [72].

 

[54] internet link:

E.F. Greneker, “Radar sensing of heartbeat and respiration at a distance with security applications” Proc. SPIE 3066, Radar Sensor Technology II, 22 (June 10, 1997); doi:10.1117/12.276106

 http://proceedings.spiedigitallibrary.org/proceeding.aspx?articleid=925719

 

Abstract: Researchers at the Georgia Tech Research Institute have developed a radar that will detect heartbeat and respiration without any physical connection to the subject. The system is capable of making these measurements at ranges exceeding 10 meters. This paper explores the use of the system for the biometric identification of personnel who work in a highly secure environment. The system, used in this application, would use the heartbeat signature of an individual as a biometric identifier. Also, the system could be used to determine the stress level being experienced by an individual on the basis of respiration and heartbeat rates.

 

More information including radar gait measurement at this publication: G. Greneker, “Radar Technology For Acquiring Biological Signals”, RADAR Flashlight, LLC, Powder Springs, Georgia 

http://truth.charleshontsphd.com/JCAAWP/2006_No_2/2006_127-134.pdf

 

@infobook tweet

 

 

Figure: Combined image generated from above publication

 

 

 

Figure: Radar Measured Gait - Doppler signatures from above publication.

 

 

 

Micro-Doppler Radar Signatures for Intelligent Target Recognition

 

#AutomaticTargetRecognition #AutomaticGaitRecognition

 

Defence R&D Canada http://www.dtic.mil/dtic/tr/fulltext/u2/a427483.pdf (p15-16)

 

When the radar transmits an electromagnetic signal to a target, the signal interacts with the target and then returns to the radar. The change in the properties of the returned signal reflects on the characteristics of the target.

 

When the target is moving, the carrier frequency of the returned signal will be shifted due to Doppler effect. The Doppler frequency shift can be used to determine the radial velocity of the moving target. If the target or any structure on the target is vibrating or rotating in addition to target translation, it will induce frequency modulation on the returned signal that generates sidebands about the target's Doppler frequency. This modulation is called the microDoppler (m-D) phenomenon.

 

The m-D phenomenon can be regarded as a characteristic of the interaction between the vibrating or rotating structures and the target body. If the target undergoes a vibration or rotation, then the Doppler frequency shift generated by the vibration or rotation is a time-varying function and imposes aperiodic time-varying modulation onto the carrier frequency.

 

The modulation contains harmonic frequencies that depend on the carrier frequency, the vibration or rotation rate, and the angle between the direction of vibration and the direction of the incidentwave. While the Doppler frequency induced by the target body is constant, the m-D due to a vibrating or rotating structure of the target is a function of dwell time.

 

The micro-Doppler effect was originally introduced in laser systems, but it can also be observed in microwave radar systems. The fundamental research of the m-D phenomenon in radar is a relatively unexplored and untested area [5]. Micro-Doppler radar signatures of battlefield engine are caused by vibration. In many cases, a target or any structure on the target may have vibrations or rotations that are referred to as micro-motion dynamics. For example, vibrations generated by a vehicle engine may be detected from the surface vibration of the vehicle.

 

From the m-D signature of engine vibration signals, one can further distinguish a gas turbine engine of a tank from a diesel engine of a bus. Vehicles produce their own types of m-D signature as do helicopters. Therefore, the m-D effect can be used to identify specific types of battlefield targets and determine their movement and engine speed.

 

(…)

 

Obtaining radar signatures of personnel is another important application of m-D. The human walking gait is a complex motion behaviour that comprises different movements of individual body parts. Since September 11, Automatic Gait Recognition (AGR) technology is growing in significance. Because gait recognition technology is so new, researchers are assessing its uniqueness and methods by which it can be evaluated. Various computer vision and ultrasound techniques have been developed to measure gait parameters [26-29]. Real-Time AGR radar systems have recently been recognized as advantageous solutions for detecting, classifying and identifying human targets at distances in all light and weather conditions. Radar has certain advantages over electrooptical (EO) systems and video cameras in that it can penetrate into clothes, does not require light, and operates in fog and other low-visibility weather conditions. However, radar-based recognition is such a novel approach that much fundamental research has yet to be done in this area. The radar sends out a signal and then measures the echo that contains rich information about the various parts of the moving body.

 

There are different shifts for different body parts, because they are moving at different velocities. For example, a walking man with swinging arms may induce frequency modulation of the returned signal and generate sidebands about the body Doppler.

 

 

Classification of micro-Doppler signatures of human motions using log-Gabor filters

http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7348901

 

 
 

 

Interesting book

 

 

 

Laser monitoring of chest wall displacement - GaAs laser

(~ 870 nm)

 

(Future studies for infant ventillation)

 

T. Kondo, T. Uhlig, P. Pemberton, and P.D. Sly, “Laser monitoring of chest wall displacement, “European Respiratory Journal, vol. 10, pp. 1865-1869, 1997 http://erj.ersjournals.com/content/10/8/1865.long

 

Excerpt from Amy Diane Droitcour’s thesis (2006):

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.7516&rep=rep1&type=pdf

 

In the study by T.Kondo et al “a laser sensor was used to measure anterioposterior chest wall motion. This is a non-contact measurement, offering no resistance to respiration and no tactile stimuli, which should ensure a noninvasive measurement of respiration that does not alter the respiratory pattern. The laser monitor measures the distance between the chest wall and the sensor, and obtains a respiratory waveform by plotting the change in distance over time. The laser monitor can track rapid changes in lung volume with almost no lag. They propose a monitor with multiple laser sensors so that they can monitor multiple points on the chest, and better model the volumes of respiration.”

 

Helpful resource on Lasers: http://www.explainthatstuff.com/lasers.html

 

Laser Therapy – Physiotherapy cf. http://www2.information-book.com/physics/lasers/

 

@infobook tweet

 

 

 

 

 

Calculating tidal volume (volume of air displaced during respiration) by magnetometers measuring displacement of body surfaces (sternum, abdomen etc)

 

Excerpt from Amy Diane Droitcour’s thesis (2006):

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.84.7516&rep=rep1&type=pdf

 

“In magnetometer measurements, one coil is driven by an oscillator to produce a weak magnetic field, while other coils are attached to the skin on the thorax and abdomen. The coils on the skin pick up the magnetic field and can determine their position in the field. Magnetometers are susceptible to rotational movement, which creates artifacts [98]. Magnetometers were used to measure the anterio-posterior motion of the rib cage and abdomen [98], and to measure displacement between the abdomen and the sternum [110]. In [110], two transmitter coils operating at two different frequencies are placed near the spine at the sternal level and on the abdomen. Two receiving coils are also placed on the body: one tuned to both frequencies is placed on the sternum to measure the sternal-umbilical displacement and the rib cage anterio-posterior displacement. The other is tuned only to the frequency of the abdominal transmitter and measures the anterior-posterior abdominal displacement. With these three measurements, after calibration, respiratory volume can be estimated using a three-degree-of-freedom model.”

 

References below  (both available upon subscription)

[98] J Med Eng Technol. 1983 Sep-Oct;7(5):217-23.

Using body surface movements to study breathing.

Gribbin HR.

http://www.tandfonline.com/doi/abs/10.3109/03091908309032586?journalCode=ijmt20

 

[110] Chest. 2002 Aug;122(2):684-91.

Tidal volume and respiratory timing derived from a portable ventilation monitor.

McCool FD1, Wang JEbi KL.

http://www.sciencedirect.com/science/article/pii/S0012369215514074

 

Excerpts from [110]: The device consists  of  a  flowmeter,  transmitter,  and receiver  circuitry. Schematic of the magnetometer device depicting coil placement is shown below. Two transmitter coils (yellow and black) oscillate at 8.97 KHz  and  7.0  KHz,  respectively.  The  red  receiver  coil  is tuned to both frequencies, and the green receiver coil is tuned only to 7.0 KHz. The received signals are processed as three channels by three detection circuits. Dual functionality of the second receiver coil  eliminates  the  need  for  a  third  pair  of  coils,  thereby simplifying design and decreasing power needs.

 

 

 

 

Figure (modified) from reference [110] mentioned above.

 

@infobookcom tweet

 

A note on magnetometers

 

Magnetometers - Degaussing/Demagnetisation

 

Excerpt from Wikipedia: https://en.wikipedia.org/wiki/Magnetometer

"Military submarines are degaussed—by passing through large underwater loops at regular intervals—to help them escape detection by sea-floor monitoring systems, magnetic anomaly detectors, and magnetically-triggered mines. However, submarines are never completely de-magnetised. It is possible to tell the depth at which a submarine has been by measuring its magnetic field, which is distorted as the pressure distorts the hull and hence the field. Heating can also change the magnetization of steel".

 

Picture from https://en.wikipedia.org/wiki/Degaussing
"RMS Queen Mary arriving in New York Harbor, 20 June 1945, with thousands of U.S. soldiers – note the prominent degaussing coil running around the outer hull".

 

 

@infobookcom tweet

 

 

 

 

Remote detection of human electroencephalograms using ultrahigh input impedance electric potential sensors

(2002)

C. J. Harland, T. D. Clark, R. J. Prance
Centre for Physical Electronics, School of Engineering and Information Technology, University of Sussex
http://www.sussex.ac.uk/ei/
 
University of Sussex link: http://sro.sussex.ac.uk/19534/
Patent (cited above): https://www.google.com/patents/WO2003048789A2?cl=en
 

Publication: https://www.researchgate.net/publication/271498241_Brain_activity_measurement_in_the_occipital_region_of_the_head_using_a_magneto-impedance_sensor

 

Excerpts:

"(...) for conventional clinical analysis, the EEG is recorded using electrodes that are in real charge current contact with the scalp tissue. In practice, this is provided by an Ag/AgCl electrode used in conjunction with an electrolytic paste. These act together to form an electrical transducer to convert the ionic current flow in the skin into an electron flow which can then be detected by an electronic amplifier [3]."

 

"This sensor operates using electric displacement, rather than real charge, current. It therefore does not require real electrical contact to the signal source (e.g., the human body) in order to function. This overcomes the disadvantages of conventional Ag/AgCl electrodes while still maintaining sufficient sensitivity ~70 nV/AHz at 1 Hz noise floor) [6] to detect body electrical signals off body."

 

"Furthermore, using similar techniques, we have been able to record the human heartbeat at distance up to 1 m off body with no electrical connection between the sensor and the body [6]. In this letter, we demonstrate that we now have the sensitivity to detect EEGs through the scalp hair with no electrical contact to the scalp. We also show that it is now possible to detect brain electrical activity with an air gap between the scalp hair and the sensor."

 

"Given that there is still much scope for improvements in sensitivity, input impedance, and size of electric potential sensors, we anticipate rapid development and exploitation of these devices in the field of brain research and diagnostics in the near future."

 

Many thanks to Skizit Gesture's YouTube channel for mentioning the paper in one of her videos and to Neal Chevrier for sharing the video.

 

@infobook tweet

 

 

Remote control brain sensor (via BBC News - 2002)

Featuring above study

 

http://news.bbc.co.uk/2/hi/health/2361987.stm

 

Excerpt:

"From a distance"
"Instead of measuring electric current flow through a fixed-on electrode, the new method takes advantage of the latest developments in sensor technology to measure electric fields from the brain without actually having to make direct contact with the head."

.

 

@infobook tweet

 

 

 

 

"Human Signature Detection" - Biometrics

"Human Thermal Fingerprint at Long Distance"

slide 13
https://www.wired.com/images_blogs/dangerroom/files/Richardson_Continuous.pdf
 
Human Biometrics - Moving Towards Thermal Imaging
https://pdfs.semanticscholar.org/9916/f3c53f62f256d9d62e9a1edf4b04cd7f8c5e.pdf