Driver Drowsiness Detection System

13 Pages·2015·0.41 MB·English

by  Uppal, Abhinav

Georgia Institute of Technology

Driver Drowsiness Detection System

Measuring heart rate variability on an embedded system

Yang Liu
Adhithya Rajasekaran
Abhinav Uppal
Yichen Wang

ECE 4781: Biomedical Sensing Systems
Design Project Report

December 3, 2015

(all members made an excellent contribution to the project)
A. Executive Summary

Sleep deprived driving is now a serious problem in the United States and worldwide,
resulting in a large number of deaths and injuries.1 Most available drowsiness monitoring
systems for drivers rely on either vehicle-based or behavioral based sensing. However, these
systems are unable to alarm drivers until after they have fallen asleep, making these devices
unreliable in preventing accidents. To that end, we have developed a physiologically-based
system that captures the electrocardiogram (ECG) of a user and assesses his / her drowsiness
level by analyzing heart rate variability (HRV.)
A user’s ECG was transduced from the body using gel-based electrodes. The signal
was then filtered using an integrated analog-front end, sampled with an ADC at 333 Hz, and
serially transmitted to computer via a USB isolator. The digitized ECG signal was processed
using Physionet’s WaveForm DataBase (WFDB) library and HRV toolkit to extract the R-R
time intervals, and hence, the heart rate variability (HRV.) Finally, the power spectrum
density of HRV was plotted to view its spectrum in the frequency domain. Our results
suggest a good correlation between the low frequency band (0.04-0.15 Hz) over high
frequency band (0.15-0.40 Hz) power density (LF/HF) ratio and drowsiness. A lower ratio
indicated that the user was in a drowsier state. Therefore, using this system, we could
physiologically monitor a driver’s drowsiness level, and issue warnings in time to reduce
accidents.
In the future, our focus would be on the development of an ear-based ECG sensing
system to improve wearability of the device. Also, an ear-based photoplethysmogram (PPG)
sensing system could also be developed to achieve the same goal without the use of
electrodes.
B. Introduction and Significance

Each year, over 37,000 people die in road crashes in the United States, and nearly 1.3
million people die in road crashes worldwide. While we hear a lot about the shocking
statistics of car accidents, we may not know some astonishing facts about the causes
underlying these crashes. One major cause of car accidents is sleep-deprived driving (i.e.
tired driving, drowsy driving, or fatigued driving). According to a poll by the National Sleep
Foundation, 60% of adult drivers admitted that they felt drowsy while driving in 2014, and
over one-third drivers said that they had actually fallen asleep while driving. According to the
National Highway Traffic Safety Administration, sleep-deprived driving is responsible for
1,550 deaths, 71,000 injuries, and monetary costs of $12.5 billion annually in the US.1
In recent years, many in-vehicle technologies have been available to address real time
monitoring of a driver’s state. However, most of these monitoring devices and systems assess
drivers’ drowsiness by measuring behavioral aspects such as eye closure, pupil occlusion, etc.
by implementing state-of-the-art computer vision technologies using remotely mounted
cameras.2,3,4 Another approach for assessing drowsiness uses vehicle based measurements.
For example, the number of micro-corrections on the steering wheel is measured for detecting
drowsiness level.5 Even though both vehicle-based and behavioral-based measures are
accurate, they become apparent only after the driver starts to sleep, which is often too late to
prevent an accident.1
A few emerging technologies have attempted to measure drowsiness using
physiological measurements such as brain wave activity and heart rate.6,7 The reliability of
physiological measurements for drowsiness detection is very high, as physiological signals
start changing in earlier stages of drowsiness. Nevertheless, one problem with current
physiological measurements is that they are intrusive, as they require electrode placements
that could interfere with driving comfort.1
To our best knowledge, only a few of the currently employed in-vehicle devices can
accurately assess and quantify drivers’ consciousness, including the state of alertness,
drowsiness, sensory awareness, etc. To this end, we worked on designing a wearable device
that can measure physical parameters correlated with stages of drowsiness, and could
potentially send automatic alerts to the driver based on its assessment of his/her drowsiness
level.
According to past studies, HRV is related with drowsiness.1,6 Deriving measures of
HRV requires accurate determination of successive heart beats. For this purpose, ECG is a
widely used non-invasive method for extracting R-R peaks for HRV measurement. Our
design thus employed ECG for HRV measurement. We also began development on an
analogous system employing PPG. The ratio of LF to HF in the ECG decreases as the driver
has an increased level of drowsiness.6,8
C. Project Narrative

This project was divided into two parallel branches, each approaching the problem of
measuring heart rate (and hence HRV) by sensing and processing either ECG or PPG. Fig. 1.
shows the hardware and software components associated with both of these branches, and the
rest of this section elaborates on the analog front-ends (AFEs), sampling approaches, and
analysis tools used for obtaining a user’s HRV at different levels of drowsiness.

Figure 1. Various stages of the project, showing the hardware and software tools utilized.

Developing a PPG Sensing Platform using AFE4400

Given the size constraints associated with performing a non-obtrusive measurement of
heart rate (variability) in a driving setting, we decided to base our sensing system on a
compact, integrated AFE. To our knowledge, the Texas Instruments (TI) AFE440010 is the
only integrated AFE available in the market dedicated to PPG capture, and with the
associated evaluation module being relatively big in size, and costing $149, we decided to
design a smaller evaluation platform based on reference designs provided by TI10.

AFE4400: Features and Design Considerations
The AFE4400 is a highly configurable AFE, with an integrated H-bridge LED driver
(on the Tx) and a transimpedance amplifier (on the Rx) with digitally adjustable parameters.
The chip supports an SPI interface for programming registers and transferring data, digital
control lines for issuing interrupts to a microcontroller, and a range of supplies for supporting
the operation of different LEDs and detectors.
To simplify our prototyping, we decided to design a board that provided pads for
easily interfacing the AFE with a microcontroller, LED, and detector by soldering connecting
wires. We also chose to breakout the supply pins for performing initial testing of the chip and
LEDs using bench top-supplies, which could subsequently be replaced with a power
regulation daughter card once the prototype’s current draw and voltage requirements have
been characterized.
For the LED(s) and detector, we chose to use Osram’s SFH-705011 in our system, as it
integrates three LEDs (red, green, and IR), and a detector in one compact package, making it
easier to experiment with different LED placements and wavelengths for sensing PPG.
Schematic Capture and Layout
An open source electronic design automation software, KiCAD12, was used to make
the platform’s schematic, generate the required footprints, and layout a 2-layer PCB. The
schematic and layout were then posted on TI’s e2e medical forum for review13. Feedback
from a TI applications engineer, Praveen Aroul, was used to revise the PCB design (including
removal of extraneous components.) We then requested the School of ECE to mill the revised
PCB, and the milling was completed on December 1, 2015. Team Morpheus would like to
thank Mr. James Steinberg for his help. Unfortunately, the PCB’s vias did not plate, hence
the board development is on hold for now.
Figure 2. Top layer of the 2-layer AFE4400 PCB. The SFH-7050 was mounted on a separate 2-layer PCB.

Designing an ECG system for extracting HRV

Being aware of the delays inherent to PCB layout and milling, we worked on an ECG-
based sensing and HRV analysis system in parallel to designing the AFE4400 board. A high-
level block diagram of this system is shown in Fig. 3.

Figure 3. ECG-based HRV analysis system, for capturing ECG and processing it on a computer.

AD8232: Single Lead ECG AFE
With the same size concern as with the PPG solution, we chose to go with an
integrated AFE for conditioning an ECG signal acquired by gel electrodes placed across a
user’s chest, with a reference electrode on the right ankle. The Analog Devices AD8232 chip
was used as the ECF AFE because of the availability of a low-cost development board from
SparkFun14.
SparkFun’s AD8232 board implements the “cardiac monitoring configuration”
provided in AD8232’s datasheet15. This configuration has a band-pass frequency response
(Fig. 4.) suitable for monitoring the shape of the ECG waveform, and it assumes that the
signal is relatively free of motion artifact (with the subject remaining stationary.) As this may
not always apply to our target users (drivers), the chip can be reconfigured to have a narrow
pass-band. For this project, however, we chose to restrain our motion during subsequent
measurements, so as to focus development efforts towards digitizing and analyzing the ECG.

Figure 4. Cardiac monitoring configuration of the AD8232 (left), highlighting its cascaded high pass and low
pass filtering stages. This configuration results in a band-pass response (right.)

A sampling of the output of AD8232 is shown in Fig. 5. The subject remained still while
these measurements were taken, and the analog output of the AFE was sampled using an
MSP432 (see next section.)

Figure 5. Amplified ECG outputted by AD8232, digitized with MSP432 and plotted in MATLAB.

Sampling AD8232’s output with MSP432 LaunchPad
The Texas Instruments MSP432 LaunchPad is a low-cost microcontroller platform
with a 14-bit 1MSPS ADC16. It also supports a back-channel UART for transmitting signals
to a computer over USB. With access to a free LaunchPad from one of Texas Instruments’
development workshops held at Georgia Tech this October, we decided to implement a
sampling routine for sending sampled data to a computer over USB.
Initially, the Arduino-based Energia IDE was used to program the MSP432 to sample
and transmit data in a loop, with fixed delays inside the loop being used to enforce a sampling
time of 4 ms. The sampling time was observed to be variable over a few milliseconds (the in-
built function was used to output time-stamps along with the sampled data.) To
millis()
get a consistent sampling rate, attempts were made to switch to TI’s CodeComposerStudio
IDE for writing an interrupt service routine for triggering ADC captures from a timer. While
these attempts were being debugged, we came across Energia’s “Multitasking” feature
employing TI’s real-time operating systems (RTOS), which was used to give a consistent
sampling period of 3 ms (Fig. 6.)
We also found generating time stamps on the MSP432 board itself to be more reliable
as compared to using the computer receiving data over USB for timing the captures.
Processing3 was run on a Windows machine to capture the serial data outputted by the
MSP432, and variability in sampling time (as stamped by Processing3) can be seen in Fig. 6.
Hence, the MSP432 was used to generate reliable time-stamps for all subsequent data
captures, and Processing3 was used to write captured ECG data to a file.

Figure 6. A comparison of intervals between consecutive timestamps generated by the MSP432 (RTOS) with
each serial (UART) write (top), and the timestamps generated by Processing3 with each line write to the data
collection file. RTOS is seen to be more reliable (only 4 out of 204828 intervals are off) for calculating the
ECG’s sampling frequency than Processing3 (running on a non-real time, Windows OS.)

QRS detection and HRV analysis with Physionet’s software tools
With access to the ECG data thus generated by the MSP432 and Processing3,
Physionet’s software tools (WFDB17 and HRV-toolkit18) were setup to perform analysis on
different team members’ computers. As a result, different parts of the ECG analysis were
performed on different platforms (Fig. 7.): Windows (running Cygwin), Ubuntu, and Debian
(kernel 3.14), Debian (kernel 3.8). Regular laptops were used to serve as the Windows and
Ubuntu machines, and two BeagleBone Blacks (BBBlack) ran different linux kernels for
Debian. For ease of use, we have begun porting all steps of Fig. 7. to the Beaglebone Blacks,
however, the ports have not been completely verified this far.

Figure 7. Physionet’s WFDB and hrv-toolkit commands used for analyzing sampled ECG signals. The tasks
were distributed among different team members’ computers, but these can also be executed on a single system
(BBBlack).

Various functions from the Physionet tool-kits were used to analyze the ECG data
collected using the MSP432 and Processing3. The sampled data was first converted to a
Physionet compatible record using the function. Then, the function was used
wrsamp gqrs
to annotate these record files with QRS peak detections. The annotations were then verified
by using , and only a few (less than 10) errors were observed over three datasets
wave
spanning approx. 10 minutes each. These errors were manually corrected using the
wave
tool.
Next, the QRS annotation files were used to obtain RR intervals with , which
ann2rr
were then plotted using the and functions for generating FFTs and lomb
hrfft hrlomb
periodograms on the Windows machine running Cygwin. The lomb periodograms for three
different datasets (corresponding to different levels of the user’s self-reported drowsiness) are
provided in Fig. 9. On the other hand, the function from Physionet’s HRV toolkit
get_hrv
was used on the same Physionet record files (transferred from Windows to BBBlack over
sftp) to obtain LF/HF ratios summarized in Fig. 10. Fig. 11. shows the functions
get_hrv
output as printed to the BBBlack’s console.

Porting all processing to one BBBlack
As BBBlack is a complete Linux computer running the Debian operating system
supporting analog inputs, we decided to implement everything from sampling the ECG to
analyzing HRV on one BBBlack (Fig. 8.) The Physionet tools have been installed and tested
independently. The entire chain runs through one node.js script, but the script is yet to be
tested with the the AD8232 board connected to the BBBlack.
Figure 8. Node.js implementation for performing ECG sampling and analysis on one embedded platform.

Figure 9. Cygwin generated Lomb periodograms (least-squares spectra) associated with three 10-minute ECG
datasets collected with different levels of drowsiness. Later data collection times were assumed to be correlated
with more drowsiness. The LF/HF ratio is seen to increase with the subject’s drowsiness (top to bottom.)
Figure 10. Comparison of LF/HF ratios obtained for three ten-minute ECG datatsets collected on the same user
reporting different levels of drowsiness over different days.

Figure 11. Screenshot of the HRV analysis results outputted to the BBBlack’s console.