Sensors

Development and validation of any new mHealth biomarker requires conducting research studies in lab and field settings to collect raw sensor data with appropriate labels (e.g., self-reports). Raw sensor data are of increasing interest as it significantly expands the useful life of the information collected. Biomedical studies often archive biospecimens in biobanks so they can be reprocessed to take advantage of future improvements in assays and support biomedical discoveries not possible at the time of data collection. In similar fashion, archived raw sensor data can be used to obtain new biomarkers that were not available at the time of data collection.

sensor approach

For example, if the activity trackers stored raw sensor data from accelerometers and gyroscopes (100+ HZ instead of few samples of activity counts per day), the same sensor data can also be used to track eating, drinking, brushing, smoking, etc., from hand gesture signatures, in addition to activity counts.

Therefore, MD2K natively supports collection of high-frequency raw sensor data and its real-time wireless streaming to the study smartphone, in order to facilitate triggering of notifications triggered by real-time computation of biomarkers from these sensor data. Such notifications may be used to confirm/refute the biomarker detection for field validation, to collect self-reports to understand the surrounding context, or to deliver a just-in-time intervention. Such high-frequency collection and streaming of mobile sensor data places significant constraints on battery life, and most consumer-grade sensors are not optimized to last the entire day in such a mode of collecting and streaming raw sensor data. To overcome this challenge, MD2K has developed a variety of new sensors that provide this capability and still last the entire day or longer.

Several sensors have been developed and deployed by MD2K in mHealth field studies:

• EasySense —It is a contactless microradar sensor that can detect heart and lung motion and assess change in the lung fluid level;

• MotionSenseHRV — It is a wrist-worn sensor that can measure hand gestures via accelerometers and gyroscopes and interbeat intervals via optical sensors for computing heart rate variability indices; and

• AutoSense — It is a chest-worn sensor suite that can measure cardiorespiratory parameters via ECG and respiration, and movement of the torso via accelerometers.

• iShadow — Our team has also developed computational eyeglasses, which are currently being evaluated for its utility in assessing fatigue and visual exposure to cues (e.g., alcohol advertisements).

Publications

James M Rehg, Susan A Murphy and Santosh Kumar (eds.).
A New Direction for Biosensing: RF Sensors for Monitoring Cardio-Pulmonary Function. pages 289–312, Springer International Publishing, 2017. URL, DOI BibTeX

@inbook{Gao2017,
	pages = "289--312",
	title = "A New Direction for Biosensing: RF Sensors for Monitoring Cardio-Pulmonary Function",
	publisher = "Springer International Publishing",
	year = 2017,
	author = "Gao, Ju and Baskar, Siddharth and Teng, Diyan and al'Absi, Mustafa and Kumar, Santosh and Ertin, Emre",
	editor = "Rehg, James M. and Murphy, Susan A. and Kumar, Santosh",
	address = "Cham",
	isbn = "978-3-319-51394-2",
	abstract = "Long-term monitoring of physiology at large-scale can help determine potential causes and early biomarkers of chronic diseases. Physiological monitoring today, however, requires wearing of sensors such as electrodes for ECG and belt around lungs for respiration, and is unsuitable for monitoring of patients and healthy adults over multiple years. In this chapter, we review advances in a novel sensing modality using radio frequency (RF) waves that can provide physiological measurements without skin contact in both lab and field environments. This chapter presents fundamentals of RF biosensing with experimental results of a new experimental bioradar platform illustrating the concepts. The focus is on new approaches to monitor heart motion and respiratory effort. Experimental results using both an articulated heart phantom and human subjects show that RF sensing modality can match the performance of state-of-the-art physiological monitoring devices in terms of retrieving features and statistics of clinical significance.",
	booktitle = "Mobile Health: Sensors, Analytic Methods, and Applications",
	doi = "10.1007/978-3-319-51394-2_15",
	url = "https://doi.org/10.1007/978-3-319-51394-2_15"
}

Saman Naderiparizi, Pengyu Zhang, Matthai Philipose, Bodhi Priyantha, Jie Liu and Deepak Ganesan.
Glimpse: A programmable early-discard camera architecture for continuous mobile vision. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. 2017, 292–305. URL BibTeX

@inproceedings{naderiparizi2017glimpse,
	author = "Saman Naderiparizi and Pengyu Zhang and Matthai Philipose and Bodhi Priyantha and Jie Liu and Deepak Ganesan",
	title = "Glimpse: A programmable early-discard camera architecture for continuous mobile vision",
	booktitle = "Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services",
	year = 2017,
	pages = "292--305",
	organization = "ACM",
	pubstate = "published",
	tppubtype = "inproceedings",
	url = "https://md2k.org/images/papers/sensors/mobisys17-Glimpse.pdf"
}

Pan Hu, Pengyu Zhang, Mohammad Rostami and Deepak Ganesan.
Braidio: An Integrated Active-Passive Radio for Mobile Devices with Asymmetric Energy Budgets. In Proceedings of the 2016 Conference on ACM SIGCOMM 2016 Conference. 2016, 384–397. URL, DOI BibTeX

@inproceedings{Hu:2016:BIA:2934872.2934902,
	author = "Pan Hu and Pengyu Zhang and Mohammad Rostami and Deepak Ganesan",
	title = "Braidio: An Integrated Active-Passive Radio for Mobile Devices with Asymmetric Energy Budgets",
	booktitle = "Proceedings of the 2016 Conference on ACM SIGCOMM 2016 Conference",
	year = 2016,
	series = "SIGCOMM '16",
	pages = "384--397",
	address = "Florianopolis, Brazil",
	publisher = "ACM",
	abstract = "While many radio technologies are available for mobile devices, one of them are designed to deal with asymmetric available energy. Battery capacities of mobile devices vary by up to three orders of magnitude between laptops and wearables, and our inability to deal with such asymmetry has limited the lifetime of constrained portable devices. This paper presents a radically new design for low-power radios — one that is capable of dynamically splitting the power burden of communication between the transmitter and receiver in proportion to the available energy on the two devices. We achieve this with a novel carrier offload method that dynamically moves carrier generation across end points. While such a design might raise the specter of a high-power, large form-factor radio, we show that this integration can be achieved with no more than a BLE-style active radio augmented with a few additional components. Our design, Braidio is a low-power, tightly integrated, low-cost radio capable of operating as an active and passive transceiver. When these modes operate in an interleaved (braided) manner, the end result is a power-proportional low-power radio that is able to achieve 1:2546 to 3546:1 power consumption ratios between a transmitter and a receiver, all while operating at low power.",
	doi = "10.1145/2934872.2934902",
	isbn = "978-1-4503-4193-6",
	keywords = "Backscatter; Wireless; Architecture; Asymmetric; Energy",
	pubstate = "published",
	tppubtype = "inproceedings",
	url = "https://md2k.org/images/papers/SSENSORS/p384-hu.pdf"
}

Addison Mayberry, Yamin Tun, Pan Hu, Duncan Smith-Freedman, Benjamin Marlin, Christopher Salthouse and Deepak Ganesan.
CIDER: Enhancing the Performance of Computational Eyeglasses. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications. 2016, 313–314. URL, DOI BibTeX

@inproceedings{Mayberry:2016:CEP:2857491.2884063,
	author = "Addison Mayberry and Yamin Tun and Pan Hu and Duncan Smith-Freedman and Benjamin Marlin and Christopher Salthouse and Deepak Ganesan",
	title = "CIDER: Enhancing the Performance of Computational Eyeglasses",
	booktitle = "Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research \& Applications",
	year = 2016,
	series = "ETRA '16",
	pages = "313--314",
	address = "Charleston, South Carolina",
	publisher = "ACM",
	abstract = "The human eye offers a fascinating window into an individual's health, cognitive attention, and decision making, but we lack the ability to continually measure these parameters in the natural environment. We demonstrate CIDER, a system that operates in a highly optimized low-power mode under indoor settings by using a fast Search-Refine controller to track the eye, but detects when the environment switches to more challenging outdoor sunlight and switches models to operate robustly under this condition. Our design is holistic and tackles a) power consumption in digitizing pixels, estimating pupillary parameters, and illuminating the eye via near-infrared and b) error in estimating pupil center and pupil dilation. We demonstrate that CIDER can estimate pupil center with error less than two pixels (0.6°), and pupil diameter with error of one pixel (0.22mm). Our end-to-end results show that we can operate at power levels of roughly 7mW at a 4Hz eye tracking rate, or roughly 32mW at rates upwards of 250Hz.",
	doi = "10.1145/2857491.2884063",
	isbn = "978-1-4503-4125-7",
	keywords = "eye tracking, low-power sensing, mHealth, pupilometry",
	pubstate = "published",
	tppubtype = "inproceedings",
	url = "https://md2k.org/images/papers/sensors/p313-mayberry.pdf"
}

Addison Mayberry, Yamin Tun, Pan Hu, Duncan Smith-Freedman, Deepak Ganesan, Benjamin M Marlin and Christopher Salthouse.
CIDER: Enabling Robustness-Power Tradeoffs on a Computational Eyeglass. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. 2015, 400–412. URL, DOI BibTeX

@inproceedings{Mayberry:2015:CER:2789168.2790096,
	author = "Mayberry, Addison and Tun, Yamin and Hu, Pan and Smith-Freedman, Duncan and Ganesan, Deepak and Marlin, Benjamin M. and Salthouse, Christopher",
	title = "CIDER: Enabling Robustness-Power Tradeoffs on a Computational Eyeglass",
	booktitle = "Proceedings of the 21st Annual International Conference on Mobile Computing and Networking",
	year = 2015,
	series = "MobiCom '15",
	pages = "400--412",
	address = "Paris, France",
	publisher = "ACM",
	abstract = "The human eye offers a fascinating window into an individual's health, cognitive attention, and decision making, but we lack the ability to continually measure these parameters in the natural environment. The challenges lie in: a) handling the complexity of continuous high-rate sensing from a camera and processing the image stream to estimate eye parameters, and b) dealing with the wide variability in illumination conditions in the natural environment. This paper explores the power--robustness tradeoffs inherent in the design of a wearable eye tracker, and proposes a novel staged architecture that enables graceful adaptation across the spectrum of real-world illumination. We propose CIDER, a system that operates in a highly optimized low-power mode under indoor settings by using a fast Search-Refine controller to track the eye, but detects when the environment switches to more challenging outdoor sunlight and switches models to operate robustly under this condition. Our design is holistic and tackles a) power consumption in digitizing pixels, estimating pupillary parameters, and illuminating the eye via near-infrared, b) error in estimating pupil center and pupil dilation, and c) model training procedures that involve zero effort from a user. We demonstrate that CIDER can estimate pupil center with error less than two pixels (0.6O), and pupil diameter with error of one pixel (0.22mm). Our end-to-end results show that we can operate at power levels of roughly 7mW at a 4Hz eye tracking rate, or roughly 32mW at rates upwards of 250Hz.",
	doi = "10.1145/2789168.2790096",
	isbn = "978-1-4503-3619-2",
	keywords = "eye tracking, near-infrared, neural network, power robustness tradeoff, pupil, wearable",
	pubstate = "published",
	tppubtype = "inproceedings",
	url = "https://md2k.org/images/papers/sensors/p400-mayberry.pdf"
}

A Mayberry, P Hu, B Marlin, C Salthouse and D Ganesan.
iShadow: Design of a Wearable, Real-time Mobile Gaze Tracker. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '14). 2014, 82–94. URL, DOI BibTeX

@inproceedings{Mayberry:2014:IDW:2594368.2594388,
author = "A. Mayberry and P. Hu and B. Marlin and C. Salthouse and D. Ganesan",
title = "iShadow: Design of a Wearable, Real-time Mobile Gaze Tracker",
booktitle = "Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '14)",
year = 2014,
series = "MobiSys '14",
pages = "82--94",
address = "Bretton Woods, New Hampshire, USA",
publisher = "ACM",
abstract = "Continuous, real-time tracking of eye gaze is valuable in a variety of scenarios including hands-free interaction with the physical world, detection of unsafe behaviors, leveraging visual context for advertising, life logging, and others. While eye tracking is commonly used in clinical trials and user studies, it has not bridged the gap to everyday consumer use. The challenge is that a real-time eye tracker is a power-hungry and computation-intensive device which requires continuous sensing of the eye using an imager running at many tens of frames per second, and continuous processing of the image stream using sophisticated gaze estimation algorithms. Our key contribution is the design of an eye tracker that dramatically reduces the sensing and computation needs for eye tracking, thereby achieving orders of magnitude reductions in power consumption and form-factor. The key idea is that eye images are extremely redundant, therefore we can estimate gaze by using a small subset of carefully chosen pixels per frame. We instantiate this idea in a prototype hardware platform equipped with a low-power image sensor that provides random access to pixel values, a low-power ARM Cortex M3 microcontroller, and a bluetooth radio to communicate with a mobile phone. The sparse pixel-based gaze estimation algorithm is a multi-layer neural network learned using a state-of-the-art sparsity-inducing regularization function that minimizes the gaze prediction error while simultaneously minimizing the number of pixels used. Our results show that we can operate at roughly 70mW of power, while continuously estimating eye gaze at the rate of 30 Hz with errors of roughly 3 degrees.",
doi = "10.1145/2594368.2594388",
isbn = "978-1-4503-2793-0",
keywords = "eye tracking, lifelog, neural network",
pubstate = "published",
tppubtype = "inproceedings",
url = "https://md2k.org/images/papers/sensors/p82-mayberry.pdf"
}

Emre Ertin, Nathan Stohs, Santosh Kumar, Andrew Raij, Mustafa and Siddharth Shah.
AutoSense: Unobtrusively Wearable Sensor Suite for Inferring the Onset, Causality, and Consequences of Stress in the Field. In Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems. 2011, 274–287. URL, DOI BibTeX

@inproceedings{Ertin:2011:AUW:2070942.2070970,
	author = "Ertin, Emre and Stohs, Nathan and Kumar, Santosh and Raij, Andrew and al'Absi, Mustafa and Shah, Siddharth",
	title = "AutoSense: Unobtrusively Wearable Sensor Suite for Inferring the Onset, Causality, and Consequences of Stress in the Field",
	booktitle = "Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems",
	year = 2011,
	series = "SenSys '11",
	pages = "274--287",
	address = "New York, NY, USA",
	publisher = "ACM",
	abstract = "The effect of psychosocial stress on health has been a central focus area of public health research. However, progress has been limited due a to lack of wearable sensors that can provide robust measures of stress in the field. In this paper, we present a wireless sensor suite called AutoSense that collects and processes cardiovascular, respiratory, and thermoregularity measurements that can inform about the general stress state of test subjects in their natural environment. AutoSense overcomes several challenges in the design of wearable sensor systems for use in the field. First, it is unobtrusively wearable because it integrates six sensors in a small form factor. Second, it demonstrates a low power design; with a lifetime exceeding ten days while continuously sampling and transmitting sensor measurements. Third, sensor measurements are robust to several sources of errors and confounds inherent in field usage. Fourth, it integrates an ANT radio for low power and integrated quality of service guarantees, even in crowded environments. The AutoSense suite is complemented with a software framework on a smart phone that processes sensor measurements received from AutoSense to infer stress and other rich human behaviors. AutoSense was used in a 20+ subject real-life scientific study on stress in both the lab and field, which resulted in the first model of stress that provides 90% accuracy.",
	acmid = 2070970,
	doi = "10.1145/2070942.2070970",
	isbn = "978-1-4503-0718-5",
	keywords = "deployment experiences, mobile health, psychological stress monitoring, wearable physiological sensors",
	location = "Seattle, Washington",
	numpages = 14,
	url = "https://md2k.org/images/papers/sensors/p274-ertin_AutoSense.pdf"
}