Dr. Santosh KumarWhen researchers use sensor data to identify when a person is smoking, they face a conundrum: How to discern the act of taking a puff from the myriad gestures a person can make?
There are a variety of sensors at the researchers’ disposal, Dr. Santosh Kumar told attendees at a research symposium held in late June as part of events held to mark the 20th anniversary of the Office of Behavioral and Social Sciences Research (OBSSR) at the National Institutes of Health.
Kumar is director of the MD2K Center, funded by the National Institutes of Health as part of its Big Data to Knowledge (BD2K) initiative. MD2K’s research is aimed at using mobile technology – sensors that are worn on or near the body or contained in the mobile phone – to help identify and predict the risk factors that lead to disease. Ultimately, the information gained through the research will be used to provide interventions aimed at improving a person’s health. The research center brings together experts in the fields of computer science, engineering, medicine, behavioral science and statistics from 12 universities and the non-profit Open mHealth.
Solving the challenge of identifying smoking gestures in mobile sensor data is one of the goals for the center.
“Gradually mobile phones have certainly become part of our habit that we take with us every day,” Kumar said. “In the future, smart watches could also become part of our habit, and if these are all part of our habit it will be easy for us to try and see what we can use to monitor and improve people’s health.”
Two areas MD2K research focuses on are improving outcomes of smoking cessation efforts and reducing hospital readmissions for congestive heart failure patients.
It’s hoped the research will ultimately lead to a more precise practice of medicine that is tailored to an individual’s personal history.
| Related article |
|---|
| Widespread Use of Mobile Tech is Game-Changer |
“Suppose we use genomics data to find that someone is at risk of hypertension,” Kumar said. “Certainly the person can become more cautious and cognizant, but still the onset of a particular disease is hard to determine unless we are monitoring the person at all time.”
Thanks to the ubiquitous nature of the mobile phone, this is now possible.
The sensors used by MD2K researchers are AutoSense (a chest band with respiration and electrocardiogram (ECG) sensors), a smartwatch that has inertial sensors, a smart eyeglass that captures visual exposure and EasySense, a contactless sensor that measures physiology through monitoring the movement of ultra-wideband radio waves through the body, which provide information on heart and lung movement and fluid levels in the lungs. These are combined with the sensors contained within the mobile phone itself – an accelerometer, gyroscope and global positioning system (GPS).
MD2K’s research hopes to employ mobile health – the use of sensor technology to improve health outcomes – along with other technologies to aid in the early detection of the disease. Then it can also help measure the progression of the disease and determine risk factors that are associated with it and at what levels they are indicators of a worsening or improvement in a person’s health. If risk factors can be identified, they can be used as predictors and help determine if and when to provide an intervention or other preventive measure.
“If we can measure progression and then we measure other risk factors together with it,” Kumar said, “ We can try to identify what risk factors at what level can lead to what worsening or improvement in disease conditions or symptoms.
“Then we can use those risk factors that are also detected by sensors as predictors and we can indeed get to the preventive medicine.”
The research will also capture the outcome to determine whether interventions that are delivered worked, whether the person complied and whether they liked receiving the intervention.
This kind of personalization of medicine made possible by mobile devices is envisioned as the future of medicine, and is one of the goals of the Precision Medicine Initiative (PMI) announced in January by President Obama. The PMI seeks to establish a research cohort of 1 million people.
In addition to detecting physiological states, mobile sensors can also determine what the user is doing at a particular time and tailor any interventions accordingly. For example, if the user is driving a car it would be dangerous to send a message to their mobile phone. Likewise, a person is going to be less likely to comply with an intervention if it is sent while they are engaged in conversation, in a meeting or on the phone.
In the study involving smoking cessation, Kumar said that researchers are able to use GPS data to determine a participant’s geoexposure, i.e., whether they are near a point of sale for tobacco or a location that might trigger tobacco use, such as a bar.
“We can also measure stress levels from physiology on a continuous basis and we can measure social conflicts by using the microphone on a mobile phone,” Kumar said. “So we have all these measurements, but how do we capture when it is that the first lapse occurs?”
Traditionally, a study participant self-reports smoking lapses. But with mobile data, they can add a time frame because the data shows when a person is driving, when they are at a gas station, at the office, indoors or outdoors, at home, or in conflict.
“But, if we depend on self-report to pinpoint the timing of the lapse, then how accurate can we be?” Kumar said. “That’s one of the questions we sought to study.”
If the goal is to develop a method to detect a smoking lapse from sensor data, Kumar said, “Then we have the precise timing of when the person lapsed, and then we can go back in time and look at the precise place where they had been prior to the lapse, at their stress levels and any other sensor derived risk factor that we have.”
That information can be used to find predictors that can be used to deliver a sensor-triggered just-in-time intervention.
“This could improve the rate of smoking cessation that we have today,” Kumar said.
But, there are challenges. For one, the sensor itself:
- It needs to be wearable so that study participants will wear it night and day, and not take it off once they are out of the lab.
- It needs to be safe, and not cause any problems for the user
- It should be reliable so that the data is trustworthy
- It should be robust so it can be worn all day while the participant is engaged in a variety of activities
- It should be versatile so that a study participant doesn’t have to wear multiple sensors all the time.
“We shouldn’t require participants to wear one sensor for the ECG, one for eating, one for activity and one for transportation,” Kumar said. “We are asking them to wear one or two sensors and from them we need to derive as many possible measurements as we can.”
In addition, the software powering the sensors should be reliable and able to retain all the data. Sensors generally operate wirelessly, so the possibility of a loss of data needs to be accounted for. The sensor batteries need to be strong enough that recharging is only required periodically.
“If this all works and we are able to get the participants to wear it, then what we get are basically just a bunch of bits and bytes, zeros and ones,” Kumar said. “From that we need to interpret: What does it represent? We need to convert that signal from the sensors into what was happening with the person.To detect smoking from the movement of a person’s wrist would mean determining that they raised their hand to their mouth, took a puff and then dropped it back down."
Right? Not so fast.
“The same thing could happen when we are eating, the same thing could happen when we scratch right here (at the chin), the same thing could happen when I’m yawning,” Kumar noted.
Similarly, if trying to detect eating, some people eat with a utensil in one hand, some pick up a sandwich with two hands. People use very similar gestures to do very different activities.
How can researchers determine which gesture indicates smoking and which does not? It’s a question data science researchers find exciting, Kumar said.
To help distinguish the smoking gesture from the random yawn, researchers combine that with data on respiration.
“What people do with smoking, they also take a deep breath; deep inhalation and exhalation,” Kumar said. “So if we can synchronize the two, and see the hand come to the mouth and as the hand is leaving the mouth, see deep inhalation and exhalation in the breathing pattern, then if the two occur together there is a much greater chance this represents smoking.”
This method was applied to a study MD2K is presently conducting, and out of 54 people, 31 reported a smoking lapse. Of those 31, the method accurately pinpointed the timing of the lapse in 27. Data for one participant was discarded due to data loss.
The next task will be to study the other sensor data to determine what else was going on at the time the lapse occurred. If researchers can identify the risk factors leading up to a lapse, they can use that information to determine the best time to deliver an intervention.
Kumar said he considers the current results to be “zero generation,” meaning they require a lot of further testing and study before they will reach the level of maturity of more traditional research methods.“They’re not here to replace our traditional measures. They’re here to complement them – to provide some information that we didn’t have.”
He said there are still a lot of questions to be answered:
- Validity: “Even when the data suggests a first lapse, how do we validate that? It’s possible that the first lapse, when that occurred the data for that was lost, and what we captured is not the first lapse, it’s the second lapse. So what method do we use to validate the temporal precision that we get? That’s an open question . . . the gold standard doesn’t exist to assess temporal precision, but we need to make do with what we have.”/li>
- Limitations: “What are the limitations? When can we use them and when can we not use them? What we develop with cigarette smoking isn’t applicable to cigar smoking. This is not applicable to several people sharing the same cigarette.”/li>
- Usability: “So we develop this method. At this point, my students are going to know how to write that code. But if it’s limited to them, then it’s not useful to the community. So the software should be such that it can be used by anybody else in the room or anyone else interested in this problem. It should work not just on one sensor, or just on Microsoft Band or Apple Watch, but it should work with a variety of sensors that are there.”/li>
- Utility: Once interventions are developed using the MD2K software, they will be refined to be scientifically useful. The concept of just-in-time interventions is a new one, so there are a lot of potential areas of research, Kumar said.
All of the software developed by MD2K will be open-source and made available to the public for its use, Kumar said.
MD2K is a collaborative effort involving 12 universities and the non-profit Open mHealth, and includes researchers from data science, statistics, engineering, behavioral health and medicine.
Here is a video of Kumar's presentation to the Symposium. It's followed by a talk by Dr. Bonnie Spring of Northwestern University, who is a professor of preventive medicine, psychology and psychiatry and Director of the NU Center for Behavior and Health. Spring also leads the MD2K research thrust on Smoking Cessation.browser does not support iframe