From extreme weather to another wave of COVID-19, forecasts give decision-makers valuable time to prepare. When it comes to COVID, though, long-term forecasting is a challenge, because it involves human behavior.
While it can sometimes seem like there is no logic to human behavior, new research is working to improve COVID forecasts by incorporating that behavior into prediction models.
UConn College of Agriculture, Health and Natural Resources Allied Health researcher Ran Xu, along with collaborators Hazhir Rahmandad from the Massachusetts Institute of Technology, and Navid Ghaffarzadegan from Virginia Tech, have a paper out today in PLOS Computational Biology where they detail how they applied relatively simple but nuanced variables to enhance modelling capabilities, with the result that their approach out-performed a majority of the models currently used to inform decisions made by the federal Centers for Disease Control and Prevention (CDC).
Xu explains that he and his collaborators are methodologists, and they were interested in examining which parameters impacted the forecasting accuracy of the COVID prediction models. To begin, they turned to the CDC prediction hub, which serves as a repository of models from across the United States.
“Currently there are over 70 different models, mostly from universities and some from companies, that are updated weekly,” says Xu. “Each week, these models give predictions for cases and number of deaths in the next couple of weeks. The CDC uses this information to inform their decisions; for example, where to strategically focus their efforts or whether to advise people to do social distancing.”
The Human Factor
The data was a culmination of over 490,000 point forecasts for weekly death incidents across 57 US locations over the course of one year. The researchers analyzed the length of prediction and how relatively accurate the predictions were across a period of 14 weeks. On further analysis, Xu says they noticed something interesting when they categorized the models based on their methodologies:
“For purely data-driven models, like machine learning and curve-fitting models, we found they do a better job predicting in the short-term, whereas theory-driven models do a better job predicting in the longer term.”
At first, this may seem strange, but Xu explains that the difference comes down to human behavior.
“It’s weird and not weird, per se,” Xu says. “If you don’t have theory and the models are just working with a bunch of data and machine learning, of course they are going to do a good job in the shorter term. But what really matters in the mid to long-term is you need to have a theory explaining why people do the things they do.”
Incorporating the behavioral component into the model was relatively simple, says Xu.
“When we looked at all those 60-70 models, we felt like there was a key behavior mechanism missing. That mechanism is when people see more death or they perceive COVID infections to be dangerous, then they voluntarily reduce their mobility or do social distancing. However, once the death rate decreases, people go back to their normal activity. In looking through the models, few of them are modeling this endogenous feedback loop.”
Insights for the Future
The researchers argue that this feedback loop, though largely overlooked by other models, offers the greatest benefit for mid- to long-term predictions.
Rahmandad says the research suggests the key to modeling for long-term predictions is not necessarily creating more complicated models, but strategically incorporating the right elements.
“To create a predictive model that is successful in the long-term, we can start small, using a simple, mechanistic model,” says Rahmandad. “We can then incorporate key mechanistic features — particularly the endogenous representation of human behavior in interaction with the evolving pandemic.”
When examining the models on the CDC hub, some do have behavioral components, says Xu, but few consider how they change over time or change as a function of how disease progresses.
“I think incorporating behaviors in infectious disease modeling and developing relevant behavioral theories is still an area that needs more research and we currently do not have comprehensive theories explaining how people behave during pandemic/infectious disease outbreak,” Xu says. “This requires collaboration among multiple disciplines, such as social scientists, epidemiologists, and methodologists.”
After incorporating the feedback loop, the researchers found the model performs very well at predicting the trajectory of COVID, and Xu emphasizes that this shows how important it is to incorporate behavioral dynamics into infectious disease modeling.
“The purpose of developing this model is not to offer real-time predictions, but it may offer insights for future prediction models. This simple model is doing even better, especially in the longer term.”
