A little over a year since its launch, ChatGPT’s abilities are well known. The machine learning model can write a decent college-level essay and hold a conversation in an almost human-like way.
But could its language skills also help first responders find those in distress during a natural disaster?
A new University at Buffalo-led study trains ChatGPT to recognize locations, from home addresses to intersections, in disaster victims’ social media posts.
Supplied with carefully constructed prompts, researchers’ “geoknowledge-guided” GPT models extracted location data from tweets sent during Hurricane Harvey at an accuracy rate 76% better than default GPT models.
“This use of AI technology may be able to help first responders reach victims more quickly and even save more lives,” said Yingjie Hu, associate professor in the UB Department of Geography, within the College of Arts and Sciences, and lead author of the study, which was published in October in the International Journal of Geographical Information Science.
Disaster victims have frequently turned to social media to plead for help when 911 systems become overloaded, including during Harvey’s devastation of the Houston area in 2017.
Yet first responders often don’t have the resources to monitor social media feeds during a disaster, following the various hashtags and deciding which posts are most urgent.
It is the hope of the UB-led research team, which also includes collaborators from the University of Georgia, Stanford University and Google, that their work could lead to AI systems that automatically process social media data for emergency services.
“ChatGPT and other large language models have drawn controversy for their potential negative uses, whether it be academic fraud or eliminating jobs, so it is exciting to instead harness their powers for social good,” Hu says.
“While there are a number of significant and valid concerns about the emergence of ChatGPT, our work shows that careful, interdisciplinary work can produce applications of this technology that can provide tangible benefits to society,” adds co-author Kenneth Joseph, assistant professor in the UB Department of Computer Science and Engineering, within the School of Engineering and Applied Sciences.
Fusing ‘geoknowledge’ into ChatGPT
Imagine a tweet with an urgent but clear message: A family, including a 90-year-old not steady on their feet, needs rescuing at 1280 Grant St., Cypress, Texas, 77249.
A typical model, such as a named entity recognition (NER) tool, would recognize the listed address as three separate entities — Grant Street, Cypress and Texas. If this data was used to geolocate, the model would send first responders not to 1280 Grant St., but into the middle of Grant Street, or even the geographical center of Texas.
Hu says that NER tools can be trained to recognize complete location descriptions, but it would require a large dataset of accurately labeled location descriptions specific to a given local area, a labor-intensive and time-consuming process.
“Although there’s a lack of labeled datasets, first responders have a lot of knowledge about the way locations are described in their local area, whether it be the name of a restaurant or a popular intersection,” Hu says. “So we asked ourselves: How can we quickly and efficiently infuse this geoknowledge into a machine learning model?”
The answer was OpenAI’s Generative Pretrained Transformers, or GPT, large language models already trained from billions of webpages and able to generate human-like responses. Through simple conversation and the right prompts, Hu’s team thought GPT could quickly learn to accurately interpret location data from social media posts.
First, researchers provided GPT with 22 real tweets from Hurricane Harvey victims, which they’d already collected and labeled in a previous study. They told GPT which words in the post described a location and what kind of location it was describing, whether it be an address, street, intersection, business or landmark.
Researchers then tested the geoknowledge-guided GPT on another 978 Hurricane Harvey tweets, and asked it to extract the location words and guess the location category by itself.
The results: The geoknowledge-guided GPT models were 76% better at recognizing location descriptions than GPT models not provided with geoknowledge, as well as 40% better than NER tools. The best performers were the geoknowledge-guided GPT-3 and GPT-4, with the geoknowledge-guided ChatGPT only slightly behind.
“GPT basically combines the vast amount of text it’s already read with the specific geoknowledge examples we provided to form its answers,” Hu says. “GPT has the ability to quickly learn and quickly adapt to a problem.”
However, the human touch, that is, providing a good prompt, is crucial. For example, GPT may not consider a stretch of highway between two specific exits as a location unless specifically prompted to do so.
“This emphasizes the importance of us as researchers instructing GPT as accurately and comprehensively as possible so it can deliver the results that we require,” Hu says.
Letting first responders do what they do best
Hu’s team began their work in early 2022 with GPT-2 and GPT-3, and later included GPT-4 and ChatGPT after those models launched in late 2022 and early 2023, respectively.
“Our method will likely be applicable to the newer GPT models that may come out in the following years,” Hu says.
Further research will have to be done to use GPT’s extracted location descriptions to actually geolocate victims, and perhaps figure out ways to filter out irrelevant or false posts about a disaster.
Hu hopes their efforts can simplify the use of AI technologies so that emergency managers don’t have to become AI experts themselves in order to use these them, and can focus on saving lives.
“I think a good way for humans to collaborate with AI is to let each of us focus on what we’re really good at,” Hu says. “Let AI models help us complete those more labor-intensive tasks, while we humans focus on gaining knowledge and using such knowledge to guide AI models.”
The work was supported by the National Science Foundation.
