AI-enabled drones expand search and rescue options

Local responders have increasingly called on drones to help in search and rescue operations. Unmanned aerial systems can cover more ground than humans and provide a birds' eye view of the operations. Besides carrying cameras, drones can be equipped with thermal imaging technology that lets them detect heat signatures arising from humans at night and in foggy or smoke-filled areas, providing vital clues to skilled search-and-rescue personnel on the ground.

To supercharge this man-machine combo, researchers at Virginia Tech are working to inject machine learning into search and rescue protocols, optimize autonomous, multi-drone teams and expand the amount of relevant data searchers can draw from.

The machine-learning component incorporates historical data of what lost people actually do, combined with typical searcher behavior and public information about the terrain searchers are covering.

Nicole Abaid, an assistant professor in the Department of Biomedical Engineering and Mechanics, built a mathematical model based on scenarios of more than 50,000 documented lost persons who behave in similar ways. "For example, people with cell phones tend to move up in elevation as they try to get service, while an elderly person might not travel very far," she said. The plan is to build over 30 lost-person profiles that account for age, mental status and the activity that led to getting lost.  The models will also incorporate topographical data from ArcGIS maps that can shed light on how people typically move through terrain.

Another part of the project involves managing multi-drone teams that can autonomously explore terrain too treacherous for human searchers and analyze information coming in from drone cameras, sensors or Lidar arrays. After analyzing and filtering that data, the drones will suggest tasks and send relevant, updated information to human searchers on the ground.

The operation requires a portable, low-cost, low-power computational infrastructure that can support analysis, storage and data sharing in the field. Researchers have plans for a wearable, backpack-based server that can provide on-the-ground processing and communications support.

"Human searchers are excellent at what they do," said Ryan Williams, an assistant professor in the Bradley Department of Electrical and Computer Engineering and project leader. "Drones are unlikely to replace people on the ground because searchers are too good at their jobs. However, what drones can do is address these niche problems of the search process by providing large-scale data that can help a search team make better decisions."

The project, "balancing collaboration and autonomy for multi-robot multi-human search and rescue," received $1.5 million grant from the National Science Foundation's National Robotics Initiative. The team has also received support from the Virginia Department of Emergency Management and will work closely with the local Black Diamond Search and Rescue Council throughout the project.