Mapping Agency Wants to ID Locations by Sound

The National Geospatial-Intelligence Agency—the government’s key intel force for pinpointing planetary happenings—announced it will dish out cash prizes for projects that can identify locations from audio and video data in an open innovation challenge.

Through the newly unveiled Soundscapes Competition, the agency will distribute up to eight awards, including a $27,000 top prize, for innovative, novel means to identify, analyze and model “sound and acoustic scene indicators” used to decipher the origination of recordings in one of eight cities. 

“This is the first time NGA has sponsored a competition exploring the potential for using non-speech sound data to narrow in on the place where video or audio recordings were made,” the agency’s Research Image and Video Pod Lead Michelle Brennan told Nextgov over email Thursday.

She explained that NGA analysts obtain a great deal of expertise in leveraging data to geolocate activities, patterns, behaviors and features that are critically important to global intelligence and national security. “Over time, NGA has received more requests to geolocate the source of audio and video recordings, yet fewer of the recordings include any clear visual geographic features,” she noted, adding that to confront the problem, the agency’s research hones in “on other, non-visual information contained in the media,” such as background sounds that could support geo-inferencing. Scientists within the organization recently conducted what Brennan called “an exhaustive literature study” to spot new assets that can help link non-speech sound to geolocating capabilities.

“Some interesting research in acoustics characterization has been conducted by academics, research institutes, and small businesses,” Brennan said. “Based on that potential, NGA initiated the prize competition as a means of uncovering creative solutions, identifying and rewarding innovators, and considering potential collaborative efforts.”  

Matching a video or audio recording to a particular point somewhere on Earth can be incredibly challenging in itself, and Brennan further suggested that the increased ease of global travel and the growing diversity of populations has essentially made it even harder, as language in a recording can no longer be used to distinguish where it was made. 

“Machine learning technology has advanced to the point that it can support models of complex acoustical patterns such as city sounds or the song of a particular species of bird,” she explained. “Acoustical analysis can benefit from the convergence of sophisticated modelling capabilities across applications with modelling techniques originally developed for computer vision and speech analysis.”

Through associating data with Earth’s specific geographical areas, those evolving, technological capabilities can help augment NGA’s continuing efforts in determining the “where” of things on this planet, in support of U.S. national security and defense aims. And relevant research into areas of sound scene characterization, environmental diversity measurement, bird species identification, and location of some specific sound events are already unfolding via a range of businesses, academic institutions, and research organizations.

“NGA is interested in working together with those who have conducted applicable research to apply it to geolocation,” Brennan said. “Potential outcomes include a library of geolocated audio sound classes and scene signatures that could support a variety of novel and innovative uses.”

The full Soundscapes Competition encompasses a five-week open period, ending November 27. During this time, those who get involved will produce and test machine learning solutions that can accurately link audio files to the city where they were recorded.

“NGA will examine the algorithms and evaluate potential for further collaborative development,” Brennan confirmed.

Winners will be selected in December and the top solvers will receive an invitation to share their results during a special showcase at a conference in 2021.

“Opportunities such as this foster increased academic and private sector interest and research into the potential of acoustics and ambient sound as a novel GEOINT data stream,” Brennan said.