ARL scientist zooms in on a better way to search confiscated images

Instead of dealing with a hierarchy of menus, users can simply pan and zoom in like with Google Maps.

VMR lets Intelligence Community analysts search through photos, such as those confiscated from enemy laptops or memory cards, and ask questions of the images, such as the identity of a person or the location of a building. But it’s a time-consuming, often frustrating business, which is what ARL researcher Dr. Jeff Hansberger set out to change with the new interface.

"The goal of DARPA's VMR program is to extract mission-relevant information, such as the who, what, where and when, from visual media captured from our adversaries and to turn unstructured, ad hoc photos and video into true visual intelligence," Hansberger said. "Our adversaries frequently use video, still and cell phone cameras to document their training and operations and occasionally post this content to widely available websites. The volume of this visual media is growing rapidly and is quickly outpacing our ability to review, let alone analyze, the contents of every image."

Hansberger came to three main conclusions following interviews with members of the intelligence community for the research and development of his design: "I discovered that like many people in other domains, the intel analysts we had interacted with also adhered to what is called the 'visual information seeking mantra,' which is that people looking within and for visual information prefer to have 1) an overview first, then 2) zoom and filter through the information and retrieve 3) details on demand.”

Hansberger utilized various touchscreen features for direct manipulation of images similar to how one might zoom in or out using Google Maps’ interface. He wanted the design to be driven by the needs of analysts, leading him to ignore technology constraints, although he eventually added keyboard and mouse operation into the design for more traditional use. 

The zooming and touchscreen application had a two-pronged effect—its simplicity saved users time compared with previous interfaces that had multistep menu options and the app supplies better pattern design support. If an interface makes a user navigate too broadly or deeply, they can get lost in the menus, Hansberger said. Giving them a flat hierarchy and the ability to zoom saves a lot of time and frustration. And the app uses visual diagrams from the images themselves for better pattern detection because the diagrams “highlight patterns and relationships across the attributes that analysts focus on such as time and the physical location the images were taken,” he said.

Hansberger said he wanted to take the stress out of the platform itself to ease the minds of users. “I try to put more effort in the design of the information and interface so the soldier spends less time thinking about the software and more time accomplishing their mission,” he said.