Mixed reality scenarios prime recruits for combat

On the ground in Afghanistan or Iraq, where the terrain is veiled in the fog of war, life or death situations can boil down to one simple observation: Something doesn’t feel right.

The ability to recognize and act on gut feelings is part of an evolving training model that incorporates the best of virtual reality and entertainment. The goal: getting as close as possible to the real thing before actual deployment.

Spiral two of the Joint Forces Command's Future Immersive Training Environment (FITE) Joint Capability Technology Demonstration program conducted at Camp Pendleton, Calif., this fall featured interaction with live actors, animatronics, projected avatars and other sensory enhancements to achieve a mixed-reality environment, designed to replicate an urban setting in Afghanistan.

“If you’re in the country or suburbs, your senses go up differently than if you’re walking down Broadway in New York City,” said Jay Reist, FITE program manager. "With FITE we place the individual or the squad in an environment with the looks, feels, smells and tastes of an urban environment, as defined by requirements from the Army and Marine Corps.”

Unlike spiral one, which used head-mounted displays, the second increment is total immersion, which can more robustly simulate the tough, split-second decision-making and situational awareness skills necessary in combat.

Spiral two developers also sought to replicate the on-the-ground communications troops would use in combat.

“Radios and communication equipment are essential tools for command and control,” Reist said. “The FITE JCTD focused on creating the radio systems or using actual radios in the simulations so that the Marines/soldiers could practice and master the skill of when and how to communicate effectively. Communications capabilities were created within the squad as well as communications to higher and adjacent units.”

In the exercise, Spiral two used a commercial equivalent of the Integrated Intra-Squad Radios (AN-PRC 153s) used by the Marine Corps, and audio output was digitized and sent to the overall After Action Review system, Reist said.

The realistic communications became part of a simulation built on real-life experiences of services members recently returned from deployment.

“We did extensive interviews with soldiers and Marines just back from combat. We asked them, ‘What were the toughest calls you had to make in those complex situations? And based on your decision, what happened next? As a leader or individual or team member, what played out after that?’ We took those responses and crafted that into the scenario of a realistic environment,” Reist said.

Those scenarios included various anomalies – situations where something was “off” – to force decision-making under pressure. Immediately afterward, while still fresh from the intensity of the exercise, teams were debriefed and shown audio and video of the experience.

“FITE is particularly for [the age group of service members] who have been in combat the last few years in a dismounted role, who have taken on a preponderance of the casualties. This is for small-unit decision-making and excellence,” Reist said. “We’re targeting younger individuals to help bring out, advance and accelerate team-building and learning for the environment they’re getting ready to get into.”

That experiential learning, enhanced by technology, creates mental models and reference points that the troops take with them into the theater, Reist said.

As the two-year FITE program wrapped up in October, Reist and his team were optimistic for significant and lasting impact on military training.

“Part of FITE was to be a stimulus across DOD for the integration of virtual capabilities. Hopefully, this will allow for further research and development in the area,” he said.