Military reaps surveillance benefits from advances in sensor technologies

Surveillance camera systems are riding an upward spiral. They’re improving their image quality, bringing more benefits, and that’s sparking more demand for information gathered by unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs).

As this increased interest drives market growth, these unmanned platforms are increasingly carrying a range of sensors, including full-motion cameras and thermal imagers. But still pictures remain the mainstay of data collection. Generally, electro-optic (EO) imagers gather high-resolution data during daylight, while infrared (IR) systems provide nighttime imagery.

However, both technologies are evolving rapidly, providing higher-resolution images while reducing size and power consumption so that cameras can be used on smaller platforms that are often less expensive. Consequently, more of them can be deployed. One of the keys that are driving this spiraling usage is the transition that occurred across America a few years ago: the changeover to high-definition digital technology.

“HD systems have been around for defense applications since 2006, but the military had to make a couple steps before they could gain the benefits of HD, such as going to systems that could deal with digital images,” said Robert Kubis, product manager for gimbal systems at FLIR Government Systems Division. “Last year, they reached the turning point where analog was being replaced, so the transition is now moving forward quickly.”

The expanding use of EO/IR sensors is prompting many system designers to integrate additional equipment that can enhance the data collected by these imagers. HD cameras have become the norm, and more platforms are adding sensors that make it easier for analysts to determine exactly where the images they’re seeing are located.

“We are introducing HD in all our payloads and integrating inertial navigation system/global positioning system (INS/GPS) sensors in many payloads,” said Igal Mevorach, marketing director at Israel Aerospace Industries’ Tamam Division. “Integrating INS/GPS enables geo-pointing and geo-registration and minimizes target location error.”

Analog Fadeout

All new imaging systems use digital technologies. That eliminates the need for data conversion while also realizing the advantages that occur when system designers can use the advances in semiconductor technology.

One example of the transition to fully digital systems has come as complementary metal-oxide-semiconductor (CMOS) imaging chips used in most consumer cameras have improved. They can now work in lower-light conditions, letting system designers get rid of image intensifiers (I²), which used analog vacuum-tube technology to improve image quality in low-light environments.

“The major change now starting to take place in EO/IR imagery integration is the revolutionary transition from analog I² tubes to digital low-light CMOS based imager technology,” said Tom Vogelsong, senior business development director for SRI International’s Engineering and Systems Group.

Low-light CMOS supplies a digital video signal that can easily be processed to improve the image. Though these devices are challenging IR for low-light applications, IR components are battling back. Vogelsong noted that the transition to digital is also occurring in IR sensors. Uncooled microbolometers and cooled mid-wave infrared and long-wave infrared sensors also provide a digital video signal.

In the Red

Currently, IR imagers lag the rapid pace set for visible-light cameras. CMOS and CCD technologies build on the advances in consumer and professional cameras, but IR systems don’t have the luxury of huge R&D budgets in those fields. That’s part of the reason that IR imagers are still moving into the HD range.

“IR is still at 1280 x 720 pixels, while color cameras are at 1920 x 1080,” Kubis said. “The 720 resolution can be more valuable because you can see better in the dark when the pixels are larger.”

Though there can be benefits to lower resolutions, there’s no stopping the trend to finer images. Some design teams are already beginning to adopt the latest generation of IR sensors, which provide HD-quality images. For example, Logos Technologies is working with IR detectors that have 1024 x 1280 resolution. But that advance can only come in platforms that have size and budget specifications that have some leeway.

“IR sensors are coming along in terms of providing more pixels, but high-pixel cameras are expensive and they’re very large,” said John Marion, director of Logos Technologies Persistent Surveillance Division.

While IR component technology is moving forward, it can’t match the dramatic pace of advances in conventional CMOS semiconductors. CMOS cameras continue to evolve rapidly, and the processors that enhance and manipulate images are also advancing quickly, driven by the huge R&D investments for consumer and business markets. Some observers predict that CMOS cameras will soon be able to provide most of the benefits of IR imagers in low-light environments while still providing daylight image collection.

“With the higher sensitivity, broader spectral coverage, higher contrast resolution, low noise and low power consumption of today’s CMOS technology, combined with image enhancement achieved with image processing, the capabilities of low-light CMOS are becoming a viable candidate for the next generation of night-vision systems,” Vogelsong said. “With the combination of a fused/enhanced video from a low-light CMOS imager with a thermal imager, the ability of our forces to see clearly under all conditions is becoming a reality.”

However, that transition could be a long time coming. CMOS parts have been making inroads into cameras for several years, but many military users still deploy time-tested charge coupled devices (CCDs). Military-grade components can cost less than CMOS devices, which must be ruggedized for harsh environments.

“Eventually, CMOS will cost less, but there are still a lot of high-quality CCD components being shipped today,” Marion said.

CCDs are expected to dominate the satellite portion of the market for quite some time. Satellites don’t need low power, and the small system savings for ruggedized CMOS isn’t as important in this low-volume, high-reliability field.

“Using CCDs is not an issue because we’re not using battery powered systems,” said David Strong, marketing vice president for FLIR Government Systems Division.

Improving images

Sensor improvements are being matched by advances in the systems that process their input. Semiconductors are able to perform more functions each second, letting engineers devise more techniques to improve the quality of images that are often collected under poor conditions such as low light and dusty atmospheres.

These chips are able to do more functions without an increase in size. Many devices use lower voltages as they shrink, which helps system designers meet the size, weight and power (SwaP) requirements of many data collection systems.

“Today’s image processors use low-power, field-programmable gate arrays and digital signal processors (DSPs) that make it possible to embed image processing functions directly behind the sensor focal plane array. These advancements are providing a path to solutions that also support the low SWaP requirements of portable defense systems,” said Mike Piacentino, technical director at SRI International’s Information and Computing Sciences.

These advances are making huge differences in system requirements. Engineers are leveraging lower power semiconductors and improving software. For example, controls and sensors quickly perform their tasks, then they go into deep sleep modes so they draw very little power when they’re inactive.

"Power is a big deal. We’re down to 200 watts for a system that was 3 kilowatts when it started,” Marion said. “The reduction comes from component improvements, as well as algorithms and software developments.”

Advanced electronics are also helping system designers eliminate the motors often used to stabilize cameras, bringing significant SWaP improvements. That makes it possible to put HD imagers on smaller, lighter UAVs and UGVs that cannot support mechanical stabilization. Many steps must be taken to ensure that high-resolution images are sharp and in focus.

“Image quality is often unusable if platform jitter is not removed,” Piacentino said. “Additionally battlefield lighting conditions are very extreme, and noise reduction algorithms become critical.”

Those lighting conditions can be addressed using lasers. Though they can’t provide light over a large area, lasers can focus in on specific areas. They can work even in the long distances of UAVs.

“We are introducing new capabilities such as electron multiplying CCD and illuminators in our new systems,” Mevorach said. “We use the laser illuminator for very short duration in conjunction with the EMCCD in order to take snapshots of the target for identification, for example reading a ship’s name.”

Sophisticated software is also being employed. As microprocessor and DSPs are able to handle more operations per second, systems can better analyze images and make enhancements. The improvements provided by sophisticated electronics continue to advance. For example, algorithms used in General Dynamics’ Imaging Through Volume Turbulence enhancement technology remove the distortions caused by dust particles and the variations caused by heated air.

The electronics aren’t the only components that must change. Mechanical components such as positioning and stabilization motors have to be more precise so the imagers can collect sharp photos.

"Everything has to change for HD. The focal plane has more pixels, so the optics have to change,” Kubis said. “The mechanics and everything that holds the camera in place need to be redesigned. With HD, nothing is hidden like it was with NTSC (National Television System Committee) or PAL (Phase Alternating Line technology).”

Many observers note that the big gains being made in imaging systems are the result of multiple fairly small improvements in many different areas. Some of the biggest gains come when components no longer need to be cooled by fans, which reduces power consumption and bulk.

“The miniaturization of optics, electronics and mechanical parts is helping reduce size and weight,” Mevorach said. “Usage of lightweight metals and composites is very helpful, as is the introduction of uncooled forward-looking infrared imagers based on uncooled IR detectors that reduce power requirements considerably.”