UAVs, C4ISR systems aren’t always reliable, Pentagon report finds

Unmanned aerial vehicles can be quickly developed and deployed, but that sometimes leads to less-than ideal performance, according to the Defense Department’s second annual report on the Defense Acquisition System, released last week. 

The report analyzes data collected from DOD’s major contracts across several years and looks at contractor performance, project reliability and cost overruns. The goal of the analysis is to find more practical insights about the acquisitions process and focus on possible incentives and techniques to improve cost, schedule and technical performance.

The report examines independent assessments of the technical performance of weapons systems conducted by the DOD Director of Operational Test and Evaluation. The systems can be labeled as operationally effective, which means that the system can do what it was designed to do, and operationally suitable, which is an indicator that the system is safe, maintainable, and reliable.

The report includes information from a variety of commodity types with programs from 1984-2013, with varying sample sizes across the types. Some notable highlights include:

Unmanned Aerial Vehicles. In a change from last year’s report, UAVs were specifically broken out of the aircraft category because they are newer and feature vertically integrated remote control systems. According to report, UAVs showed the worst technical performance out of the different commodity types, with 71 percent of programs rated as “effective” and only 29 percent of programs rated as “suitable”.

The report notes that UAVs have a tendency to grow out of technology demonstration programs that focus more on immediate production and fielding, and are less rigorous than standard acquisition programs. As a result, “a lack of engineering discipline and rigorous reliability growth programs is an important factor in these results,” the report states. Coincidentally, UAVs had the highest price growth since contract award for development contracts from 2000-2012.

Space. Among space-related programs, satellites scored the highest on performance. Satellite programs achieved 100 percent “effective” ratings and 80 percent “suitable” ratings, meaning that they were the most consistent in terms of effectiveness.

C4ISR. Also in a change from the previous report, C3I and sensors were combined into the larger group of C4ISR because they are closely connected from a mission perspective. In terms of program ratings in operational testing, these systems were rated higher than UAVs – with an 80 percent “effective” rating and a 56 percent “suitable” rating. But C4ISR systems still had lower percentages than every other category (which also included fixed-wing aircraft, helicopters, munitions, ships and ground vehicles). 

Meanwhile, the report uses the Nunn-McCurdy threshold as a barometer to measure overspending. Introduced in 1982, the amendment required that increases of 15 percent over original costs per unit required notification to Congress, and that 25 percent increases would result in program termination unless the Defense secretary could submit appropriate reasoning for the increased costs.

For programs from 1997-2013, UAVs had a 33 percent breach rate, or cost increase, compared with the 41 percent breach rate of fixed-wing aircraft. C4ISR programs had a breach rate of only 22 percent, being outperformed only by missile defense programs, at 13 percent. In terms of space-related programs, satellites had the fourth highest breach rates – 42 percent. Space launch was found to have a 100 percent breach rate, but only because it has a single data point – the Evolved Expendable Launch Vehicle program. Overall, 32 percent of all programs since 1997 had some sort of breach.

In assessing primary contractors, the analysis indicated that many of the defense giants rank at the bottom in terms of development performance, reports DOD Buzz.

For cost overruns on developmental programs, Northrop Grumman had a weighted price growth average of 41 percent, followed by Lockheed Martin at 37 percent and Raytheon at 32 percent.

Meanwhile, Raytheon was plagued by the longest development delays, with an average schedule growth of 2.8 years. Lockheed followed closely at 2.5 years, and Northrop Grumman’s schedule growth clocked in at 0.9 years.

Despite this, the larger defense firms seem to be doing OK, with about a 9 percent profit margin, which has been largely consistent over the past five years.

“It is clear that defense firms generally are earning adequate margins that on the whole do not vary significantly year to year,” the report states. “Within these results, there is adequate opportunity to provide effective incentives to industry without changing aggregate returns for defense firms in general.”