Why DOD needs DevOps to accelerate IT service delivery

The Department of Defense is struggling to deliver IT services on schedule.

That’s the conclusion of a June evaluation of 15 major IT programs by the Government Accountability Office’s Defense Acquisitions Annual Report in which weapon acquisition officials “consistently acknowledged software development as a risk item in their efforts to develop and field capabilities to the warfighter.”

Between 2018 and 2019, a staggering two-thirds of projects experienced significant schedule delays,  averaging two years. These slowdowns were attributed to a failure to incorporate cybersecurity testing into development cycles, application performance issues and workforce recruitment challenges.

The watchdog’s findings are not surprising. Although DOD has made strides in the adoption of DevOps and DevSecOps practices, many of its IT programs cling to waterfall development practices -- characterized by complex, immutable, and lengthy schedules -- or even a hybrid of an agile/waterfall approach.

Considering this, here are three ways DOD can continue to improve and address the challenges causing scheduling disruptions.

1. Bake cybersecurity into the development process

As defense systems are increasingly connected, best practices in cybersecurity are important for preventing hacks and breaches that could compromise missions or even cause loss of life. Yet during the rush to build and deploy IT systems, security is often an afterthought.

Although DevOps practices can speed the development processes, development teams aren’t always focused or charged with securing their software. Instead, security testing has historically been a separate, siloed process much later in the application lifecycle. This legacy approach adds unnecessary delays and reduces accountability for security issues.

This has led to the DevOps culture incorporating a security focus. DevSecOps addresses these challenges by enhancing DevOps with tooling that weaves security into the continuous iteration/continuous delivery (CI/CD) process. With DevSecOps, developers can automatically identify security flaws, bugs and vulnerabilities and resolve them in real time before the code is released into production. Ultimately, security becomes part of the mindset and practice.

Continuous security must also persist once the application is deployed. Using continuous penetration testing, DevOps teams can quickly identify issues and address them in the next software iteration.

This merging of the security and development functions breaks downs siloes and creates a culture of “security as code,” significantly reducing development cycles, saving money and protecting weapon systems from increasingly sophisticated cybersecurity threats.

Unfortunately, the GAO report found less than half of the 15 projects reviewed reported conducting developmental cybersecurity testing, and those that did were inconsistent or delayed. Referring to the DOD’s Cybersecurity Testing and Evaluation Guidebook, the GAO stressed that “not conducting developmental cybersecurity testing puts programs at an increased risk of cost and schedule growth and poor program performance.”

2. Integrate application monitoring with DevOps tools

Any application or system must be monitored 24/7, but in a DevOps world these capabilities must be implemented in a more agile way. If a code change is introduced via the CI/CD process, operations teams must know within minutes if something is misbehaving or suspicious behavior is afoot. To do so, they must augment traditional security and application monitoring tools accustomed to keeping an eye on “live” systems with solutions designed to enhance visibility and collaboration across the development cycle.

Next-generation monitoring technologies, for instance, give DevOps teams visibility and control over everyone’s access rights within the development environment -- without the need to engage IT. They also monitor for suspicious account activity and web or server performance issues that may delay schedules from the early stages of development through delivery to production environments.

Only with full visibility and command over activity across operating systems, the network, applications  and the user experience can teams identify problems and quickly fix them.

3. Enable a cultural shift

Finding and hiring both government and contractor staff with the required agile, DevOps and cyber expertise is a serious challenge for program officials, according to GAO. This leads to a greater reliance on existing skillsets. But for developers accustomed to waterfall methodologies, embracing an agile transformation requires a significant mental and culture shift.

To overcome these challenges, the agile development process must be engaged from the ground up. Whether tasked with building new software or migrating from a monolithic application, officials must identify the right skillsets from within and assemble a new team committed to a CI/CD approach to software development. With training, mentoring and small wins under their belt, DevOps teams can bring other individuals or small teams into the fold, slowly extending this quicker, iterative way of working to the entire organization.

Resilient IT systems keep pace with the evolving mission

The shift to agile methodologies like DevOps, CI/CD, and DevSecOps can seem overwhelming. However, they are critically important if the Pentagon is to overcome its decades-long struggle with cost-overruns and scheduling delays. With new skills and the agility to develop and deploy applications in days or weeks (not months or years), DOD can innovate and accelerate time to delivery. The result: resilient, state-of-the art capabilities that are continuously and securely deployed in line with evolving mission needs.