Tell Everyone How to Measure Cyber Risk, DOD Begs NIST

It’s time the National Institute of Standards and Technology point to how organizations should be assessing the risk they’re associating with systems when deciding what security controls to implement for their protection, according to the Defense Department.   

“Enhance Section 4.0 (Self-Assessing Cybersecurity Risk with the Framework) to integrate guidance on how [Special Publication 800-30, revision 1] can be leveraged to perform the risk measurement to assign a value,” wrote Michele Iversen, director of risk assessment and operational integration at DOD’s chief information office for cybersecurity. “It appears that [the Cybersecurity Framework] depends on measuring, or assessing risk, but [avoids] alignment to the NIST standard commonly used to assess cybersecurity risks.”

Iversen’s comment is in response to a request for information NIST issued toward a second update of the agency’s landmark cybersecurity framework. NIST on Friday released a summary of the comments it’s received—over 130, mostly from industry—since the request in February. 

Originally issued in 2014, the Cybersecurity Framework, or CSF, points to various security controls organizations should consider implementing. But the document leaves it up to the user to determine which of those to prioritize, depending on how much risk they’re looking to address, or are willing to accept. And the question of how to measure whether use of the framework was successful was never really answered.

“Further guidance for measuring the performance of an entity in establishing and improving a cybersecurity program was a key need expressed in the RFI responses,” NIST wrote. “As with previous RFIs, comments on drafts, and discussions at NIST forums, metrics and measurement remain a lively topic among respondents. Many recognize that cybersecurity program implementation and improvement are not a pass/fail exercise, and that an effective program must be able to assess, coordinate and report measurable activities. Others stated that such detailed metrics, such as specific control objectives, ‘defeat the broad applicability and flexibility that make the CSF valuable.’” 

That tension between the desire for broad applicability and specific guidance is another general challenge for the framework, with groups like BSA | The Software Alliance asking for examples of how federal agencies have used it, as required.

“The level of detail and specificity in the CSF reflects the scalability and flexibility necessary to meet the needs of a wide range of stakeholders—small and large organizations in various sectors,” NIST wrote. “There were more than 500 references in the comments supporting the need for more guidance to support CSF implementation, and many users expressed a desire for greater detail in the CSF while maintaining a non-prescriptive approach. Identifying the proper balance between simplicity and detail in updates to the CSF is a key takeaway that will need further discussion.”

From DOD’s perspective, measurement is “NIST’s core competency” and the agency should be doing more to facilitate whole-of-government risk assessments which also consider the supply chain components of commercial information and communications technology. 

“The current practice of departments and agencies developing their own overlays results in variability … The individual department or agency may be operating at low risk to their mission w/o realizing how others may be impacted by the residual risks that they manage,” read the Defense Department comments. “Whole-of-government activities (national security, national commerce, etc.) need a capstone resource to enable integrated risk assessments grounded in the broader/shared uncertainties associated with observation and measurement particularly for their common operating space of ICT, cyber and cyber-security.”