A central point for collection of information that relates to computer security. Including, but not limited to, security advisories from the major vendors, major data breaches, “phishing” alerts, commentary regarding staffing levels. etc. etc.
Cyber Security Metrics and NIST
September 11, 2009 at 9:12 pm #80235
NIST publication (PDF File)
Title: Directions in Security Metrics Research
Author: Wayne Jansen
More than 100 years ago, Lord Kelvin insightfully observed that measurement is vital to deep knowledge and understanding in physical science. During the last few decades, researchers have made various attempts to develop measures and systems of measurement for computer security with varying degrees of success. This paper provides an overview of the security metrics area and looks at possible avenues of research that could be pursued to advance the state of the art.
Related article from Government Computer News
How to measure security? NIST maps out the emerging field of IT metrology
* By William Jackson
* Sep 10, 2009
Information technology security is a hot topic, but attention usually focuses on the lack of it. What is missing is an objective, quantifiable way to effectively measure it.
“Security can be looked at in different ways by different people,” said Wayne Jansen, a computer scientist at the National Institute of Standards and Technology’s IT Laboratory. There is quality control for code developers, the process of deploying a system, and its maintenance by users. “These are all different aspects,” and they do not lend themselves to traditional methods of measurement used in physical science, he said.
Jansen has examined the status of efforts to develop security metrics, identified challenges and suggested a course for future research in a recent NIST report, “Directions in Security Metrics Research.”
There have been a number of efforts to establish metric systems for security, including the international Common Criteria, the Defense Department’s Trusted Computer System Evaluation Criteria, the European Communities’ Information Technology Security Evaluation Criteria, and the International Systems Security Engineering Association’s Systems Security Engineering Capability Maturity Model.
“Each attempt has obtained only limited success,” Jansen wrote. “Compared with more mature scientific fields, IT metrology is still emerging.”
The issue is complicated because security means different things to different people and organizations. “Security is risk- and policy-dependent from an organizational perspective; the same platform populated with data at the same level of sensitivity, but from two different organizations, could be deemed adequate for one and inadequate for the other,” he wrote. “The implication is that establishing security metrics that could be used for meaningful system comparisons between organizations would be extremely difficult to achieve.”
There is no standardized terminology for discussing or describing security, Jansen said. The Federal Information Security Management Act’s criteria for rating systems as low, medium or high impact is subjective, and assigning them numerical rankings can blur the distinction between qualitative and quantitative measures.
It is difficult to remove subjectivity from IT security. Security measures can be correctly implemented yet still not be effective. “Effectiveness requires ascertaining how well the security-enforcing components tie together and work synergistically, the consequences of any known or discovered vulnerabilities, and the usability of the system,” the report states. In other words, what is effective for one system might not be for another.
Are meaningful security metrics even achievable?
“The answer is yes,” Jansen said, “but they might not be as satisfying as you want.”
He identified two broad areas of research — process and organizational maturity — that focus on the care and maintenance of IT systems, and the intrinsic characteristics or properties of the systems. “I think we can make good progress on the maturity aspect,” he said. Research on security characteristics is not as far along.
There is not likely to be a single system of security metrics anytime soon because of the need to address different elements of security separately. Jansen cited the Federal Information Processing Standard 140 for cryptographic modules as a workable metric “because it bites off a manageable chunk.” The much broader Common Criteria, on the other hand, is less effective, he said.
“The issue of how to do this is going to be with us for the foreseeable future,” he said.
Challenges to effective security metrics identified in the report include:
* The lack of good estimators of system security.
* The entrenched reliance on subjective, human, qualitative input.
* The protracted and delusive means commonly used to obtain measurements.
* The dearth of understanding and insight into the composition of security mechanisms.
Promising lines of research for improved metrics include:
* Formal models of security measurement and metrics.
* Historical data collection and analysis.
* Artificial intelligence assessment techniques.
* Practicable concrete measurement methods.
* Intrinsically measurable components.
© 1996-2009 1105 Media, Inc. All Rights Reserved.
September 12, 2009 at 10:42 am #80237
have uploaded the NIST document
You must be logged in to reply to this topic.