Disparate Conclusions

I’m always amazed (frustrated?) at how often two or more groups can investigate a situation and walk away with very different conclusions, each substantiated by “facts”. One would think in this age of data and analytics it would be ever easier to have groups come to the same or, at least, similar conclusions because, in theory, the more data, the less assumption.

However, in practice, this does not (yet) seem to be the case. To illustrate my point, I recently read over this article from InformationWeek Government regarding a report released by the Inspector General of DOE. The report details the finding of the investigation into the data breach DOE suffered in July.

Of all the interesting nuggets, what caught my attention was the notion that the “data breach may be more extensive than realized.” Ignoring the obvious reason this caught my attention, I zeroed in on the fact that the IG and DOE officials are in disagreement over the number of possibly affected people. Of course, this is in the media, so, I expect there’s an element of PR here, but regardless of that, what this suggests to me is a data problem. I’m not referring to the data stolen, but, the data collected, stored and analyzed (or not) by the various security systems and sensors. Perhaps the IG and the security team are looking different data sources?

I find myself asking questions such as:

• Did the IG and the security team have access to ALL the same data?
• Was the data from multiple sources integrated together in the same manner?
• How much data was used? How far back does the data go? Are there gaps in the data? Where did the data come from?
• What does the analytics environment look like? Did the IG and the security team use the same tools?
• What was the skillset of the analysts on both teams?

As the frequency and sophistication of cyber attacks continue to grow, access to data, well integrated, detailed data from ALL possible sources, will become more critical than ever. Not only for the purposes of forensics activities and after action investigations, but, also to support detection and remediation, it is such an approach that will offer organizations the ability to ask questions and “see” answers that have been previously “hidden” across disparate data repositories.

Of course, this is not in any way unique to the world of cyber security. Consider how many opportunities you have in your world to bring together data from disparate data sources.

The more complete the data, the accurate relevant the insight, the more relevant the conclusions…