July 18, 2011 at 1:52 pm #135909
There is a lot of data in the world these days… more than most of us know what to do with. And there is a lot of talk about ‘data driven decision making.’ But is this a reality in the public sector? What could be done to encourage policy makers, decision makers and managers utilize data more day to day, and would this improve how we do work?
October 11, 2011 at 7:49 pm #135921
The GPRA Modernization Act of 2010 requires quarterly reviews on federal Agency Priority Goals (APGs) beginning by June 30, 2011. While APGs include just a fraction of performance measures, the processes for reviewing them can be more comprehensive. Depending, of course, on how they are managed, they could also improve communication, data quality, and decision-making.
October 11, 2011 at 9:11 pm #135919
I’m not sure how big of an issue this is in the Federal Government. In my local government It is a huge problem exacerbated by an aging infrastructure that regularly fights against the type of integration needed to perform analytics.
October 12, 2011 at 12:34 am #135917
From my experience – it certainly is an issue. There’s a lot of data, but not a lot of people with the knowledge on how to access it, and analyze it, to make decisions. I think this is definitely echoed at state and local levels.
October 12, 2011 at 6:06 pm #135915
The problem is that it relies on a lot of people to behave like scientists, and weigh the preponderance of evidence. Without wishing to slight anyone, I’m not so sure that constitutes an important element of MBA or MPA programs.
Conversely, those who DO have training in the social, physical, or life sciences, are not always well-equipped to see the policy implications of research data in clear fashion.
We often hear about political interference with scientists or science reports in government agencies. To my mind this happens because policy folks, and scientists (empiricists), tend to occupy two different epistemological worlds, and have differing views on how data and information is to be treated.
Policy is almost always about coherence. As such, there is a need and hunger for consistency, and especially consistency of message. If an agency has policies, then any apparent expression of those policies should be the same message until the policy is changed.
By contrast, scientists abide in a world where completely contradictory results can be found in the same issue of the journal, or within the same session/workshop at a conference, and it’s no big deal. Anomalies are welcome, and lead not to confusion, but to better theories that are able to handle more data/phenomena.
As you can imagine, that results in conflict and grumblings. The empiricists have little to offer in the way of how best to translate the data into policy, and the policy wonks are too often blinded to data by ideology or corporate commitments.
To end on a sardonic note, I was attending a buddy’s retirement party last year, and in one of the testimonial speeches, a long-time colleague of the retiree said “Murray always believed in evidence-based decision making…as opposed to decision-based evidence-making”. A whole lot wrapped up in that clever little turn of phrase.
October 13, 2011 at 2:21 pm #135913
@amelia: Thanks for starting this topic. When I first came on to GovLoop, I tried to start discussions on evidence-based management but there wasn’t much interest at the time. Good to see that the topic has come around again.
One major issue in data driven policy is the GIGO principle – garbage in, garbage out. Sure, we have lots of data but what data should we pay attention to and how does it actually affect policy? I can give you a long academic reply on this but you are better off reading or seeing “Moneyball” to understand how the right measures can have a dramatic effect on outcomes. But you have to have the right measures which have a real causal link to your policy.
And there is also the issue that data in the social sciences is not as precise as data in the physical sciences. The force of gravity can be measured to a much higher degree of accuracy then the effects of the stimulus bill on the economy. In fact, I have commented before that our current economic theory is fundamentally flawed (due to physics envy) that is almost worthless for good decision making. We collect a lot of economic data measures but what good are they if the assumptions are wrong?
Thanks for starting the discussion.
October 13, 2011 at 2:45 pm #135911
One approach I’ve observed to data-driven decision-making is organizational: create a team that provides objective and transparent analyses of Agency programs for strategic decision-making and budget decisions. This has been tried at various times in the Department of Defense and some civilian agencies, going by the name “Program Analysis and Evaluation (PA&E).” Both the Office of the Secretary of Defense (OSD) in DoD and NASA have used a PA&E function, yet in recent years both have abandoned it for organizations that focus more on cost. That begs the question in my mind: is data-driven decision-making as an organizational function too difficult or of questionable value? I don’t have any answers…maybe some others out there do.
You must be logged in to reply to this topic.