November 18, 2012 at 1:12 pm #173414
Phuong Le Callaway, PhDParticipant
I often question the results for each accomplishment that comes out from a leadership or management report, especially in the public sectors. How can you measure the bottom line results for intangible products in public organizations? For my own satisfaction, for intangible products, tangible results should be realized and must be measurable in some ways. How can you measure an intangible initiative to ensure that the bottom line results are in fact achieved rather than just to take the face values? I often engage all employees in the feedback loop and provide a comfortable environment so they can give honest feedback. Employee engagement is one way to measure the success!! Listen to their voices!
November 19, 2012 at 1:43 am #173422
Start by specifying what quantitative measures you plan to impact with the result of the project. Measure the baseline values prior, and then again after you have delivered the project. ROI is usually pretty clear – sometimes it is an objective yes/no answer to a question though, like did you implement something that now meets regulatory compliance.
November 19, 2012 at 2:15 pm #173420
A terrific little paper I read a few years ago looked across a variety of agencies in the New Zealand PS, and observed that the more concrete and directly measurable the goals, deliverables, and outcomes, the more stable their accountability framework was. The more abstract the goals and outcomes, the more frequently the means for measuring were modified.
I suspect one of the reasons for this happening is partly because those assembling the accountability framework have different visions of what the abstract concept is, as well as there being some hydrants that every passing management dog has to lift their leg on. But, less cynically, when the outcomes are more concrete, it is easy to get immediate feedback on whether one’s approach to measuring them is suitable, and sensitive enough. When the outcomes are more abstract, it is quite likely that quarterly and annual measurement of those outcomes will be puzzling and probably unsatisfyingly vague, leading to regular changes in the measurement and tweaking of the accountability framework.
Of course, that something takes a couple of years to gel and show impact, does not stop anyone from wanting to demonstrate impact on an annual basis, in time for the next budget (particularly if there is performance pay riding on it). It sounds trite, but a little bit of patience goes a long way.
November 19, 2012 at 3:15 pm #173418
Erik G EitelParticipant
Interesting thought. I think though that everything can be measured in some way or another. Employee satisfaction score? Sales numbers? Efficiency? I guess it all depends on what you’re looking to measure. And depending on how much effort you want to put in to things, you could go as far as having an independent auditor measure the results of an employee engagement survey (if it’s just something as simple as improving employee satisfaction). Otherwise, I think Josh hit the nail on the head with his comment.
November 19, 2012 at 4:08 pm #173416
There’s measureability, and then there is when you measure. The more intangible the outcomes, the harder it is to judge when something should be measured, even when you can achieve some consensus on how it could be measured.
My sense is that a great number of government projects and initiatives are launched prior to any in-depth discussion of those particular challenges. The intentions and hoped-for outcomes are blessed by all the right folks for all the right reasons – no criticism there – but a suitable measurement framework only gets thought about well after the implementation of the project/initiative. One really needs to think about the implications for measurement of outcomes from the get-go. For example, perhaps there is administrative/systems data that can be built in to provide effective indicators. If there is any sort of suitable and needed baseline data to be acquired to judge effectiveness, one would need to have a measurement plan in place well before launch, so as to gather the optimal baseline information.
Not that projects/products/ initiatives with more “tangible” outcomes are no-brainers, with respect to evaluation, but in many instances I suspect the outcomes to use as performance indicators are readily available. As both Josh and Erik indicate, there will always be something one can measure, you just have to put more forethought into it for the more abstract projects/products/initiatives, since you can’t guarantee their availability uness you plan for them to be there.
You must be logged in to reply to this topic.