March 14, 2012 at 2:41 pm #156174
We are attempting to determine benchmarks for our Earned Value Management (EVM) reporting, but I haven’t been able to find any data on reported actual performance (if such a thing exists). I am primarily looking at scheduling statistics in service industries, as I work for the federal government in an accounting agency and labor cost and schedule performance are our only concerns (we generally don't have to budget for consuming “material”). Does anyone know of a resource that provides this? (For example, the average SPI of XXX company/industry’s projects is .95)
March 14, 2012 at 3:44 pm #156196
Hmm...I don't think so.
Performance indexes will vary over time and I don't think anything other than the benchmark of 1.00 should be a guide for any organization.
There are too many variables in the way a baseline is created I think, for any meaningful relative comparison across organizations like this.
I think the more relevant benchmarks include the specifics of HOW the baseline is formulated in the first place, and best practices for monitor & control during execution.
March 14, 2012 at 4:20 pm #156194
Thanks for the response, I figured that was the case!
We have a customer that we convinced to use EVM for reporting metrics and they wanted us to help establish performance levels (green-yellow-red, my agency is a big believer in stoplight-inspired reporting, lol). An SPI of 1.0 is unattainable due to the complexity of the project (long term audit preparation & execution), so we have to create standards that will fall short of this benchmark.
March 15, 2012 at 1:02 pm #156192
Bryan - I am in agreement with Josh, and honestly, your client is missing the point of EVM. The calculated metrics are a moving target, with the only defined variables the indices. These indices are already baselined, so the results of the EVM analysis should be used for decision making. Calculate the indices, what are they, and what does that tell you? (i.e. CPI is greater than 1, now what do we do?)
Trying to baseline moving targets is not really possible. It is a better strategy to have maximum and minimums around the established index, that takes into account risk tolerance and performance metrics/acceptability. That way you can have a "range" of acceptability based on the outcomes of the analysis. Perhaps that is the way to go.
March 15, 2012 at 2:17 pm #156190
I don’t think it is necessarily a problem of “missing the point of EVM at the project manager/scheduler level. The issue is about effectively communicating results to the senior executive sponsors and customers, most of whom have very little PM knowledge.
The data must be delivered in a common format, which is the “red/yellow/green” tiered format. Inevitably, the questions will be posed “What is a reasonable scale for this metric? What results do similar operations consider as satisfactory performance?”
March 16, 2012 at 4:33 pm #156188
If one special aspect is [I cite you]: “The issue is about effectively communicating results to the senior executive sponsors and customers, most of whom have very little PM knowledge”, in this case, and generally speaking for a C-level management presentation, one of the recommended methods for EVM benchmarking with single slide/dashboard presentation is the BSC-Balanced Score Card method: http://www.strategy2act.com/solutions/project_management_excel.htm
The BSC acts as a benchmark for the EVM using the 4 Perspectives with a lot of specific selected metrics (KPI) and weights for each of its 4 Perspectives (Financial, Customer, Internal Business, Growth & Learning) in order to compute the performance values.
Concerning the 4 perspectives tables of a BSC Reporting, in case of a program/project, they may be filled up by different management persons according to their specific Project/Program Management roles and purposes, at any corporative level, from both the “supplier” and the “customer” in a project/program.
Anyhow, care should be taken when using the EVM formulas, as they should be commonly accepted by the provider of the report and the beneficiary of it, and if they were updated no matter if they are still recommended by standards. Also, in case of a governmental/agency project it should be checked if the project is covered by the official EVM policy.
For example, the last EVM Gold Card was updated on February 2012 (https://acc.dau.mil/CommunityBrowser.aspx?id=19577 ), and for example it mentions a more clear computation of the Baseline Execution Index (BEI), while the computation formula the EAC(CPI) = BAC/CPI(CUM) was removed; anyhow this EAC formula is still valid and proposed by the PMI PMBOK V4 ANSI/PMI 99-001-2008 on page 184. Also, the EVM policy is clearly indicated in the updated EVM Gold Card.
In another way, great care should be taken on the referenced baselines when computing different performance indexes (e.g. CPI, SPI, TCPI, etc.) at different moments of time. The baseline maintenance is a somewhat complex but very necessary continuous process to be performed when computing EVM values. A good presentation module on the subject of baseline maintenance is: https://acc.dau.mil/docs/evm/module5/frame.htm
Also, a good reference for a project forecast block based on the EVM computed values, and thresold-level colouring scheme proposal may be found at page 6 of this document:
Concerning statistics resources for average values of company/industry project's EVM indicators, usually they may be found in PMO and PPMO databases of those companies which implemented such organizational structures and are performing such statistics for corporate interests. Again, the warning is put on the reasonable computations of such indicators and the moments of time when they were computed vs the related updated baselines.
March 18, 2012 at 8:04 pm #156186
Did you try asking PMI's COP's (communities of practices) member's?
March 19, 2012 at 2:45 pm #156184
Just curious: How active is that community, Jennifer?
March 19, 2012 at 3:05 pm #156182
Hi Andrew -
It depends on which COP you are in; however most are pretty active. I just think the membership is so worth it. There is an Earned Value Management Community of Practice.
March 19, 2012 at 4:27 pm #156180
Thank you so much for this outstanding reply! I appreciate the time that it took,it really provides me with some excellent ideas.
March 19, 2012 at 4:29 pm #156178
I'll take a look, thank you!
March 19, 2012 at 5:09 pm #156176
You are welcome !
Thank you very much for your appreciations.
Dr. Mihail Sădeanu
APMG PRINCE2® Registered Practitioner, PMI PMP, IAPPM CPD, AAPM MPM, IPMC CIPM
6 Sigma Master Black Belt, ExpertRating TQM,
PMI, IPMA, ASAPM, IAPPM, PMOC, ACM CR
Scientific Researcher 1-st Rank
You must be logged in to reply to this topic.