A group who shares ideas and experiences employing innovative acquisition practices, collaborative methods and use of Web2.0 technologies to transform federal acquisition.
Source Selection Evaluation – What is your experience with the good, the bad, and the ugly?
May 7, 2009 at 8:37 pm #71582
I am a practicing capture and proposal manager, and am very connected with the community of similar professionals. The one question that is always in everyone’s mind is what is it like to be on the other side – reading the stuff that we put together? Of course, we know about the obvious offenders – the misspelllings, the cut-and-paste errors, and such – but what is it that makes the evaluators select one proposal, and toss another to the side? What specific things got you to tell stories to this day of something you have read years ago? What amazed you, and what completely turned you off?
By the way, here is my sketch of an evaluator for my upcoming book on “writing dynamite executive summaries.”
The caption is: “Three cups of coffee and I still cannot get through the executive summary… I don’t look forward to this day…”
May 7, 2009 at 8:54 pm #71604
Peter G. TuttleParticipant
Great question Olessia. Having been on both sides of the table, what always diluted my interest was a proposal filled with meaningless management-ese or techo-babble, since flowery and overly-complicated narratives obscured any of the contractor’s discussion of their knowledge about the requirements or their solution(s) to our problems. Perhaps, one might say, this writing style was purposeful in some cases. Another space-waster, in my opinion, is the constant addition of self-aggrandizing contractor marketing and sales information sprinkled throughout a proposal. The funny part of this is that now that I’m on the other side of the partnership, it’s sometimes difficult to have people actually write in understandable english or write from the point of view of the buyer/customer. Buyers want to know what we are doing to solve their problems – they start getting suspicious when we continue to tell them how wonderful we are. Cheers. Peter G. Tuttle, CPCM
May 7, 2009 at 8:55 pm #71602
John van SantenParticipant
0) you are very talented! I like your drawing…
1) It is critical to prevent cost bias from creeping in during a technical evaluation. The fact remains that some representations are unbelievable (from an implementation cost POV) but are certainly technically desirable. The TECHNICAL SSEB must not estimate the price of the technical proposals – or be led to, even unintentionally.
2) There is some subjectivity in assesing the capabilities of technical personnel against the RFP requirements.
3) It is critical to avoid comparing one proposal against another – the standard is the RFP. Worse, the comparison must be against what the RFP states, not what it should have stated, or what is (perceived) as really needed.
4) The weight (comparative importance) of the evaluation criteria stated in the RFP MUST be followed. Create the weights and scale first, then test it, then apply it to actual proposals.
May 7, 2009 at 9:13 pm #71600
As someone now on the “other side,” I enjoyed the cartoon. 🙂
You ask, “what is it that makes the evaluators select one proposal, and toss another to the side?”
The answer must always be found in the RFP. The source selection must be based on the factors in the RFP. When we sit down and brief the unsuccessul offerors, we must be able to trace our decision to the what the Government requested in the RFP. If we cannot do that, then we should expect a Protest that we cannot win.
I am sorry if this answer is not entertaining, amazing or amusing, but it is true. The answer is in the RFP.
May 7, 2009 at 11:29 pm #71598
Also being on both sides of the fence, the one thing that is irritating during evaluation, and that I try to avoid when writing proposals, is rehashing what is in the RFP without explaining the solution. So many times I have read RFPs that vendors write in response to proposals where they just repeat what is in the RFP word-for-word, with no value add. Why do companies think this is appropriate?
Many companies, especially small businesses who may not have vast experience, seem to forget their audience and who will be reviewing their proposal. Adding value by offering a solution to a problem is critical to differentiation, which is of utmost importance to winning. Cutting and pasting what is in the RFP wastes the evaluators time.
Here is the link to an article I wrote on this subject that may be of interest: http://tinyurl.com/dkxtvw
May 8, 2009 at 1:30 am #71596
To be honest, the Executive Summary should not be a factor in the evaluation unless requested.
This thread is interesting and useful to the extent that it shows the lack of understanding of the marketing community of what we do.
May 8, 2009 at 11:15 am #71594
Yes, as someone already said, you are very talented!
Here’s what I tell folks, as a gov CO… the “story” needs to be laid out as follows:
1. The government’s requirement
2. How Firm X will meet / service this requirement (demonstrates approach)
3. For whom has Firm X performed a similar requirement (and how it was similar, not generalities – such as similar IT environment, etc.) (demonstrates understanding)
4. The metrics for past / current clients vs. these metrics, and outcomes / expected outcomes
5. Value adds (WILL do), efficiencies & suggestions for optimal performance IF gov is amenable to it (but don’t force it on them unless you know they’re open to it, such as going from one tool to another that you think is more efficient)
That’s the “story”.
The “theme” should incorporate a best practice (industry-based such as ITIL, or even corporate unique is okay as long as it’s solid and proven) that is the glue for holding together the story.
Something I’ll add from personal experience is that often, industry will ask government what its “pain points” are. That’s fine, but ultimately you don’t know who will be on the panel, and if they actually believe what someone else has defined as a “pain point”, or if the cure for the pain will be acceptable to the agency’s decision makers, and even if it is, how long it would take to implement (so it shouldn’t be presented as a do or die solution).
May 10, 2009 at 10:45 pm #71592
Jaime – great point. I have read/evaluated countless proposals that simply rehash the language in the RFP with no real value add or descriptive solutions to the requirements provided. I suppose this seems like a way to “ensure” the proposal seems responsive and clear but really just screams “no clue”. With that said, I often criticize the government for not taking the time to do due diligence and provide the private sector with enough time, information and dialogue to really understand the requirement and develop a responsive solution. The gov often seems to be in a rush to get proposals in when in fact we are doing ourselves a disservice. One of the best acquisitions I worked on was one in which the government spent the time (6 months) to engage in discussions with industry to frame the requirement, understand available solutions, good practices, metrics, etc and as we developed the requirement, continued the dialogue with industry to continue to refine the RFP. By the time we got to the solicitation stage, we had given industry enough time to understand what we wanted to accomplish, decide if they should bid, put teams together, prepare proposals, and respond. We held oral presentations to continue the dialogue and answer questions on both sides. We received responsive, creative proposals and ended up with a team of contractors that are still meeting our needs today. I think the time we spend on the front end (that we never seem to think we have) is really what makes the difference and helps make us all successful.
May 11, 2009 at 12:31 am #71590
I begin by acknowledging I have not read all the other responses. Knowing some of the respondents I am sure they provide sound advice but there is a twist I would like to offer : a good proposal should make the case for a best value decision by the Contracting Officer. If you consider the relationship of Sections C, F, L and M of the RFP then a good proposal will provide the CO with the kind of information that will support a best value decision in which the better technical, but not necessarily lost cost/price proposal is selected. Having oversee many a technical evaluation panel on best value procurements contractors consistently fail to help themselves by not helping the panel and CO discern the technical strenghts and the associated benefits of their proposal. Best value is a form of ROI – in which the CO and panel must discern how the technical strengths, identified in M, provide a greater value for any difference in cost/price with the other proposals. Contractors spend consider time writing their own press clippings and believing them – creating a false sense of confidence and self-congratulatory appreciation for their own empty written words.What a well written proposal will offer are the nuggets of value that an informed evaluator will recognize and then build upon in the evaluation. Keep in mind the evaluation should be accomplished without knowledge of the cost/price – it therefore can argument the strongest technical merits without the need to present at that stage the complete value proposition that also considers the cost/price. I personally am of the belief that a good panel and CO will find those nuggets of value and build a business case that is reasonable and consistent with the evaluation plan in Section M. A business case that when objectively presented leads other offerors to accept the logic of the selection.
May 31, 2009 at 5:19 am #71588
From the contractor point of view – I can understand why the government has gotten some real loser props from industry. In some cases it is because the requirements are vague; evaluation criteria misleading; and Section L disagrees with M and both don’t align with C. And there is nothing I like more than getting a 20 page list of requirements – many of which are in bullet format – and being given 25 pages to write a response in 12 point font with 1 inch margins. Or given 5 page of requirements and a 300 page limit for the response. lol.
Throw in an inexperienced; poorly resourced or uninterested proposal team and you can get some real interesting responses!!!
From my end – as a contractor – I think the biggest problem is “insecurity” so we try to do too much in whatever amount of space is provided…”hey one evaluator might be into topic X we ought to talk about it – but another may be into topic Y – oh throw that in too.”
Imagine eating a salad made by twenty people who are often working on divergent paths and with different visions; requirements; goals; biases and viewpoints. I imagine it wouldn’t taste too good unless everyone agreed on the exact vision for the salad upfront.
That is what it must be like for the government to review a proposal made by a capture manager; proposal manager; project manager; operations executive; solution architect; countless writers; pricing; contracts; subcontracts; teammates; that SME consultant someone hired; the guy that used to work there and “knows what they want”; and two or three review teams with their own divergent recommendations (often with very senior execs forcefully stating uniformed decisions). The odds say sometimes it must read like Shakespeare – as written by a roomful of monkeys on typewriters.
May 31, 2009 at 1:14 pm #71586
Just wanted to say – thank you so much for your inputs; I really appreciate the insights. By the way – what exactly dictates the requirements to write something really brief (20 pages) when the Statement of Work is 60 pages – and do you normally get solid responses that would enable you to select a great contractor?
June 4, 2009 at 4:44 pm #71584
Taking enough time to understand and frame requirements is always a good idea. Sometimes, though, agency personnel use a pre-proposal period to learn about the industry at the expense of the industry,
Spending time to educate agency personnel is usually a legitimate sales expense, of course, but smaller companies can be at a disadvantage since they can’t afford to make available the same amount of pre-sales time given that technical staff in smaller companies usually perform a mix of business development and technical work.
Perhaps if agency personnel were more open about what they are learning during this period, issues like this might be overcome.
You must be logged in to reply to this topic.