Forum Replies Created
June 26, 2011 at 9:33 pm #133850
Great questions! My team has been experimenting with a number of collaborative tools as we’ve executed a major system acquisition in the Coast Guard.
We consulted with the tribe here late in 2009 to get their advice on sharing two of our key requirements documents prior demonstrations we held in March of 2010 as part of our market research. We used SlideShare as a place to share the Concept of Operations and the Operational Requirements Document. There was a bit of hand-wringing within the government team on whether it was appropriate to share the draft documents, but ultimately the program management, legal, and a contracting representatives agreed it was in the government’s interest to provide industry the early visibility into our requirements.
During the demonstrations, we used a commercial screensharing tool to allow government people to view the demonstrations and participate in the government discussion afterward without the cost and hassle of travel. We used GoToMeeting, but there are many similar tools available.
As the Acquisition plan was fleshed out and the cost estimates were completed, we realized that we needed to consider alternative strategies to divide the projects into smaller chunks that would reduce risk, cost, and time to market.
We wanted industry involvement in this re-assessment process. We needed a way to provide everyone the same information on the issues we were struggling with, and a way for them to share ideas with us.
We used a wiki hosted by GSA on their citizen.apps.gov platform to support the collaboration on the acquisition strategy.
I’ve shared some of the lessons learned from that experiment in blog posts here on GovLoop:
I described the rationale for starting the wiki here:
I shared feedback from the HCA here:
After the three months study was completed, we used SurveyMonkey to ask participants for their feedback. The complete results are available here:
Here are a few things we heard:
In response to the question “Should the government continue to use a forum like this to develop further CG-LIMS RFI’s or RFP’s has been doing recently?”
“The wiki forum seems to be much more open and if it includes historical information / brainstorming type information like it has for this wiki, I think it provides valuable insight that is usually guessed at or derived. I think this leads to solutions that are more targeted to the customers needs. One caveat though is that it doesn’t protect industry intellectual property for those that want to contribute.”
“For private sector everyone stays informed… cuts way down on guessing.”
“With the weekly blog postings and wiki ther has been no ambiguity on what the Government is intending to do. As an Integrator it allows to me to focus more clearly on what my competitive approach will be.”
Proving that you can’t please everyone, we heard both:
“In person meetings are better”
“Provides good interface with Government in lieu of face to face meetings.”
We also asked: “These survey results will be shared with focus groups implementing the “25 Point Implementation Plan to Reform Federal Information Technology Management.” Your feedback is particularly relevant to the team implementing Point 25: “Launch interactive platform for pre-RFP agency-industry collaboration.” Based on your experience with this Acquisition Strategy wiki, is there anything you would like to share with that group? Do you have any input that hasn’t been captured above?”
“The efficiency of a wiki as compared to a series of time displacing meetings for many people, is hard to imagine, until you see it happen.”
“Government needs to use wiki as a tool for shaping and informtaion gathering. Yet not be shaped by highly active, biased, contributors. Need to understand motivations of contributors. Uncertain if there is a way to gather that data or develop accountability for content updates. Would there ever be cause for protest, based on level of wiki contributions?”
We continued to use the wiki to keep industry informed as our strategy further evolved in response to an expected reduction in out-year funding.
When we began to execute the strategy, we used the wiki to manage the questions and answers that followed the release of an RFQ for COTS software on GSA eBuy. We gave industry the option of posting questions directly to the wiki or e-mailing them to the Contracting Officer. As soon as the questions arrived through e-mail, the CO posted them to the wiki. This instant visibility reduced the number of repeat questions and let the government team focus on answering the questions without the distraction of repeat questions or multiple status updates. Here’s the RFQ Q&A page: https://wiki.citizen.apps.gov/CGLIMS/index.php/Software_RFQ_Questions_and_Answers
Here are some of the challenges I see:
1. Resistance to change: Government acquisition professionals are under tremendous pressure to know and follow an increasing number of rules. It will require education to help them understand that it is okay to do things differently.
2. Solution pollution: After the pilot and experimenting phase, I think we need to settle on some small number of tools that industry and government should be expected to learn and use. Using MediaWiki has a learning curve, but if that emerges as the platform of choice for acquisition professionals, we can learn it. It seems to work for Wikipedia, as well as plenty of government communities like Intelink, MilSuite, and many others.
3. Need to share? Need to share! Many acquisition professionals I work with still believe we need to develop a complete requirement before providing visibility to industry. As a community, I don’t think we’re convinced of the imperative to engage early with industry if we’re going to meet the government’s needs in the most effective way in declining budgets.
I’d like to see us move to institutionalize collaboration with industry, especially in the pre-RFP phase of acquisition. I’d love to see the pilots and experiments like BetterBuy grow into a mature and supported way of doing business that’s used as widely as FedBizOpps.
Until then, I’ll keep an eye on what leaders in this tribe are doing, keep stealing ideas, keep pushing the envelope, and keeping sharing what I learn!
February 9, 2011 at 3:54 pm #122632
Great question. Here’s some quick feedback. I’ve been using a MediaWiki-based wiki hosted by GSA’s apps.gov platform for past three months to get industry engagement and work transparently and openly to develop the strategy for a major system acquisition within the the Coast Guard.
I described my goals in this post: https://www.govloop.com/profiles/blogs/help-from-everywhere﻿
Share as much as we can to help industry help us deliver CG-LIMS
Solicit input on acquisition strategy
Provide a public forum for Questions and Answers
In addition the huge substantive help, I got some feedback that can be used for future work. This was a question posted by industry and my answer on the Q&A page:
Q: Have you thought about a combination of wikis (where kimonas are opened to some degree) and BAAs (where disclosure is not made to the Competition) that give your crew a much broader and honest access to the minds and hearts of Industry?”
A: We’ve done two RFI’s, and we’re likely to do more engagements where we get private input and/or capability demonstrations from industry.
Also got some feedback on the “Whiteboard” page:
“I look forward to more dialog on this. Thank you for the opportunity to share some ideas.”
“First of all I wanted to say that your open approach to discussion with wiki, blog, etc is excellent. So thanks for putting this out there. “
“Thanks for this Wiki. This is a great idea and hope these inputs can help. “
“Thanks very much Captain Taylor and please all of us contributing here if we are helping. Looks like some very good feedback and want to make sure we are getting to the points you need input on. “
“Thanks again for giving us this opportunity to provide input to the USCG CG-LIMS team.”
I also did a survey after almost three months of use. The results are at https://wiki.citizen.apps.gov/CGLIMS/index.php/Survey_Results. There are some useful insights there from govt and industry.
For those who like stats, go to https://wiki.citizen.apps.gov/CGLIMS/index.php/Special:Statistics
Here are some highlights
Active users (Users who have performed an action in the last 7 days): 15
Views per edit: 21.51
I had 1 off topic message (an advertisement) that I had to remove in the whole three months. Going into it, I knew that might be an admin burden, but experience proved it to not be the case.
My personal input: The MediaWiki platform on apps.gov was the best way for me to quickly meet the need. As a govt Program Manager, the top three priorites for me for any future platform are:
Ease of use, including easy set up by a program office
Transparency, including a way to for registered users to share anonymously
March 26, 2010 at 12:13 am #95849
Fantastic idea. I’m sure many of us are looking forward to see what kind of response this experiment generates.
March 25, 2010 at 1:22 am #88910
This is a quick note of thanks to all who provided insight in this thread. Tomorrow we’ll complete the 27th demonstration from firms responding to the sources sought described above. There’s no doubt in my mind that the responses have been much more focused than they could have been without the CONOP and ORD. Several of the companies told us how much they appreciated the early visibility into requirements.
I also stole an idea from BetterBuy that’s worked out great for us: Someone suggested live streaming pre-solicitation meetings triggered a thought that I ought to explore using some sort of desktop sharing to allow government folks outside the room to participate in the market research demos. We decided to use GoToMeeting and it has worked out great for us. Without any government travel, we’ve been able to include stakeholders and end users from throughout the Coast Guard participating in the process. It seems obvious know, but I wouldn’t have thought of trying it if I hadn’t seen the related idea suggested in BetterBuy. So thanks all for helping me think smarter!
February 11, 2010 at 1:59 am #91922
I agree — folks who are able to take advantage of the flexibility provided by telecommuting infrastructure should be expected telework when the gov’t is closed it has been for past three days… but the leaders should make that expectation clear as part of the telework agreement.
I did some preliminary digging to find out what the policy was from OPM and within the Coast Guard. I think the relevant OPM guidance is in the Washington, DC Area Dismissal or Closure Procedures.
In the description of Telework on page 3, it says, “Any requirement that a teleworker continue to work if the agency closes (or dismisses employees early) on his or her teleworkday or on any of his or her regularly scheduled workdays should be included in the employee’s written telework agreement.”
I’ve signed telework agreements for some of the civil servants on my team to telework one day every two weeks, but I don’t remember what, if anything, the agreement said about working when the agency was closed. The expectation should have been clear in the agreement. If I’d done that, it would have removed the “it’s not fair” from the equation.
These past few days have been a wakeup for many of us to ensure teleworking expectations are clarified so the agency can be more effective for… grandson of snowmageddon?
January 15, 2010 at 1:46 am #88914
Thanks to all for your advice / caution / encouragement / support / assistance. I appreciate being able to use this forum to engage with professionals on common challenges. It’s been helpful for both me as the PM and for Contracting Officer.
I’m pleased to share that we got the right heads together, made a decision this afternoon, and have updated the Sources Sought notice on FedBizOpps. That’s the forum we’re using for discussions with industry about the substance of the Acquisition.
Our answers to industry’s questions, the DRAFT CONOP, and DRAFT ORD are here:
I appreciate the spotlight that Steve Kelman pointed at this forum and the forward leaning leaders who started it. His post and the responses are here:
Celebrating little steps in the right direction! Dan
January 11, 2010 at 3:29 pm #88918
I work just a few floors below Pat. I’ll say, hi for you! She’s not directly in the CO chain for this project, but I see her regularly.
January 11, 2010 at 3:14 pm #88922
Pete, Roger the workload risk. As the PM, I can’t afford to waste my time or the CO’s.
One of the risks I didn’t share had to do with minimizing workload as well… minimizing workload involved in protecting release of acquisition sensitive information while still ensuring we have appropriate involvement from agency SME’s (both gov’t and contractor). My thought (which we’re still working through) is that you can eliminate that information protection workload by sharing the early CONOP and ORD openly.
January 11, 2010 at 2:51 pm #88926
If it helps anyone to help me, here was my quick evaluation of benefit and risk before I asked our contracting shop to consider sharing the drafts. There are additional risks that I deleted that impact other acquisitions but don’t change the basic issue. If there are additional downsides that are obvious to some of you who have done this, I welcome that feedback as well.
-Industry has better idea what we want, without us having to do any extra work to spell it out for them.
-We’re more like to get focused feedback from industry that’ll help us put together RFP or help sponsor with next update to ORD.
-Consistent with our approach to do this as transparently as possible. The contract specs will be acq sensitive until awarded, the ORD isn’t. Putting it out there will also level the playing field.
-Consistent with gov’ts goal of transparency
-CONOP and ORD are still draft, final version will change, industry may not like change. I think we can accept that.
-Contract language will be different from ORD. I think we can accept that. I think everyone understands CONOP and ORD are not contractual documents.
-Someone may ask for an extension to the current deadline. We can mitigate that by just adding a week to the response deadline.
-Could generate lots more questions, creating admin burden for KO and PM. Accept risk. We can manage how we respond. We’ve shown that we can be brief. Just knowing what the questions are will help us put together solicitation down the road
January 11, 2010 at 2:41 pm #88928
Peter, I appreciate the reply. You’re absolutely right about BetterBuy project discussions. I’ve already found one idea I’m going to try to do with this project. Frankly it was some of the discussion on that site that got me thinking about sharing the draft ORD and CONOPs to support the sources sought notice.
My project staff has done some experimenting with a wiki tool to collaborate on (non-acquisition sensitive) planning documents within the project office. Putting the Sponsor’s draft CONOP and ORD out with the solicitation is just a baby step in the right direction in my mind…. if the government is not willing to share these in draft form, it’ll be hard to get support for collaboration using a wiki.
Thanks for your feedback!
January 11, 2010 at 1:21 pm #88932
I appreciate the quick feedback. The FAR was the first place I’d looked to see if anything prevented sharing the info, and as you say, nothing does. The thing that makes this different from the norm is that the CONOPs and ORD are broader than what will eventually be in the solicitation (or draft solicitation). The purpose it would serve at this point in the process for us is to focus the vendor demonstrations that we’ve announced we’re soliciting in March (the announcement is in FedBizOpps http://bit.ly/cglims_sources).
We don’t yet have a draft solicitation to share, but I do have a draft CONOP and ORD that will help government and industry get the most value out of the demonstrations.
I think part of the reluctance to share is that since those planning documents are not and never will be part of the contract, it’s best not to share them, and to simply wait and only share what will eventually be part of a solicitation. I’ve always been of the “if there’s no rule against it, get it out there” camp, but I know this community is full of folks who have “been there and done that,” who may be able to share either success stories or horror stories of sharing government CONOPs and ORDs for IT systems.