Monitoring and continuous evaluation of OpenGov forums
As part of my ongoing efforts to monitor the various feedback channels that have been launched as part of the Open Government Directive, I'll be tracking some basic metrics. I thought I'd share the raw data here on a regular basis so people can chime in and add any other insights they've come across.
23 agencies are using IdeaScale, which exposes the following in real-time:
- Number of ideas
- Number of comments
- Number of votes
- Number of registered users
Aside from cold, hard numbers, I will be looking for examples that illustrate how various agencies are doing in terms of the other nine items on my list. Here it is again in short form:
- Expectation management
- Community ground rules
- Level of convener involvement/participation
- Quality of moderation
- Quantity of participation over time
- Outreach and diversity of participants
- Conclusion and impact
- Tech support
- Project communications
- Mood
I'm using this RSS feed for easy scanning but don't expect to catch everything. Please share your impressions in the comments.
A set of screenshots will be available on Flickr.
Tags: e-participation, gov20, ideascale, metrics, ogd, open government, opengov, participation, public, uservoice
Replies to This Discussion
-
Permalink Reply by Tim Bonnemann on -
Ok, not pretty but here goes...
These are the numbers as of tonight.
Agency # Ideas # Comments # Votes # Users
http://openusaid.ideascale.com 0 1 2 7
http://opencommerce.ideascale.com 5 7 16 16
http://opendefense.ideascale.com 4 4 21 19
http://openeducation.ideascale.com 3 2 7 21
http://openenergy.ideascale.com 2 2 4 13
http://openepa.ideascale.com 8 19 31 37
http://opengsa.ideascale.com 7 9 21 32
http://openhomelandsecurity.ideascale.com 20 3 55 53
http://openhud.ideascale.com 3 2 4 10
http://openinterior.ideascale.com 20 57 329 79
http://opendoj.ideascale.com 16 33 106 44
http://www.opendol.ideascale.com 5 4 13 13
http://opennasa.ideascale.com 45 142 589 116
http://www.opennsf.ideascale.com 4 26 28 12
http://opennrc.ideascale.com 8 5 18 23
http://openopm.ideascale.com 7 10 25 35
http://openostp.ideascale.com 0 4 5 17
http://opensba.ideascale.com 2 10 11 7
http://opensocialsecurity.ideascale.com 5 60 40 23
http://openstate.ideascale.com 6 14 53 77
http://opendot.ideascale.com 22 11 16 25
http://opentreasury.ideascale.com 9 1 12 11
http://openva.ideascale.com 22 13 84 61
Total (change over last 24-hour period): 223 (+81) ideas, 439 (+128) comments, 1,490 (+722) votes, 751 (+261) users
NASA, DOI and DOJ are leading the pack in terms of total activity. DHS, VA and NCR saw a big jump today.
-
Permalink Reply by Joe Sanchez on -
Nice work, Tim.
Given the way the IdeaScale OpenGov agency "Agree" and "Disagree" vote boxes are designed, you cannot tell how many "Votes" "Agree" or "Disagree" with the "Ideas" that are posted. This is a design flaw that IdeaScale needs to fix.
-
Permalink Reply by Tim Bonnemann on
-
I'm taking the number from their standard metrics dashboard, see for example here: http://opendefense.ideascale.com/a/ideafactoryStats.do
I would think that they expose the total number of votes (both "agree" and "disagree"), not the balance.
Checking the OpenDefense forum seems to confirm that. The dashboard shows 21 votes total, yet addeding the scores across all four ideas gives 9. So 12 vote cancel each other out.
But good catch, probably a good idea to have that confirmed (otherwise my numbers make little sense). ;-)
-
Permalink Reply by Adriel Hampton on -
Very interesting. The numbers, which are very low, go to the extreme difficulty of actually driving community engagement. If you build it, they won't come. I'll be interested to see what you have to say and what plays out on "Level of convener involvement/participation" as this goes forward.
-
Permalink Reply by Tim Bonnemann on
-
I expect the numbers to go up significantly over the next couple of weeks (that's how long it took during last year's Open Government Dialogue, see my post from back then).
Also, as Jeffrey Levy (EPA) tweeted last night: "No surprise in relatively low participation so far . Thx to the snow, promotion hasn't yet begun (e.g., our email to 130K ppl)."
I guess that goes for the majority of agencies.
-
Permalink Reply by Adriel Hampton on
-
Ah, thanks for that info. If the promotion hasn't begun, the numbers are actually pretty good :)
-
Permalink Reply by Tim Bonnemann on
-
Had an email from EPA in my inbox this morning, which links to their /open site and mentions the idea forum: http://www.flickr.com/photos/planspark/4346416009/in/set-7215762337...
-
Permalink Reply by Adriel Hampton on
-
Very nice! Thanks for sharing.
-
Permalink Reply by Andrew Krzmarzick on -
Thanks for posting this information, Tim.
Acknowledging that this is the first phase, I'd be curious to learn if agencies have established metrics for the number of ideas that are implemented by each agency.
Related: do they have a road map for them to think through which ideas best match the mission, accomplish strategic goals/objectives, achieve a targeted cost savings, etc....followed by a (relatively quick!) vetting process and action plan for accomplishment?
Gathering ideas is an excellent first step...and wondering about the subsequent metric markers along the journey to open government. I think those are the metrics that really matter - to what degree do we make good on the promise that we'll listen and deliver on citizen input?
Thoughts anyone?
-
Permalink Reply by Tim Bonnemann on
-
I filed this under "impact", and given the fairly aggressive timeframe the Open Government Directive is on, I wouldn't be surprised if we saw a timely follow-up (within a few months after these online consultations have concluded).
I agree, the ultimate success matter is: did it matter?
-
Permalink Reply by Tim Bonnemann on
-
According to this post by IBM Center for The Business of Government Blog, the agencies have "more than 500 moderators participating in the event."
-
Permalink Reply by Tim Bonnemann on
-
Numbers as of tonight:
Agency Ideas Comments Votes Users
http://openusaid.ideascale.com 2 1 4 12
http://opencommerce.ideascale.com 5 7 18 20
http://opendefense.ideascale.com 6 5 23 24
http://openeducation.ideascale.com 21 25 255 67
http://openenergy.ideascale.com 4 2 8 17
http://openepa.ideascale.com 11 24 32 42
http://opengsa.ideascale.com 10 17 30 49
http://openhomelandsecurity.ideascale.com 40 70 423 177
http://openhud.ideascale.com 5 2 6 13
http://openinterior.ideascale.com 28 84 506 97
http://opendoj.ideascale.com 19 37 115 51
http://www.opendol.ideascale.com 7 4 15 18
http://opennasa.ideascale.com 57 176 803 147
http://www.opennsf.ideascale.com 6 27 30 22
http://opennrc.ideascale.com 8 5 24 27
http://openopm.ideascale.com 10 11 31 38
http://openostp.ideascale.com 2 4 7 20
http://opensba.ideascale.com 4 11 13 10
http://opensocialsecurity.ideascale.com 7 60 42 26
http://openstate.ideascale.com 8 17 67 99
http://opendot.ideascale.com 24 11 18 28
http://opentreasury.ideascale.com 12 1 15 17
http://openva.ideascale.com 31 40 185 85
Popular Content
Latest Activity
I vote for #7! Great piece!
Simple: I believe in our mission at EPA. In my 17 years here, I've literally helped save the world by working to protect the ozone layer, helped people understand acid rain, and run our response websites for 9/11 and Katrina. And now I lead our enti…
Both Government Workers and Federal Contractors network through Web 2 and 3.0 - this group embraces the federal contractor on GovLoop.
We're swinging into the final weekend of summer. Looks like we are safe from the hurricane, so to keep the season going as long as we can, time to list your favorite summer songs. What makes you feel like school's out? What is a song that defined a…
San Francisco's aggressive open data efforts were on display this week, as civic and technology leaders took the stage at sf.govfresh, an event highlighting technology innovation in City government. City CIO Chris Vein (who also was recently intervi…
I want to have both my votes to go to # 8 good job Kyle
Top Content
© 2010 Created by GovLoop.
