Reader Note: This is the second in a series of blog posts about online public comment forums (OPCFs) – that are becoming increasingly prevalent across the country. Other articles in the series discuss the justification and best practices for implementing OPCFs. The author, Michael Alvarez Cohen, is a co-founder of Peak Democracy Inc — a company that has powered over 700 OPCFs, with over 70,000 attendees, for about 40 local government agencies and elected officials.
Government officials that are augmenting and diversifying feedback from their community via the internet should be aware of several challenges and potential pitfalls. These challenges include keeping the forums legal, civil, and fair — and equally important, preventing a pitfall with crowd-sourcing known as the Referendum Effect.
This blog post starts with a brief description of the Referendum Effect, and then focuses on how it can be impeded using online public comment forums (OPCFs).
What is the Referendum Effect?
The Referendum Effect characterizes the loss of decision-making autonomy that government leaders incur when they are pressured to make decisions that are based on the majority opinion that is expressed via public feedback — regardless of whether that feedback reflects the opinions of the majority of their constituents. More specifically, the Referendum Effect occurs when public feedback that doesn’t accurately reflect the community’s opinion, usurps the decision-making independence of government leaders.
This dynamic is prevalent in conventional public hearings, and can also arise when public feedback is gathered using online crowd-sourcing techniques in which participants are encouraged to vote on comments.
The referendum effect can be especially intense when public feedback is one-sided. One-sided feedback puts pressure on elected officials because if their decision goes against the sentiments of the one-sided feedback, then the officials can be accused of not supporting the (apparent) will of the people (even when the one-sided feedback doesn’t accurately reflect the community’s opinion).
The pressure to comply with the non-representative public feedback is typically inflicted by the people giving the feedback, but it can also be self-inflicted by elected officials because they think that the people that are not providing feedback are indifferent, apathetic and/or most importantly – are less likely to vote.
How to Minimize or Prevent the Referendum Effect Online:
There are techniques that can minimize the potential of the Referendum Effect in online forums. The most straightforward technique is to caveat the forum with messaging that explicitly addresses expectations. For example, Peak Democracy Inc’s Open Town Hall OPCFs integrate the following message in the user interface: As with any public comment process, participation in Open Town Hall is voluntary. The statements are not necessarily representative of the population, nor do they reflect the opinions of any government agency or elected officials.
Another straightforward technique to minimize the Referendum Effect is to exclude the word “vote” from the user interface — as the “v-word” can create an expectation that feedback with the most votes wins.
An additional and more sophisticated approach to minimizing the potential for the Referendum Effect is to structure the online forum to solicit only qualitative feedback (as opposed to quantitative feedback). For example, instead of the online forum requiring participants to indicate “yes” or “no”, or option 1 or 2, the online forum can simply ask for a comment.
Structuring an OPCF using a qualitative format can eliminate the Referendum Effect, but if the qualitatively formatted forum garners lots of participation, then it can be difficult for decision makers to read all of the comments. This challenge can be addressed with clever analytical tools. For example, Peak Democracy Inc’s Open Town Hall OPCFs can be configured to enable participants to support comments. The comments can then be listed in order of most to least supported, and links to similarly supported comments can be provided. This “related comments” graph enables decision makers to synthesize voluminous online feedback.
Enabling users to support other comments makes the OPCF structure slightly more quantitative. However, the risk of the forum becoming a vote for the most popular comment can be reduced by not showing the number of supporters that each comment obtains, and instead only listing comments in order of most to least supported.
In summary, caveating online forums, not using the v-word, and structuring forums for qualitative feedback can prevent the Referendum Effect, and thereby enable government leaders to leverage OPCFs without the risk of losing their decision-making authority. To learn more about the Referendum Effect and ways to prevent it, contact Mike at PeakDemocracy.com.
My own feelings in this area are guided by research in organizational justice. One of the contemporary notions in the psychology of perceived fairness in individual/organizational transactions is that of “referent cognitions”. Stated fairly simply, the model proposes that we judge the fairness of the outcome, and how the outcome was arrived at, by comparing it to other relevant counterfactual examples we may know about.
Without wishing to demean it, it’s pretty much the basis of “But Mike’s dad said he could go on the camping trip! You’re so UNFAIR! (door-slam)”. So, if we can imagine what we believe are other realistic possible outcomes/procedures/treatments that we find preferable to the one we encountered, we evaluate that which we did encounter as less “fair”.
The challenge with crowd sourcing and referenda is that it is not guaranteed to go the commenters way. “I told them. They could have listened, but did they listen? NO!!” is a very real risk. Unless the person in the crowd being sourced has a VERY realistic sense of the various behind-the-scenes contingencies weighing on the opinion-solicitor’s shoulders, and the scope of information received, the odds go up that any hoped-for trust-building to be achieved by crowd-sourcing will be squandered, and maybe even reduce trust.
It is a very risky proposition to ask a lot of people for their opinions or ideas on something, create the impression that they will all be seriously considered, and then end up having to look the other way. There may well be some very compelling stuff in there – the needle in the haystack – but there is the risk of alienating a lot of people unless it is done right.
This is NOT an argument against democracy, but an argument against thinking democracy alone will do the job. Like a great many things, democracy has to be conducted just right to accomplish the magic it does.
Last year, I was living in a the neighborhood called Jamaica Plain in Boston. Whole Foods announced it was going to move into a building recently occupied by Hi-Lo Foods which was a Latin American Market.
This “conversation” ended up splitting the community into two halfs. The Pro-Whole Foods side, mostly, white, well-do-to folks. The other side called themselves Whose Foods, who were made up of a cultural and economically diverse group of activists.
The concern was that Whole Foods would further displace many of the long-time African-American and Latino neighbors, whose population, according to the 2010 Census has already dipped 25% in the last 10 years. In essence it the “experienced” forced the neighborhood to “vote” on yuppy convenience or social justice.
When I found out this I posted a blog post titled Whole Foods or Trader Joe’s: We have voice, let it be heard!!
My initial inclination was to get people talking, which I was very successful at, but I was not aware of nor did I employ any of the techniques Michael or Mark have shared. When I realized my initial approach was dividing the community, I changed tactics and posted What impacts should be considered and what should the community process look like if Whole Foods is really planning to move into JP?
Here is what I learned and now do differently:
In public participation, it’s always good to ask at least these two questions:
As for the use of online tools, as long as there is a clear purpose for public input and expectations are set accordingly there is room for a wide variety of gestures and activities (liking, voting, rating, ranking, commenting etc.) and multiple ways to expose this information to the participants (or not).
Btw, thanks for sharing a look behind the scenes of Open Town Hall. I’ve added this post as a reference here: http://participatedb.com/references/153
Mark, Joseph and Tim,
Thanks for sharing your insights. You have an advanced perspective on how to anticipate and mitigate or outright eliminate the Referendum Effect in online (as well as conventional) forums.
However, I think that this awareness is an educational process for much of the government community. My favorite example of the risk of the Referendum Effect is a crowd-sourcing effort by Seattle Mayor Mike McGinn — that resulted in the feedback that Seattle needs to legalize marijuana and have more clothing-optional beaches. The Seattle Mayor subsequent took down the website. Check-out this article: http://www.kboi2.com/news/95121534.html .
In a future GovLoop blog post, I’d like to discuss the trade-offs between the quantity and quality of online (and conventional) feedback — especially as it pertains to offering reward/prizes and gamification techniques to encourage civic engagement.