Scott, as part of a leadership continuum at my employer, I had a 360 eval, and I have participated in three as contributors. It’s a worthwhile evolution.
The idea is to get a statistically relevant sample (at least 12, and 20 is the best number) of inputs from friends, colleagues and co-workers from across your experience. The survey I have seen takes about 40 minutes, and it covers nearly two dozen separate areas of personality, leadership, management and task accomplishment. Many of the areas are related, and that helps for correlation of responses, and it gives more insight into particular areas of interest.
After compiling inputs, the results form a report of 25-30 pages, showing how respondents graded the person. The results are frequently complicated to interpret, so a one-on-one session is a necessity to get the most from the packet.
The advantage of a 360 eval is that you get more, and more honest, feedback than from simple evaluation sessions with your immediate supervisor. The disadvantage is that it’s expensive, both in time and resources, and you have to provide follow-up training to address employees’ weak areas.
If your organization wants to encourage professional growth and you have resources and the will to carry it through, then it’s definitely a worthwhile exercise. If your employees see it as something other than an opportunity for personal and professional growth, then don’t do it – opposition will skew results and spoil the opportunity to shape and improve the organization.