The art of 360 feedback rating scales

After having implemented literally hundreds of different 360 degree feedback customisations, we’ve learned some really interesting lessons about how people fill them in and what works and doesn’t work. (In case you don’t know, a typical 360 degree feedback consists of some open text questions plus ratings of how well or how often an individual demonstrates various behaviours. This feedback is gathered together and presented back to the individual in the form of a report. The rated questions obviously need to be expressed on a scale of some sort.)

I was recently asked a question similar to one that many clients ask, about the rating scale used in a 360 feedback.  This particular client uses a frequency scale, where the reviewers are asked to say how often the participant demonstrates the various behaviours given the opportunities to do so. Here’s what they asked…

Thank you from me for all the work you and your team have done! The reports look great and really do provide a rich source of feedback. I was just wondering about the scale used in the 360 questionnaire. Currently it is:

1. Almost never 2. Sometimes 3. Often 4. Almost always 5. Always

 I wonder if these descriptors tip the balance towards the higher levels of frequency? The indicator “sometimes”, although a 2, is still quite a positive statement.

Now I absolutely know that 1000s of experts better qualified than myself in these matters will have very good reasons for this scale! However, my idea (just wanted to share it) might be as below (ie sandwiches “sometimes” in the middle).

1. Never 2. Quite rarely 3. Sometimes 4. Quite often 5. Always

What do you think?

This is a very common perception – that the scale ought to be evenly balanced between the ‘terrible’ ratings and the ‘excellent’ ratings. However, there is a very good reason why the scale is rather more positive: to counter the fact that most people generally always give very positive ratings to each other.  So we need to work very hard to counter that effect! Here’s a graph showing the distribution from this specific client’s feedback results:

As you can see, despite the fact that the labels against this scale sound quite positive, a large proportion of people gave scores on just the top 2 options, and almost no-one used the bottom end of the scale. In this particular case, the ratings people gave themselves hardly touched either of the bottom 2 options. (This is not always the case – in most organisations the average person rates themselves about as highly as other raters them.)

The pattern you can see above, with most ratings being generally positive and almost no-one using the bottom point of the scale, is virtually universal – we have seen this in over a hundred different designs of 360 feedback in hundreds of different organisations.

The interesting question to ask is: why do 360 feedback ratings tend to be so positive? We think there are two main factors at work:

  1. People generally like their colleagues and are nice to them.
    Clearly there are exceptions, but most organisations are the kinds of places where people get on well with their colleagues, and inevitably this is evident in the way they rate each other. Some organisations see more positive ratings than others, and no doubt part of this variation will be down to cultural difference in this area. Some companies are full of nice people and are very positive, other companies less so! Despite the anonymity of a 360 feedback process, people are normally as nice about their colleagues as they are to their faces. The corollary of this is the neglect of the bottom of the scale: to give someone a rating with the ‘worst possible’ score feels very harsh, so people just don’t do it unless they have a grudge to bear and want to stick the knife in!
  2. People generally do jobs that suit them.
    People seek out roles and companies that will value their skills and appreciate their style. Companies hire people with skills and personalities that suit the roles needed and fit the culture. And if people are performing poorly they are normally given support to get better, moved into a different role or removed from the organisation. The upshot of all this is that people are generally good at their jobs, and inevitably when their colleagues give ratings of their performance they’ll tend to give them good scores.

Another, related thing that customers often suggest is the use of a 4-point scale, because they think they should avoid giving a ‘middle’ option that people might revert to without having to think much. We always advise against this, and the chart above vividly illustrates why: people never touch the bottom end of the scale, so a 4-point scale tends to only be used in the top 3 of the 4 points, so instead of removing a ‘middle’ option you’ve actually created one! And on top of that feedback participants don’t think of the middle of the scale as the ‘middling’ rating anyway: because they think most of their colleagues are good, giving them the middle rating is effectively a ‘nice’ way of saying that they’re not good at it.

So why are 360 feedback scales so positive? It’s because of the human condition – people are nice to each other and people are generally good at their jobs.

So if you ever fill in a 360 feedback form and think the scale seems too positive, now you know why. And if you fill in a 360 feedback form and think the scale seems perfectly evenly-balanced between positive and negative ratings, ask HR to check the stats on their questionnaire design, because it might be in need of a tune-up!

Read more about rating scales – Is a rating scale of 1 to 5 most effective?

For more insight on setting up a 360 degree feedback come along to our Full Monty introduction to 360 degree feedback

Elva Ainsworth


One thought on “The art of 360 feedback rating scales

  1. Occasionally I come across products or services online
    that are so good that I feel compelled to share them with
    my friends and clients.
    I recently went searching for a customer rating and
    review service, because I wanted to gather ratings and reviews
    for my website. My search led me to shopperapproved.com, where I was able to easily start gathering ratings and
    feedback for my website in about 5 minutes. If you understand the power of social
    proof, this is the place to go.|I just got you free
    access to an amazing tool that helps you gather customer
    ratings and reviews.

    I’m going to use it for all of my sites, so I can start builidng social proof for my websites.

    Don’t know if you’d be interested too, but Ive put the
    link in my signature if you want to check it out

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s