360° Feedback Questionnaire Design 

The quality of feedback the participant receives will relate directly to the quality of the questionnaire. There’s nothing that contributes more to a sinking sensation than the realization that instructions were misunderstood due to lack of clarity or that questions were open to interpretation. There is no going back to “correct” the data, so it’s critical to ensure everything “works” the first time out. Here are some ideas to help ensure there are no unpleasant surprises: 

1. Provide clear instructions. Participants and respondents will receive a variety of instructions on ways to select participants, orient them to 360° feedback, access the questionnaire site, complete forms, answer questions, and interpret results. Be brief, uncomplicated, and to the point. Cover all the bases, use illustrations, and use the active voice.

2. Use unambiguous, behavioural questionnaire items.

• Evaluate observable behaviour rather than intangible values. For example, behaviour such as “My manager relates expectations of me to department goals” is easier to assess than “My manager has a global perspective.”
• Use terminology everyone will understand – speak the organization’s language.
• Evaluate only one behaviour per statement.
• Keep statements neutral in tone, and let the assessment scale measure quality. For example “My manager is excellent at linking my performance objectives to department goals” won’t provide as meaningful data at the “strongly disagree” end of a rating scale as “My manager links my performance objectives to department goals.”
• Test questions to ensure that over the short term, the same respondent will respond in the same manner. This will ensure your questions are clear and provide valid and consistent measurements.

3. Select an appropriate rating scale, and use it consistently throughout the questionnaire. Consider the following questions when selecting a rating scale:

• Do you intend to measure quality (strength vs. weakness, for example) or frequency (not at all to always)? We prefer quality, because frequency does not indicate skill or strength, nor do respondents apply consistent selection criteria to frequency.
• Do you want respondents to select a neutral midpoint, such as “neither a strength nor a weakness” or “neither agree nor disagree” or take a firm position on agreement or disagreement/strength or weakness? Odd numbered scales provide a midpoint, such as 1 – 5, while even numbered scales don’t.
• How wide a range of possible ratings do you want? 2 point scales (agree/disagree) provide a clear assessment, but don’t show progress over time, or significant variances among respondent groups. We favour a 5 to 10 point scale, to provide latitude in the assignment of ratings, clearly display variances, and allow for progress where participants carry out the 360° process repeatedly over the long term.
• Do you want to know which items are most important to respondents? We ask them to identify the top 3 items on which an individual should focus, to help the participant identify priorities for action for each respondent group.

4. Consider requesting qualitative comments. Written comments can provide participants with important background information about ratings they receive, as well as possible directions to take in terms of improvement and development. Comments may need to be edited before being included in the Feedback Report, to preserve respondents’ anonymity. We ask 3 questions, to facilitate categorization of responses:

• What should the participant continue doing?
• What should the participant start doing?
• What should the participant stop doing?

5. Test everything. To ensure the questionnaire and report provide the desired data and calculate results correctly:

• Test the questionnaire and report using predetermined input, to ensure input is processed correctly, and results tabulated correctly.
• Use a small pilot group to ensure participants and respondents understand the instructions, interpret the questions appropriately, and that any glitches are resolved.
• Have a backup process in place to ensure that the potential for loss of data is minimized. For example, if responses are recorded in a web database, also have them sent to the administrator by email as a backup.