Skip to content

Writing survey questions that do not lead

Verifiedintermediate7 min

Wording, scales, and order effects — the bias you build in before anyone answers.

A survey measures what you asked, not what you meant. Most of the damage happens in the wording.

Leading questions. "How much did the new method improve your focus?" presupposes improvement. Ask "How did the new method affect your focus?" with a scale that runs both ways.

Double-barrelled questions. "Was the task clear and enjoyable?" cannot be answered by someone who found it clear but dull. Split it.

Vague quantifiers. "Often" means different things to different people. Use "3–5 times per week".

Unbalanced scales. Options running from "good" to "excellent" have no room to disagree. Balance the poles and include a neutral midpoint only if genuine neutrality is possible.

Order effects. Early questions prime later ones. Randomise question order where you can, and never put your key outcome question after a block that frames it.

Social desirability. People under-report unflattering behaviour. Anonymity helps; asking about "people your age" rather than "you" helps more.

Pilot the instrument on five people who are not your friends, and ask each of them to say aloud what they thought every question meant. You will rewrite at least two.

More in Human Participants