The Best Survey Tool Isn’t the Simplest One

When managing a survey, the field team will often ask you to simplify it.

And usually, they have a point. Something is genuinely confusing, or hard to explain to a respondent in the field, or doesn’t quite fit the local context.

I’m currently working on an endline survey for a multi-country evaluation of climate-smart agricultural interventions. During testing, our data collection partners flagged a few questions for review, including questions around household assets and facilities which had response options that were confusing and combined multiple items into a single response.

But this question was part of an established poverty index, with set question and response texts used to calculate the probability a household is living in poverty. Change the question or response and you’re no longer using a comparable index. Your endline no longer compares to your baseline. The cross-country comparisons you’ve been building towards fall apart quietly, and you might not notice until it’s too late.

So, we kept the tool as designed. And every flagged question became a training point instead.

This isn’t a new lesson for me. I spent three and a half years running large-scale surveys across East Africa, and published research on how enumerator behaviour shapes data quality. The problems aren’t always in the instrument. They’re just as often in the gap between the instrument and the person holding the tablet.

The best survey tool isn’t the simplest one. It’s the one your team actually understands.

When field teams flag something as difficult, it’s worth asking: is this a design problem, or a training problem?