Pilot Testing

Pilot Testing Data Collection Instruments

What is pilot testing?

Basically, pilot testing means finding out if your survey, key informant interview guide

or observation form will work in the “real world” by trying it out first on a few people.

Why pilot test?

The purpose is to make sure that everyone in your sample not only understands the

questions, but understands them in the same way. This way, too, you can see if any

questions make respondents feel uncomfortable. You’ll also be able to find out how

long it takes to complete the survey in real time.

When to pilot test?

Once you have a version your project is happy with, but before training your data

collectors about how to administer it. (During data collector training you will want to

conduct another type of test—a field test. In that case you are testing your team, rather

than the instrument. But that is another story...)

Who to test it on?

People that represent the various subgroups within your intended

sample. Remember, you want to make sure that everyone will

understand your data collection instrument in the same way. So if you plan to conduct a representative public opinion survey, you might want to include both old and young people, well-educated and less educated, native English speakers and recent immigrants in your test sample. At a minimum, involve several people from each subgroup

so their views can be confirmed by others.

How to conduct the test?

There are a number of ways to find out how people are understanding the data

collection instrument.

Retrospective Interview

1.Administer the survey in the same way and under similar conditions as you plan

to do for actual data collection. Record the start and end time so you can tell

how long it took to complete each survey.

2.Pay attention to instances when respondents hesitate to answer or ask for

clarification, as this may be an indication that questions or answers are too vague, difficult to understand or...