Would you rather discover that question 14 is misunderstood by half your participants during a pilot of ten people, or after you have collected 300 questionnaires? That is the case for a pilot study in one sentence. A small rehearsal of the main study exposes problems with instructions, timing, recruitment, data entry and analysis while they are still cheap to fix.
This article is part of our guide How to Write a Master’s Thesis by the Mastermind PhD team and relates to our Survey & Questionnaire Design service. Sources last checked: November 2026.
This guide explains what a pilot study is for and what it is not for, compares four published approaches to sample size, shows how to define success criteria in advance and gives a template for reporting the pilot in your methodology chapter.
What is a pilot study? A pilot study is a small-scale version of a planned study, conducted beforehand to test its feasibility: whether participants can be recruited, whether instruments and procedures work as intended, how long data collection takes and whether the planned analysis can be carried out. Its purpose is to improve the main study, not to test its hypotheses.
What is a pilot study for?
Thabane and colleagues (2010), in a widely cited tutorial, group the reasons for piloting into four areas:
| Area | What the pilot tests | Thesis example |
|---|---|---|
| Process | Recruitment, consent, retention, clarity of instructions | Response rate to the invitation email; how many participants complete all sections |
| Resources | Time, cost, equipment, staff | Average time to complete the questionnaire; interview length |
| Management | Data handling, coding, logistics | Whether online responses export cleanly into SPSS |
| Scientific | Instrument performance, variability of outcomes, safety | Item clarity, preliminary reliability, spread of scores |
For most theses, the pilot is a feasibility check on the instrument and procedure: are the questions understood as intended, does the online form work on phones, are response options exhaustive, and how long does it take?
What a pilot study cannot tell you
A pilot study is not a small version of the hypothesis test. Leon, Davis and Kraemer (2011) warn against using pilot results to test hypotheses or to estimate the effect size for the main study’s power calculation, because estimates from small samples are too imprecise. A large effect in ten participants can easily be a small or null effect in 200.
Base the power analysis of your main study on the smallest effect that would matter in practice, or on effect sizes from published studies, not on the pilot. Report any pilot statistics as descriptive feasibility information.
📱 Get your questionnaire reviewed before fielding — Send the draft on WhatsApp; we point out wording and scale issues before you collect data.
How large should a pilot study be? Four rules
There is no single correct number. Four approaches appear in the literature:
- Twelve per group. Julious (2005) proposed a rule of thumb of 12 participants per group, based on the precision of the estimates of mean and variance.
- At least 30. Browne (1995) recommended at least 30 participants when the pilot is used to estimate variance for sample size calculation.
- 10 to 40 per group, depending on the aim. Hertzog (2008) evaluated samples of 10 to 40 per group and showed that the general “10% of the main sample” guideline may be inadequate for aims such as assessing instruments or providing estimates for a larger study.
- Confidence intervals around the key estimate. Also from Hertzog: decide which quantity matters most (a recruitment rate, a completion rate, a reliability coefficient) and choose the sample that estimates it with acceptable precision.
For a questionnaire pilot in a thesis, a common practical choice is 10 to 30 participants from the target population, but justify the number by the pilot’s aim. If you also want a preliminary reliability estimate, the larger end of the range is safer, and the estimate should still be treated as provisional.
Participants in the pilot are usually excluded from the main sample, especially if the instrument changes after piloting. State your decision either way.
How do you set success criteria?
Decide before the pilot what result means “proceed”, “proceed with changes” or “stop and redesign”. Examples:
- At least 70% of invited participants respond within two weeks.
- The median completion time is under 20 minutes.
- No item is skipped by more than 10% of respondents.
- No participant reports misunderstanding an item in the cognitive interviews.
The CONSORT extension for randomized pilot and feasibility trials (Eldridge et al., 2016) recommends prespecified criteria for judging whether to proceed to a future trial. Although it was written for trials, the principle applies to any thesis pilot: the criteria come first, the data second.
How do you run a pilot study?
- Recruit participants who resemble your target population but will not be in the main sample.
- Use the exact procedure planned for the main study: same invitation, consent form, platform and instructions.
- Collect feasibility data: time taken, items skipped, questions asked by participants, technical problems.
- Where possible, add short cognitive interviews: ask a few participants to explain in their own words what each question means to them.
- Enter and analyze the data as you plan to in the main study, to test coding and analysis scripts.
- List every problem found and the change made in response.
How do you report a pilot study in the thesis?
In the methodology chapter, a short subsection usually covers six elements:
- Aim: what the pilot study was designed to test.
- Participants: number, how they were recruited and how they resembled the target population.
- Procedure: what was done, including any cognitive interviews.
- Criteria: the success criteria set in advance.
- Results: feasibility findings, such as completion time, response rate, problem items and, if computed, a provisional reliability coefficient.
- Changes: what was modified as a result and why.
A sentence such as “Three items were reworded after four of 15 pilot participants interpreted ‘workload’ as hours worked rather than perceived pressure” shows examiners that piloting improved the study. If ethics approval covered the pilot, say so.
📱 Ask on WhatsApp now — 5 years of experience across all disciplines and 14 academic services. Two-stage payment (80/20); revisions are free until supervisor approval, within 30% of the chapter.
Frequently asked questions
Is a pilot study required for a master’s thesis?
Not always. Many programs expect one when you develop or translate an instrument, or when the procedure is complex. Check with your supervisor and your program’s guidelines.
Can I include pilot data in my main analysis?
Generally no, especially if the instrument or procedure changed. If nothing changed, some researchers merge the data, but this must be justified and reported.
Is a pilot study the same as a feasibility study?
The terms overlap. Eldridge and colleagues describe pilot studies as a subset of feasibility studies: a pilot is a smaller version of the future study run in the same way, while a feasibility study may ask broader questions about whether the study can be done.
Do I need ethics approval for a pilot?
Usually yes, because it involves human participants. Many students include the pilot in the main ethics application.
Can a qualitative study have a pilot?
Yes. Piloting one or two interviews tests the interview guide, recording equipment and the length of the conversation, and often reveals questions that need rewording.
Sources
- Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., Robson, R., Thabane, M., Giangregorio, L., & Goldsmith, C. H. (2010). A tutorial on pilot studies: The what, why and how. BMC Medical Research Methodology, 10, 1. https://doi.org/10.1186/1471-2288-10-1
- Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011). The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research, 45(5), 626–629. https://doi.org/10.1016/j.jpsychires.2010.10.008
- Julious, S. A. (2005). Sample size of 12 per group rule of thumb for a pilot study. Pharmaceutical Statistics, 4(4), 287–291. https://doi.org/10.1002/pst.185
- Hertzog, M. A. (2008). Considerations in determining sample size for pilot studies. Research in Nursing & Health, 31(2), 180–191. https://doi.org/10.1002/nur.20247
- Browne, R. H. (1995). On the use of a pilot sample for sample size determination. Statistics in Medicine, 14(17), 1933–1940. https://doi.org/10.1002/sim.4780141709
- Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., & Lancaster, G. A. (2016). CONSORT 2010 statement: Extension to randomised pilot and feasibility trials. BMJ, 355, i5239. https://doi.org/10.1136/bmj.i5239
Sources last checked: November 2026. This article is general educational guidance and does not replace your university regulations or your supervisor’s instructions. How we prepare our content.