Why do so many thesis questionnaires produce data that cannot answer the research questions they were written for? Usually because the items came first and the research questions were fitted around them afterwards. A questionnaire is a measuring instrument. Like any instrument it needs a specification (what exactly it measures and for which research question), components that work (items people understand the same way), a response format that matches the analysis, and a test run before it is used for real.
This article is part of our guide How to Write a Master’s Thesis by the Mastermind PhD team and relates to our Survey & Questionnaire Design service. Sources last checked: November 2026.
This guide takes you through those stages of questionnaire design in order, with a specification table, eight item-writing rules illustrated by bad and good versions, a comparison of response formats, a pilot-testing protocol and the reliability figures examiners expect to see.
What is questionnaire design? Questionnaire design is the process of translating research questions and constructs into a set of written items, response formats and instructions that respondents can complete without help and that yield data suitable for the planned analysis. It covers specification, item writing, scaling, layout and ordering, expert review, pilot testing and the estimation of reliability and validity before the main data collection.
Where do the items come from? Start with a specification table
Good questionnaire design starts before the first item is written: build a table that links each research question to the constructs it needs, the dimensions of each construct, the number of items, the source of the items and the analysis you plan. The table forces three decisions that students usually postpone: what each section measures, how many items it needs, and whether existing scales already exist.
| Research question | Construct | Dimensions | Items | Source | Response format | Planned analysis |
|---|---|---|---|---|---|---|
| RQ1: What is the level of job satisfaction among nurses? | Job satisfaction | Pay, supervision, workload, growth | 4 × 4 = 16 | Adapted from a published scale (with permission) | 5-point agreement | Means, SDs, interpretation bands |
| RQ2: Does satisfaction differ by shift pattern? | Shift pattern | Day, rotating, night | 1 | New demographic item | Single choice | One-way ANOVA on the composite score |
| RQ3: Does satisfaction predict turnover intention? | Turnover intention | One dimension | 4 | Adopted scale, unchanged | 5-point agreement | Linear regression |
| Description of the sample | Demographics | Age, gender, experience, hospital | 4 | New | Categories | Frequencies |
Twenty-five items for four research questions is a typical size in questionnaire design for a master’s thesis; the table shows an examiner that no section is decorative.
Should you adopt, adapt or build a scale?
| Option | What it means | Advantages | Costs and conditions |
|---|---|---|---|
| Adopt | Use a published, validated scale unchanged | Known reliability and validity; results comparable with earlier studies | Requires permission or an open license; wording may not fit your context |
| Adapt | Change wording, context or number of items in a published scale | Keeps the structure; fits your population | Validity evidence no longer transfers in full; needs a pilot and your own reliability figures |
| Build | Write new items from the literature and interviews | Fits the construct exactly when no scale exists | Needs expert review, pilot testing and factor analysis; adds weeks |
For Arab students administering an English scale in Arabic, translation is a form of adaptation in questionnaire design: translate, have a second person back-translate, compare the two English versions, and report the procedure. The reliability of the translated version must be re-estimated on your own data.
📱 Get your questionnaire reviewed before fielding — Send the draft on WhatsApp; we point out wording and scale issues before you collect data.
What are the item-writing rules?
Each rule below is violated in the “bad” column and fixed in the “good” column. The rules follow the questionnaire design guidance consolidated by Krosnick and Presser (2010) and Dillman, Smyth and Christian (2014).
| Rule | Bad item | Good item |
|---|---|---|
| One idea per item (no double-barreled items) | My supervisor is supportive and communicates clearly. | My supervisor communicates clearly. (Supportiveness becomes a separate item.) |
| No leading or loaded wording | Do you agree that the new, improved appraisal system is fair? | The appraisal system is fair. |
| Avoid negatives and double negatives | I do not think the training was not useful. | The training was useful. |
| Use words every respondent knows | The organization’s KPIs are cascaded effectively. | I know which targets my unit is expected to meet. |
| Specify the time frame | How often do you work overtime? | In the past month, how many days did you work more than your scheduled hours? |
| Avoid absolutes | I always receive feedback after completing a task. | I receive feedback after completing a task. |
| Do not assume a behavior or state | How satisfied are you with the feedback your manager gives you? | Does your manager give you feedback on your work? If yes, how satisfied are you with it? |
| Keep items short and concrete | Taking everything into account, including the various aspects of my role, I would say that my overall level of engagement with the job is high. | I feel engaged in my work. |
Two further habits of good questionnaire design help: write the item as a statement respondents can disagree with (an item everyone agrees with carries no information), and keep the vocabulary at about the level of a 12-year-old reader for general populations.
Which response format should you use?
| Format | Example | Use it for | Notes |
|---|---|---|---|
| Agreement scale (Likert type) | Strongly disagree to strongly agree, 5 or 7 points | Attitudes, perceptions, self-reports | Label every point; 5 points are easier on phones, 7 add discrimination with educated samples |
| Frequency scale | Never, rarely, sometimes, often, always | Behaviors | Anchor with numbers where possible (“1 to 2 times a week”) |
| Semantic differential | Unfriendly 1 2 3 4 5 6 7 Friendly | Evaluations of a brand, person or service | Bipolar adjectives must be true opposites |
| Single choice | Shift pattern: day, rotating, night | Facts and categories | Make categories exhaustive and mutually exclusive; add “other” |
| Ranking | Rank these five benefits from 1 to 5 | Priorities | Hard beyond 5 or 6 options; analyze with medians |
| Open-ended | What would most improve your working conditions? | Explanations, new ideas | Limit to 2 or 3 per questionnaire; plan content analysis |
Two recurring decisions deserve a sentence in the methodology chapter. First, the midpoint: a neutral option lets undecided respondents answer honestly, while removing it forces a direction; either is defensible if stated. Second, a “don’t know” or “not applicable” option should be offered whenever some respondents genuinely cannot answer, and coded as missing rather than as the midpoint.
How should you order and lay out the questionnaire?
- Open with a short cover statement: purpose, who is conducting the study, anonymity or confidentiality, time needed, voluntary participation and a contact address. This is your consent text; ethics committees check it.
- Put easy, interesting, non-threatening items first; sensitive items (income, health, misconduct) in the middle or late; demographics last unless they are screening questions.
- Group items by construct under a short heading and keep the response format constant within a block.
- Aim for 10 to 15 minutes of completion time, which in practice means about 25 to 40 items; every additional page costs responses.
- Number every item, use one column, and keep the scale labels visible on each screen or page.
- Mix the direction of a few items only if you will reverse-code them, and expect those items to perform worse in the pilot.
How do you pilot test the questionnaire?
In questionnaire design, pilot testing happens in three short rounds.
- Expert review. Three to five people who know the construct or the population read the draft and judge each item for relevance and clarity. Record their ratings and the changes you made; this is your evidence of content validity.
- Cognitive interviews. Ask five to eight people from the target population to complete the questionnaire while explaining what each item means to them. This catches words that are read in two ways and items that assume a situation respondents are not in.
The first two rounds are qualitative; the third produces numbers:
- Pilot administration. Give the revised version to a sample resembling the main sample; Johanson and Brooks (2010) recommend about 30 respondents as a reasonable minimum. Record completion time, item non-response, items with everyone at the ceiling, and comments.
From the pilot data compute, for each multi-item section, Cronbach’s alpha and the corrected item-total correlation of every item. Items with correlations below .30 (a widely used cutoff; see Nunnally & Bernstein, 1994) or that lower alpha when retained are candidates for rewriting or removal. Pilot respondents are excluded from the main sample.
Which reliability and validity figures do you report?
| Evidence | How you obtain it | What examiners expect |
|---|---|---|
| Content validity | Expert review with recorded ratings | Who the experts were, how many items changed |
| Face validity | Cognitive interviews | Summary of misunderstandings found and fixed |
| Internal consistency | Cronbach’s alpha per section, pilot and main sample | Alpha with the number of items; DeVellis (2017) treats .65 to .70 as minimally acceptable, .70 to .80 as respectable and .80 to .90 as very good |
| Construct validity | Exploratory factor analysis in the main sample (if the scale is new or heavily adapted) | A factor structure matching the specification table |
| Stability (optional) | Test-retest with 20 to 30 pilot respondents two weeks apart | Correlation between the two administrations |
Alpha values well above .90 usually indicate redundant items rather than better questionnaire design, and DeVellis (2017) suggests shortening the scale in that case.
How do you describe the instrument in the methodology chapter?
Template: “Data were collected with a self-administered online questionnaire of 29 items in four sections. Section A (16 items) measured job satisfaction on a 5-point agreement scale (1 = strongly disagree, 5 = strongly agree) and was adapted from [Author, year] with the authors’ permission; three items were reworded for the hospital context. Section B (4 items) measured turnover intention using the scale of [Author, year] unchanged. Section C recorded shift pattern and Section D collected demographic data.”
“The draft was reviewed by four experts in nursing management, tested in cognitive interviews with six nurses and piloted with 32 nurses from a hospital not included in the main study. Alpha in the pilot was .87 for job satisfaction and .81 for turnover intention; two satisfaction items with item-total correlations below .30 were reworded.”
Which questionnaire design mistakes do examiners flag?
- No link between items and research questions, so whole sections are never analyzed.
- Double-barreled items analyzed as if they measured one thing.
- A new scale with no pilot and no reliability figures, or alpha reported for the whole questionnaire instead of per construct.
- Demographic categories that overlap (20 to 30, 30 to 40) or leave gaps.
- Translating a scale alone, without back-translation or a note on the procedure.
- Running factor analysis on 60 respondents with 40 items, then reporting a “validated” instrument.
- Copying a published scale without permission or without citing its source.
📱 Ask on WhatsApp now — 5 years of experience across all disciplines and 14 academic services. Two-stage payment (80/20); revisions are free until supervisor approval, within 30% of the chapter.
Frequently asked questions
How many items should a thesis questionnaire have?
Most master’s questionnaires run 25 to 40 items, including demographics, and take 10 to 15 minutes. Each construct needs at least three or four items for a reliable composite score, and each demographic variable you will actually analyze needs one item.
Should I use a 5-point or a 7-point scale?
Both are acceptable. Five points are quicker and work better on phones; seven points give more spread with educated respondents and slightly higher reliability. Use the same number of points that the original scale used if you adopt or adapt one, and label every point.
Do I need permission to use a published questionnaire?
For copyrighted commercial scales, yes. Many academic scales are free for non-commercial research if you cite the source; the original article or the author’s website usually says so. Write to the author when the terms are unclear and keep the reply for your appendix.
How many people do I need for the pilot?
About 30 respondents from the target population is a common minimum for estimating reliability, plus five to eight cognitive interviews and three to five expert reviewers. Pilot respondents are not included in the main sample.
Can I put the questionnaire in Google Forms?
Yes, if your university permits the tool for research data and your ethics approval names it. Switch off email collection for anonymous surveys, export to a spreadsheet, and recode text responses to numbers before importing into SPSS or R.
Sources
- Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Wiley.
- DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). SAGE.
- Fowler, F. J. (2014). Survey research methods (5th ed.). SAGE.
- Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In P. V. Marsden & J. D. Wright (Eds.), Handbook of survey research (2nd ed., pp. 263–313). Emerald.
- Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
- Johanson, G. A., & Brooks, G. P. (2010). Initial scale development: Sample size for pilot studies. Educational and Psychological Measurement, 70(3), 394–400. https://doi.org/10.1177/0013164409355692
- Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, 149. https://doi.org/10.3389/fpubh.2018.00149
Sources last checked: November 2026. This article is general educational guidance and does not replace your university regulations or your supervisor’s instructions. How we prepare our content.