Key points at a glance
- Reliability concerns measurement consistency.
- Validity concerns the justification of an interpretation.
- One statistic cannot replace consideration of content and context.
Two different questions about quality
Reliability concerns whether a procedure produces sufficiently consistent measurements under suitable conditions. Validity concerns whether the intended inference from those scores is justified. A procedure can behave consistently while missing the characteristic of interest.
OpenStax contrasts both concepts when discussing data collection. In your thesis, go beyond definitions. Explain the kind of score your instrument produces and what claim you intend to draw from it.
An original example: consistent but unsuitable
Suppose you want to measure understanding of a digital tool. A question counting logins may give consistently recorded answers. Yet it does not directly show whether the person understands the functions. Many logins might reflect technical difficulties rather than competence.
A task asking participants to explain or perform a function could be closer to understanding, but it also needs clear evaluation rules. State which aspects of “understanding” your measure covers and which remain unobserved.
Gather appropriate forms of evidence
Internal-consistency statistics can help evaluate a multi-item scale. Alone, they cannot show that its items cover the intended content rather than repeating very similar wording. Agreement between raters may matter for coded judgements; repeated measurement may matter for stable characteristics.
For validity, consider content, intended use and relationships with other suitable observations. A pretest can reveal misunderstandings, while comparison with an appropriate external criterion can support a particular interpretation. No one check proves universal validity.
Match the check to the instrument. Internal consistency for a single item is not calculated in the same way as for a scale. Evidence from one population also cannot simply be transferred to substantially rewritten questions or another target group.
Describe measurement limitations concretely
Identify the instrument, version, population, scoring rule and available quality evidence. If you translate or shorten items, document the change and assess its consequences. A source about the original instrument does not justify your modified version automatically.
Phrase limitations as boundaries on interpretation: “The login indicator records use events but cannot establish functional understanding.” This helps readers more than a generic statement that validity cannot be guaranteed.
Final submission context
Formal details can feel like a separate writing task, but they matter just as much for a printed submission. Anything missing, misplaced or inconsistently formatted in the document will also appear in the bound copy.
Use this guide together with your cover page, table of contents, page numbers, source notes and appendices. Prepare the final PDF for printing and binding only after the complete file has been checked.
Practical check before PDF export
- Do headings, chapter structure and page numbers match?
- Are sources, figures and tables included completely?
- Are required elements such as declarations, appendices or the cover page included where required?
- Did you open and check the final PDF after exporting it?
Checklist
- Define the target construct.
- Connect measurement with the intended claim.
- Choose a reliability check suitable for the instrument.
- Provide substantive validity evidence.
- Report changes and limits concretely.
Common mistakes
- Treating high internal consistency as proof of content validity.
- Applying a scale statistic indiscriminately to one item.
- Transferring validation evidence after major instrument changes without examination.
Print and bind your finished thesis
Once you have checked the content and final PDF, configure the printed copies to match your submission requirements.
Configure printing and bindingRelated content
Explore related guidance on thesis structure, sources, formatting and final submission.
Frequently asked questions
Can a measure be reliable but invalid?
Yes. It may repeatedly produce similar results while measuring something other than intended. The particular interpretation still needs its own support.
Must I always calculate Cronbach’s alpha?
No. It addresses certain questions about multi-item scales and has assumptions and limitations. Select a quality check that matches the actual instrument and question.
Is a questionnaire valid after a pretest?
A pretest can expose comprehension problems and support revisions. It does not automatically provide every form of evidence required for later interpretations.