12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

The 4 Types of Reliability in

Research | Definitions &
Examples
Published on August 8, 2019 by Fiona Middleton. Revised on June 22, 2023.

Reliability tells you how consistently a method measures

something. When you apply the same method to the same sample
under the same conditions, you should get the same results. If not,
the method of measurement may be unreliable or bias may have
crept into your research.

There are four main types of reliability. Each can be estimated by

comparing different sets of results produced by the same method.

Test-retest

The same test over time.

Interrater

The same test conducted by different people.

Parallel forms

Different versions of a test which are designed to be equivalent.

Internal consistency

The individual items of a test.

https://www.scribbr.com/methodology/types-of-reliability/ 1/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

Test-retest reliability
Test-retest reliability measures the consistency of results when
you repeat the same test on the same sample at a different point in
time. You use it when you are measuring something that you
expect to stay constant in your sample.

A test of color blindness for trainee pilot applicants should

have high test-retest reliability, because color blindness is a
trait that does not change over time.

Why it’s important

Many factors can influence your results at different points in time:
for example, respondents might experience different moods, or
external conditions might affect their ability to respond accurately.

Test-retest reliability can be used to assess how well a method

resists these factors over time. The smaller the difference between
the two sets of results, the higher the test-retest reliability.

How to measure it
To measure test-retest reliability, you conduct the same test on the
same group of people at two different points in time. Then you
calculate the correlation between the two sets of results.

https://www.scribbr.com/methodology/types-of-reliability/ 2/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

Test-retest reliability example

 Table of contents
You devise a questionnaire to measure the IQ of a group of
participants (a property that is unlikely to change
significantly over time).You administer the test two months
apart to the same group of people, but the results are
significantly different, so the test-retest reliability of the IQ
questionnaire is low.

Improving test-retest reliability

When designing tests or questionnaires, try to formulate
questions, statements, and tasks in a way that won’t be
influenced by the mood or concentration of participants.
When planning your methods of data collection, try to
minimize the influence of external factors, and make sure all
samples are tested under the same conditions.
Remember that changes or recall bias can be expected to
occur in the participants over time, and take these into
account.

Prevent plagiarism. Run a free

check.

Try for free

https://www.scribbr.com/methodology/types-of-reliability/ 3/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

Interrater reliability
 Table of contents
Interrater reliability (also called interobserver reliability) measures
the degree of agreement between different people observing or
assessing the same thing. You use it when data is collected by
researchers assigning ratings, scores or categories to one or more
variables, and it can help mitigate observer bias.

In an observational study where a team of researchers

collect data on classroom behavior, interrater reliability is
important: all the researchers should agree on how to
categorize or rate different types of behavior.

Why it’s important

People are subjective, so different observers’ perceptions of
situations and phenomena naturally differ. Reliable research aims
to minimize subjectivity as much as possible so that a different
researcher could replicate the same results.

When designing the scale and criteria for data collection, it’s
important to make sure that different people will rate the same
variable consistently with minimal bias. This is especially important
when there are multiple researchers involved in data collection or
analysis.

How to measure it
To measure interrater reliability, different researchers conduct the
same measurement or observation on the same sample. Then you
calculate the correlation between their different sets of results. If
all the researchers give similar ratings, the test has high interrater
reliability.

Interrater reliability example

https://www.scribbr.com/methodology/types-of-reliability/ 4/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

A team of researchers observe the progress of wound

healing
 Table of in patients. To record the stages of healing, rating
contents
scales are used, with a set of criteria to assess various
aspects of wounds. The results of different researchers
assessing the same set of patients are compared, and there
is a strong correlation between all sets of results, so the test
has high interrater reliability.

Improving interrater reliability

Clearly define your variables and the methods that will be
used to measure them.
Develop detailed, objective criteria for how the variables will
be rated, counted or categorized.
If multiple researchers are involved, ensure that they all have
exactly the same information and training.

Parallel forms reliability

Parallel forms reliability measures the correlation between two
equivalent versions of a test. You use it when you have two
different assessment tools or sets of questions designed
to measure the same thing.

Why it’s important

If you want to use multiple different versions of a test (for example,
to avoid respondents repeating the same answers from memory),
you first need to make sure that all the sets of questions or
measurements give reliable results.

In educational assessment, it is often necessary to create

different versions of tests to ensure that students don’t have
access to the questions in advance. Parallel forms reliability
means that, if the same students take two different versions

https://www.scribbr.com/methodology/types-of-reliability/ 5/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

of a reading comprehension test, they should get similar

results
 Table of in both tests.
contents

How to measure it
The most common way to measure parallel forms reliability is to
produce a large set of questions to evaluate the same thing, then
divide these randomly into two question sets.

The same group of respondents answers both sets, and you

calculate the correlation between the results. High correlation
between the two indicates high parallel forms reliability.

Parallel forms reliability example

A set of questions is formulated to measure financial risk

aversion in a group of respondents. The questions are
randomly divided into two sets, and the respondents are
randomly divided into two groups. Both groups take both
tests: group A takes test A first, and group B takes test B
first. The results of the two tests are compared, and the
results are almost identical, indicating high parallel forms
reliability.

Improving parallel forms reliability

Ensure that all questions or test items are based on the same
theory and formulated to measure the same thing.

Internal consistency
Internal consistency assesses the correlation between multiple
items in a test that are intended to measure the same construct.

You can calculate internal consistency without repeating the test or

involving other researchers, so it’s a good way of assessing
reliability when you only have one data set.
https://www.scribbr.com/methodology/types-of-reliability/ 6/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

Why it’s important

When
 Table you devise a set of questions or ratings that will be
of contents
combined into an overall score, you have to make sure that all of
the items really do reflect the same thing. If responses to different
items contradict one another, the test might be unreliable.

To measure customer satisfaction with an online store, you

could create a questionnaire with a set of statements that
respondents must agree or disagree with. Internal
consistency tells you whether the statements are all reliable
indicators of customer satisfaction.

How to measure it
Two common methods are used to measure internal consistency.

Average inter-item correlation: For a set of measures

designed to assess the same construct, you calculate the
correlation between the results of all possible pairs of items
and then calculate the average.
Split-half reliability: You randomly split a set of measures into
two sets. After testing the entire set on the respondents, you
calculate the correlation between the two sets of responses.

Internal consistency example

A group of respondents are presented with a set of

statements designed to measure optimistic and pessimistic
mindsets. They must rate their agreement with each
statement on a scale from 1 to 5. If the test is internally
consistent, an optimistic respondent should generally give
high ratings to optimism indicators and low ratings to
pessimism indicators. The correlation is calculated between

https://www.scribbr.com/methodology/types-of-reliability/ 7/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

all the responses to the “optimistic” statements, but the

correlation
 Table of contents is very weak. This suggests that the test has low
internal consistency.

Improving internal consistency

Take care when devising questions or measures: those
intended to reflect the same concept should be based on the
same theory and carefully formulated.

Receive feedback on language,

structure, and formatting
Professional editors proofread and edit your paper by
focusing on:

 Academic style
 Vague sentences
 Grammar
 Style consistency

See an example

Which type of reliability applies to my

research?
It’s important to consider reliability when planning your research
design, collecting and analyzing your data, and writing up your
research. The type of reliability you should calculate depends on
the type of research and your methodology.

Measuring a property that you expect to stay the same over time.

Test-retest

https://www.scribbr.com/methodology/types-of-reliability/ 8/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

Multiple researchers making observations or ratings about the same

 Table topic.
of contents

Interrater

Using two different tests to measure the same thing.

Parallel forms

Using a multi-item test where all the items are intended to measure the
same variable.

Internal consistency

If possible and relevant, you should statistically calculate reliability

and state this alongside your results.

Other interesting articles

If you want to know more about statistics, methodology, or
research bias, make sure to check out some of our other articles
with explanations and examples.

 Statistics

Normal distribution
Skewness
Kurtosis
Degrees of freedom
Variance
Null hypothesis

 Methodology
https://www.scribbr.com/methodology/types-of-reliability/ 9/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

Discourse analysis
 Table ofControl
contents
groups
Mixed methods research
Non-probability sampling
Quantitative research
Ecological validity

 Research bias

Rosenthal effect
Implicit bias
Cognitive bias
Selection bias
Negativity bias
Status quo bias

Frequently asked questions about types

of reliability

What’s the difference between reliability and validity? 

How can I minimize observer bias in my research? 

Why are reproducibility and replicability important? 

Why is bias in research a problem? 

Cite this Scribbr article

https://www.scribbr.com/methodology/types-of-reliability/ 10/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

If you want to cite this source, you can copy and paste the
 Table citation or click the “Cite this Scribbr article” button to
of contents
automatically add the citation to our free Citation Generator.

Middleton, F. (2023, June 22). The 4 Types of

Reliability in Research | Definitions & Examples.
Cite this
Scribbr. Retrieved December 9, 2024, from
article
https://www.scribbr.com/methodology/types-
of-reliability/

Is this article helpful?

1868 131

Fiona Middleton
Fiona has been editing for Scribbr since August 2016. She has a
bachelor's degree in geology and is currently working towards a
master's degree in marine sciences. She loves working with
students based around the world to refine their writing.

https://www.scribbr.com/methodology/types-of-reliability/ 11/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

https://www.scribbr.com/methodology/types-of-reliability/ 12/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

https://www.scribbr.com/methodology/types-of-reliability/ 13/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

https://www.scribbr.com/methodology/types-of-reliability/ 14/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

https://www.scribbr.com/methodology/types-of-reliability/ 15/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

https://www.scribbr.com/methodology/types-of-reliability/ 16/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

https://www.scribbr.com/methodology/types-of-reliability/ 17/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

https://www.scribbr.com/methodology/types-of-reliability/ 18/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

https://www.scribbr.com/methodology/types-of-reliability/ 19/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

 Table of contents

Other students also liked

Reliability vs. Validity in Research | Difference,

Types and Examples
Reliability is about a method's consistency, and validity is about its accuracy. You
can assess both using various types of evidence.

4609

What Is Quantitative Research? | Definition,

Uses & Methods
Quantitative research means collecting and analyzing numerical data to describe

characteristics, find correlations, or test hypotheses.

2240

Data Collection | Definition, Methods &

Examples
Data collection is the systematic process of gathering observations or
measurements in research. It can be qualitative or quantitative.

1790

Scribbr
Our editors
https://www.scribbr.com/methodology/types-of-reliability/ 20/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

Jobs
 Table of contents Partners
FAQ
Copyright, Community Guidelines, DSA & other Legal Resources

Our services
Plagiarism Checker
Proofreading Services
Citation Generator
AI Proofreader
AI Detector
Paraphrasing Tool
Grammar Checker
Spell Checker
Essay Checker
Punctuation Checker
Free Text Summarizer
Paragraph Rewriter
Sentence Rewriter
Rewording Tool

Contact
[email protected]
 +1 (510) 822-8066


 
 


Excellent

https://www.scribbr.com/methodology/types-of-reliability/ 21/22
12/12/24, 4:13 PM The 4 Types of Reliability in Research | Definitions & Examples

Terms of Use

 Table of contents
Cookie preferences

Privacy Policy

Happiness guarantee

https://www.scribbr.com/methodology/types-of-reliability/ 22/22