I.

Introduction
Assessments are a crucial tool in various fields, including education, employment, and healthcare.
They help measure knowledge, skills, and abilities, informing decisions that can have a significant
impact on individuals and organizations. However, for assessments to be effective, they must be valid,
reliable, and free from bias. Testing an assessment for these essential qualities is critical to ensuring
that the results are accurate, fair, and meaningful.

II. Objectives
At the end of the topic, you should be able to:
 express the importance of testing an assessment for validity, reliability, and bias.

III. Unlocking of Difficulties

 Validity- Measures whether a test is actually measuring what it is intended to measure.
 Reliability- the consistency of a measurement tool, meaning it produces similar results
when repeated under similar conditions. Measures consistency of results if the same
test is administered multiple times under similar conditions.
 Bias - Represents a systematic error that favors a particular outcome, potentially
distorting the results of a study.

IV. Body of the Report

Understanding reliability vs validity
Reliability and validity are closely related, but they mean different things. A measurement can
be reliable without being valid. However, if a measurement is valid, it is usually also reliable.

What is reliability?
Reliability refers to how consistently a method measures something. If the same result can be
consistently achieved by using the same methods under the same circumstances, the
measurement is considered reliable.

You measure the temperature of a liquid sample several times under identical conditions. The
thermometer displays the same temperature every time, so the results are reliable.
A doctor uses a symptom questionnaire to diagnose a patient with a long-term medical
condition. Several different doctors use the same questionnaire with the same patient but give
different diagnoses. This indicates that the questionnaire has low reliability as a measure of the
condition.
What is validity?
Validity refers to how accurately a method measures what it is intended to measure. If research
has high validity, that means it produces results that correspond to real properties,
characteristics, and variations in the physical or social world.

High reliability is one indicator that a measurement is valid. If a method is not reliable, it
probably isn’t valid.

If the thermometer shows different temperatures each time, even though you have carefully
controlled conditions to ensure the sample’s temperature stays the same, the thermometer is
probably malfunctioning, and therefore its measurements are not valid.

If a symptom questionnaire results in a reliable diagnosis when answered at different times and
with different doctors, this indicates that it has high validity as a measurement of the medical
condition.
 However, reliability on its own is not enough to ensure validity. Even if a test is reliable, it may
not accurately reflect the real situation.

The thermometer that you used to test the sample gives reliable results. However, the
thermometer has not been calibrated properly, so the result is 2 degrees lower than the true
value. Therefore, the measurement is not valid.
A group of participants take a test designed to measure working memory. The results are
reliable, but participants’ scores correlate strongly with their level of reading comprehension.
This indicates that the method might have low validity: the test may be measuring participants’
reading comprehension instead of their working memory.
Validity is harder to assess than reliability, but it is even more important. To obtain useful
results, the methods you use to collect data must be valid: the research must be measuring
what it claims to measure. This ensures that your discussion of the data and the conclusions
you draw are also valid.

strategies for estimating validity and reliability.

How are reliability and validity assessed?

Reliability can be estimated by comparing different versions of the same measurement.
Validity is harder to assess, but it can be estimated by comparing the results to other
relevant data or theory. Methods of estimating reliability and validity are usually split up
into different types.

Types of reliability
Different types of reliability can be estimated through various statistical methods.

Types of reliability

Type of reliability What does it assess? Example

Test-retest reliability The consistency of a measure across time: do you A group of participants complete
get the same results when you repeat the a questionnaire designed to measure personality
measurement? traits. If they repeat the questionnaire days, weeks
or months apart and give the same answers, this
indicates high test-retest reliability.

Interrater reliability The consistency of a measure across raters or Based on an assessment criteria checklist, five
observers: do you get the same results when examiners submit substantially different results for
different people conduct the same measurement? the same student project. This indicates that the
assessment checklist has low inter-rater reliability
(for example, because the criteria are too
subjective).
Types of reliability

Type of reliability What does it assess? Example

Internal consistency The consistency of the measurement itself: do You design a questionnaire to measure self-esteem.
you get the same results from different parts of a If you randomly split the results into two halves,
test that are designed to measure the same thing? there should be a strong correlation between the
two sets of results. If the two results are very
different, this indicates low internal consistency.

Types of validity
The validity of a measurement can be estimated based on three main types of evidence.
Each type can be evaluated through expert judgement or statistical methods.

Types of validity

Type of validity What does it assess? Example

Construct validity The adherence of a measure to existing theory and A self-esteem questionnaire could be assessed by
knowledge of the concept being measured. measuring other traits known or assumed to be
related to the concept of self-esteem (such as social
skills and optimism). Strong correlation between
the scores for self-esteem and associated traits
would indicate high construct validity.

Content validity The extent to which the measurement covers all A test that aims to measure a class of students’
aspects of the concept being measured. level of Spanish contains reading, writing and
speaking components, but no listening component.
Experts agree that listening comprehension is an
essential aspect of language ability, so the test
lacks content validity for measuring the overall
level of ability in Spanish.

Criterion validity The extent to which the result of a measure A survey is conducted to measure the political
corresponds to other valid measures of the same opinions of voters in a region. If the results
concept. accurately predict the later outcome of an election
in that region, this indicates that the survey has
high criterion validity.

To assess the validity of a cause-and-effect relationship, you also need to consider internal
validity (the design of the experiment) and external validity (the generalizability of the
results).

Calculate and describe the properties of the following measures of variability; frequencies and percents,
variance, and standard deviation

Here's how to calculate and describe the properties of each measure:

 Frequencies and Percents
Frequencies and percents are measures of variability that describe the
distribution of a categorical variable.
Calculation:
1. Count the number of observations in each category.
2. Calculate the frequency by dividing the count by the total number of
observations.
3. Multiply the frequency by 100 to convert to a percentage.
Properties: -
Frequencies and percents describe the proportion of observations in each
category. –
They are useful for categorical variables with a small number of categories. –
They do not provide information about the spread or dispersion of the data.
Variance
Variance is a measure of variability that describes the spread or dispersion of a
continuous variable.
Calculation:
1. Calculate the mean of the data.
2. Subtract the mean from each observation to calculate the deviation.
3. Square each deviation.
4. Calculate the average of the squared deviations.
Properties: -
Variance describes the spread or dispersion of the data. –
It is sensitive to outliers, which can greatly increase the variance. –
It is not in the same units as the original data.
Standard Deviation
Standard deviation is a measure of variability that describes the spread or
dispersion of a continuous variable.
Calculation:
1. Calculate the variance of the data.
2. Take the square root of the variance.
Properties: -
Standard deviation describes the spread or dispersion of the data. –
It is in the same units as the original data. –
It is less sensitive to outliers than variance.

Example: Frequency and Percent

Score Frequency Percentage

5 15 50%
4 6 20%
3 3 10%
2 3 10%
 1 3 10%
N= 5 Ʃf = 30 Ʃ=100%

Variance

Example:

Score Mean
(X) (x̄ )
(x- x̄ ) (x- x̄ )2
7 12 -5 25
11 12 -1 1
8 12 -4 16
8 12 -4 16
19 12 7 49
15 12 3 9
7 12 -5 25
9 12 -3 9
9 12 -3 9
20 12 8 64
17 12 5 25
14 12 2 4
ƩX= 144 Ʃ(x- x̄ )2 = 252

2
( x− x̄)
Variance = ∑
N
Standard Deviation = √ Variance

V. Summary of the Report ( Sum up the report)

VI. Recommendation (at least 3 recommendations Based on Findings of the Report)
VII. References (Include all resources used)

Output: Post the Report for printing