ANALYZING THE TEST

Romeo T. Quintos, Jr.

[email protected]
• After administering and scoring the test, the teacher should
also analyze the quality of each test item in the test. Through
this you can identify the item that is good, item that needs
improvement or items to be removed from the test. But when
do we consider that the test is good? How do we evaluate the
quality of each item in the test?
• Why is it necessary to evaluate each item in the test?
• To help teachers determine the quality of a test item known as
item analysis. One of the purposes of item analysis is to
improve the quality of the assessment tools. Through this
process, we can identify the item to be retained, revised or
rejected and also the content of the lesson that is mastered or
not.
• There are two kinds of item analysis, quantitative item analysis
and qualitative item analysis
 ITEM ANALYSIS
• Item analysis is a process of examining the student’s
response to individual item in the test.
• It consists of different procedures for assessing the
quality of the test items given to the students.
 ITEM ANALYSIS
• Uses of Item Analysis
1. Item analysis data provide a basis for efficient class
discussion of the test results.
 ITEM ANALYSIS
• Uses of Item Analysis
1. Item analysis data provide a basis for efficient class
discussion of the test results.
2. Item analysis data provide a basis for remedial work.
 ITEM ANALYSIS
• Uses of Item Analysis
1. Item analysis data provide a basis for efficient class
discussion of the test results.
2. Item analysis data provide a basis for remedial work.
3. Item analysis data provide a basis for general
improvement of classroom instruction.
 ITEM ANALYSIS
• Uses of Item Analysis
1. Item analysis data provide a basis for efficient class
discussion of the test results.
2. Item analysis data provide a basis for remedial work.
3. Item analysis data provide a basis for general
improvement of classroom instruction.
4. Item analysis data provide a basis for increased skills in
test construction.
 ITEM ANALYSIS
• Uses of Item Analysis
1. Item analysis data provide a basis for efficient class
discussion of the test results.
2. Item analysis data provide a basis for remedial work.
3. Item analysis data provide a basis for general
improvement of classroom instruction.
4. Item analysis data provide a basis for increased skills in
test construction.
5. Item analysis procedures provide a basis for
constructing test bank.
 ITEM ANALYSIS
• Types of Quantitative Item Analysis
There are three common types of quantitative item
analysis which provide teachers with three different types
of information about individual test items. These are:
1. difficulty index
2. discrimination index
3. response options analysis
 ITEM ANALYSIS
• 1) Difficulty Index
It refers to the proportion of the number of students in the
upper and lower groups who answered an item correctly.
The larger the proportion the more students who have
learned the subject is measured by the item. To compute
the difficulty index of an item use the formula:
𝒏
DF =
𝑵
where:
DF =difficulty index
n-=number if the students selecting item correctly in the upper
and lower group
N=Total number of students who answered the test
 ITEM ANALYSIS
• Level of Difficulty
• To determine the level of difficulty of an item, find first
the difficulty index using the formula and identify the
level of difficulty using the range given below.
Level of Difficulty of an Item
Index Range Difficulty Level
0.00 – 0.20 Very Difficult
0.21 – 0.40 Difficult
0.41 – 0.60 Average/Moderately Difficult
0.61 – 0.80 Easy
0.81 – 1.00 Very Easy
 ITEM ANALYSIS
• 2) Discrimination Index
The power of the item to discriminate the students
between those who scored high and those who scored low
in the overall test. In other words, it is the power of the
item to discriminate the students who know the lesson
and those who do not know the lesson. It also refers to the
number of students in the upper group who got an item
correctly minus the number of students in the lower group
who got an item correctly.
 ITEM ANALYSIS

Divide the difference by either the number of the students

in the upper group or number of students in the lower
group or get the higher number if they are not equal.
Discrimination index is the basis of measuring the validity
of an item.
This index can be interpreted as an indication of the extent
to which overall knowledge of the content area or mastery
of the skills is related to the response on an item.
 ITEM ANALYSIS
• Types of Discrimination Index
1. Positive Discrimination happens when more students
in the upper group got the item correctly those students in
the lower group.
2. Negative Discrimination occurs when more students in
the lower group got the item correctly those students in
the upper group.
3. Zero Discrimination happens when a number of
students in the upper group and lower group who answer
the test correctly are equal. hence, the test item cannot
distinguish the students who performed in the overall test
and the students whose performance are very poor.
 ITEM ANALYSIS
• Discrimination Index Formula

𝑪𝑼𝑮 −𝑪𝑳𝑮
𝑫𝑰 = 𝑫

where:
DI =discrimination index value
CUG = number of the students selecting the correct answers in the UG
CLG = number of the students selecting the correct answers in the LG
D = number of students in either the lower or upper group
 ITEM ANALYSIS
• Level of Discrimination
Index Range Discrimination Level
0.19 and below Poor item, should be eliminated or
need to be revised
0.20 – 0.29 Marginal item, needs revision
0.30 – 0.39 Reasonably good item but possibly
for improvement
0.40 and above Very good item
 ITEM ANALYSIS
• Steps in Solving Difficulty Index and Discrimination Index
• 1. Arrange the scores from highest to lowest.
• 2. Separate the scores into UG and LG. There are different
methods to do this:
• A) if a class consists of 30 students who takes an exam,
arrange their scores from highest to lowest, then divide
them into two groups. The highest score belongs to the
upper group. The lowest score belongs to the lower
group.
• B) other literatures suggested to use 27%, 30%, or 33% of
the students for the UG and LG. However, in the LET the
test developers always used 27% of the students who
participated in the examination for the UG and LG.
 ITEM ANALYSIS
• Steps in Solving Difficulty Index and Discrimination Index
• 3. Count the number of those who chose the
alternatives in the upper and lower group for each item
and record the information using the template:

Options A B C D E
Upper Group
Lower Group
 ITEM ANALYSIS
• Steps in Solving Difficulty Index and Discrimination Index
• 4. Compute the value of the difficulty index and the
discrimination index and also the analysis of each
response in the distracters.
• 5. Make an analysis for each item.
 ITEM ANALYSIS
• Checklist for Discrimination Index
• It is very important to determine whether the test item will
be retained, revised or rejected. Using Discrimination Index
we can identify the non-performing question items; just
always remember that they seldom indicate what is the
problem. Use the given checklist below:
Yes No
1. Does the key discriminate positively?
2. Does the incorrect options discriminate negatively?
• If the answers to questions 1 and 2 are either YES or NO,
revise the item.
• If the answers to questions 1 and 2 are both NO, eliminate or
reject the item.
 ITEM ANALYSIS
• 3) Analysis of Response Options
• Aside from identifying the difficulty index and
discrimination index, another way to evaluate the
performance of the entire test item is through the
analysis of the response options. It is very important to
examine the performance of each option in a multiple-
choice item. Through this, you can determine whether
the distracters or incorrect options are effective or
attractive to those who do not know the correct answer.
The attractiveness of the incorrect options is
determined when more students in the lower group
than in the upper group choose it. Analyzing the
incorrect options allows the teachers to improve the
test items so that it can be used again in the future.
 ITEM ANALYSIS
• Distracter Analysis
• 1) Distracter
• Distracter is the term used for the incorrect options in the
multiple choice type of test while the correct answer
represents the key. It is very important for the test writer to
know if the distracters are effective or good distracters.
Using quantitative item analysis we can determine if the
options are good or if the distracter are effective.
• Item analysis can identify non-performing test items, but this
item seldom indicates the error or the problem in the given
item. These are factors to be considered why students failed
to get the correct answer in the given question.
 ITEM ANALYSIS
• A. it is not taught in the class properly.
• B. it is ambiguous.
• C. the correct answer is not in the given options.
• D. it has more than one correct answer.
• E. it contains grammatical clues to mislead the students.
• F. The student is not aware of the content.
• G. The students were confused by the logic of the
question because it has double negatives.
• H. the student failed to study the lesson.
 ITEM ANALYSIS
• 2) Miskeyed Item
• The test item is a potential miskey if there are more
students from the upper group who choose the
incorrect than the key.
• 3) Guessing Item
• Students from the UG have equal spread of choices
among the given alternatives. Students from the UG
guess their answers because the following reasons.
• A. the content of the test is not discussed in the class or in
the test.
• B. the test item is very difficult.
• C. the question is trivial.
 ITEM ANALYSIS
• 4) Ambiguous Item
• This happens when more students from the UG choose
equally an incorrect option and the keyed answer.
 ITEM ANALYSIS
• Qualitative Item Analysis
• Qualitative Item Analysis is a process in which the
teacher or expect carefully proofreads the test before it
is administered, to check if there are typographical
errors, to avoid grammatical clues that may lead to
giving away the correct answer, and to ensure that the
level of reading materials is appropriate. These
procedures can also include small group discussions on
the quality of the examination and its items, with
examinees that have already took the test.
 ITEM ANALYSIS
• IMPROVING TEST ITEMS
• To improve multiple choice test item we shall consider
the stem of the item, the distracters and the key answer.
• How to Improve the Test Item

• Example 1: A class is composed of 40 students. Divide

the group into two(UG/LG). Option B is the correct
answer. Based from the given data on the table, as a
teacher, what would you do with the test item?
Options A B* C D E
Upper Group 3 10 4 0 3
Lower Group 4 4 8 0 4

• 1) Compute the difficulty index.

𝑛 14
n = 10 + 4 = 14 DF = 𝑁 DF = 40 DF = 0.35
N = 40
 Level of Difficulty of an Item

Index Range Difficulty Level

0.00 – 0.20 Very Difficult
0.21 – 0.40 Difficult
0.41 – 0.60 Average/Moderately Difficult
0.61 – 0.80 Easy
0.81 – 1.00 Very Easy
• How to Improve the Test Item

• Example 1: A class is composed of 40 students. Divide

the group into two(UG/LG). Option B is the correct
answer. Based from the given data on the table, as a
teacher, what would you do with the test item?
Options A B* C D E
Upper Group 3 10 4 0 3
Lower Group 4 4 8 0 4

• 1) Compute the difficulty index.

𝑛 14
n = 10 + 4 = 14 DF = 𝑁 DF = 40 DF = 0.35
N = 40 Difficult item
• How to Improve the Test Item

• Example 1: A class is composed of 40 students. Divide

the group into two(UG/LG). Option B is the correct
answer. Based from the given data on the table, as a
teacher, what would you do with the test item?
Options A B* C D E
Upper Group 3 10 4 0 3 DF = 0.35
Lower Group 4 4 8 0 4

• 2) Compute the discrimination index.

CUG = 10 CLG = 4 𝐶𝑈𝐺 − 𝐶𝐿𝐺 10 − 4
𝐷𝐼 = 𝐷𝐼 = = 0.3
D = 20 𝐷 20
 ITEM ANALYSIS
• Level of Discrimination
Index Range Discrimination Level
0.19 and below Poor item, should be eliminated or
need to be revised
0.20 – 0.29 Marginal item, needs revision
0.30 – 0.39 Reasonably good item but possibly
for improvement
0.40 and above Very good item
 • 3. Make an analysis about the level of difficulty,
discrimination and distracters.
Options A B* C D E
Upper Group 3 10 4 0 3 DF = 0.35
Lower Group 4 4 8 0 4 DI = 0.30
• a. Only 35% of the examinees got the answer correctly,
hence, the item is difficult.
• b. More students from the upper group got the answer
correctly. Hence, it has a positive discrimination.
• c. Retain options A, C and E because most of the
students who did not perform well in the overall
examination selected it. These options attract most
students from the lower group.
Options A B* C D E
Upper Group 3 10 4 0 3 DF = 0.35
Lower Group 4 4 8 0 4 DI = 0.30
• 4. Conclusion: Retain the test item but change option
D, make it more realistic to make it effective for the
UG and LG. At least 5% of the examinees choose the
incorrect option.
• How to Improve the Test Item

• Example 2: A class is composed of 50 students. Use 27%

to get the upper and the lower groups. Analyze the item
given the following results. Option D is the correct
answer. What will you do with the test item?
Options A B C D* E
Upper Group(27%) 3 1 2 6 2
Lower Group(27%) 5 0 4 4 1

• 1) Compute the difficulty index.

𝑛 10
n = 6 + 4 = 10 DF = 𝑁 DF = DF = 0.36
28
N = 28
• How to Improve the Test Item

• Example 2: A class is composed of 50 students. Use 27%

to get the upper and the lower groups. Analyze the item
given the following results. Option D is the correct
answer. What will you do with the test item?
Options A B C D* E
Upper Group(27%) 3 1 2 6 2
Lower Group(27%) 5 0 4 4 1
DF = 0.36
• 2) Compute the discrimination index.
CUG = 6 CLG = 4 𝐶𝑈𝐺 − 𝐶𝐿𝐺 6−4
𝐷𝐼 = 𝐷𝐼 = = 0.14
D = 14 𝐷 14
 • 3. Make an analysis about the level of difficulty,
discrimination and distracters.
Options A B C D* E
Upper Group(27%) 3 1 2 6 2
Lower Group(27%) 5 0 4 4 1
DF = 0.36 DI = 0.14
• a. Only 36% of the examinees got the answer correctly,
hence, the item is difficult.
• b. More students from the upper group got the answer
correctly, hence, it has a positive discrimination.
• c. Retain options A and C because most of the students
who did not perform well in the overall examination
selected it. A and C are effective distracters.
• 3. Make an analysis about the level of difficulty,
discrimination and distracters.
Options A B C D* E
Upper Group(27%) 3 1 2 6 2
Lower Group(27%) 5 0 4 4 1
DF = 0.36 DI = 0.14

• d. Modify option B and E because mores students from

the upper group chose them compare with the lower
group, hence, they are not effective distracters because
most of the students who performed well in the overall
examination selected them as their answers.
Options A B C D* E
Upper Group(27%) 3 1 2 6 2
Lower Group(27%) 5 0 4 4 1
DF = 0.36 DI = 0.14

• 4. Conclusion: Revised the item by modifying options B

and E.
• How to Improve the Test Item

• Example 3: A class is composed of 50 students. Use 27%

to get the upper and the lower groups. Analyze the item
given the following results. Option E is the correct
answer. What will you do with the test item?
Options A B C D E*
Upper Group(27%) 2 3 2 2 5
Lower Group(27%) 2 2 1 1 8

• 1) Compute the difficulty index.

𝑛 13
n = 5 + 8 = 13 DF = 𝑁 DF = DF = 0.46
28
N = 28
 Level of Difficulty of an Item

Index Range Difficulty Level

0.00 – 0.20 Very Difficult
0.21 – 0.40 Difficult
0.41 – 0.60 Average/Moderately Difficult
0.61 – 0.80 Easy
0.81 – 1.00 Very Easy
• How to Improve the Test Item

• Example 3: A class is composed of 50 students. Use 27%

to get the upper and the lower groups. Analyze the item
given the following results. Option E is the correct
answer. What will you do with the test item?
Options A B C D E*
Upper Group(27%) 2 3 2 2 5
Lower Group(27%) 2 2 1 1 8
DF = 0.46
• 2) Compute the discrimination index.
CUG = 5 CLG = 8 𝐶𝑈𝐺 − 𝐶𝐿𝐺 5−8
𝐷𝐼 = 𝐷𝐼 = = −0.21
D = 14 𝐷 14
 • 3. Make an analysis about the level of difficulty,
discrimination and distracters.
Options A B C D E*
Upper Group(27%) 2 3 2 2 5
Lower Group(27%) 2 2 1 1 8
DF = 0.46 DI = – 0.21
• a. Only 46% of the examinees got the answer correctly,
hence, the item is moderately difficult.
• b. More students from the lower group got the answer
correctly, so, it has a negative discrimination.
• c. No need to analyze the distracters because the item
discriminates negatively.
 • 3. Make an analysis about the level of difficulty,
discrimination and distracters.
Options A B C D E*
Upper Group(27%) 2 3 2 2 5
Lower Group(27%) 2 2 1 1 8
DF = 0.46 DI = – 0.21

• d. Modify all the distracters because they are not

effective. Most of the students in the upper group
choose the incorrect options. The options are effective
if most of the students in the LG chose the incorrect
options.
 Options A B C D E*
Upper Group(27%) 2 3 2 2 5
Lower Group(27%) 2 2 1 1 8
DF = 0.46 DI = – 0.21

• 4. Conclusion: Reject the item because it has a negative

discrimination index.
• POTENTIAL MISKEYED ITEM
• AMBIGUOUS ITEM
• GUESSING ITEM
• INEFFECTIVE DISTRACTERS
 ITEM ANALYSIS
• Level of Discrimination
Index Range Discrimination Level
0.19 and below Poor item, should be eliminated or
need to be revised
0.20 – 0.29 Marginal item, needs revision
0.30 – 0.39 Reasonably good item but possibly
for improvement
0.40 and above Very good item
 Level of Difficulty of an Item

Index Range Difficulty Level

0.00 – 0.20 Very Difficult
0.21 – 0.40 Difficult
0.41 – 0.60 Average/Moderately Difficult
0.61 – 0.80 Easy
0.81 – 1.00 Very Easy
• Potential Miskeyed Item

• Example 4: Make an item analysis about the table below.

Options A* B C D E
Upper Group 1 2 3 10 4
Lower Group 3 4 4 4 5

• 1) Compute the difficulty index.

𝑛 4
n=1+3=4 DF = 𝑁 DF = DF = 0.10
40
N = 40
• Potential Miskeyed Item

• Example 4: Make an item analysis about the table below.

Options A* B C D E
Upper Group 1 2 3 10 4
Lower Group 3 4 4 4 5

DF = 0.10
• 2) Compute the discrimination index.
CUG = 1 CLG = 3 𝐶𝑈𝐺 − 𝐶𝐿𝐺 1−3
𝐷𝐼 = 𝐷𝐼 = = −0.10
D = 20 𝐷 20
 • 3. Make an analysis about the level of difficulty,
discrimination and distracters.
Options A* B C D E
Upper Group 1 2 3 10 4
Lower Group 3 4 4 4 5
DF = 0.10 DI = – 0.10
• a. More students from the UG choose option D than
option A even if A is the correct answer.
• b. Most likely teacher has written the wrong answer
key.
• c. The teacher checks and finds out that he did not
miskey the answer that he thought is the correct
answer.
 • 3. Make an analysis about the level of difficulty,
discrimination and distracters.
Options A* B C D E
Upper Group 1 2 3 10 4
Lower Group 3 4 4 4 5
DF = 0.10 DI = – 0.10
• d. If the teacher miskeyed it, he must check and retally
the scores of the students’ test papers before giving
them back.
• e. If option A is really the correct answer, revise to
weaken option D, distracters are not supposed to draw
more attention than the keyed answer.
 • 3. Make an analysis about the level of difficulty,
discrimination and distracters.
Options A* B C D E
Upper Group 1 2 3 10 4
Lower Group 3 4 4 4 5
DF = 0.10 DI = – 0.10
• f. Only 10% of the students got the answer correctly,
hence the test item is very difficult.
• g. More students from the LG got the item correctly,
therefore, a negative discrimination resulted.
• h. No need to analyze the distracters because the test
item is very difficult and discriminates negatively.
 Options A* B C D E
Upper Group 1 2 3 10 4
Lower Group 3 4 4 4 5
DF = 0.10 DI = – 0.10
• 4. Conclusion: Reject the item because it is very difficult
and has a negative discrimination index.
 • Ambiguous Item

• Example 5: Make an item analysis about the table below.

Options A B C D E*
Upper Group 7 1 1 2 8
Lower Group 6 2 3 3 6

• 1) Compute the difficulty index.

𝑛 14
n = 8 + 6 = 14 D =
F 𝑁 DF = 39 DF = 0.36
N = 39
 • Ambiguous Item

• Example 5: Make an item analysis about the table below.

Options A B C D E*
Upper Group 7 1 1 2 8
Lower Group 6 2 3 3 6

DF = 0.36
• 2) Compute the discrimination index.
CUG = 8 CLG = 6 𝐶𝑈𝐺 − 𝐶𝐿𝐺 8−6
𝐷𝐼 = 𝐷𝐼 = = 0.10
D = 20 𝐷 20
• Ambiguous Item

• Example 5: Make an item analysis about the table below.

Options A B C D E*
Upper Group 7 1 1 2 8
Lower Group 6 2 3 3 6
DI = 0.10 DF = 0.36
• a. Only 36% of the examinees got the answer correctly,
hence, the item is difficult.
• b. More students from the upper group got the answer
correctly, so, it has a positive discrimination.
• c. About equal numbers of top students went for option A
and option E, this implies that they could not tell which is
the correct answer. The students do not know the content
of the test, thus, a reteach is needed..
• Ambiguous Item

Options A B C D E*
Upper Group 7 1 1 2 8
Lower Group 6 2 3 3 6
DI = 0.10 DF = 0.36
• 4. Conclusion: Revise the test item because it is
ambiguous.
 • Guessing Item

• Example 6: Make an item analysis about the table below.

Options A B C* D E
Upper Group 4 3 4 3 6
Lower Group 3 4 3 4 5

• 1) Compute the difficulty index.

𝑛 7
n=4+3=7 D =
F 𝑁 DF = 39 DF = 0.18
N = 39
 • Guessing Item

• Example 6: Make an item analysis about the table below.

Options A B C* D E
Upper Group 4 3 4 3 6
Lower Group 3 4 3 4 5

DF = 0.18
• 2) Compute the discrimination index.
CUG = 4 CLG = 3 𝐶𝑈𝐺 − 𝐶𝐿𝐺 4−3
𝐷𝐼 = 𝐷𝐼 = = 0.05
D = 20 𝐷 20
• Guessing Item

• Example 6: Make an item analysis about the table below.

Options A B C* D E
Upper Group 4 3 4 3 6
Lower Group 3 4 3 4 5
DI = 0.05 DF = 0.18
• a. Only 18% of the examinees got the answer correctly,
hence, the item is very difficult.
• b. More students from the upper group got the answer
correctly, so, it has a positive discrimination.
• c. Students respond about equally to all alternatives, an
indication that they are guessing.
• d. If the test item is well-written but too difficult,
reteach the material to the class.
• Guessing Item

• Three possibilities why student guesses the answer on

a test item:
• a. the content of the test item has not yet been
discussed in the class because the test is designed in
advance;
• b. test item was badly written that students have no
idea what the question is really all about; and
• c. test item was well-written but too difficult, as shown
by the DI.
• Guessing Item

Options A B C* D E
Upper Group 4 3 4 3 6
Lower Group 3 4 3 4 5
DI = 0.05 DF = 0.18

• 4. Conclusion: Reject the item because it is very

difficult, and the discrimination index is very poor, and
options A and B are not effective distracters.
 • Ineffective Distracters

• Example 7: Make an item analysis about the table below.

Options A B C* D E
Upper Group 5 3 9 0 3
Lower Group 6 4 6 0 4

• 1) Compute the difficulty index.

𝑛 15
n = 9 + 6 = 15 D =
F 𝑁 DF = DF = 0.38
40
N = 40
 • Ineffective Distracters

• Example 7: Make an item analysis about the table below.

Options A B C* D E
Upper Group 5 3 9 0 3
Lower Group 6 4 6 0 4

DF = 0.38
• 2) Compute the discrimination index.
CUG = 9 CLG = 6 𝐶𝑈𝐺 − 𝐶𝐿𝐺 9−6
𝐷𝐼 = 𝐷𝐼 = = 0.15
D = 20 𝐷 20
• Ineffective Distracters

• Example 7: Make an item analysis about the table below.

Options A B C* D E
Upper Group 5 3 9 0 3
Lower Group 6 4 6 0 4
DI = 0.15 DF = 0.38
• a. Only 38% of the examinees got the answer correctly,
hence, the item is difficult.
• b. More students from the upper group got the answer
correctly, so, it has a positive discrimination.
• c. Options A,B and E are attractive and effective
distracters.
• d. Option D is ineffective; therefore, change it with
more realistic one.
• Ineffective Distracters

Options A B C* D E
Upper Group 5 3 9 0 3
Lower Group 6 4 6 0 4
DI = 0.15 DF = 0.38
• 4. Conclusion: Revise the item by changing option D.