Assessment and Evaluation in Mathematics

Administering, Analyzing and Improving Mathematics Tests Page 1 of 14

Topic: Administering, Analyzing and Improving Mathematics Tests

I. INTRODUCTION:

Item analysis provides statistics on overall performance, test quality, and individual questions. This
data helps you recognize questions that might be poor discriminators of student performance. In this module,
you will learn how to analyze test items and improve its quality.

II. OBJECTIVES:

At the end of this module, you will be able to:

* define Item Analysis,
* discuss the use of item analysis, and
* analyze test items and improve its quality through steps.
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 2 of 14

III. PRELIMINARY ACTIVITIES:

Before you proceed to the main lesson, test yourself in this activity.

Activity

Answer:
* Why item analysis is important? Explain your answer.

GREAT!!!
You may now proceed to the main lesson.
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 3 of 14

IV. LESSON PROPER

Item Analysis Defined

Item analysis is the act of analyzing student responses to individual exam questions with the intention
of evaluating exam quality. It is an important tool to uphold test effectiveness and fairness. Item analysis is
likely something educators do both consciously and unconsciously on a regular basis

An item analysis provides three kinds of important information about the quality of test items.
 Item difficulty: A measure of whether an item was too easy or too hard.
 Item discrimination: A measure of whether an item discriminated between students who knew the
material well and students who did not.
 Effectiveness of alternatives: Determination of whether distractors (incorrect but plausible answers)
tend to be marked by the less able students and not by the more able students.
Item difficulty, item discrimination and the effectiveness of distractors on a multiple-choice test are
automatically available with Par Score’s item analysis. An illustration of Par Score’s “Standard Item Analysis
Report” printout is attached.

Uses for item analysis:

 Improve questions for future test administrations or to adjust credit on current attempts
 Discuss test results with your class
 Provide a basis for remedial work
 Improve classroom instruction

Steps in item analysis (relative criteria tests)

1. Award of a score to each student

A practical, simple and rapid method is to perforate on your answer sheet* the boxes corresponding
to the correct answer. By placing the perforated sheet on the student's answer sheet the raw score (number
of correct answers) can be found almost automatically.
This format corresponds to a multiple-choice question test. However, the “item analyses” technique
can also be used for other types of assessment instrument.
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 4 of 14

2. Ranking in order of merit

Assuming that the scores of 21 students have been obtained (alphabetical list on the left), this step
consists merely in ranking (listing) students in order of merit (in relation to the score) proceeding from the
highest to the lowest score. Let us assume the list as under A and then rank the students to obtain distribution
B, ranging from 4 to 27.
A B
Student Score Order Student Score
Albert 7 1 Patrick 27
Alfred 13 2 Frank 26
Andrew 19 3 Ann 25
Ann 25 4 Emily 24
Brian 16 5 Jennifer 21
Christine 19 6 Christine 19
Elise 17 6 Andrew 19
Emily 24 8 Elise 17
Felicity 16 8 Ian 17
Frances 14 8 Fred 17
Frank 26 11 Brian 16
Fred 17 11 Felicity 16
Harriet 11 11 Margaret 16
Ian 17 11 Philip 16
John 14 15 Frances 14
Jennifer 21 15 John 14
Margaret 16 17 Alfred 13
Michael 9 18 Harriet 11
Patrick 27 19 Michael 9
Peter 4 20 Albert 7
Philip 16 21 Peter 4
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 5 of 14
3. Identification of high and low groups
Ebel suggests the formation of “high” and “low” groups comprising only the first 27% (high group)
and the last 27% (low group) of all the students ranked in order of merit.
Ebel, R.L. (1965) Measuring educational achievement, Prentice Hall, pp. 348 - 349.
Why 27%? Because 27% gives the best compromise between two desirable but contradictory
aims:
1. making both groups as large as possible;
2. making the two groups as different as possible.
Truman Kelley showed in 1939 that when each group consists of 27% of the total it can be said with
the highest degree of certainty that those in the high group are really superior (with respect to the quality
measured by the test) to those in the low group. If a figure of 10% were taken, the difference between the
two means of the competence of the two groups would be greater but the groups would be much smaller and
there would be less certainty regarding their mean level of performance.
Similarly, if a figure of 50% was taken the two groups would be of maximum size but since the basis
of our ranking is not absolutely accurate, certain students in the high group would really belong to the low
group, and vice versa.
While the choice of 27% is the best, it is, however, not really preferable to 25% or 33%; and if it is
preferred to work with 1/4 or 1/3 rather than with the somewhat odd figure of 27% there is no great
disadvantage in so doing.
For the rest of our analysis we shall use 33%.

4. Calculation of the difficulty index of a question

Difficulty index
Index for measuring the easiness or difficulty of a test question. It is the percentage (%) of
students who have correctly answered a test question; it would be more logical to call it the easiness
index. It can vary from 0 to 100%.

Calculation
The following formula is used:

where H = number of correct answers in the high group

L = number of correct answers in the low group
N = total number of students in both groups
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 6 of 14
5. Calculation of the discrimination index of a question

Discrimination index
An indicator showing how significantly a question discriminates between “high” and “low”
students.

It varies from -1 to +1.

Calculation

The following formula is used:

6. Critical evaluation of a question

This is based on the indexes obtained.
Difficulty index: the higher this index the easier the question; it is thus an illogical term. It is
sometimes called “easiness index”, but in the American literature it is always called “difficulty index”.
In principle, a question with a difficulty index lying between 30% and 70% is acceptable (in that range,
the discrimination index is more likely to be high).
Some authors give values between 35% and 85%.
If for a test you use a group of questions with indexes in the range 30% - 70%, then the mean index
will be around 50%. It has been shown that a test with a difficulty index in the range of 50% - 60% is very
likely to be reliable as regards its internal consistency or homogeneity.
Discrimination index: the higher the index the more a question will distinguish (for a given group of
students) between “high” and “low” students. When a test is composed of questions with high discrimination
indexes, it ensures a ranking that clearly discriminates between the students according to their level of
performance, i.e., it gives no advantage to the low group over the high group. In other words, it helps you to
find out who are the best students.
It is most useful in preparing your question bank. Using the index, you can judge questions as
follows:
0.35 and over : Excellent question
0.25 to 0.34 : Good question
0.15 to 0.24 : Marginal question - revise
under 0.15 : Poor question - most likely discard
Remember that the index has an indicative rather than an absolute value.
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 7 of 14

Activity 1:
* What is Item Analysis?
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 8 of 14
Activity 2:
* Why item analysis is important?

* What are the uses of item analysis?

Activity 3
* What are the steps of item analysis? Explain each step.
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 9 of 14

We had just finished the discussion on Basic Concepts in

Mathematical Assessment. Let’s move on to the next higher level of
activity/ies or exercise/s that demonstrate your potential skills/knowledge of
what you have learned.

V. ANALYSIS, APPLICATION AND EXPLORATION

Name:________________________________ Grade & Section: ____________

Make an item analysis of the result of quiz of 14 students in Math class.

Student Score
Bong 20
Jessa 19
Randy 15
Ann 11
Ransel 20
Rissa 12
Jess 6
Rina 11
Danna 12
Rosa 9
Frank 18
Peter 15
Rica 13
Ian 17

John 14
Finally, let us summarize the lesson of what we had discussed today.
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 10 of 14

VI. GENERALIZATION

Make a Journal Writing through MRI Approach:

M-Memory - this is all about the things that you have learned in the lesson
R-Reflect - this is for all the topics that gave impact to your life. Make a list of those
topics.
I-Integrate - this is on how you apply the topics that you have learned in your daily life.

MRI APPROACH

KUDOS!
You have come to an end of Module 8.
OOPS! Don’t forget that you have still an assignment to do.
Here it is….
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 11 of 14

VII. ASSIGNMENT

Name:______________________________ Grade & Section: _______________

Based on the result of 10 student in 50 item Prelim exam, do we need to improve the test questions
or not? Why?
Boy – 19 Ron – 37 Jess - 31 Eliz - 21
Jane – 26 Dina – 15 Dorry - 41
Tess – 34 Joy – 19 Bon - 23

After your long journey of reading and accomplishing the module, let us now
challenge your mind by answering the evaluation part of this module.
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 12 of 14

VIII. EVALUATION

Name:_________________________________________ Grade & Section: ______________

Direction: Choose and write the letter of the correct answer.

1. Item Analysis is the most important to increase test effectiveness.

A. True
B. False
Answer: _____

2. In Item Analysis, each items contribution is analyzed and assessed.

A. True
B. False
Answer: _____

3. To write effective items, it is not necessary to examine whether they are measuring the fact,
idea, or concept for which they were intended.
A. True
B. False
Answer: _____

4. Item Difficulty measure whether an item is too easy or too hard.

A. True
B. False
Answer: _____

5. Item Discrimination delaminates distractors tend to be marked by the less able students and
not by the more able students.
A. True
B. False
Answer: _____

6. Item Difficulty, Item Discrimination and the Effectiveness of Distractors on a multiple-choice test
are automatically available with Par Score’s item analysis.
A. True
B. False
Answer: _____
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 13 of 14
7. After Item Analysis, the questions: (1) Is the wording of the question confusing? (2) Are the
answer options unclear? Are being answered.
A. True
B. False
Answer: _____

8. After awarding the score to each student, it should be rank in order to merit.
A. True
B. False
Answer: _____

9. The Difficulty Index measures the easiness and difficulty of test questions.
A. True
B. False
Answer: _____

10. In Discrimination Index, if the range is 0.35 and over, the question is poor.
A. True
B. False
Answer: _____

11. Which of the following are the functions of Item Analysis?

A) Improve questions for future test administrations or to adjust credit on current attempts.
B) Improve classroom instruction
C) Discuss test result and basis for remedial work
D) All of the above
Answer: _____

12. Which of the following is the first step of Item Analysis?

A) Award of a score to each student
B) Ranking in order of merit
C) Identification of groups: high and low
D) Calculation of the difficulty index of a question.
Answer: _____

13. Who said that when each group consist of 27% of the total, it can be said with the highest degree
of certainty that those in the higher group are really superior?
A) Truman Valley
B) Truman Kelley
C) Kelly Williams
D) Kelly Brint
 Assessment and Evaluation in Mathematics
Administering, Analyzing and Improving Mathematics Tests Page 14 of 14

14. Which of the following is the formula of computing Difficulty Index?

A) H+L X100
N
B)
H-L X 100
N

C) H+L / 100
N

D) H+L / 100
N
Answer: ____

15. Which of the following is the formula of Discrimination Index?

A.) 2 x (H - L)
N

B) 2 x (H + L)
N

C) 2 + (H - L)
N

D) 2 - (H - L)
N

Answer: _____
CONGRATULATIONS on reaching the end of this module!
You may now proceed to the next module.
Don’t forget to submit all the exercises, activities and portfolio
on ___________________.
KEEP UP THE GOOD WORK.
Well Done!!!