Testing: Assessment and

Evaluation

GOOD VS. BAD TEST ITEMS

 Selecting Appropriate Types of Items

A. The Objective Test Item

1. Supply Types
a. Short Answer
Ex: What is the name of the author of
Moby Dick?
b. Completion
Ex: The pattern of the sentence is
________.
 Selecting Appropriate Types of Items

2. Selection Types

a. True-False or Alternative Response

b. Matching Items
c. Multiple Choice
Selecting Appropriate Types of Items

3. Extended-Response Essay Questions

Ex: Explain why “Romeo and Juliet” is
considered a tragedy.
4. Restricted-Response Essay Questions
Ex: Name 2 elements to prove that “Romeo and
Juliet” is an example of a tragedy.
 Selecting Appropriate Types of Items

B. Other Types of Performance Assessment

1. Extended-Response Performance Tasks

Ex: Present a one-act play based on Shakespeare’s

“Macbeth”.
Selecting Appropriate Types of Items

2. Restricted-Response Performance Tasks

Ex: Memorize and deliver in class the famous

“Shylock’s Speech”.
 Considerations in Preparing Relevant Test Items

1. Matching Items and Tasks to Intended

Outcomes
Ex: Specific Learning Outcome: Identify an
example of hyperbole.
Which of the following is an example of
hyperbole?
A. She is an angel.
B. She has cried a river.
C. She roars like a lioness.
D. She died so that we may live.
 Considerations in Preparing Relevant Test Items

2. Obtaining a Representative Sample of Items and

Tasks
 We expect students to know thousands of
facts, but we can test for only a limited
number of them…we expect students to
develop thinking skills that will enable them
to solve a variety of problems, but we can test
their problem-solving ability with only a
limited number of problems.
 Considerations in Preparing Relevant Test Items

3. Eliminating Irrelevant Barriers to the Performance

 Some Possible Barriers in Test Items

 Ambiguous statements
 Excessive wordiness

 Difficult vocabulary

 Complex sentence structure

 Guidelines: Some Possible Barriers in Test Items

 Some Possible Barriers in Test Items (cont’d)

 Unclear instructions
 Unclear illustrative material

 Racial, ethnic, or gender bias

 Considerations in Preparing Relevant Test Items

4. Avoiding Unintended Clues in Objective Test Items

 Guidelines: Some Common Clues in Test

Items
 4. Avoiding Unintended Clues in Objective Test Items (cont’d)

 Grammatical inconsistencies
Ex: A verb is an
A. action word.
B. linking word.
C. helping word.
D. descriptive word.
Ex: A piece of land that is completely surrounded by
water is known as an ___.
 4. Avoiding Unintended Clues in Objective Test
Items (cont’d)
 Verbal associations
Ex: Which of the following words is used to connect
ideas?
A. Modifiers
B. Connectors
C. Prepositions
D. Antecedents
 4. Avoiding Unintended Clues in Objective Test
Items (cont’d)
 Specific determiners
 use of words like sometimes, usually, generally, always, never,
none, only, etc.

 Phrasing of correct responses

 Length of correct responses
 Location of correct responses
 General Suggestions for Writing Test Items

1. Use your test specifications as a guide.

2. Write more items as needed.
3. Write the items well in advance of the testing
date.
4. Write each test item so that the task to be
performed is clearly defined and it calls forth
the performance described in the intended
learning outcome.
 General Suggestions for Writing Test Items
(cont’d)

5. Write each item at an appropriate reading

level.
6. Write each item so that it does not provide
help in responding to other items.
7. Write each item so that the answer is one that
would be agreed upon by experts.
8. Whenever a test item or assessment task is
revised, recheck its relevance.
Constructing Objective Test Items: Simple
Forms

A. Short-Answer Items
1. Short Answer
2. Completion
Uses of Short-Answer Items
 Knowledge of Terminology
 Knowledge of Specific Facts
 Knowledge of Principles
 Constructing Objective Test Items: Simple Forms (cont’d)

Uses of Short-Answer Items (cont’d)

 Knowledge of Method or Procedure
 Simple Interpretations of Data
 Ability to Solve Numerical Problems
 Skill in Manipulating Mathematical Symbols
 Ability to Complete and Balance Chemical Equations
 Suggestions for Constructing Short-Answer Items

1. Word the item so that the required answer is both

brief and specific.

Poor: An adjective is a word that _____.

Better: An adjective is a word that describes a/an

_____.
 Suggestions for Constructing Short-Answer Items (cont’d)

2. Do not take statements directly from textbooks to

use as a basis for short-answer items.

Poor: Fiction is _____.

Better: Fiction may be classified into two general
categories namely _____.
 Suggestions for Constructing Short-Answer Items (cont’d)

3. A direct question is generally more desirable than an

incomplete statement.
Poor: Poetry flourished in _____.
Better: When did poetry flourish?
Best: In what year (or decade or century or period) did
poetry flourish?
Suggestions for Constructing Short-Answer Items (cont’d)

4. If the answer is to be expressed in numerical

units, indicate the type of answer wanted.
Poor: In “One Hundred Years of Solitude”, how long
did the rain last? _____.
Better: In “One Hundred Years of Solitude”, how
long did the rain last? _____ yr/s., _____
month/s.
 Suggestions for Constructing Short-Answer Items (cont’d)

5. Blanks for answers should be equal in length and in

a column to the right of the question.
Poor: What is the name of the part of speech that
connects words, clauses, and sentences?
_________
What is the name of the part of speech that
declares, asserts, or predicts something? ____
 Suggestions for Constructing Short-Answer Items (cont’d)

5. Blanks for answers should be equal in length and in

a column to the right of the question.
Better: What is the name of the part of speech that
connects words, clauses, and sentences?
_________
What is the name of the part of speech that
declares, asserts, or predicts something?
_________
Suggestions for Constructing Short-Answer Items (cont’d)

6. When completion items are used, do not include too

many blanks.
Poor: A _____ is a group of words that _____ a
_____ thought.

Better: A _____ is a group of words that expresses a

complete thought.
 B. True-False or Alternative-
Response Items

Uses of True-False Items:

Ability to identify:
 the correctness of statements of facts
 definitions of terms
 statements of principles
 Suggestions for Constructing True-False Items

1. Avoid broad general statements if they

are to be judged true or false.
Poor: The president of the Philippines is usually
elected to that office. T F

2. Avoid trivial statements.

Poor: Rumor has it that Shakespeare was a
homosexual. T F
Suggestions for Constructing True-False Items (cont’d)

3. Avoid the use of negative statements, especially

double negatives.

Poor: None of the steps in the process was

unnecessary. T F
Suggestions for Constructing True-False Items (cont’d)

4. Avoid long, complex sentences.

Poor: Every time you read a work, you encounter
assumptions, beliefs, and perspectives that are
different from your own and which make up that
work’s general repertoire.
Better: As a reader, you too have a general
repertoire. T F
Suggestions for Constructing True-False Items (cont’d)

5. Avoid including two ideas in one statement, unless

cause-effect relationships are being measured.

Poor: A verb is not just an action word because it also

functions in different ways. T F
Suggestions for Constructing True-False Items (cont’d)

6. Avoid using opinion that is not attributed to some

source, unless the ability to identify opinion is
being specifically measured.

Poor: J.K. Rowling is considered the richest fiction

writer of the decade. T F
Better: According to Forbes Magazine, J.K. Rowling
is considered the richest fiction writer of the
decade. T F
Suggestions for Constructing True-False Items (cont’d)

7. Avoid using true statements and false statements

that are unequal in length.

 There is a natural tendency for true statements to be

longer because such statements must be precisely
phrased in order to be absolutely true.
Suggestions for Constructing True-False Items (cont’d)

8. Avoid using disproportionate numbers of true

statements and false statements.
 Constructing a test with an approximately equal
number of true statements and false statements will
prevent response sets from unduly inflating or
deflating the students’ scores.
 C. Matching Exercises

Uses of Matching Exercises:

 To measure factual information based on simple associations

 To relate two things that have some logical basis for
association
 Suggestions for Constructing Matching Exercises

1. Use only homogenous material in a

single matching exercise.

Column A Column B
1. Known as the Father a. Nick Joaquin
of detective stories b. D. H. Lawrence
2. Known as Quijano c. Edgar Allan Poe
de Manila d. Carlos Palanca, Sr.
3. Considered as the e. William Shakespeare
Renaissance Man f. Kerima Polotan
Tuvera
 Suggestions for Constructing Matching Exercises (cont’d)

2. Include an unequal number of responses and

premises, and instruct the student that responses
may be used once, more than once, or not at all.
3. Keep the list of items to be matched brief, and place
the shorter responses on the right.
Suggestions for Constructing Matching Exercises (cont’d)

4. Arrange the list of responses in logical order.

Place the words in alphabetical order and
numbers in sequence.

Column A Column B
1. The Puritan Age a. 13th Century
2. The Elizabethan b. 14th Century
Period c. 15th Century
3. The Renaissance d. 16th Century
Period e. 17th Century
f. 18th Century
 Suggestions for Constructing Matching Exercises (cont’d)

5. Indicate in the directions the basis for matching the

responses and premises.
Ex: Directions: On the left line to the left of each
literary period in Column A, write the letter from
Column B that identifies the specific century when
the event occurred. Each date in Column B may be
used once, more than once or not at all.
 Suggestions for Constructing Matching Exercises (cont’d)

6. Place all of the items for one matching exercise on

the same page.
Constructing Objective Test Items: Multiple-Choice Forms

Characteristics of Multiple-Choice Items:

 The problem may be stated as a direct question or
an incomplete statement and is called the stem of
the item.
 The list of suggested solutions may include words,
numbers, symbols, or phrases and are
alternatives (also called choices or options).
 The correct alternative in each item is called the
answer, and the remaining alternatives are called
distracters (also called decoys or foils).
 Item Types

 Direct-Question Form

 Incomplete-Statement Form

 Best-Answer Type
 Uses of Multiple-Choice Outcomes

1. Measuring Knowledge Outcomes

 Knowledge of Terminology
 Knowledge of Specific Facts
 Knowledge of Principles
 Knowledge of Methods and Procedures
Uses of Multiple-Choice Outcomes (cont’d)

2. Measuring Outcomes at the Understanding and

Application Levels

 Ability to Identify Application of Facts and Principles

 Ability to Interpret Cause-Effect Relationships
 Ability to Justify Methods and Procedures
Suggestions for Constructing Multiple-Choice Items

1. The stem of the item should be meaningful by

itself and should present a definite problem.
Poor: South America
A. is a flat, arid country.
B. imports coffee from the US.
C. has produced great fiction writers.
D. was settled mainly by colonists from
Spain.
Suggestions for Constructing Multiple-Choice Items (cont’d)

Better: Most of South America was settled by colonists

from
A. Spain.
B. France.
C. Holland.
D. England.
Suggestions for Constructing Multiple-Choice Items (cont’d)

2. The item stem should include as much of the item as

possible and should be free of irrelevant material.
Suggestions for Constructing Multiple-Choice Items (cont’d)

Poor: Most of South America was settled by colonists

from Spain. How would you account for the large
number of Spanish colonists settling there?
A. They were adventurous.
B. They wanted lower taxes.
C. They were in search of wealth.
D. They were seeking religious freedom.
Suggestions for Constructing Multiple-Choice Items (cont’d)

Better: Why did Spanish colonists settle most of South

America?
A. They were adventurous.
B. They wanted lower taxes.
C. They were in search of wealth.
D. They were seeking religious freedom.
Suggestions for Constructing Multiple-Choice Items (cont’d)

Best: Spanish colonists settled most of South America

in search of
A. wealth.
B. adventure.
C. lower taxes.
D. religious freedom.
Suggestions for Constructing Multiple-Choice Items (cont’d)

3. Use a negatively stated stem only when significant

learning outcomes require it.
Poor: Which of the following plays was not written by
Shakespeare?
A. Macbeth
B. Julius Caesar
C. Romeo and Juliet
D. Waiting for Godot
Suggestions for Constructing Multiple-Choice Items (cont’d)

Better: Which of the following plays was written by

Shakespeare?
A. Antigone
B. Macbeth
C. Waiting for Godot
D. Death of a Salesman
Suggestions for Constructing Multiple-Choice Items (cont’d)

4. All of the alternatives should be grammatically

consistent with the stem of the item.
Suggestions for Constructing Multiple-Choice Items (cont’d)

Poor: Flashback is a literary device that is used

A. for remembering past events.
B. to hint at significant past events.
C. so the writer can inject important
events in the past.
D. in writing to provide readers
background about past events.
Suggestions for Constructing Multiple-Choice Items (cont’d)

Better: Flashback is a literary device that is used to

A. recall past events.
B. hint at significant past events.
C. inject important events in the past.
D. provide readers background about
past events.
Suggestions for Constructing Multiple-Choice Items (cont’d)

5. An item should contain one correct or clearly best

answer.
Poor: The protagonist is considered a complex
character because he
A. has changed from good to evil.
B. makes the story more interesting.
C. makes his audience identify with him.
D. has transformed into a different person.
Suggestions for Constructing Multiple-Choice Items (cont’d)

Better: The protagonist is considered a complex

character because he
A. has changed from good to evil.
B. has introduced novelty in acting.
C. makes the story more interesting.
D. makes his audience identify with him.
Suggestions for Constructing Multiple-Choice Items (cont’d)

6. Items used to measure understanding should

contain some novelty, but beware of too much.
 The situations must be new to the students, but not
too far removed from the examples used in class.
Suggestions for Constructing Multiple-Choice Items (cont’d)

7. All distracters should be plausible. The purpose of

a distracter is to distract the uninformed from the
correct answer.

Ways to Make Distracters Plausible:

 Use the students’ most common errors.
 Use important-sounding words (e.g. significant,
accurate) that are relevant to the item stem. But
don’t overdo it!
 Ways to Make Distracters Plausible:
(cont’d)

 Use words that have verbal associations with the

item stem (e.g. politician, political)
 Use textbook language or other phraseology that
has the appearance of truth.
 Use incorrect answers that are likely to result from
student misunderstanding or carelessness (e.g.,
forgets to convert from feet to yard)
 Ways to Make Distracters Plausible:
(cont’d)

 Use distracters that are homogenous and similar

in content to the correct answer (e.g., all are
inventors)
 Use distracters that are parallel in form and
grammatically consistent with the item’s stem.
 Make the distracters similar to the correct answer
in length, vocabulary, sentence structure, and
complexity or thought.
Caution: Distracters should distract the
uninformed, but they should not result in trick
questions that mislead knowledgeable students.
Suggestions for Constructing Multiple-Choice Items (cont’d)

8. Verbal associations between the stem and the

correct answer should be avoided.
Suggestions for Constructing Multiple-Choice Items (cont’d)

9. The relative length of the alternatives should not

provide a clue to the answer.

Poor: What is the major purpose of the United

Nations?
A. To form new governments
B. To provide military control
C. To establish international law
D. To maintain peace among the peoples of
the world
Suggestions for Constructing Multiple-Choice Items (cont’d)

Better: What is the major purpose of the United

Nations?
A. To develop a new system of international law
B. To maintain peace among the peoples of the
world
C. To provide military control of nations that have
recently attained their independence
D. To establish and maintain democratic forms of
government in newly formed nations
Suggestions for Constructing Multiple-Choice Items (cont’d)

10. Use sparingly special alternatives such as “none of

the above” or “all of the above.”
Suggestions for Constructing Multiple-Choice Items (cont’d)

11. Do not use the multiple-choice items when other

item types are more appropriate.
Thank you very much
AND

GOOD LUCK!!!