Assessing Writing

By : KMS M Sandi Hidayatullah

GENRES OF WRITI'EN LANGUAGE
1. Academic Writing 3. Personal Writing
• Papers and general subject reports • Letters, emails, greeting cards, invitations
• Essays and compositions • Messages, notes
• Academically focused journals
• Calendar entries, shopping lists, reminders
• Short-answer test responses
• Financial documents (e.g., checks, tax forms, loan applications)
• Technical reports (e.g., lab reports)
• Forms, questionnaires, medical reports, immigration documents
• Theses and dissertations
• Diaries, personal journals
2. Job-related Writing
• Messages (e.g., phone messages)
• Fiction (e.g., short stories, poetry)

• Letters and emails

• Memos (e.g., interoffice)
• Reports (e.g., job evaluations, project reports)
• Schedules, labels, signs
• Advertisements, announcements
• Manuals
TYPES OF WRITING PERFORMANCE
Four categories of written performance capture the range of written 3. Responsive Writing
production
• Tasks require limited discourse level performance, connecting
1. Imitative Writing sentences into paragraphs and creating logical sequences
• Focus on fundamental tasks of writing letters, words, punctuation, • Genres include narratives, descriptions, reports, summaries, and
and brief sentences responses to reading
• Mastery of spelling and phoneme-grapheme correspondences • Writer has some freedom of choice and focuses on discourse
conventions, with attention to context and meaning
• Form is the primary focus, while context and meaning are
secondary 4. Extensive Writing
2. Intensive (Controlled) Writing • Mastery of all writing processes and strategies for various
purposes, up to essay or research paper length
• Skills in producing appropriate vocabulary, collocations, idioms,
and correct grammar up to sentence length • Focus on achieving purpose, organizing ideas logically, using
details, and demonstrating syntactic and lexical variety
• Some consideration of meaning and context, but tasks are mostly
form-focused and controlled • Limited focus on grammatical form, occasional editing or
proofreading
MICRO- AND MACROSKIILS OF
WRITING
• Taxonomy of micro- and macroskills helps define Macroskills:
assessment criteria 1.Use rhetorical forms and conventions of written
• Microskills are more relevant to imitative and intensive discourse.
writing tasks 2.Achieve communicative functions according to form and
• Macroskills are essential for responsive and extensive purpose.
writing 3.Convey links and connections between events and
Microskills: communicate relations such as main idea, supporting idea,
new information, given information, generalization, and
1.Produce graphemes and orthographic patterns of English.
exemplification.
2.Write at an efficient speed for the purpose.
4.Distinguish between literal and implied meanings in
3.Use appropriate word order patterns and produce a core writing.
vocabulary. 5.Convey culturally specific references in the context of the
4.Apply acceptable grammatical systems, patterns, and text.
rules. 6.Develop and use writing strategies: audience assessment,
5.Express meaning in different grammatical forms. prewriting devices, drafting with fluency, using
6.Use cohesive devices in written discourse. paraphrases and synonyms, seeking feedback, and
revising/editing based on feedback.
DESIGNING ASSESSMENT TASKS:
IMITATIVE WRITING
• Tasks in Handwriting Letters, Words, and
Punctuation
Considerations:
Types of assessment tasks: • Authenticity: Some tasks lack
1.Copying: Test-takers are instructed to copy letters authenticity, while others reflect
or words.
real-life situations.
2.Listening cloze selection tasks: Test-takers listen
to a passage and select missing words from a list. • Criterion: Specify the criteria for
3.Picture-cued tasks: Test-takers write the word that
corresponds to a displayed picture.
assessment and proceed with
4.Form completion tasks: Test-takers fill out a
caution.
simple form with personal information. • Balance: Ensure a balance
5.Converting numbers and abbreviations to words:
Test-takers write out numbers or abbreviations. between reading and writing
components in tasks.
DESIGNING ASSESSMENT TASKS:
IMITATIVE WRITING
Spelling Tasks and Detecting Phoneme-­Grapheme 4. Matching phonetic symbols: Students write correctly
Correspondences spelled words based on given phonetic symbols.
1. Spelling tests: Traditional dictation-based tests where • Works well with non-one-to-one correspondence
students write words from dictation. letters
• Focus on correct spelling
• Use caution with students unfamiliar with the phonetic
• Choose words encountered before to control for listening alphabet
errors
Considerations:
2. Picture-cued tasks: Pictures are displayed to assess spelling
of familiar but unpredictable words. • Use familiar words and relevant contexts for
• Opportunity to present challenging word pairs assessment tasks
• Relevant for assessing spelling conventions • Balance between formative reinforcement and
assessment of spelling skills
3. Multiple-choice techniques: Words and phrases presented
as multiple-choice tasks with a follow-up writing component. • Recognize the value and limitations of phonetic
symbols for literacy-level students
• Reinforce spelling conventions
• Can include homonyms for added challenge
DESIGNING ASSESSMENT TASKS:
INTENSIVE (CONTROLLED) WRITING
• Controlled writing: Also known as form-focused writing, Dicto-Comp
grammar writing, or guided writing
Dicto-Comp is a related form of controlled writing, requiring
• Display writing: Producing language to showcase recall and rewriting skills
language competence without conveying new information
Related form of controlled writing: Dicto-Comp
Dictation and Dicto-Comp
• Reading a paragraph at normal speed, followed by
Dicatation rewriting from memory
Dictation assesses listening and writing integration with a • Variation: Providing key words as cues for students in a
focus on correct spelling handout
• Dictation as imitative writing, focusing on writing what is • Dicto-Comp classified as an intensive or responsive
heard writing task
• Importance of correct spelling in dictation assessments Benefits of dictation and dicto-comp:
• Consideration of dictation as a controlled or intensive • Assessing listening and writing skills simultaneously
form of writing due to listening and inserting punctuation
• Testing comprehension and retention of content
• Encouraging recall and paraphrasing abilities
DESIGNING ASSESSMENT TASKS:
INTENSIVE (CONTROLLED) WRITING
Grammatical Transformation Tasks Advantages of grammatical transformation tasks:
Various versions of grammatical transformation tasks: • Easy administration and practicality
• High scorer reliability
• Changing tenses in a paragraph
• Measure knowledge of grammatical forms
• Converting full verb forms to contractions performed through writing
• Transforming statements to yes/no or wh-questions Considerations for using grammatical
• Changing questions into statements transformation tasks:
• Combining sentences using relative pronouns • Justifiable when assessing the ability to produce
grammatical forms
• Converting direct speech to indirect speech
• Limited in assessing communicative competence
• Changing from active to passive voice and meaningful language use
DESIGNING ASSESSMENT TASKS:
INTENSIVE (CONTROLLED) WRITING
Picture-Cued Tasks Picture sequence description task:
Picture-cued sentence writing: • Test-takers are shown a sequence of
• Test-takers see a drawing of a simple pictures depicting a story line
action • Write a description of the sequence
• Write a brief sentence describing what using multiple sentences
is happening Scoring considerations for picture-
Picture description task: cued tasks:
• Test-takers are presented with a • Designing a rating scale to account for
complex picture grammatical and lexical choices
• Asked to describe the picture using • Examples of different responses and
specific prepositions their corresponding scores
DESIGNING ASSESSMENT TASKS:
INTENSIVE (CONTROLLED) WRITING
Vocabulary Assessment Tasks Ordering Tasks
• Authenticity constraints in using a word in a • Appeals to those who enjoy word games and puzzles
sentence task: Test-takers need to come up with • Task involves reordering scrambled words into correct
appropriate sentences quickly sentences
Vocabulary writing task examples: • Ordering tasks appeal to those who enjoy word games and
puzzles
• Writing two sentences using given words
• Task involves reordering scrambled words into correct
• Writing three words that fit in the blank sentences
• Writing correct endings for word in given sentences • Task's authenticity and justification depend on factors like
sentence complexity and prior practice
Form-focused nature of vocabulary assessment
tasks Factors that can make the task more justifiable:

• Creative linkage of target word, collocations, and • Using simple sentences

morphological variants • Limiting the number of words in the sentence
• Tasks provide opportunities for assessing lexical • Ensuring only one possible sentence can be formed
choice and coherence • Providing practice in class
DESIGNING ASSESSMENT TASKS:
INTENSIVE (CONTROLLED) WRITING
Short-Answer and Sentence Completion Tasks
• Reading performance plays a significant role in completing these tasks
• Short-answer and sentence completion tasks require both reading and writing skills
• Reading performance is prominent in the first three tasks but less influential in the last three
• Scoring on a 2-1-0 scale can be an effective approach for evaluating responses
Examples of limited response writing tasks
• Task 1: Completing a dialogue by providing short answers about a person's identity and origin
• Task 2: Responding to a question about someone's field of study
• Task 3: Restating sentences using specific words
• Task 4: Asking questions about quantities in a kitchen scenario
• Task 5: Describing activities before and after a certain event using a schedule
• Task 6: Describing personal preferences in different scenarios
ISSUES IN ASSESSING RESPONSIVE
AND EXTENSIVE WRITING
• Assessing responsive and extensive writing involves addressing issues of
authenticity, scoring, and time constraints
• The discussion will cover task types, timed impromptu tests, scoring methods, and
the role of editing and responding
• Examples: full-length essays, term papers, project reports, theses, and dissertations
Assessment issues in responsive and extensive writing
1. Authenticity: Ensuring face and content validity to bring out the best in the writer
2. Scoring: Evaluating both form and function of the text, considering impact and
effectiveness
3. Time: Challenges of timed assessments for a skill that benefits from drafting,
revising, and editing
DESIGNING ASSESSMENT TASKSt
RESPONSIVE AND EXTENSIVE WRITING
• Responsive and extensive writing tasks are part of a continuum of Paragraph Construction Tasks:
possibilities. • Reading performance is important for effective paragraph writing.
• They range from lower-end tasks to more open-ended tasks. • Assessment criteria include topic sentence writing and
• Tasks include writing short reports, essays, summaries, and development within a paragraph.
responses. • Main and supporting ideas across paragraphs are evaluated.
• Evaluation criteria include addressing the topic, organizing
Paraphrasing: supporting ideas, using appropriate details, and demonstrating
• Paraphrasing is an important skill for second language learners. language proficiency.
• It involves saying something in one's own words and avoiding
plagiarism. Strategic Options:
• Test designers can elicit paraphrases of sentences or paragraphs. • Writers need to be aware of the task and genre of writing.
• Scoring focuses on conveying the same or similar message. • Attending to the task involves fulfilling the criterion of the task.
• Attending to the genre requires following the conventions of the
Guided Question and Answer: specific genre.
• A lower-order task that guides learners without dictating the form • Assessment criteria depend on the genre and may include format,
of the output. purpose, organization, language use, and effectiveness.
• The test administrator poses a series of questions as an outline for
the written text. Test of Written English (TWE):
• Questions stimulate the sequence of sentences in a narrative. • A standardized test of writing ability.
• Example questions include setting, characters, events, reasons, • Impromptu test with a 30-minute time limit.
opinions, climax, and moral. • Topics are prepared by experts and represent university-level
discourse.
Guided Writing Texts: • Test-takers are advised to follow specific steps for success.
• Two or three paragraphs with guided writing prompts. • Scoring is based on holistic evaluation criteria addressing topic,
• Can be scored on analytic or holistic scales. organization, development, language proficiency, and
• Used as prompts for initial drafts of writing. grammatical correctness.
• First drafts can undergo editing and revising stages.
SCORING METHODS FOR RESPONSIVE
AND EXTENSIVE WRITING
Holistic Scoring Primary Trait Scoring
• Single score representing overall assessment • Focuses on achieving the primary purpose or trait of an essay
• Score based on effectiveness in achieving that goal
• Reader's general impression • Evaluation based on specific function of the text
• Advantages: • Advantages:
• Fast evaluation • Allows focus on function
• Provides feedback potential
• High inter-rater reliability
• Easily interpreted by laypersons Analytic Scoring
• Emphasizes writer's strengths • Scores multiple subcategories of writing
• Enables learners to identify strengths and weaknesses
• Applicable to different disciplines • Provides more detailed feedback
• Five major categories with five different levels each
• Disadvantages:
• Masks differences across subskills Benefits of Analytic Scoring
• Offers more washback to the writer's learning process
• No diagnostic information available • Helps identify areas for improvement
• May not apply equally well to all genres • Provides more information about writing
• Requires extensive training for evaluators • Requires more time for teachers but gives students valuable
feedback
BEYOND SCORING: RESPONDING TO
EXTENSIVE WRITING
Assessing Initial Stages of the Process of Assessing Later Stages of the Process of
Composing: Composing:
1.Focus on meaning, main idea, and 1.Comment on clarity and strength of main and
organization. supporting ideas, argument, and logic.
2.Comment on the introductory paragraph. 2.Point out minor grammatical and mechanical
errors, directing the writer to self-correct.
3.Make general comments on clarity of main
idea and logic/organization. 3.Comment on word choices and expressions that
lack clarity or directness.
4.Ignore minor grammatical and lexical errors.
4.Identify problems with cohesive devices within
5.Indicate major errors and allow the writer to and across paragraphs.
make corrections.
5.Comment on documentation, citation of
6.Use questions to probe meaning instead of sources, evidence, and support if applicable.
rewriting unclear or awkward sentences.
6.Assess adequacy and strength of the conclusion.
7.Comment on features irrelevant to the topic.