Document Origin: [Link]

uk/research/romtopics/rom_teachingandlearning/afl/

Assessment for learning: putting it into practice

What happens when teachers try out ideas which research suggests could benefit their students? This month, we feature a study which reported the experiences of a group of teachers who started from research findings to develop formative assessment in their classrooms. The study showed the positive change in their practice. The study is: Assessment for Learning: putting it into practice Black, P., Harrison, C., Lee, C., Marshall, B., and Wiliam, D. (2003) Open University Press, Maidenhead This study built on earlier work by two of the authors:

* a review of published research on formative assessment (Black and Wiliam, 1998a) which provided evidence that formative assessment raises standards * the booklet 'Inside the Black Box' (Black and Wiliam, 1998b), a research review which explored how formative assessment contributes to effective teaching. 'Inside the Black Box' featured as an earlier Research of the Month summary. See Raising standards through classroom assessment. In 'Inside the Black Box', the authors pointed out the gap they noticed between the positive research findings on formative assessment and the low take up of the ideas by practicing teachers. They suggested that if research was to make an impact on practice, teachers needed structured support: 'Teachers will not take up attractive sounding ideas, albeit based on extensive research, if these are presented as general principles which leave entirely to them the task of translating them into everyday practice What they need is a variety of living examples of implementation, by teachers with whom they can identity and see concrete examples of what doing better means in practice'. This follow up study was designed to help bridge the gap between evidence and practice. The study vividly

Page: 1

demonstrates how teachers can transform research results into new and effective practice which has a positive impact on the way students learn together and on their attainment. The authors conclude with suggestions for bringing about whole school change, based on their experience gained during the project.

The study
What is assessment for learning? The term 'assessment' is commonly assumed to mean 'testing'. Summative tests, such as national curriculum tests or GCSE examinations, are used to classify students numerically and to compare them with each other. These formal tests tend to be infrequent, isolated from normal teaching and learning, and carried out on special occasions. The meaning of 'assessment for learning', the focus of this study, is quite different. Assessment for learning is usually informal, rooted in teaching and learning, and can occur many times in every lesson. 'Assessment', in the context of this study, refers to all the activities undertaken by teachers and their students which provide feedback that shapes and develops the teaching and learning activities engaged in by the class. This becomes 'formative assessment' when the evidence is used to adapt the teaching work to meet the students' learning needs. There is no one simple set of activities that constitute formative assessment it is not a quick fix process with a single formula. Rather it can involve multiple ways in which teachers can find out what their students know, can do and need to do and then use this in feedback to students and for future lesson planning. How did teachers aim to create greater feedback on teaching and learning in their classrooms? The teachers in this study planned individual strategies to generate feedback about learning, which they could then build on to modify their teaching and improve their students learning. The strategies were based on the areas suggested in the earlier work of the research team Inside the Black Box, which was summarised in our RoM on classroom assessment.

Although the strategies were not all explored to the same extent by all the teachers, collectively, the strategies the teachers implemented in their classroom involved:

* teachers use of questioning; * feedback through marking; * peer and self assessment by students; and * formative use of summative tests. In the following sections we discuss the impact of the changes on teaching and learning. Later in the RoM we describe why and how teachers brought about the changes to their practice.

Page: 2

What was the evidence of improved students achievement? This study grew out of an earlier comprehensive review of research of assessment that established important links between effective formative assessment and students learning. (See Black, P. and Wiliam, D. (1998b) in Further Reading . Also see our earlier RoM on classroom assessment ) In this study the researchers looked at the test results of twenty-five classes of students whose teachers participated in formative assessment activities. The tests included national tests at key stage 3, internal school examinations and GCSE results. The researchers compared the performance of each class with that of a similar class that had not undergone the intervention. They were then able to measure the effects of the intervention on students' academic achievement. The thirty-six project teachers were classified by the researchers into one of four groups, depending on the use they made of formative assessment strategies in their classrooms. The groups were:

* 'experts' formative assessment strategies embedded in and integrated with practice; * 'moving pioneers' teachers who were successful with one or two key strategies and were looking for other ways to augment their practice; * 'static pioneers' teachers who were successful with one or two key strategies who had restricted themselves to these; * 'triallers' teachers who had attempted strategies, but had not embedded any into their practice. The researchers found that:

* there was a positive and significant effect on attainment in all but four classes (the mean attainment of the experimental groups was about 15% better than that of the control groups); * there was no difference in the mean effect size for groups of different ages, but all the Year 11 classes (who took GCSE examinations) had positive effect sizes; and * the gains in the classes of the teachers who were more successful at using assessment strategies were more consistent than the gains in the other teachers' classes. The researchers suggested reasons why positive effects were not found in four of the twenty-five classes two of the teachers had not attended all the available in-service sessions and had made only limited use of formative assessment, while the other two teachers' classes had been compared with teachers who were much more experienced. The researchers were cautious about claiming positive effects of the interventions on attainment. They pointed out the difficulty they had had in ensuring fair comparisons between classes. The best form of comparison was between the attainments of the intervention class and that of a parallel class taught by the same teacher the previous year, but it was not always possible to achieve this. Some of the intervention classes were compared with classes taught by a different teacher, where the positive effect may have been due to the project teacher being a better teacher. Sometimes, colleagues of the project teachers copied some of the new practices, which may have reduced the size of the effect. Despite these reservations, the researchers believed the study provided evidence that improving formative assessment did produce tangible benefits in national tests. Based on this research, they suggest that improvements equivalent to between one-quarter and one-half of a GCSE grade per student per subject are achievable, depending on how successful the teacher is in using assessment for learning.

Page: 3

How did students become active rather than passive learners? A key finding, reported by the teachers, was that students grew from passive receivers of knowledge delivered by the teacher to active learners who were able to, and were expected to, take responsibility for their own learning. Evidence about the ways in which students changed their approach to learning came both from the thirty-six teachers and their classes who participated in this study and from the larger evidence-base in our earlier RoM on this subject. Evidence suggested, that across the classrooms, an increased number of students:

* took a greater part in their own learning than previously they became aware of their learning goals and how they could meet them; * participated in shared learning with all students involved in discussing solutions with their peers - as a common practice in their lessons; * became more confident and effective communicators; and * learnt to work in, and helped to create, an ethos in which expressed misunderstandings were received empathetically by the whole class . The researchers suggest that the ability to monitor one's own learning may be one of the most important benefits of assessment for learning that learning how to learn is more important for raising achievement in the long run than how to improve scores on particular tests. It was clear for most of the students that this was the case. One of the science teachers Roger recorded the effects assessment for learning had on his class of Year 9 students. The first three quotes are taken from his journal. Roger explained how the students helped to create a shared learning environment:

'We all had to be aware of everyone else, be prepared not just to listen, but to listen with an empathetic ear, we all had to understand and invest in a common goal, which was to move learning forward'. In this new learning environment:

'students felt safe to give wrong answers and to express, freely, their lack of understanding the classroom ceased to be a habitat where only the brightest survived and flourished, but one where, with careful grouping and good questioning, every student could feel themselves making progress through a lesson'. Roger noticed how the students began to take risks allowing themselves to be questioned in a more rigorous manner. In his diary, he recorded:

'they themselves seem to be becoming more confident learners. Attitudes changing faster in some than others, need to tackle "quiet-head-down" low achievers. It has become unacceptable in groups to be loud-low-achievers. Loudhigh achievers the norm'. Students growth in confidence was also observed. Another teacher commented on one student, for example:

Page: 4

The second [result] is with a girl with a statement for being unable to talk or communicate with an adult. Having got used to the extra thinking time, she now offers answers orally and will now tentatively explain her answers. (Gwen, Waterford School) How did teachers thinking move away from transmission teaching? The researchers felt implementing assessment for learning had a profound effect upon the culture of the project teachers' classrooms. The emphasis shifted from teaching about the subject to helping the students learn how to learn. Although the teachers changed their beliefs and practices radically, the change was gradual and slow many teachers had changed only small details in their practice after a year of the project. The changes made by the teachers included:

* moving away from being dominated by the curriculum; * changing their beliefs about students learning capacities from fixed to incremental; * encouraging more thinking among their students; and * letting go of some of the control they had previously exerted over teaching and learning. All the project teachers reported that they had changed the way that they thought of their teaching, seeing their role as helping students learning, rather than feeling that they had to 'get through' the curriculum. By identifying what they wanted the students to learn, the teachers could then see the 'gap' between the students' current learning and the learning goal and were able to monitor the 'gap' as it closed'.

'I enjoyed classroom teaching again I was not facing piles of marking. I was looking forward to being creative and to planning the next day I was focusing on the girls' understanding and not on their behaviour. I often found that once the understanding was there, the behaviour followed'. (Gwen, Waterford School) Teachers may find it helpful to look at a case study featuring a group of teachers who saw their role as helping students learning. They developed diagnostic probes to help them identify what their students knew already about particular science topics which also helped them to determine what they needed to learn. The teachers reported that the extra time they spent on ensuring students understood something was compensated by covering other areas more quickly, because the students were able to understand them more quickly as a result of the earlier work. Many of the teachers moved away from viewing their students as having a fixed level of ability to seeing them as being able to improve with appropriate help and support a shift from a fixed to an incremental view of ability. If the required learning did not seem to be happening, the teachers tried a different approach and gave the students more time. They did not think that the students were unable to understand because they lacked ability. They expected all students to be successful as result of the project. The teachers also expected their students to do more thinking in lessons in the belief that the more students think, the more they learn. The teachers aimed to strike a better balance between the thinking that the teacher did in a lesson and the thinking the students did. This teacher attempted to do this by involving more students in class discussions:

Page: 5

The class knows that they will all contribute at some point and most responses are requested by me rather than volunteered. (James, Two Bishops School) The teachers gave students more control in the lessons, over what they needed to learn in a topic, how long they would spend on the topic and what activities were to be engaged in to aid the learning. Teachers expected students to take more responsibility for their learning. So, for example, in classroom questioning, every student was expected to have an opinion and be prepared to express it. The teachers expected students to monitor and manage their own learning:

'I have thought carefully about pupils taking ownership of their own learning. I have now thought more about letting pupils know what the intention of the lesson is and what they need to do to achieve it. This way they have to think what they know and take more responsibility for their own learning'. (Angela, Cornbury Estate School) What was different about the ways in which the teachers used questioning? Drawing on earlier research (for example Rowe, 1974, see Further Reading (click to page 24)) the researchers helped teachers analyse their practice in terms of wait time (i.e. time teachers allowed students to answer) and the effectiveness of their style of questioning. The teachers became aware of shortcomings in their approaches, as this quote shows:

'I'd become dissatisfied with the closed Q & A [question and answer] style that my unthinking teaching had fallen into, and I would frequently be lazy in my acceptance of right answers they and I knew that if the Q & A wasn't going smoothly, I'd change the question, answer it myself, or only seek answers from the 'brighter students'. I had always been wary of wrong answers I want to encourage children to answer in class so I'd usually end up with a lame 'That's an interesting way of looking at it'. No-one learnt anything from this exchange except the willing student, who gradually learnt to be less willing'. (James, Two Bishops School) During the workshops, project teachers worked on developing approaches to questioning that fostered reflection and discussion in their classrooms, including:

* preparing quality questions designed to explore and challenge common misunderstandings, and to create some conflict requiring discussion (see example open questions below); * using incorrect answers from both class work and homework as discussion points to be taken up by the whole class; * giving students more time to think and sometimes time to discuss their thoughts with peers; * encouraging students to put mistakes right together, reaching answers collaboratively; * abandoning the usual practice of students putting their hands up to volunteer answers to questions by insisting on hands down to discourage competition; and * asking open questions so that all the students felt they had something to say. These are two examples of how teachers can change closed questions that emphasise recall of facts into 'big' open questions promoting reflection and discussion:

Page: 6

Closed, factual questions Open, discussion questions Some people describe friction as the opposite of slipperiness. Do you agree or disagree? Some people describe friction as the opposite of slipperiness. What do you think? What is the equation for photosynthesis? If plants need sunlight to make food, how come the biggest plants dont grow in deserts, where its sunny all the time? Open questions allowed all students to begin to formulate ideas and to be part of a discussion, not just the students who knew of, or understood, for example, key terms such as photosynthesis and friction. The use of open questions that challenged students existing ideas echoes the emphasis on cognitive challenge in thinking skills approaches. See our earlier RoMs: Cognitive Acceleration in Science Education (CASE) project (click to RoM) and Social interaction as a means of constructing learning: the impact of Lev Vygotskys ideas on teaching and learning(click to RoM) Many of the improved questioning styles are highlighted in case study 2 in which a teacher changes practice to using questioning as a formative tool. Project teachers commented on the many benefits they noticed with the new approach to questioning. For example:

'I now use the 'hands down' strategy and this has made a big difference to my classroom discussions. In particular, it has broadened the range of participation and removed (at its best) the curious mixture of envy and relief which characterised the mood of the group while the usual people answered questions'. (Paul, Cornfields School) 'because they have to explain their answers each time orally, this has carried through to their written work and now they set out their answers fully without being prompted'. (Gwen, Waterford School)

'My whole teaching style has become more interactive. Instead of showing how to find solutions, a question is asked and students are given time to explore answers together'. (Nancy, Riverside School) What did the teachers think about the way they marked their students' work? Marking is crucial to assessment for learning. When reflecting on their marking practice the teachers concluded that:

Page: 7

* students rarely read comments, preferring to compare marks with peers, as their first reaction on getting work back; * teachers rarely give students time in class to read comments that are written on work and probably few, if any, students return to consider these at home; * often the teachers' comments are brief and/or not specific, for example 'Details?'; * the same written comments frequently recur in a student's book, implying that students do not take note of or act on the comments. The teachers wanted to find the best way to communicate to the students about what they had achieved and what they needed to work on next. They also wanted to find ways of encouraging and supporting the students to act upon the feedback. At the start of the project, the comments given by teachers tended to be one or a combination of the following types:

* a general evaluation, such as good or well done; * geared to improving presentation, for example, title?, date?, space out the questions on the page, rule off each piece of work, or use a pencil and ruler for diagrams; or * instructions to complete the work, for example, please finish or answer all the questions. The teachers' comments tended not to indicate what had been achieved or what steps to take next. OFSTED (1998) (Further Reading) indicated that this kind of practice is common. The report stated: 'Marking is usually conscientious, but often fails to offer guidance on how work can be improved'. The students were also clear about the way they wanted their teachers to give feedback. Students wanted their teachers to:

* write legibly so that the comments could be read; * write statements that could be understood; and * not use red pen because students felt that it ruined their work. How did the way the teachers marked their students' work change? During workshops, the teachers worked on feedback strategies from a number of perspectives including:

* designing more challenging activities which created opportunities for teachers to give a much broader range of comments to students; * familiarising themselves with the sorts of comments that were effective because they explained to the students what they had done well and what they needed to improve upon, and gave guidance on how to make that improvement; * thinking up ways of making the increased feedback economical in time; and designing procedures for monitoring students responses to feedback. By the end of the project, the teachers had become more adept at writing comments, which prompted students to take action and reflect on their work. The authors gave the following examples:

* Susan, you have got the right idea here about trying to explain your rule. Think: does it apply to all triangles? * Go back to your notes from 29th September and look up where chlorophyll is and what it does. * Well explained so far but add reasons why the Haber process uses these conditions. Practitioners may like to read a case study showing how a child improved her story in response to written comments

Page: 8

provided by a teacher, but delivered by a computer, which we featured in an earlier summary about effective teachers of literacy. Teachers noticed that some tasks were useful in revealing students' understandings and misunderstandings, and that others focused mainly on conveying information. This led the teachers to design activities that created opportunities for giving a range of different comments. How did teachers find time for giving extra feedback as well as teaching? The teachers found time-saving ways of accommodating the increased amount of feedback they provided by, for example:

* entering marks into record books, but not writing them in the students books; giving marks only after a student had responded to the teachers comments; using lesson time to allow students to redraft their work; * not marking some pieces of work; and * involving students in checking their answers to straightforward tasks. The teachers also devised ways of monitoring the students' responses to their comments. For example:

'I was keen to try and have a more easy method of monitoring pupils response to my comments without having to trawl through their books each time to find out if theyd addressed my comments. I implemented a comment sheet at the back of my Year 8 class's books. It was A4 in size and the left-hand side is for my comments and the right-hand side is for the pupils to demonstrate by a reference to the page in their books where I can find the evidence to say whether they have done the work The comments have become more meaningful as the time has gone and the books still only take me one hour to mark'. (Sian, Cornbury Estate School) The teachers' initial fears that students and parents might react unfavourably to not receiving marks turned out to be ungrounded, as these teachers explained:

'At no time during the first 15 months of comment-only marking did any of the students ask me why they no longer received grades I found this amazing, particularly considering just how much emphasis students place on the grades and how little heed is taken of the comments generally'. (Derek, Century Island School)

'The whole faculty moved to a comment-only assessment policy in September I am not aware of a single parent expressing concern over a lack of grades/levels/marks'. (James, Two Bishops School) Evidence for the effectiveness of comment only marking is given in a case study in our earlier RoM on assessment for learning.

Page: 9

In what ways did the teachers use peer and self-assessment skills to improve learning? The researchers suggested that self- and peer-assessment could be a means of helping students understand what their learning goals were and the approach they needed to take to meet them. Specifically, they regarded peer assessment as a means of helping students to develop the detachment they need for self-assessment because:

* peer-assessment improves the students' motivation to work more carefully; * peers use the same language and can provide models of achievement; * students accept criticisms more readily from their peers than from their teachers; * peer-assessment helps improve communication between students and their teacher about their learning; * peer-assessment helped the students to identify learning goals and what had to be done to achieve them, skills they then transferred into self-assessment; * peer-assessment activities give teachers time to observe, reflect and frame helpful interventions; and * students learn by taking the roles of teachers and examiners of others. The project teachers devised a variety of strategies to develop their students' peer and self-assessment skills, including:

* asking students to mark their peers investigational work using the national curriculum level descriptions. From this peer assessment, the students produced their own targets which they then used to help them rewrite their investigational work; * asking students to 'traffic light' a piece of their own work, then to indicate by a show of hands whether they put green, amber or red (according to whether they think they have good, partial or little understanding). The teacher then paired up the students with amber and green lights to help each other improve their work, whilst the teacher worked with the remaining group of red students; * using criteria supplied by examination boards for grading their peers or their own work; and * using summative tests as a resource for formative assessment. Teachers may find it helpful to look at a case study, which reports how a teacher helped her students improve their French writing skills by developing their self-assessment skills. Teachers may like to read four vignettes showing how teachers developed peer- and self-assessment. These vignettes were featured in the main study. Teachers wanting to find out more about the positive effects of peer-assessment may like to look at a case study about students who worked with writing buddies to produce a short story.

Link to case studies

The researchers suggested that whilst summative tests (such as national curriculum tests and GCSE examinations) are usually seen as being incompatible with formative practices, they can be used as a resource to give students the opportunity to gain helpful feedback in a number of ways, including:

* using peer and self-assessment to make revision more effective;

Page: 10

* setting practice test questions; and * students marking test answers. The process of using summative tests formatively helped students to see tests in a different way:

'They feel that the pressure to succeed in tests is being replaced by the need to understand the work that has been covered and the test is just an assessment along the way of what needs more work and what seems to be fine'. (Belinda, Cornbury Estate School) What challenges did the teachers have to face? Students in some classes were resistant to the changes their teachers wished to implement. One teacher Barbara explained the problem she faced when she changed school:

'The school I now teach in is a grammar school and I find that students who have already achieved well in a 'traditional' classroom are not so keen to get involved with their learning on a grand scale! The first time I asked a Year 10 top set to work in groups to examine the errors they had made in a test and to help each other to understand fully what was being asked of them, it was unsuccessful. They were not used to the ethos of taking that level of responsibility. That lesson I heard several times, Why don't you just tell us? I needed them to see that their actual test scores were not the issue; what we did about improving their understanding was much more important'. Barbara persevered with the class. She found the idea of a 'traffic light' on each piece of work helped all her students take time to reflect on their understanding and decide for themselves whether they were red, amber or green. She recorded traffic lights for each student in her mark book (a system she found much more visual than a mark) and used the traffic lights to group students according to their own perceived understanding. The other element she found helpful was questioning. She explained:

'By accepting every oral answer with the same body language and then asking another student to comment on it, the programmed responses of students are challenged. (What is the answer? Is that right? Why is it right/wrong?) The ethos of the class has to be right non-judgmental and all striving for understanding and when it is, this opens up so much discussion and teases out misunderstandings'. Teachers also found it difficult to change their practice, as this teacher explained:

'Increasing waiting time after asking questions proved difficult to start with due to my habitual desire to 'add' something almost immediately after asking the original question. The pause after asking the question was sometimes 'painful'. It felt unnatural to have such a seemingly 'dead' period, but I persevered. Given more thinking time, students seemed to realise that a more thoughtful answer was required. Now, after many months of changing my style of questioning I have noticed that most students will give an answer (and an explanation where necessary) without additional prompting. (Derek, Century Island School) How might other schools plan for a whole school change of practice?

Page: 11

The authors suggested some starting points for implementing whole school change, based on their experiences gained during the project. We have divided this process into two main parts reflecting activities related to what happens or needs to happen in schools to bring about changes in teachers practice and how the changes can be monitored and measured. Briefly, the activities designed to bring about changes in teaching and learning involve:

* a clear vision of the aims of the strategy; * a review of existing practices of the school; * putting the plan into action; * providing support for those involved; (including through CPD) ongoing evaluation and dissemination (see next section); and * an overall timetable (see next section). Aims and vision The authors recommended that schools should centre their plans on a clear vision of the changes that will be brought about in the roles of the teachers and students. Review of existing practices The authors highlighted the value of a review of:

* the existing strengths of the school in promoting students' learning; and * the extent to which staff are close to, or a long way from, the new practices. Implementing the plan The authors suggested that schools should begin on a small scale with a few groups perhaps selecting two departments, or a group of teachers from different departments, with each teacher choosing to develop one of the activities. Gradually, after studying and evaluating the experience of the teachers involved in the project, other colleagues can become involved and the number of activities can be expanded. Support The authors suggested the programme for improvement will need the support of senior management including:

* initial support such as INSET in which training is provided by outside experts to the first group of teachers; * sharing experiences regularly with colleagues to reflect on immediate experiences in the light of the overall vision; * support from a member of the school management team, with knowledge about, and enthusiasm for, formative assessment; * release from school policies that might hinder the innovation; * consultation with parents so they understand the effects on their children; and * INSET for the whole school to make all staff aware of the innovation. The head of department will be a key figure in the change process providing support and time for discussion. Outside agencies can also support school developments. For example, the LEA or local universities can provide access to expertise and experience from other schools, from its own officers and from links with research. Publishing results through established networks within the authority can also make it possible for other schools to learn from the developing expertise of the teachers involved. Continuing Professional Development The change strategy contains clear implications for the key features of successful continuing professional development (CPD) and closely embodies the findings of a recent systematic review of CPD.

Page: 12

Among the factors contributing to successful CPD outcomes for teachers and students, the review identified specifically:

* the role of external experts in providing access to research findings * developing the CPD around issues of concern identified by teachers * building on what teachers know * creating conditions for teacher control of the CPD; and * using peer observation and feedback. The full report can be found at: [Link] Teachers may like to read a case study of how one school went about a whole school change of practice. How can you ensure that you know what effect the intervention is having? Evaluation The efforts of the first group of teachers participating in the project should be evaluated, to track changes in teaching and learning. Because many complex variables are involved it is easy to forget key aspects. Change takes time to embed and teachers can easily feel they have failed when, in fact, students are moving towards the teachers goals in important ways. On the other hand, they may feel they have finished long before all the key features of formative assessment are in place. Tracking helps to monitor progress teachers formatively assess their own learning. Evaluation could consist of pre and post collection of data to give evidence of changes and questionnaires to students about their perceptions of the classrooms in which changes are being implemented, perhaps comparing with questionnaires with students in other classes. Progress might be helped by setting and checking on targets. A target could be, for example, allowing ample 'wait time' for at least some questions asked in a class and the achievement of such targets could be checked by peer-assessment among teachers who observe one another's classrooms. Timetable The authors suggested that evidence of outcomes from the initial intervention is likely to occur after about a year of work and, on this basis, the whole school could adopt formative assessment during the second year. Evaluation plans should take into account evidence that might inform subsequent decisions about the development and diffusion of changes across the school. How was the research designed? Twelve science and twelve mathematics teachers, and, at a later stage, twelve teachers of English, from six comprehensive schools in two local authorities (Oxfordshire and Medway) worked with a research team from King's College London. The project, which became known as the 'King's Medway Oxford Formative Assessment Project' (KMOFAP) ran for two and a half years from January 1999 until the summer of 2001. The findings relating to changes in students and teachers learning in this study corroborate the evidence collected in the earlier Black and Wiliam study Inside the Black Box. (See Further Reading (click to page24)) After the researchers had introduced the teachers to findings highlighted by their earlier studies into formative assessment, the teachers planned individual intervention strategies which they developed with a Year 7, Year 8 or Year 10 class of their own choice. The research team worked with the teachers involved in the project in two main ways:

Page: 13

* teachers, researchers and LEA officers met during twelve INSET days to share ideas, formulate action plans and discuss progress; and * the research team visited schools to observe lessons, give feedback and discuss with the teachers the changes they were making to their practice in order to help them reflect. The researchers gathered evidence for their study from the following sources:

* student attainment data; * interviews (transcribed) with individual teachers at the beginning, middle and end of the project; * lesson observations; * documents and observations from the INSET meetings; * the teachers' action plans; * closing statements from each school about plans to extend and disseminate the work; * journals kept by many of the teachers; and * the closing statements by the teachers, summing up their reflections on the experience. Implications for practice Leaders wanting to help their colleagues bring about improved student learning through a change of assessment practice may want to consider the following questions:

* has formative assessment featured as an area for collaborative professional development in your department or school would it be helpful to create opportunities for teachers to work on developing new strategies? * could you do more to support colleagues trying out formative assessment strategies for the first time, by for example, organising workshops and inviting external help, such as practitioners from university education departments or other schools? Teachers wanting to create more opportunities for improving their students learning may wish to consider the following questions about their current practice. When conducting question and answer sessions, could you do more to:

* help more students contribute to discussions, by giving them extra time to think about their answers? * make a special point of seizing on students misconceptions as a positive contribution to the learning process in order to provide a building block for further discussion? * ask more open questions designed to help develop students understanding, as in the photosynthesis example given earlier? * avoid questions that seek descriptions and technical names to focus instead on causes and explanations and let names and specialist vocabulary emerge in the process? When marking your students work, could you do more to:

* encourage students to show their understanding of the key features of what they have learnt, such as making a short presentation to the rest of their group? * adopt a comments only method? Could you do more to help your students learn self-assessment skills, by for example:

Page: 14

* asking students to work with a partner on activities, and encourage students to comment on each other work? * getting students to level each others work according to national assessment criteria (perhaps simplified and set out on card)? and * trying the traffic lights strategy (click to page 10 above) during revision? Your Feedback Have you found this study to be useful? Have you used any aspect of this research in your own classroom teaching practice? We would like to hear your feedback on this study. Click on the link below to share your views with us. research@[Link]

Case studies

We have selected the following vignettes and case studies to illustrate what teachers have done to change their practice. Case studies 2 and 4 were part of the study itself. Case studies 1, 3, 5 and 6 were drawn from independent classroom-based research conducted by teachers. Case study 1: Using 'diagnostic probes' to identify students understanding in science We have chosen this case study because it is an example of how science teachers in a school identified what their students needed to learn and how they used the information to break the learning of science topics into a series of steps. This study describes how the teachers created a tool diagnostic probes' to assist them in their planning and teaching, by effectively and efficiently identifying students' levels of understanding within particular areas of science. The study involved 200 Year 7, 8 and 9 pupils from ten classes. The Year 7 and 8 pupils were taught in mixed ability, mixed gender groups, but were grouped by attainment in Year 9. The sample groups were taken from the whole ability range. The teachers focused on particular areas of science such as: 'inheritance', 'burning' and 'the Earth in space and gravity'. They reviewed previous research on children's understanding in science to identify possible categories of ideas which pupils usually have. 'Probes' were developed in each of the chosen areas. These were trialled with a small group of pupils and refined before being used with the main test groups of pupils. Developing 'the Earth in space and gravity' diagnostic probes The review of previous research on students' understanding of the earth in space and gravity for example, revealed:

* a clear pattern of development in students' ideas from a flat earth to a spherical model; * gravity pulls objects down this may be in conflict with the idea of pulling towards the centre of mass; * gravity does not exist in space; * the universe takes the form of a sphere made of all the other objects lying outside the solar system; and * great confusion exists regarding the sizes and distances of objects in the universe. The teachers created the following five probes for the earth in space and gravity topic to probe their own students'

Page: 15

understanding: Models the probe involved the use of a range of different sized balls to probe the problems children have with scale in the universe - in particular the solar system. Order this involved the use of a card sort in which students were asked to sequence bodies (sun, galaxy, solar system etc) in order of size. This was to identify the ideas pupils have about the orders of size of bodies and systems in the universe. Solar system this involved the use of a card activity in which students could choose bodies (star, planet, comet, galaxy, milky way etc) that they would find in our solar system. This was used to identify what students knew and believed about the objects present in the solar system. Gravity (ball) students were provided with a diagram and asked to explain what would happen when a ball was released in different places. This was intended to identify the ideas that students had about why objects fall. Gravity (objects) students were provided with a diagram on which different objects were shown around the earth and they were asked to explain what would happen when the objects were released. This was used to identify the ideas that students have about the effect of gravity on different objects. When the teachers used the earth in space and gravity probes with the students, they found that many of the students:

* encountered a major problem with scale when thinking about the universe and found three dimensional models even harder to use than two dimensional diagrams; * believed that the solar system is the largest system in the universe and believed that galaxies and other stars would be found in our solar system; did not associate 'falling' under gravity with mass, but were confident with the ideas that heavy objects will fall and light objects will not; * believed that gravity does not exist in space they will change their ideas of falling to meet the context in which it is considered a ball will fall on earth, but will not fall on the moon. Learning grids Having found out what the students did and did not know about a particular science topic, the teachers developed learning grids to identify possible learning routes through the topic recognising that the students have to go through some significant steps in learning. For example, they developed the following learning grid for the Earth in space and gravity topic:

The place of diagnostic probes in teaching The teachers recognised the diagnostic probes they developed could be used for many purposes, including:

* measuring students' initial understanding prior to teaching a topic; * a learning activity for challenging and stimulating thought during teaching; * evaluating the teaching of a topic at the end of the topic; * informing colleagues of the ideas that students may hold about a topic;

Page: 16

* assisting teachers to review and develop schemes of work; * assisting teachers in target setting for individuals and groups of pupils; * challenging teachers' own thinking and understanding; and * stimulating and motivating teachers through being more effective. Reference David Nixon, Hilary Kirk and Richard Needham, Brooksbank School, Elland, West Yorkshire, The use of 'diagnostic probes' to aid teaching and learning in science [Link]

Case study 2: Improving question and answer sessions We chose this case study because it shows the change in a science teacher's use of questioning which correspondingly changed the students' learning experience. This example featured in the main study. The authors of the main study commented on transcripts of the start of two Year 7 (11-12 year olds) science lessons where a question and answer session was followed by a practical activity. In the first lesson, the teacher asked individual students closed questions, requiring brief factual answers. In the second lesson, students were involved in a discussion the teacher encouraged the students to comment on each others' answers, giving them the opportunity to consolidate or modify their answer. This first transcript was from the start of a lesson about electricity, which took place near the beginning of the project. The students had been studying electric circuits for two weeks before this lesson and were familiar with the setting up of series and parallel circuits, but the teacher did not try to elicit their understanding in this extract. Instead, he engaged in conversation with a few children to see if they could guess the facts which he had in his head and which he wanted them to grasp before they began their experiment.

Teacher: Anyone know what we call these and where you might find one? Starts to walk round and show groups the ammeter. Two hands go up in the class. Teacher: Look carefully. Where have you seen something like this? You might have seen something like it before. What is it involved with? Its got a special name . . . Three more hands go up. The teacher selects one of these students. Teacher: Yes. . . Jay? Jay: In electricity, sir. Teacher: Thats right. You can use these in electric circuits. Anyone know what it is called? This word here helps. Can you read what it says? Carolyn? Carolyn: Amps. Teacher: And what is this instrument called that measures in amps ? Pause of 2 seconds. No hands go up. Teacher: No? No-one? Well, its an ammeter because it measures in Amps. The teacher played a fast pace question-and-answer game, in which some students scored points because they guessed what the teacher wanted them to say, while others were highlighted for not paying attention: Teacher: And where do we find these ammeters? Monica? Monica shrugs her shoulders. Six children have their hands raised.

Page: 17

Teacher: No idea. Tell her Rebecca. Rebecca: In electric circuits. Teacher: Good. I am starting to spot which of you are sleeping today. Are we with it now Monica? Monica nods. Teacher: Right. Now we are going to use these ammeters in our practical today and so gather round and I will show you how it works. Quietly please. Over the next few months, this teacher worked on:

* extending wait time; * involving a larger number of students in whole-class question-and-answer sessions: * dealing with incorrect answers rather than ignoring them. This second transcript taken from a lesson about photosynthesis seven months later shows a change in the same teachers use of questioning. The teacher began by showing the class two geranium plants one which was healthy and large, the other quite spindly. First he asked the students to discuss in pairs why the plants had grown differently; then he asked the pairs to share their ideas with the others in a whole-class discussion.

Teacher: Okay. Ideas? About half the class put up their hands. Teacher waits for 3 seconds. A few more hands go up. Teacher: Monica your group? Pair? Monica: That ones grown bigger because it was on the window. [Pointing] Teacher: On the window? Mmm. What do you think Jamie? Jamie: We thought that.. . Teacher: You thought. . .? Jamie: That the big un had eaten up more light. Teacher: I think I know what Monica and Jamie are getting at, but can anyone put the ideas together? Window Light Plants? Again about half the class put up their hands. The teacher chooses a child who has not put up his hand. Teacher: Richard. Richard: Err yes. We thought, me and Dean, that it had grown bigger because it was getting more food. Some students stretch their hand up higher. The teacher points to Susan and nods. Susan: No it grows where theres a lot of light and thats near the window. Teacher: Mmmm. Richard and Dean think the plants getting more food. Susan . . . and Stacey as well? Yes. Susan thinks its because this plant is getting more light. What do others think? Tariq. Tariq: Its the light cos its photosynthesis. Plants feed by photosynthesis. The teacher writes photosynthesis on the board. Teacher: Who else has heard this word before? The teacher points to the board. Almost all hands go up. Teacher: Okay. Well can anyone put plant, light, window and photosynthesis together and tell me why these two plants have grown differently? The teacher waits 12 seconds. Ten hands went up immediately he stopped speaking. Five more go up in the pause. Teacher: Okay. Carolyn? Carolyn: The plant . . . The big plant has been getting more light by the window and cos plants make their own food by photosynthesis, its . . . Jamie: Bigger.

Page: 18

This transcript showed a marked change in the way that the teacher approached questioning. For example the teacher: no longer used questioning to seek terms and descriptions or to support classroom management through revealing those students who failed to listen or refrained from taking part as he had in the lesson about electricity. Rather, he tried to explore students understanding;

* created opportunities for the students to exchange ideas, articulate their thoughts and to modify answers in a supportive environment; * extended 'wait time' which encouraged more students to participate and give longer answers containing indications of their conceptual understanding, rather than of their knowledge of names and terms; * changed the way the students participated in the classroom dialogue the aim was not for discrete right answers to be celebrated, but for a discussion of the ideas to be explored. Case study 3: Developing self-assessment strategies in writing in KS4 French We have chosen this case study because it is an example of how a teacher developed her Key Stage 4 students' self assessment skills which helped them to improve their writing in French. The study involved four Year 10 French groups two experimental groups (a top set and a mixed ability set) and two parallel control groups. Over the year, the students in the two experimental groups were introduced to a combination of learning and checking strategies, which they practised frequently and regularly during lessons. These included:

* translating a sentence written in French back into English, to see if the writing made sense, revising the sentence where appropriate; * brainstorming before starting to write, relying on what the students already knew (without access to a dictionary) and using this as a plan for writing; * checking verbs in sentences in a systematic way, using colour-coding; * monitoring use and effectiveness of the dictionary when composing; and * assessing their own coursework and setting own targets, based on feedback given to the whole class. The students were helped to develop these strategies through:

* using A4 whiteboards for writing in pairs, practising verbs, brainstorming vocabulary on a particular theme or a group of words such as conjunctions, memory games and writing drafts; * learning how to use verb tables and dictionaries; using word processing packages to redraft particularly lengthening sentences and making sentences more interesting; * whole class brainstorming activities in both English and French, whole class writing activities and whole class feedback; * planned lesson time for review and correction in response to feedback. At the beginning and the end of Year 10, all the students were asked to produce a piece of writing in French, using only a dictionary for support. Their pieces of writing were analysed and compared. All the students completed a questionnaire asking them to reflect on the progress they had made during the year and on their performance in the end of year writing examination. The students in all four groups made progress during the year, but those in the mixed ability experimental group

Page: 19

displayed the greatest improvement. Their average sentence length improved by 2.5 words per sentence (from 5.3 to 7.8) and the percentage of totally accurate verbs by 11% (from 55% to 66%). The top set experimental group showed greater confidence in and enjoyment of French and were producing original, sophisticated and creative pieces of writing by the end of the year. Some of the students were reluctant to adopt some of the strategies until after the summer examination, leading the teacher to conclude that it may be necessary to explain to students the purposes of the strategies and at what stage in their writing they would be helpful. Reference Belinda Bartley, Developing learning and checking strategies in writing in Key Stage 4 French. [Link]/professionaldevelopment/resourcesandresearch/bprs/search Case study 4: Examples of four lessons using peer and self-assessment skills We have chosen these vignettes because they illustrate the variety of ways peer assessment activities can be set up to help students learn self-assessment skills. The vignettes featured in the main study. The lessons are examples of formative practice because they created opportunities for students to reveal their own understanding of the criteria for success to their peers and then to improve it. The opportunities were the result of careful planning the tasks given to the groups had a sharp focus, with clear requirements of the students. Piers All the students in Pier's mixed-ability Year 9 class had just completed a mock Key Stage 3 test. Piers had read through the scripts and identified several questions on which all the students had done badly in the reading paper. He handed the papers back unmarked and asked them to first think about, then discuss, a mark scheme for awarding levels. They agreed on the following mark scheme:

* Level 3: Good punctuation and spelling and be able to answer the required question. * Level 4: All of the above, good use of PEE (Point, Evidence, Explanation), good length answer. * Level 5: All the above, good use of PEE, well explained answer. * Level 6: All the above, good use of language and time. * Level 7: All the above, couldn't be better. Piers then identified particular questions from the test and asked them to identify what they would need to change to go up one level. During the activity, the students made comments such as:

I think in our comments we could have explained more. We didn't use enough quotes; we could have had one per paragraph. The activity was followed by a detailed re-working of one of the questions which the students then re-levelled. Mary Mary's top-set Year 11 English class were preparing for timed examination essay answers. The class spent the first half of the lesson on a timed essay on 'poetry from other cultures'. Mary collected in the essays and then redistributed the papers randomly. She asked the students to:

Page: 20

* identify the important parts of the essay title, which she then set against the qualities that the examiners would be looking for (structure, language, interest, culture and use of references); * mark in the margin of their peer's work where they felt the person showed evidence of these elements and to note anything they felt the person had missed; and * give verbal feedback. The students completed the task in almost silence, except for the odd comment, such as 'I needed to do that' or 'I haven't done that' the peer assessment exercise had triggered reflection about the strengths and weaknesses of their own work. Derek Derek began his Year 11 lesson on plant nutrition by getting the students, in groups of four, to do poster-size concept maps for the topic. The groups were then split into pairs; one pair stayed with their poster, while the other group went round the different posters asking the pairs remaining with their poster why they had included some of the terms and why they had linked some terms together. The pairs in each foursome then swapped places and repeated the exercise. This was followed by a 'feedback' session chaired by Derek where students reported on aspects of the activity such as the terminology selected, pointing out links they felt were good or confusing. The students were then given the following question to discuss:

'If a villain got hold of a chemical that could destroy chlorophyll, what effect would this have on plants if released?' Each group of four were asked to record between three and five criteria they felt were needed for a good written answer to the question. These criteria were discussed by the whole class and a final list of criteria was drawn up:

* say what chlorophyll does; * explain photosynthesis as a process; * put in the equation for photosynthesis; * describe the effect on plants of no chlorophyll; and * add any secondary effects from photosynthesis stopping. The students then wrote their answers for homework. Derek marked the work by writing comments and not allocating grades. In another lesson, the students read the comments in pairs to check they understood the teacher's suggestions then redrafted and improved their answer. Philip Philip asked his Year 10 class to revise for homework, work done so far on human nutrition, circulation and excretion. In the next lesson, he split the class into groups of five. Each group was given a set of five topic cards absorption, blood, circulation, digestion and enzymes. Each student took one of the cards and spent five minutes preparing a short talk on that topic which they would then give to the other four members of the group. To help them prepare their talk, each card gave three or four suggestions for what to include. For example, the 'blood' card had the following suggestions:

* what are the main components of blood? * briefly what job does plasma do? * briefly what job do red cells do? * how do white blood cells defend the body against pathogens?

Page: 21

The rest of the group had to secretly 'traffic light' each presentation and give details to justify their decision, like this:

* Green: better than I could have done it/I learnt something from this. * Amber: about the same as I could have done it/no major omissions or mistakes. * Red: not as good as I could have done it/some serious omissions or mistakes. The students gave their talks in alphabetical order and feedback was delayed until they had all given their talks. The suggestions on the cards were used to support the discussion, and sometimes disagreement about the traffic lights given. Philip then facilitated a five minute whole-class discussion, in which some students were asked to explain why their talk had not been awarded 'green' and what they needed to do to improve. The students who accepted that they were 'red' or 'amber' for their talks wrote out an improved version, whilst the students who were 'green' chose another card and wrote a talk on that topic. The students who finished before the end of the lesson read the work of other students to check that they had included all the points. Case study 5: Peer and self-assessment in creative writing We have chosen this case study because it illustrates how asking students to work with a partner in specific, structured, planned ways can have a positive impact on students' understanding, performance, motivation and self esteem. Two Year 8 classes an above average and a below average literacy set took part in the study. The students worked on producing a short story. This was an extended project, which spanned several weeks. The teacher developed the creative writing project in the following sequence:

* the students chose "Writing Buddies" whom they would work with for the whole project; * they were helped to compose assessment criteria based on the genre work done previously; * the writing buddies were encouraged to work together on planning explaining their ideas orally, swapping outline plans etc before the writing process began; * during writing, partners swapped books at frequent intervals and were encouraged to question and constructively criticise the writing using the assessment criteria as a reference; * when the final draft was complete, each student produced a word-processed copy; * the students were asked to write an assessment of their own and their partner's finished story, referring directly to the original agreed assessment criteria; and * the students assigned a national curriculum level using a checklist to help them, having previously practiced using level descriptions to make assessments of model texts. Evaluating the impact of the project, the teacher found:

* assessments made before and after the project showed clear gains in achievement with an average gain of approximately two thirds of a national curriculum level and similar gains being made by the most and least able students; * the students' attitudes were very positive, with many pupils feeling that the project had increased their self-esteem as writers; * the process of ongoing discussion about stylistic issues during the writing process led to many opportunities to teach grammar and structure within a real context, according to individual need; * most pupils felt that they gained a much greater understanding of what constitutes 'quality' in writing and were able to apply it to their own work; * most pupils were very accurate in the levels they assigned to their work; and * the students' behaviour was exemplary, even where problems might have been expected.

Page: 22

Reference Rachel Swaffield, Self-assessment in creative writing [Link] Case study 6: Working towards a whole school change of practice We have chosen this case study because it is an example of how a school set about implementing a change in practice. The school wanted to improve communication with students about their learning. The change was implemented in three stages. First, teachers belonging to the schools' Effective Learning Group identified 11 new strategies by reflecting on ideas from research. Next, 37 teachers who taught two Year 9 tutor groups volunteered to trial the strategies. Their questionnaire and interview responses helped the school identify successful actions. Finally, it was anticipated that all the staff at the start of the following academic year would implement a modified set of strategies. The strategies The 11 strategies and their outcomes were:

* using checking questions to make sure that students were saying what they meant to say many of the teachers felt using checking questions worked well by helping discussion and deepening thinking * using 'why' questions to check what was going on in the student's mind a small proportion of teachers reported having difficulty using the 'why' questions in a non-threatening way * setting up opportunities for peer-peer reflection this was not well understood by all the teachers, but was reported to work well by one teacher * using questionnaires in which students rated their level of understanding in writing this was not well received by teachers as they found it hard to use * when speaking to students, emphasising the validity of the communication by using eye contact, smiles and affirmative nods most teachers recognised the value of this strategy * when speaking to students on a one-to-one basis, being on the same physical level as they are the teachers recognised this as a valuable strategy though found it difficult to implement it in small rooms * giving negative messages using non-verbal signals only and giving positive signals verbally or non-verbally this was found to be a successful strategy by some teachers * reducing the isolation of students from the teacher by using a 'U' shape for seating the teachers found this impracticable in crowded classrooms * using checklists to enable the same points to be made to a number of students to help ensure that positive comments are not forgotten these were found to be practicable when developed collaboratively, and when checklists were used, they provided insights for teachers and seemed to lead some students to improve their presentation * encouraging students to write comments and questions in their books some teachers did not like encouraging students to write comments and questions in their books because they felt it defaced the student's work * adopting mechanisms for the teacher to encourage students to see them without thinking they had done something wrong this was unsuccessful with some of the boys.

In general the evaluation suggested:

* the teachers wanted practical suggestions to improve practice, but they needed to be involved in the change process to avoid them rejecting the suggestions

Page: 23

* the teachers who were successful at implementing difficult strategies could be a useful learning resource helping others move forward and establishing a learning ethos * more account should have been taken of the change required of students in accommodating the change in teachers' practice. The modified strategies Based on the experiences of the group of teachers who trialled the strategies, the following six modified strategies were identified:

* check students say what they mean to say by asking checking questions such as "So what I think you are saying is is that right?", "Can you say why you think [what they have just said] is true what makes you know that?" * set up opportunities for peer-peer reflection through a scoring system for students to express their understanding giving themselves a score out of five by holding up their hands with a number of fingers showing * when speaking to students on a one-to-one basis, give full attention to the student through eye contact, smiling and affirmative nods, and try to be on the same level as the students (standing or sitting) * use non-verbal communication for negative communication and use verbal or non-verbal communication for positive * use checklists for marking to make it more likely that positive comments will not be forgotten and design spaces for students to write comments or questions on the checklists and in their books * use mechanisms such as asking students to collect smiley faces or "smart" stamps from the teacher to encourage students to use them without thinking they have done something wrong. The school felt that the modified strategies would improve communication between teachers and students and intended to have all teachers in the school implement them at the start of the following academic year. Reference The Effective Learning Group, John Mason School, Abingdon, Oxon, Communication between students and their teachers about learning [Link]/php/[Link]?sectionid=82&articleid=689

Your Feedback Have you found this study to be useful? Have you used any aspect of this research in your own classroom teaching practice? We would like to hear your feedback on this study. Click on the link below to share your views with us. research@[Link]

Further reading

1. What else might I enjoy reading?

Assessment in Education, 5(1), pp.7-71 Black, P. and Wiliam, D.,(1998)

School Science Review, June, 82(301) 55 61

Page: 24

Black, P. and Harrison, C.,(2001)

Inside the Black Box: raising standards through classroom assessment Black, P. and Wiliam, D.,(1998) London: School of Education, King's College

School Science Review, September, 83(302) 4349 Black, P. and Harrison, C.,(2001)

Working Inside the Black Box: Assessment for learning in the classroom Black, P., Harrison, C., Lee, C., Marshall, B., and Wiliam, D. ,(2002) London: Department of Education and Professional Studies, King's College 2. The project teachers looked at several studies on the importance of oral feedback. The study that created the most discussion was:

Enhancing and undermining intrinsic motivation Butler, R.,(1988) British Journal of Educational Psychology, 58, 1-14 Butler found that: students learned more from teachers comments on their work than from marks or a combination of comments and marks feedback given as rewards or grades encouraged students to compare themselves with others and to focus on their image and status rather than to think about the work itself and how they can improve it. The authors reported that the project teachers' initial reactions to these findings ranged from shock and disbelief to considering why and how comment-only marking might raise achievement. 3. The project teachers looked at this study about increasing wait-time during question and answer sessions:

Wait time and rewards as instructional variables, their influence on language, logic and fate control. Rowe, M.B., ,(1974) Journal of Research in Science Teaching, 11 81-94 Rowe found that teachers, on average, waited less than a second between asking a question and, if they received no answer from the class, intervened again with another question or a prompt. The research found that increasing the 'wait time' had the following effects on students: they gave longer answers their responses were more confident their failure to respond decreased they challenged and/or improved the answers of other students they offered more alternative explanations.

Page: 25

4. The teachers looked at research which showed that self-assessment is important to learning - that students need to understand their learning goal and what they need to do to reach it. For example:

Formative assessment and the design of instructional systems Sadler, R.,(1989) Instructional Science, 18, 119-44 5. Practitioners wanting to find out details of Ofsteds findings regarding the way teachers usually mark their students work, may like to read:

Secondary education a review of secondary schools in England 1993-1997 OFSTED,(1998) London: HMSO 6. Where else might I find information online? King's College website Further information about publications and other resources for formative assessment by the research team can be obtained on the King's College website in the pages of the King's Formative Assessment Group. Some of the publications can be downloaded from this site. There are also references to other useful websites. The address is: [Link] Raising standards through classroom assessment 'Inside the Black Box' featured as an earlier Research of the Month topic: [Link] Toolkit 98 Groups of teachers wanting to think through the issues before developing formative assessment in their school may find it helpful to look at this 'toolkit' of readings, overheads, exercises and handouts: [Link] Assessment under pressure Practitioners may also find it helpful to read some case studies of how university lecturers have used formative assessment with their students. The case studies are available from: [Link] DFES (1) Teachers' and students' roles in formative assessment A digest of some of the earlier work by the authors is available on the DfES Standards web site: [Link] DFES (2) Developing formative assessment in the classroom The DfES standards web site features another digest of research into formative assessment: [Link]

Page: 26

Appraisal
Black, P., Harrison, C., Lee, C., Marshall, B., and Wiliam, D. (2003) Open University press, Maidenhead Robustness This study built on earlier work by two of the authors (Black and Wiliam) now widely known as 'Inside the Black Box' which explored how formative assessment contributes to effective teaching see our earlier RoM Raising standards through classroom assessment. This follow up study was designed to help bridge the gap between evidence and practice. It reported the experiences of 12 science and 12 mathematics teachers, and, at a later stage, 12 teachers of English, from six comprehensive schools in two local authorities (Oxfordshire and Medway) who worked with a research team from King's College London. The project, which became known as the 'King's Medway Oxford Formative Assessment Project' (KMOFAP) ran for two and a half years - from January 1999 until the summer of 2001. After the researchers had introduced the teachers to findings highlighted by their earlier studies into formative assessment, the teachers planned individual intervention strategies for development with a Year 7, Year 8 or Year 10 class of their own choice. The findings relating to improvements in students and teachers learning in this study corroborate the evidence collected in the earlier Black and Wiliam study. The researchers gathered evidence for their study from a number of sources, including:

* student attainment data * interviews (transcribed) with individual teachers at the beginning, middle and end of the project * lesson observations * journals kept by many of the teachers. Relevance Our impression, supported by, for example, feedback from the RoM website, is that the topic continues to grow in importance and popularity. Assessment for learning is now embedded in the national key stage 3 strategy. Whilst this study involved teachers of year 7, year 8 and year 10 classes and their students, the approaches described and explored in the report are relevant to all age groups from primary to post-16. The authors also offer suggestions for bringing about whole school change, based on their experience gained during the project, which lead to several implications for teacher professional development, that teachers and leaders will find useful. Applicability The report is practically focused and gives lots of illustrations about work by the teachers in classrooms with which teachers at all levels of experience will identify. Teachers wishing to shift the emphasis from teaching about their subject to helping the students learn how to learn will find these examples helpful for their own practice. The reported gains in students attainment will help to boost teachers confidence in both the theory underpinning assessment for learning and its practice. The report contains detailed illustrations about questioning and feedback. This includes allowing sufficient time for students to answer questions, asking open questions and comment only marking. The report also looks at peer and self- assessment, including the use of traffic light strategies to highlight levels of understanding of content material.

Page: 27

Writing The report is readable and there is little by way of statistical analysis. Findings are described in ways teachers can readily access. The authors have aimed to make their report of the study user-friendly for example, they provide an explicit guide to the structure of the book, and also present explanations of terms such as assessment, formative, etc. The processes involved in the study and the outcomes of the study are clearly distinguished.

Page: 28