CENTRE FOR DISTANCE EDUCATION

TEACHING AND RESEARCH AWARDS ROUND 6

PROJECT REPORT

IMPLEMENTING IPSATIVE
ASSESSMENT
By
Dr Gwyneth Hughes
Kaori Okumoto
Elizabeth Wood

Faculty of Policy and Society

Institute of Education
20 Bedford Way
London
WC1H 0AL
Tel: 0207 911 5308
Email: [email protected]

October 2011
Implementing Ipsative Assessment

Implementing Ipsative assessment

A report on a Centre for Distance Education (CDE) funded study by Gwyneth Hughes, Kaori
Okumoto and Elizabeth Wood

Summary

Distance learners rely on effective assessment and feedback for motivation and for learning.
There is evidence from a previous study that an ipsative approach to assessment (assessment
based on a comparison with a learner’s   previous performance) might both motivate distance
learners by developing self-awareness of progress and help them to take a longer-term
approach to academic study. In this project an ipsative feedback scheme was developed with
distance learning tutors and implemented using an ipsative feedback form for students to
complete. These forms and subsequent tutor comments were evaluated and analysed for
changes in feedback practice. Students and tutors were also interviewed about the value of the
scheme. Findings indicated that ipsative feedback on progress has the potential to motivate
learners and to encourage them to act on developmental feedback, but further intervention is
needed to manage student expectations and to ensure that this intervention is sustainable and
scalable.

1. Introduction

Distance learners are dependent on good feedback from tutors, yet assessment processes and
feedback are often deemed unsatisfactory by students. Hughes (2011) has proposed that
ipsative assessment (assessment based on a comparison with a previous performance rather
than external criteria) may offer a solution. A previous CDE funded study explored the feedback
that distance learners received (Hughes, et al. 2010) and found that there was very little
evidence of ipsative feedback although learners would welcome more information about their
progress. Tutors agreed that an ipsative approach which informs learners of how well they have
implemented the recommendations from previous feedback would be helpful too, particularly for
those students who are struggling with writing. Such feedback might also be especially
motivating and supportive for learners who do not receive encouragement through high grades,
but who nevertheless are progressing (Hughes, 2011). An ipsative approach requires careful
tracking  of  a  learner’s  progress  over  several  modules  and  could alert tutors to any students who
were not acting on the feedback they have been given.

However, the preliminary study indicated that distance learning tutors do not have enough
information about learner progression to provide learners with cumulative and ipsative feedback.
An intervention was therefore planned to take the exploratory work further. This intervention
included technological innovation with the current VLE to make all electronic feedback available
to all tutors and the design of a new feedback submission form to explicitly address learner
progress.

This report begins with a review of literature on assessment for distance learners and details of
the rationale for the intervention. The report next describes how the intervention was evaluated
using interviews and a tool to analyse written feedback. The main finding is that use of the form
prompted some students to reflect on previous feedback and encouraged ipsative feedback
from tutors that was motivational. However, there were further issues to be addressed and

1
Implementing Ipsative Assessment

recommendations arising from this intervention are presented along with suggestions for further
work.

2. Literature Review: ipsative assessment and distance learning

Assessment has been receiving increasing attention in Higher Education with recognition that
assessment drives learning as well as measuring outcomes. While much literature has
addressed issues of reliability and validity of summative assessment practice (Bloxham et
al.,Gibbs,2006), formative assessment has also received attention. Formative feedback aims to
reduce the discrepancy between performance and desired goals by informing learners where
they are now and where they need to go next (Hattie & Timperley, 2007; Sadler, 2005) and is
especially important for the retention and motivation of distance learners (Simpson,
forthcoming). However, there is plenty of evidence that formative feedback does not always
perform this function effectively in higher education. Although students are usually interested in
marks or grades, they might not be willing to engage with, or act upon, developmental feedback
(feed forward). This may be because they have been de-motivated by feedback or because
feed forward is not useable or it is unclear (Walker, 2009).

The  impact  of  assessment  on  a  learner’s  sense  of  self-worth has a significant and often under-
recognised influence on motivation. Comments and grades give individuals powerful messages
about themselves as learners (Ivanic. et al.,2000; Stobart, 2008), and for many learners
assessment produces negative emotions (Falchikov & Boud, 2007). Even when there are
attempts   to   ‘sweeten’   negative   or   challenging   feedback   and   results   with   positive   comments,  
there are indications from other studies that many learners do not act on critical feedback
(Crisp, 2007), either because they have focused only on the praise at a personal level, or
because they have been overwhelmed by excessive critique (Lizzio & Wilson, 2008).

Handley, et al. (2008) reported that students were more likely to act upon feedback if they could
see the immediate utility of it, for example, in helping with the next piece of work. But such task
specific feed forward is only useful if the task is to be repeated in the short term and comments
addressing   task   level   criteria   are   not   easy   to   apply   to   a   new   piece   of   work.   Walker’s   (2009)  
study from the UK Open University further supports a claim that developmental comments are
more useful if they do not solely perform a short-term corrective function. Feedback comments
on generic skills, such as thinking or writing skills, are most useable: these can be used for
other assignments as well as improving on draft work and general statements can always be
illustrated with points from the current assignment content. In the previous study, Hughes et al.
(2010) also suggested that generic feedback illustrated by task specific examples is very helpful
for distance learners.

To address problems with feedback clarity, Nicol & Macfarlane-Dick (2006) argue that feedback
should be directed towards enabling learners to become self-regulating. Dialogue is the basis of
a social constructivist rather than a transmissive view of learning and they suggest that
discussion between teacher and learners or amongst peers about the criteria and standards can
be very helpful and this is one of their key principles. However, individual face-to-face dialogue
is difficult to implement in pressurised mass higher education and is not readily available for
distance learners except by telephone or occasional meetings if tutors are pro-active (Simpson,
2008). Facilitating dialogue in mainly written communication is an alternative, but not easy to
achieve. However, there is a different approach to assessment which might provide a solution:
ipsative assessment and feedback.

2
Implementing Ipsative Assessment

Ipsative feedback to motivate and to encourage learner autonomy

Ipsative feedback has been proposed as a new approach that might address some of the
current problems with formative assessment (Hughes, 2011). There are several possible
benefits. Firstly, ipsative feedback closes the feedback loop in that learners can be informed on
how far they have responded to previous feedback and progressed see fig 1. Such a scheme
can initiate dialogue between learner and tutor without necessitating a meeting.

Fig. 1 Ipsative cumulative scheme for 3 sequential assessments

Feed forward 1 Feed forward 2

Assessment 1 Assessment 2 Assessment 3

etc.

Feedback on feed forward 1 Feedback on feed forward 2

by comparing assessments by comparing assessments
1 and 2 2 and 3

Feed forward 2 can build on feed forward 1 if remedial action has not been taken.

Secondly, ipsative feedback is longitudinal and gives learners and their assessors a way of
monitoring progress over time involving several assessments. This is particularly helpful in a
modular system where learners can select modules which are not strongly interconnected.
Finally, not all learners will be high achievers and ipsative feedback could be motivational for
learners who do not receive immediate rewards for effort through high grades.

Assessment for distance learners and the role of technology

There are inevitably differences between assessment practice for on-site students and distance
learners and provision of feedback is of particular significance. Differences between on-campus
and distance assessment regimes are found in the administration of assessment, for example,
submitting coursework remotely rather than in person or taking examinations in a distant
location (Morgan & O’Reilly, 1999). Technological innovation has much to offer distance
learners. Online courses which engage with a range of technologies may be more innovative in
summative assessment methods, for example using e-portfolios or blogs, but this will probably
vary with discipline. While written feedback is almost always provided at a distance, often
electronically, verbal feedback arising from face to face tutorials and from teachers and peers in
class may not have a direct distance learning or e-learning equivalent.

Ipsative assessment could help provide the missing dialogue for distance learners, but one of
the main arguments against using ipsative feedback is that it requires much more organisation
than conventional feedback because the assessor must have access to previous feedback and
assignments to make comparisons. However, there are ways of making this easier using
technology. For example, Nicol & Milligan (2006) suggest that if assignments, drafts and
feedback are submitted electronically, then they could be stored in one easy to access place
such as a virtual learning environment. Feedback comments could also be selected from
electronic menus.

3
Implementing Ipsative Assessment

Previous work on ipsative assessment for distance learners

A preliminary scoping study on ipsative feedback and distance learners (Hughes el al. 2010)
aimed firstly, to establish the extent to which ipsative assessment is used on a distance learning
Masters programme; and secondly, to explore both learner and tutor views on the benefits of
using ipsative assessment. This study concluded that ipsative assessment was little used on
this distance learning programme and tutors were not proactive in informing learners of
progress. However, while learners were self-assessing on progress, they indicated that they
would find ipsative feedback helpful and motivational. Tutors tended to provide generic
feedback that was very criteria focused suggesting that obtaining a good grade is viewed as the
primary purpose of assessment, but learners found much of this feedback vague, and specific
examples of what to improve would be more useful for their learning. Hughes (2011) proposed
that ipsative grading might shift the focus away from outcomes onto progress, but in the study
ipsative grades were perceived to be problematic by tutors and many students. The study also
recommended that there might need to be a built in mechanism for including ipsative feedback if
the criteria referenced grading process remains unchanged.

Because ipsative assessment is not explicitly practised in higher education, there is little
evidence for its potential benefits. Therefore this intervention project was established to
introduce ipsative feedback into a distance learning programme and to evaluate the new
approach. The intervention aimed to:

1. Discuss ipsative assessment and its potential benefits with the tutors and then agree with
them a process for providing the distance learners with cumulative ipsative feedback.

2. Evaluate the impact of the ipsative feedback intervention process on motivation and learning
from the perspective of both learners and tutors.

3. Consider the scalability, transferability and sustainability of the process.

Ipsative grades were not introduced as this would require changes to the institutions
regulations.

3. Methodology and background to the project

This study builds on an investigation of a distance learning Masters programme: the MA in
Applied Educational Leadership and Management (AELM), delivered wholly online to a fixed
cohort at a research intensive university. This scoping study had focused on tutor feedback for
the two core module assignments and included questionnaires to students, an analytical tool
applied to feedback and an invitation to tutors to respond to the findings (Hughes et al., 2010).

The second implementation phase of the study took place during the 2011 spring and summer
terms,   i.e.   halfway   through   the   programme’s   cycle   and   focused on implementing an ipsative
feedback process for the two core (year 1) and two optional (year 2) module assignments. A
meeting of the research team, which included the programme leader, discussed the systems
that might need to be in place to support the change in feedback. It was also agreed that it was
important to involve the tutor team, at an early stage, and so the next programme academic
group meeting included an item on ipsative feedback, where the purpose of ipsative feedback
was discussed and the findings of the initial research were presented. As a result of the
meetings and discussions, the programme leader instigated the development of a new
Assignment Submission Form (Appendix 1) which would enable students to inform tutors of how
they had acted on previous feedback and   a   new   data   base   of   current   students’   assignment  

4
Implementing Ipsative Assessment

mark sheets, located on the Virtual Learning Environment (VLE) accessible to tutors. Both of
these interventions could provide tutors with a means of comparing current and previous
assignments. Although   developing   the   data   base   required   extra   work   from   the   programme’s  
administrator, it was set up to be self-supporting with tutors uploading mark sheets.

Data collection and analysis

The sample

The total sample of 28 mostly international students comprised 17 first-year students and 11
second-year students and 17 females and 11 males. The email interview questions were sent
out to these 28 students and 13 replied. Out of 13, seven were first-year students and six were
in the second year.

The feedback forms and written summative feedback of 27 students were analysed (28 were
selected but one deferred the assessment), including feedback items from six tutors. A list of
students and the degree to which they participated is given in appendix 3.

Three sets of data were collected: the use of the Assignment Submission Form and the written
feedback received by students, email interviews with student volunteers and e-mail interviews
with the tutors of these programmes. Draft questionnaires and interview schedules were shared
amongst the research team before final versions were agreed.

Analysis of completed forms and written feedback

Draft and final module assignments were   submitted   to   an   ‘Assignment   Drop-box’   in   the   VLE.    
Tutors retrieved students’   scripts,   prepared their feedback and then upload it into the same
space so that students could receive it. Assignments were double marked, but feedback was
provided by the first marker who sometimes added additional comments from the second
marker on a single form. An independent researcher used an analytical tool based on the Hattie
& Timperley (2007) model for effective feedback, distinguishing between feedback on current
performance and feed-forward for the next assignment. Criterion-­‐based and ipsative feedback
were identified and task specific and more generic comments were also distinguished (see
appendix 2). The students taking only the option modules (most of second year students) were
unable to retrieve the Submission Form from the VLE and therefore were not able to complete
it. However, their feedback was analysed and some volunteered for interviews.

Student Interviews

Interviews were conducted by email by an independent researcher. Some students interviewed

had not been able to complete the form but these results were informative and therefore were
included in the study.

Samples of ipsative and feed forward comments were sent to students to reflect upon as part of
the email questions. The email questions (Appendix 4) were divided into three sections: 1) new
assignment submission form; 2) styles of feedback, usefulness and motivation; 3) recognition of
progress and ipsative feedback.

5
Implementing Ipsative Assessment

3. Tutor Interviews

Five tutors were asked to respond to interview questions which were sent by the programme
leader via e-mail correspondence after all 2010-11 assignments had been marked. The
programme leader was judged to be more suitable than an independent research in this case
because she had initiated the discussion and use of the form with the team. The programme
leader, who tutored one of the optional modules and marked assignments across all optional
modules, was also not included in the sample because she was part of the research team. Two
tutored the core module, one tutored both core and optional modules, and two tutored optional
modules. The two tutors, who tutored the optional modules only, were not able to find the time
to respond to the questions so three of the tutors were interviewed (2 males and 1 female).

The  questions  were  developed  out  of  the  students’  responses  to  the  specific  feedback  they  had  
received.    Four  questions  explored  tutors’  approaches  to  the  new Submission Form, their views
on   the   use   of   the   VLE   to   further   students’   understanding   of   the   assessment   process,     how  
ipsative feedback could be managed, and how the programme team might support tutors in
developing an ipsative feedback approach. Tutors were also given the opportunity to make
additional comments. The e-mail sent to tutors also included specific comments about use of
peer feedback taken from the student interviews, for example, about the difficulty some students
have interacting with others and their uncertainty about how to give peer feedback (See
Appendix 5).

Data Analysis:

The three datasets were systematically organised for interpretation through thematising and
summarising. The following themes emerging from the feedback analysis:

 use of the form;

 student response to previous feedback;
 tutor response to actions listed on the form;
 ipsative feedback without use of the form.

These themes were then cross-­‐referenced with themes emerging from the student interviews:

 the new Submission Form and usefulness;

 motivation for distance learners;
 short-term grade improvement;
 continuity between tutors.

Themes emerging from the analysis of tutor interviews were also cross-referenced:

 change in practice for a new feedback approach;

 supporting an ipsative feedback process;
 supporting student understanding of assessment and managing expectations.

6
Implementing Ipsative Assessment

4. Results of analysis of the feedback form and written feedback

Numbers using Assignment Submission Form

11 students completed the ipsative feedback sections  on  the  form  entitled:  ‘Feedback given on
previous assignments’ and   ‘Formative   feedback   on   this   assignment’.   These   students also
provided an account of how they had addressed the previous feedback in the section entitled:
‘Response to formative feedback’.  A further 5 students requested feedback on a specific issue,
but did not complete the new sections of the form or in one case used an old form without the
ipsative feedback sections. 11 of the 27 students submitting work did not complete the form.
Non-completion was not obviously linked to achievement and from student accounts was mostly
because there were technical problems downloading the form from the VLE for the option
modules.

The ‘Request   for   specific   feedback’   section had been introduced at the beginning of the year
and so was more familiar to the students than the other ipsative sections. As some students
filled in this section but not the new ipsative sections, it is reasonable to assume that more
students might fill in the ipsative sections as they become more accustomed to the new form.

Student response to previous feedback

Those who filled in the sections about previous feedback tended to list points from both previous
summative feedback and formative feedback for the current assignment. For example from
student 1:
-to be more argumentative

-to draw more on relevant literature to inform and support the analysis, so it is less descriptive

-to ensure the assignment stays within the guidance provided on word length

In the ‘Response  to  formative  feedback’ section some students gave a very general response to
claim that they had addressed all the points raised. Student 11 claimed to have addressed all
the points from previous feedback and wrote:
I have tried to act on all the feedback received.

This could indicate a superficial reflection on the part of the students and there is scope here for
tutors to points out any over-inflated or inaccurate claims. In the example of student 1 above,
the student claimed to have addressed all the points:
I feel I have attempted to address the points made, especially about the Literature Review and
the Methodology to the best of my ability and understanding

But, the issue of word-length had not been addressed as this student incurred a penalty for
exceeding the word-length. There was no ipsative feedback to highlight the inadequate
response to feedback.

Others students gave details on how they had acted on the feedback. For example, student 22:
-In the academic review section I have added an additional section which focuses solely on
external factors....

-In the methodology section I have added more detail and have given further thought to sampling
and validity of data

7
Implementing Ipsative Assessment

Student 27 explained:
The ethical issues raised in the methodology section were resolved by changing the participation
in the research from compulsory to fully voluntary

These students have taken note of the feedback and made confident changes to their writing
and/or content but Student 13 was less certain and is clearly opening up dialogue:
I  tried  to  show  more  connection  of  the  literature  to  my  research,  but  I’m  not  sure if  it’s  this  the  right  
way.

Tutor responses to student actions and provision of ipsative feedback

Three tutors (A, B and C) taught the core modules which used the new form. Their responses to
the students’ claims to have acted on feedback were supportive and helpful:
- It is clear that you took careful note of the advice you received on the draft and the previous
assignment. You have worked hard to improve the narrative flow and have taken steps to develop
your critical engagement with the sources. (Tutor A).

This was definitely an improved version, compared to the draft you previously submitted for
formative assessment. You have managed to shift the focus from a description of an intervention
on enhancing school/family links (which was not the focus of this study) to a small scale study
addressing one particular research question. (Tutor C)

In another case this tutor provided a positive ipsative comment but also added that there was
more work to do:
Although this assignment was an improvement compared to the previous version you submitted
for formative feedback and you have moved from discussing the establishment of a research
centre  to  proposing  a  small  scale  research  exploring   staff  and  students’  views,  there  are  issues  
that could be further addressed to improve   your   work  further  …   establishing a strong narrative
thread that will allow you to capitalise on your command of the field of study and second,
ensuring that your referencing is appropriate (Tutor C)

The tutors had already agreed to provide ipsative feedback and the results suggests that the
form was instrumental in prompting them by enabling students to open up dialogue with their
tutors about acting on feedback and progress.

But, the results were not consistent: in 3 cases the tutor (tutor B) did not respond to the student
claims about acting on feedback. More surprisingly in other cases, where students did not
complete or were unable to access the form, there was still evidence of ipsative feedback.

Ipsative feedback without use of the form

The limitation of such a form is that tutors may be dependent on it to review previous feedback
and thus rely on student interpretation of that feedback. A positive ipsative comment particularly
requires comparison with previous work to see that an improvement has been made and this
requires a new process for the assessor. But there was evidence that tutors provided feedback
on progress, or lack of progress, even when the form was not used.

For student 8 there was a positive comment as well as continuing issues:

Occasionally I detect examples of overly long indented quotation which was also an issue in your
first assignment. However, overall your writing has moved on in range and detail, both from the
early outline draft and from your earlier assessed work. (Tutor B)

8
Implementing Ipsative Assessment

For lower achieving students such as student 20, ipsative feedback potentially provided
motivation beyond a grade:
You managed to effect major improvement from the draft submission and have cleared the first
hurdle. You have secured a pass grade and I recognise the huge commitment on your part in the
face of adversity that has got you there. (Tutor A)

But for student 7 there was a warning:

It is a pity you did not have the chance to revise your assignment on the basis of the feedback
provided earlier, as this would probably result in a better grade. (Tutor C)

If the students do not use the ipsative feedback form, then the tutor will need to refer back to
previous feedback to provide an ipsative response. Comments such as the above provide some
evidence that tutors had done so and tutor accounts confirmed this.

There was further evidence that tutors did not rely completely on the information provided by
students on the form. For example, for student 27 the list of points to address from previous
assessments was:
-to pay more attention to ethical issues

-to improve the conclusion to make it fit the requirements of the brief

-to include an appendix listing the questions for the questionnaires -cutting down the usage of the
personal pronoun

But although tutor A recognised progress on the points above, the tutor also raised a new area
of progress:
-It is good to see that you have moved on from the frequent, quite lengthy quotations of the draft,
though perhaps you need to think carefully about how you develop a narrative thread which links
together the ideas of various authors without it becoming a little like a list. (Tutor A)

The point about length of quotations does not come from the form, so either the tutor
remembers this issue from the draft or has looked at the written feedback given for the draft and
then added this comment.

In another example student 16 did not provide an account of previous feedback, but the tutor
response was detailed:
this assignment was certainly an improvement compared to the previous version you submitted
for formative feedback. You avoided repetitions (especially when referencing) in a revised
Literature Review section, which appeared focused on your purpose. In addition, you provided a
well-documented research design in support of your stated aims. (Tutor C)

Even with small numbers it would be unlikely that the tutor would remember this level of detail
and again this   tutor   has   probably   referred   back   to   the   comments   s/he   wrote   on   the   student’s  
draft assignment.

Referring  back  to  one’s  own  previous feedback may not be unusual. However, in a fully ipsative
scheme tutors might need to refer to comments written by other tutors. Tutors do not routinely
make their feedback available to colleagues - giving feedback is possibly a more solitary activity
than teaching - and there was no evidence of reviewing the feedback provided by other tutors in
this study.

9
Implementing Ipsative Assessment

5. Results of analysis of student interviews

Relevant themes emerging from student interviews were: the new assignment submission form,
feedback and motivation and short versus longer-term views of assessment.

The Assignment Submission Form and its usefulness

Students who used the form identified two particular aspects as beneficial. One was that the
form allowed them to reflect on how and to what extent they had responded  to  tutors’  comments  
in   the   formative   feedback.   For   example,   one   student   commented:   ‘It [the form] allowed me to
show how I had attempted to improve on my last assignment and also encouraged me to reflect
more  deeply  on  my  submission’  (Student 22). The other benefit was that the new form facilitated
a systematic approach to keep track of changes made which enabled self-assessment:

The two step process was like a self assessment. The first step was how I had approached and
understood  the  assignment.  After  incorporating  my  tutors’  inputs  the  second  step  allowed  me  to  
explain  the  changes  I  had  incorporated.  The  difference  between  the  two  steps  helped  ‘focus’  the  
areas of change (Student 19).

There was one more sceptical comment about the new form being:

..for the convenience for tutors to check quickly what advice each student have received, and
check what  extent  each  student  respond  in  their  assignment’  (Student 14).

Student 14 continued to explain that the form would become useful only after the second time
students fill it in – because the first time benefits only the tutor:

the new form is not useful for the students who use it for their first time….  I  expect  from  the  next  
time tutors will give me comments more in detail based on their deep understanding about my
capacity  to  understand  tutor’s  comment  and  to  respond  it  (Student 14).

But the student’s  further comment suggested that there is a longer-term expectation of improved
feedback through dialogue and this will be discussed further below.

Overall, the usefulness of the new form was recognised by those who had used it, albeit a small
sample. A tool such as this form has a potential to improve the mechanism of assessment by
offering students an opportunity for reflection and to provide both students and tutors with a
space for better communication.

Feedback and motivation for distance learners

Feedback was important for these distance learners, whether or not they received ipsative
feedback. They often claimed to take it  ‘very  seriously and always act on it’ (Student 10). They
were very dependent on their tutors for feedback as few mentioned peer feedback and those
that did were uncertain of the value of peer feedback.

Students intended to act on feedback but were sometimes unable to follow through on part-time
distance learning course because of workload. In this case getting access to sources was also
an obstacle:

10
Implementing Ipsative Assessment

I have been very pleased with the feedback received. It was clear, concise and helpful. I do not
think it made reference to improvement with reference to the draft or previous assignment (maybe
I did not improve) I tried to act upon the recommendations but at times the task seemed daunting
when I could not get hold of additional sources to support my findings (Student 26).

The problem here is not about feedback, but about the learning environment.

This student did not receive ipsative feedback and was easily daunted by lack of progress, but
other students were very much aware of their progress and, as in the previous study, progress
was linked to grades, but there was also awareness of improved learning:

I had specifically tried to improve by following the advice given and it was helpful to know my
changes had my (made) a difference and reflected in a higher mark for my second assignment
(Student 22).

There was evidence that ipsative feedback was particularly motivational by building confidence.
8 out of 13 students mentioned that they had received clear comments from tutors about how
their work improved or not improved since the previous piece of work. With one exception, all
received   a   comment   saying   the   work   had   improved,   which   they   felt   ‘helpful   and   encouraging’  
(Student 22).

When tutors explicitly identify areas of improvement with reference to the previous version of the
assignment, students seem to appreciate their own progress and to be able to build their
confidence in acting on feedback in the longer-term:

Ipsative comment from tutor: this assignment was certainly an improvement compared to the
previous version you submitted for formative feedback. You avoided repetitions (especially when
referencing) in a revised Literature Review section, which appeared focused on your purpose.

Tutee’s  response:  [the  comment]  gave  me  confidence and made me feel proud of my work – this
was very important as it was the first assignment I had handed in and I was very unsure about it!
It also made me aware of the changes that I made that improved my work so I will know what to
do next time (Student 16).

As was the case in the findings from the previous research, generic  guidance,  such  as  ‘greater  
depth   in   analysis   required’   (Student   13) was sometimes confusing. Specific comments and
concrete examples were helpful: a ‘more  direct  type  of  feedback’  (Student 1)  such  as  ‘you  need  
to  look  at  this’  or  ‘if  you  are  including  this,  then you  should  look  at  this’ (Student 24).

Ipsative feedback was also appreciated alongside task relevant examples:

Ipsative comment from tutor: You are certainly writing with improving clarity but there some points
at which a lack of easy fluency weakens the impact of your narrative. An example of this would be
the paragraph on page 6 where good points based on Fidler needed better expression.

Tutee’s   response:   Because improving clarity is one of the biggest issues for me, and [this
comment] was helpful for me to understand this issue because tutor suggesting an example to be
improved (Student 14).

But, although illustrations are clearly valuable, students’   preference   for direct examples might
stem from a focus on short-term grade improvement.

Short-term grade improvement and long-term views on feedback

11
Implementing Ipsative Assessment

Many tutors’  generic comments addressed longer-term strategies but some tutees did not seem
to appreciate such long-term views. Instead, several students were seeking to take action to
quickly raise the quality of the essay:

My tutor [Tutor C] gave particularly good feedback after my draft assignment. It was easy to
understand and follow and was very specific so I could see exactly what I needed to improve and
how. I think her feedback made a big difference to my final mark (Student 16).

Many were disappointed not to gain extra marks by acting on feedback on draft work. One high
achieving student wrote:

I do find that the generous positive comments, motivating as they are, don't always translate into
higher marks. Perhaps the marking is very strict but after all the hard work, the positive feedback
encourages hopes for higher marks and I wasn't quite sure why I eventually only grazed into an A
grade. Grateful as I am to be doing relatively well, in this regard, given my exact same marking, I
wasn't sure whether I had progressed at all during the year (Student 13).

There was further evidence of student expectations that acting on feedback would produce
short-term rewards in terms of improved marks. In this example the student was also proactive
in informing the tutor that s/he had acted on the feedback and did not agree with the ipsative
feedback provided:
…the tutor reminded me of not having addressed an issue raised about my first draft. I did email
the tutor and highlighted an extract from the final essay where I had covered the suggestion
made. I believe the tutor had missed that point and the comment was then removed from my final
feedback. I was told, however, that this would not result in a revised mark (Student 13).

However, another student was clearly under pressure to finish the essay, but was aware that
the benefits might be longer term:

I   wasn’t   always   entirely   sure   how   to   act   on   the   feedback,   apart   from   by   trying   to   reflect on the
advice/comments. To be honest, I felt more relieved to just to finish the essays in the first place!
However, all the feedback has helped when I have gone on to do subsequent writing. I have tried
to consider where I went off tangent before (Student 3).

This is useful as taking a short-term view of expecting higher grades immediately can be
demotivating if expectations are not met. While students are motivated by grades, it would seem
that introducing ipsative feedback does not necessarily engender a longer-term view of progress
and learning, but this longer term view might develop as the students become more familiar with
the ipsative process. Tutors were aware of this and had some suggestions.

Continuity between tutors

In response to the questions on ipsative feedback, a couple of students identified discrepancies

between feedback on a draft and the summative feedback which came from a different tutor:

I received such [ipsative] comment once and it was from [Tutor C]. Other tutors would rarely
follow  up   on  feedback  once  given.  In  one   incident,  the  final  assignment’s  results  had  little  to  do  
with the feedback given on the draft, and I lost some marks on points not raised in the initial
feedback, which surprised me (Student 24).

12
Implementing Ipsative Assessment

Another recognised that tutors may not necessarily have access to previous feedback and this
will prevent ipsative feedback:

As different tutors are responsible for different courses it may be difficult for them to do this
[ipsative feedback]. Unless there could be a stock of comments kept on students previous essay
work [by] previous tutors? (Student 3).

Receiving new feed forward in the summative feedback that had not been mentioned in the
formative feedback was also criticised by some. For example:
After the first draft for both assignments, the first feedback from them was detailed. This helped
me to reflect, redo and improve the assignment. The second feedback sometimes had additional
findings which were not mentioned earlier (Student 19).

There is a dilemma here for tutors who fear too much feedback in one go might be
counterproductive and take a ‘drip   feed’   approach   providing   students   with   new   areas   for  
progress each time. Feed forward must be achievable and realistic. It is not helpful to provide
lengthy feedback to a student who is a borderline pass on what they would need to do to
achieve the top grade. However, it may be that the tutor missed an important piece of feedback
first time round. Using the Submission Form might make such practice more visible and easier
to change. Again this highlights a need to encourage learners to take a long-term view of
developmental feedback and not focus on short-term marks which is difficult when the short-
term marks are high stakes. Tutors were aware of this dilemma as we shall see in the next
section.

6. Results of analysis of tutor interviews

The responses to the questions were categorised into three sections: change in practice to
respond to the new form; supporting an ipsative feedback process and supporting   students’  
understanding of the assessment process.

Change in practice to respond to the new form

Each tutor responded differently to the new Submission Form. Although they agreed that it
changed  the  way  that  they  gave  feedback,  ‘up  to  a  point’  (Tutor  B),  one  tutor  commented  that:

I was already on the journey to that approach. When providing feedback I tended to approach it
in my normal way and then made sure that I adapted things to meet the requirements of the new
front sheet. (Tutor A).

Tutor C, who had been tutoring on the programme for a shorter period of time, had changed
more radically by carefully checking whether each student had addressed the points made in
previous   formative   feedback.     This   tutor   also   commented   that   ‘students   had   made   significant  
efforts   to   address   the   points   raised’ which is consistent with both the comments made by
students, although less consistent with the evidence from the analysis of the written feedback
which suggested that students may generalise claims to have addressed feedback or may only
partially address feedback.

An issue was raised about the process being introduced half way through the year. Student
feedback had suggested that the benefits could be limited because they were half-way through
a cycle, but tutors felt it gave them the opportunity to get used to a modified process and so they
would be more confident in applying it at the beginning of the next year.

13
Implementing Ipsative Assessment

Supporting an ipsative feedback process

Tutor interviews suggested that a positive outcome of the implementation of ipsative feedback
was   ‘further   dialogue   over   the   nature   of   progression’   (Tutor A) and in particular how ipsative
feedback,   once   fully   understood   as   a   development   tool,   could   contribute   to   students’ learning
(Tutor B). This is consistent with some student reports of using the form for dialogue with tutors
about their learning.

A further benefit was the way in which information was shared amongst the tutor team. The
core tutor team, who taught   both   modules,   ‘invested   a   lot   of   time’   (Tutor A) in sharing
information about individuals and their work, but there had been no mechanism in place to
share this knowledge with tutors who took over for the optional modules.

The tutors indicated a readiness to further explore the use of ipsative feedback, however, tutor
B also warned, that it could increase the burden of work.

Providing systematic cumulative feedback is a desirable objective, but the system we adopt
should be efficient as well as effective and not place unsustainable burdens on students of (or)
tutors. (Tutor B)

This rather more sceptical view of use of the form, from Tutor B, is also consistent with the
greater use of the form, made by Tutors A and C identified in the analysis of the tutor written
feedback.

Supporting  students’  understanding  of  the  assessment  process  

There   was   a   belief   that   students’   expectations   of   the   assessment   process   needed   to   be  
managed as these expectations could range from   the   ‘reasonable   to   the   unreasonable’   with  
students immediately seeking solutions to raising their grades (all tutors).

I could see from the comments that some students expected their revised draft to lead to an A,
and it was subsequently disappointing for these students to gain a sense of progress without an
A. (Tutor C)

The tutors’ view   that   it   was   difficult   to   manage   students’   expectations   and   deepen   their  
understanding of the process was consistent with some student expectations of continuity
between tutors and the dissatisfaction at not receiving an immediate grade increase when they
acted on feedback.

A  further  issue  that  emerged  was  the  programme’s  diverse  intake  (which  the  team  views  as  a  
positive feature of the programme) and the impact this  has  on  the  student’s  ability  to  understand  
that emerging issues are generic, irrespective of the context. This was an issue that would
need careful consideration when developing and improving the ipsative feedback process.

There was also concern about   students’   understanding   of   the   different requirements made by
consecutive assignments and the different skills they would need to evidence. For example, the
assignment for Core Module 1 focuses on developing the students’  ability to critique literature,
identify themes and apply them to their own workplace, whilst the assignment for Core Module
2, additionally, develops the ability to design, but not carry out, small scale research. Tutor A
commented:

14
Implementing Ipsative Assessment

The two [core module] assignments are very different in nature, draw on overlapping but not
identical skill sets and each poses its own challenges. It is perfectly feasible to have picked up (in
the   student’s   view)   on   earlier   guidance   only   to   fall   down   on   something   which is specific to the
second of the assignments. Perhaps we need to make this clearer. (Tutor A)

Some students also pointed out that even when they addressed the feedback there was more to
do even for the same assignment, and this supports a recommendation that assessment needs
to be viewed as a longer-term developmental process where new challenges will arise.

The tutors made suggestions for broadening views of assessment that included: putting
together a statement of the goal/aims of ipsative feedback, exploring the addition of peer
feedback to supplement tutor feedback, and revisiting the contribution of the VLE discussions to
students’   understanding   of   the   assessment   process. There is evidence that reading student
reflections on the form provoked these ideas but it is not possible to attribute all the
development in tutors’  thinking about assessment to use of the ipsative feedback form becuase
taking part in the research process may also have contributed.

7. Discussion of further implementation of ipsative feedback

The findings above indicate that a formalised process for providing ipsative feedback has shifted
the way in which some tutors write feedback and has been helpful and motivational for learners.
This pilot implementation process is therefore worth scaling up and continuing. There are
several points for further discussion.

Use of a form to promote ipsative assessment

Use of a form both to make the ipsative feedback process explicit and to involve students in a
dialogue on feedback was effective. This is particularly important when ipsative feedback is new
to both tutors and students. A dialogue between tutors and students is key to effective
assessment (Nicol & Macfarlane-Dick, 2006) and for distance learners where there is little direct
contact, use of a process such as this is very valuable. However, the form was not consistently
used by students and some tutors did not provide ipsative feedback even when students did
complete the form. Further investigation will determine whether this problem will disappear as
both tutors and students become more accustomed to the ipsative process. Ipsative feedback
did arise without the form being used and it may be that once a longitudinal approach to
assessment becomes established, such a tool would be superfluous.

Ipsative feedback as motivational but raising expectations

There was evidence that ipsative feedback is motivational for students and there is some direct
evidence of enhancement of learning through use of the form. However, the process used in
this study also raised student expectations as student believed that if they acted on previous
comments then they would gain immediate higher marks. A longer-term view of feedback will
need to be established so that learners will view progress over time and not have expectations
of immediate results. However, it might be difficult to persuade learners, who are accustomed to
relying on marks, to gauge their progress and to view progress in the longer-term as the
ultimate goal. It may be that introduction of some form of ipsative grading will be necessary
(Hughes, 2011).

15
Implementing Ipsative Assessment

Making a comparison with previous assessments

Comparison with previous assessment is problematic for two reasons. Firstly, access to
previous feedback and assessment is not always easy and may be time-consuming for tutors.
Inviting the students to provide the summary of previous assessments is helpful, but it may not
be accurate and not all students may see the advantages of filling in a form which may appear
bureaucratic. Other ways of comparing assessments are therefore worth exploring as well as
use of technology to make the comparisons easier.

Secondly, there is the problem of consecutive modules assessing different skills. This means
that comparison is not straightforward. It can also lead to students’ disaffection because they
believe that they have acted on previous feedback but the response they get from the assessor
is that there is more work to do. Broadening understanding of assessment to view it as a
longitudinal journey throughout a programme, where a learner is expected to make continual
improvements is therefore imperative. Again, as above, this may be challenging in current
competitive and summative assessment-led regimes, but   the   tutors’   suggestions   for clarifying
the ipsative process and using peer feedback may help.

Scalability, sustainability and transferability

In the previous study of this programme (Hughes et al. 2010) there was very little evidence of
ipsative feedback being provided. The introduction of the ipsative feedback form has
significantly increased the amount of ipsative feedback given to learners, albeit from some
tutors more than others. This is the first time the innovation has been tried and it might be
expected that continued use of the form might increase the numbers of students engaging with
issues of progress and encourage more consistent use of ipsative assessment by tutors through
greater familiarity with the process and a greater understanding of the benefits. Tutors
interviewed were willing to continue with ipsative feedback.

Numbers on this programme are relatively small, but using a form such as this means that there
are possible scalable benefits in that keeping a record of feedback means that a different
assessor could continue the process in future modules and students from much larger cohorts
could be tracked. However, this would need a much more open and collegial approach to giving
feedback. Further studies are needed to confirm the scalability of such an ipsative assessment
process.

Using technology may provide some solutions to managing large volumes of student
assessment data as Nicol & Milligan (2006) suggest. For example, making previous feedback
and assignments readily accessible via a VLE or other technology might help students and
tutors quickly identity progress. However, although a folder was added to the VLE containing
this information, there was limited evidence that tutors referred back to previous assignments
and feedback although some tutors reviewed their own previous comments. It may be that if
ipsative assessment was fully embedded and became normal practice, then referring to
previous assessments might be useful, especially when students do not provide information
about previous feedback on the form or when student representations of their previous feedback
and of their responses are inaccurate. Reference to previous assignments could, however,
increase the time taken for the formative assessment process, but may be offset by
encouraging students to act on cumulative feedback and thus needing less detailed feedback as
they progress through the programme.

Although this study applies to Masters students, there is nothing to suggest that this approach
might not also be successful for students at undergraduate level. Undergraduates are less likely

16
Implementing Ipsative Assessment

to be self-reliant than postgraduates and many take a similarly short-term and grade dominated
view (Higgins et al. 2002). They may be less motivated to complete a form or less able to
complete it accurately and tutors may need to be more proactive, but over the longer-time span
of degree programmes the benefits of increased motivation and enhanced learning might
outweigh this.

This was an education programme and other disciplines in social sciences and humanities could
similarly benefit from an ipsative scheme such as this. But, that is not to say that scientific and
technical disciplines might not also apply ipsative assessment to motivate students through
making longer term progress visible, perhaps using a different methodology.

8. Conclusion and recommendations

This study has documented the second, implementation phase of a study of the potential for
using ipsative feedback to motivate distance learners. A new assignment submission form was
designed that enabled learners to reflect on addressing previous feedback and encouraged
tutors to provide ipsative feedback, although use of the form was not essential. The ipsative
feedback was well received by those students who completed the form and helped with both
motivation and dialogue between tutor and students. However, a view of progress as short-term
grade improvement limited the potential of ipsative feedback and although a few students
appeared to take a longer term view of assessment, perhaps promoted by the new ipsative
process, this needs more encouragement and embedding.

Further work will consider the following recommendations which emerge from this report:

1. Ipsative feedback potentially has motivational and learning benefits for both tutors and
students and should be embedded on this programme.

2. The ipsative process at programme level needs to be made clear to students and
inclusion of some ipsative grading and peer assessment might help.

3. The sustainability of ipsative assessment and longer-term approaches to assessment on

the AELM programme should be monitored.

4. The use of technology to facilitate ipsative feedback needs to be embedded e.g.

recording feedback in the VLE as a standard practice, but tutors also need to be
encouraged to share feedback with colleagues.

5. Use of ipsative feedback could be tested with other programmes at different levels, scale
and for different disciplines.

17
Implementing Ipsative Assessment

Bibliography

Bloxham,   S.   Boyd,   P.   &   Orr,   S.   2009.   Mark   my   words:   An   analysis   of   HE   lecturers’   essay  
marking approaches. Challenging Higher Education: knowledge, policy and practice. Society for
Research in Higher Education conference papers, Newport, Wales.

Carless, D. 2006. Differing perceptions in the feedback process. Studies in Higher Education. 3,
no. 2: 219-233.

Crisp, B. R. 2007 Is it worth the effort? How feedback influences students' subsequent
submission of assessable work. Assessment and Evaluation in Higher Education 32, no.5: 571-
582.

Falchikov, N. & Boud, D. 2007. Assessment and emotion: the impact of being assessed. In
Rethinking Assessment for Higher Education: learning for the longer term, eds. David Boud and
Nancy Falchikov, 144-152. Abingdon: Routledge.

Gibbs, G. (2006) Why Assessment is Changing. In Innovative Assessment in Higher Education,

eds. C. Bryan and K. Clegg,11-22. Abingdon: Routledge.

Handley, K. Price, M. & Millar, J. 2008. Engaging Students with Assessment Feedback. FDTL5
Project Final Report, Oxford Brookes University
https://mw.brookes.ac.uk/download/attachments/2851361/FDTL+Final+report+to+HEA+-
+September+2008.pdf?version=1

Hattie, J. & Timperley, H. 2007. The Power of Feedback. Review of Educational Research 77
no. 1: 81-112.

Higgins, R., Hartley, P. & Skelton, A. 2002. The Conscientious Consumer: reconsidering the
role of assessment feedback in student learning. Studies in Higher Education 27, no. 1: 53-64.

Hughes, G. 2011. Aiming for Personal Best: a Case for Introducing Ipsative Assessment in
Higher Education Studies in Higher Education 36 (3): 353 - 367

Hughes, G., Okumoto, K. & Crawford, M. 2010. Ipsative Assessment and Motivation of Distance
Learners. CDE Project Report http://cdelondon.wordpress.com/2010/07/28/use-of-ipsative-
assessment-in-distance-learning/

Lizzio, A. & Wilson, K. 2008. Feedback   on   assessment:   student’s   perceptions   of   quality   and  
effectiveness. Assessment & Evaluation in Higher Education 33 no.3:263-275.

Morgan,  C.  &  O’Reilly.  M.  1999  Assessing Open and Distance Learners London: Kogan Page.

Nicol, D. & Macfarlane,-Dick, D. 2006. Formative assessment and self-regulated learning: a

model and seven principles of good feedback practice. Studies in Higher Education 31 no. 2:
199-218.

Nicol, D. & Milligan, C. 2006. Rethinking technology-supported assessment practices in relation

to the seven principles of good feedback practice. In Innovative Assessment in Higher
Education, eds. Cordelia Bryan and Karen Clegg, 64-80. London: Routledge.

Poulos, A. & Mahony, M. J. 2008 . Effectiveness of feedback: the students' perspective.

Assessment and Evaluation in Higher Education 33 no. 2: 143-154

18
Implementing Ipsative Assessment

Sadler, R. D. 2005. Interpretations of criteria-based assessment and grading in higher

education. Assessment & Evaluation in Higher Education, 30 no.2:175-194.

Simpson, O. forthcoming. Technology Supported Assessment for Retention. In Lynn Clouder,

Christine Broughan, Steve Jewell & Graham Steventon (Eds) Improving Student Engagement
and Development through Assessment Theory and Practice in Higher Education. London:
Routledge.

Simpson, O. 2008. Motivating learners in open and distance learning: do we need a new theory
of learner support? Open Learning 23(3): 159-170.

Walker, M. 2009. An investigation into written comments on assignments: do students find them
useable? Assessment and Evaluation in Higher Education 34 no. 1: 67-78.

19
Implementing Ipsative Assessment

Appendix 1: New Assignment Submission Front Sheet (elements added for ipsative
feedback are in italics. Spaces in text boxes have been removed)

MA Applied Educational Leadership and Management

Assignment submission form

*Please copy and paste this form into the first page of your assignment file*

Name

Student ID

Module title

Title of
assignment

Assignment word
count
(excluding
references and
appendices)

Is this a draft? Is this a resubmission?

Yes / No (delete as appropriate) Yes / No (delete as appropriate)

If this is later than

deadline, please
Date of
give submission
submission
date agreed with
course director

Please indicate what feedback you were given, for your last assignment, in terms of how you
could improve:

20
Implementing Ipsative Assessment

Please indicate what feedback you were given, for your draft of this assignment, in terms of how
you could improve (if applicable):

Please comment on the extent to which you feel you have responded to feedback:

If you would like feedback on any particular aspects of your current assignment please make a
note of them here:

CHECKLIST: Before you submit your assignment please check that you have followed all the
guidelines so that we can identify, print and mark your assignment. Mark each box with an X
symbol.

Have you:

saved your document with your name and module in the  title  (e.g.”SJacksonMod1assign”)?

inserted page numbers ?

remained within the word length limits stipulated for the assignment ?

included your word count on the submission form ?

set left + right margins to at least 2.4 cm?

followed guidelines for referencing ?

read and understood plagiarism code ( p58 - 61 of handbook)?

completed the feedback section of the submission form

completed an online course evaluation (for final assignment not draft)?

21
Implementing Ipsative Assessment

Appendix 2: Analytical Tool

Module:
Feedback given on
previous assignments
F

E Formative feedback given

on this assignment
E
To all or some
D
Response
B to
formative How
A
feedback
C
Other
K
Student’s  
request
S
Request
H for Tutor’s  
particular response
E feedback
E

Comparison with past performance

Comparison with assessment
e.g. comment on student's response
criteria or standards
to feedback on Feedback Sheet

Task level Task level

SUMMATIVE
FEEDBACK
Grade: Generic level Generic level
1st-marker:

22
Implementing Ipsative Assessment

Appendix 3: Details of students involved in the study

Student number Completed form Y/N Interviewed Y/N

1 Y Y
2 Y N
3 Y (request for feedback section only) Y
4 N N
5 Y Y
6 Y (old) N
7 N N
8 N N
9 N N
10 N Y
11 Y Y
12 Y (old) N
13 Y Y
14 Y Y
15 N Y
16 Y Y
17 N N
18 N Y
19 Y Y
20 N N
21 Y N
22 Y Y
23 Y (request for feedback section only) N
24 N Y
25 N N
26 N Y
27 Y N
28 deferred

Totals submitted assignment(s) 27, completed form 15, no form submitted, 12, interviewed 13 (7
used form and interviewed, 6 did not complete form and were interviewed).

The breakdown was as follows:

Core modules: no. of students

Leading and Managing Educational Change and Improvement 6
Leadership for the Learning Community 7

Optional modules:
Developing Leadership and Management Skills 5
Exploring Educational Policy 2
Values, Vision and Moral Purpose in Educational Leadership 3

23
Implementing Ipsative Assessment

Appendix 4: Student Interview Questions

Ipsative Assessment Project

Email Questions

These questions are part of a research project investigating new assessment arrangements on
your programme. The changes have been made in response to tutor and student interviews
from last year. The questions mainly refer to the modules listed below. Please give as much
detail as you can. We might wish to follow up the answers you give.

Your answers will be confidential and anonymous and will not in any way influence the
marking of your work.

Please tick the assignments you have submitted during this academic year.
Leading and Managing Educational Change and Improvement
Leadership for the Learning Community
Developing Leadership and Management Skills
Exploring Educational Policy
Values, Vision and Moral Purpose in Educational Leadership

New Assignment Submission Form

1. Have you used the new Assignment Submission Form (cover sheet)? If NO, please go to 4.
2. If you have found the new form useful, please explain in which way.
3. If you have not found the new form particularly useful, please explain why.
4. Please give any reason/s why you did not use the new form.

Styles of feedback, usefulness and motivation

5. Who gives you useful feedback, e.g. your tutor, other students, or do you prefer to give
yourself feedback? Please explain your choice/s.
6. What is the most useful feedback you have received on the modules?
7. Here are two pieces of feedback you were given. Was either of comments more helpful than
the other? If so, why? Did one of them motivate/encourage you more than the other?
a) An example of ipsative feedback.
b) An example of feed forward feedback.
How did you, or are you intending to act on the above feedback comment?

Recognition of progress and ipsative feedback

8. Did you receive clear comments from your tutor about how your work has improved or not
improved since the previous piece of work? If so, was it helpful to receive such comments?
9. Did tutors inform you when you have not addressed the issues raised in previous feedback?
If so, was it helpful to receive such comments?
10. What informed you of progress during the modules?

24
Implementing Ipsative Assessment

Appendix 5: Tutor Interview Questions and letter

Dear Colleagues,

If you remember, we have been trialling ipsative feedback to assignments submitted in the
spring term, with a view to setting up a system that is simple for both tutors and students to use,
and that would hopefully benefit all.

We have received the responses to the questionnaires and interviews from the students. They
have proved very interesting, and the insights they are providing should inform development of
the programme in several areas, not just assessment. I have attached the interview comments
in a separate document.

The issues that are emerging are:

Draft and final assignment feedback

 Feedback  should  comment  on  the  student’s  ability  to  explore  concepts   as  well  as  their  
development of writing skills.

 Comprehensive feedback for the first assignment, at draft and final assignment stage, is
very helpful in understanding how to improve next time.

 The use of examples is helpful in understanding good practice and how improvements
can be made.

 A mix of comments, about what is good work and where to improve next time, is
motivating.

 There is not always consistency between feedback for the draft and for the final
assignment, i.e. new issues emerge. This is both confusing and demotivating.

 It is demotivating to act on guidance and see no change in grade.

VLE activity

 There appears to be variability in the response to how useful on-line activities are, with
students finding it difficult to interact with/help each other (see the responses at the top
of page 3, in section 5: who gives you useful feedback?)

Further issues that we are aware of and trying to address are:

 The need for more specific research methods training.

 The problems of writing an assignment when starting a new module.

I would find it very helpful if you would read the attached summary of comments about ipsative
feedback and then answer the following questions. Your responses will inform the research and
help me to think about how I can support your work as tutors and develop the programme to
meet  students’  needs.    Your  responses  will  be  treated  as  confidential  and  if  we  want  to  quote  
you, we will first ask your permission and ensure the quote cannot be attributed to you.

25
Implementing Ipsative Assessment

These are the questions that we would like you to consider and respond to:

1. Did you find that the new front sheet changed the way that you gave feedback? Please
explain your response.

2. How have the comments from section 5 (see highlighted comments below), informed your
thinking   about   the   contribution   of   the   VLE   to   students’   understanding of the assessment
process?

3. How would you suggest that we emphasise that ipsative feedback is for long term and not
necessarily immediate benefits?

4. Is there any way in which the programme team might support you in developing an ipsative
feedback approach to the assessment process?

5. Do you have any further comments that you would like to make?

Student feedback was inserted here

Thank you very much for your help, we will share our report with you when it is complete.

6. How would you suggest that we emphasise that ipsative feedback is for long term and not
necessarily immediate benefits?

7. Is there any way in which the programme team might support you in developing an ipsative
feedback approach to the assessment process?

8. Do you have any further comments that you would like to make?

Student feedback was inserted here

Thank you very much for your help, we will share our report with you when it is complete.

26