UK Quality Code

for Higher Education

Advice and Guidance
Monitoring and Evaluation

UKSCQA
UK Standing Committee
for Quality Assessment
UK Quality Code - Advice and Guidance
Monitoring and Evaluation

Contents
Regulatory contexts for the Quality Code 1
Terminology 1
Expectations and Practices 2
Guiding principles 4
Practical advice 5
Advisory group 9

Regulatory contexts for the Quality Code

The Quality Code articulates a set of principles that apply across the UK through four Expectations.
These Expectations are then explained and contextualised through Core and Common practices in
a way that allows institutions to demonstrate them. The Expectations, Core and Common practices
are not regulatory requirements in England, but the Practices should be demonstrated by providers
operating in Wales, Scotland and Northern Ireland.
National regulators and QAA are not bound by the information in this advice and guidance and will not
view it as containing indicators of compliance. This guidance does not interpret statutory requirements.

Terminology
Monitoring: The routine collection and analysis of information that focuses on an area of work, project
or programme/course, undertaken while the area of work, project or programme/course is ongoing.
Evaluation: The periodic, retrospective assessment of an organisation, an area of work, project or
course, that might be conducted internally or by external independent evaluators. Evaluation uses
information from monitoring, current and historic, to develop an understanding and inform planning.
Providers: Any organisation involved in the provision of higher education to students and apprentices.
Degree-awarding bodies: Organisations empowered to award higher education awards under relevant
UK statutes.

1
 Expectations and Practices
The advice underneath the Expectations and Practices is not mandatory for providers but illustrative of a range of possible approaches.

Monitoring and evaluation of higher education is an essential process within providers, forming a
fundamental part of the academic cycle. It can, and should, look at all aspects of the higher education
experience. All higher education providers are involved in course monitoring and review processes as
these enable providers to consider how learning opportunities for students may be improved.

The academic standards of courses The value of qualifications awarded

meet the requirements of the relevant to students at the point of qualification
national qualifications framework. Expectations and over time is in line with
for standards sector-recognised standards.

Monitoring and evaluation ensures Monitoring and evaluation is an

that providers’ academic provision enables essential process within a provider’s
students to achieve the intended learning internal quality assurance mechanisms,
outcomes of courses. They evaluate covering all provision that leads to their
student attainment of academic standards awards and assuring the standard of those
and allow providers to confirm that their qualifications. Relevant sector-recognised
portfolio aligns with their mission and standards form a baseline for monitoring
strategic priorities. and evaluation systems.

Core practices
The provider ensures that the threshold standards for its qualifications are consistent with the
relevant national qualifications frameworks.
In practice, this means that providers in Scotland, Wales and Northern Ireland refer to the relevant
national qualifications framework when designing monitoring and evaluations policies and processes,
and ensure they assess whether threshold standards are being achieved by their graduates.
Providers based in England and registered with the Office for Students (OfS) should refer to the OfS
sector-recognised standards when considering how their academic frameworks and regulations
adhere to regulatory requirements in that nation.
The provider ensures that students who are awarded qualifications have the opportunity to achieve
standards beyond the threshold level that are reasonably comparable with those achieved in other
UK providers.
In practice, this means that providers collect and analyse data that allows for comparison and
make appropriate use of externality and sector guidance on, for example, degree classifications and
algorithms.
Where a provider works in partnership with other organisations, it has in place effective
arrangements to ensure that the standards of its awards are credible and secure irrespective of
where or how courses are delivered or who delivers them.
In practice, this means that degree-awarding bodies ensure monitoring and evaluation arrangements
are tailored to HE delivery in other organisations and learning environments, which may include
satisfying themselves that those organisations’ own monitoring systems are sufficient and feed in to
their own appropriately.

Common practice
The provider reviews its core practices for standards regularly and uses the outcomes to drive
improvement and enhancement.
In practice, this means that providers have formal systems in place with the express purpose of using
the outcomes from monitoring and evaluation activity as a mechanism for enhancing their provision
beyond meeting threshold regulatory requirements.

2
 Courses are well-designed, provide From admission through to completion,
a high-quality academic experience all students are provided with the
for all students and enable a student’s Expectations support that they need to succeed in
achievement to be reliably assessed. for quality and benefit from higher education.

Effective monitoring and evaluation Monitoring and evaluation systems

allows providers to consider objectively look at all stages of the student
whether their courses are in fact well- experience and consider the support
designed and high-quality, and can provided to students.
consider whether other systems and
processes are effective in ensuring
reliable assessment.

This theme gives guidance to providers Core to helppractices

support the provision of effective, high-quality learning
opportunities for all students, wherever or however the learning is enabled and whoever enables it.
It applies
The to any
provider learning
designs opportunity
and/or that leads tocourses.
delivers high-quality the award of a UK higher education qualification
or academic credit, from short courses involving single modules
In practice, this means that providers routinely monitor their to provision
course multi-yeartocourses. Learning and
allow objective
teaching enables
assessment students’
of whether achievement
this practice to be
is being reliably evaluated through assessment, calibrated to the
achieved.
national reference points, for example the Frameworks for Higher Education Qualifications[link].
The provider supports all students to achieve successful academic and professional outcomes.
In practice, this means that providers ensure that the support given to their students is scrutinised
through their monitoring activity.
The provider actively engages students, individually and collectively, in the quality of their
educational experience.
In practice, this means providers ensure effective collection, collation and analysis of student
perspectives and feedback in monitoring and evaluation, and feed outcomes into strategic planning
and course design as appropriate.

Common practices
The provider reviews its core practices for quality regularly and uses the outcomes to drive
improvement and enhancement.
In practice, this means that providers regularly review and enhance their provision, reflecting on a
range of data sets as they relate to quality to ensure courses and wider services remain fit for purpose
and to take account of changing circumstances, demands and pedagogical developments.
The provider’s approach to managing quality takes account of external expertise.
The provider engages students individually and collectively in the development, assurance and
enhancement of the quality of their educational experience.
In practice, this means that providers identify and involve key internal and external stakeholders,
including students and external examiners, to the design, implementation and operation of
monitoring and evaluation, as well as the analysis and communication of findings.

3
 Guiding principles
The guiding principles given here are not mandatory for any provider. They are a concise expression of
the fundamental practices of the higher education sector, based on the experience of a wide range of
providers. They are intended as a framework for providers to consider when establishing new or looking
at existing higher education provision. They are not exhaustive and there will be other ways for providers
to meet their requirements.

Providers agree strategic principles for monitoring and evaluation to ensure processes are
1 applied systematically and operated consistently.

Providers have strategic guiding principles that set out why monitoring and evaluation takes place
and what it intends to achieve. They ensure that activities are relevant, useful, timely and credible.
The processes used, and the results from monitoring and evaluation activity, are recorded clearly
and are themselves reviewed periodically to ensure they remain fit for purpose.

Providers normalise monitoring and evaluation as well as undertaking routine

2 formal activities.

Effective monitoring and evaluation is an ongoing activity incorporated into everyday,

standard practice. Formal activities - such as periodic review of courses, annual monitoring
and workshops - are set in place on a routine basis. Progress against plans developed in
response to the outcomes of monitoring and evaluation is checked.

3 Providers clarify aims, objectives, activities and actions, and identify the key indicators,
issues, questions, targets and relevant information/data.

Providers decide and prioritise what they will monitor and evaluate, fitting with internal and
strategic priorities and external requirements, and within available resources, establishing systems
which are ongoing and pre-emptive rather than simply reactive.

Providers decide whom to involve in the different stages of monitoring and evaluation,
4 clearly defining roles and responsibilities and communicating them to those involved.

Providers identify key internal and external stakeholders, particularly students, allowing
consideration of how they can be involved and contribute to the design and implementation
of monitoring and evaluation systems, analysis of data and the communication of findings.

5 Providers evaluate, analyse and use the information generated from monitoring to
learn and improve.

Providers ensure their processes periodically analyse and assess data they collect to generate
evidence used in quality assurance and enhancement, internal decision-making, planning and
learning processes.

Providers communicate outcomes from monitoring and evaluation to staff,

6 students and external stakeholders.

Providers put in place mechanisms to share, discuss and interpret findings, for example, periodic
internal meetings and specific evaluation workshops. These systems may also facilitate conversations
on actions required and draw out learning points for the provider and other stakeholders.

Providers take account of ethics and data protection requirements when designing
7 and operating monitoring and evaluation systems.

Providers have informed consent from any participants, ensure anonymity in the communication
of findings, and respect GDPR and all other data protection laws.

4
 Practical advice
This section provides practical, contextualised advice to providers on monitoring and evaluation.
Where relevant, we have indicated which guiding principles the advice will help you achieve.
Please bear in mind that this guidance is illustrative and intended to inform the approaches
you consider and ultimately implement.

Establishing strategic principles for monitoring and evaluation

(Guiding principles 1, 3)
Providers develop strategic principles to ensure that monitoring and evaluation is relevant, useful,
timely and credible. The starting point is what the provider is hoping to learn from the process
- once this is established, systems and procedures can be designed that are fit for this purpose.
The following examples of what monitoring and evaluation can do are not exhaustive but may
provide a starting point for establishing or reviewing providers’ strategies.
Effective monitoring and evaluation can:
¡ help develop clear, attainable outcomes, objectives, targets and goals
¡ assess and demonstrate effectiveness in achieving them
¡ improve organisational planning, performance, enhancement and decision making
¡ empower and motivate staff and key stakeholders
¡ promote ownership and engagement at appropriate levels of the organisation
¡ influence strategic decisions, policy development, process improvements, learning and
teaching activity and assessment, and provide opportunities to test their effectiveness
¡ identify problems at provider, faculty and course level, and seek where appropriate to
effect an early remedy
¡ contribute to the national evidence base about the effectiveness of HE provision
¡ help assess levels of satisfaction among students, alumni and employers
¡ help collate and disseminate good practice and techniques across faculties within a provider,
and identify internal or external factors that may be facilitating or constraining progress
¡ ultimately help sustain successful courses and effective systems in the longer term.

Figure 1: A cyclical model of monitoring and evaluation.

PL
AN

Setting
N

NI

the vision
TIO

NG
LUA
EVA

Defining the
results map and
Managing results-based
and using management
evaluation framework
Stakeholder (RBM)

Participation

Implementing Planning for

and using monitoring and
monitoring evaluation

MONITORING

5
 Reflective questions

¡ How do your strategic principles for monitoring and evaluation take account of internal
and external stakeholders?
¡ How do you ensure that your systems for monitoring and evaluation are sustainable
and effective?

Features of effective monitoring and evaluation (Guiding principles 2, 4, 5, 6)

Effective monitoring and evaluation analyses or interprets collected data to produce a systematic and
objective assessment, taking place at specific times. It draws conclusions on relevance, effectiveness,
efficiency, impact and sustainability, asks whether providers are achieving what they set out to do, and
identifies what could be done better or differently. These conclusions feed into course review.
Providers apply their monitoring and evaluation processes systematically and operate them
consistently; the processes are capable of being applied to all higher education offered by a provider,
but recognise and respect differences between subjects, modes and levels of study.
Providers use examples of sound practice and innovation, and relevant guidance from sector bodies,
to keep their strategic approach to learning and teaching under review, to modify it as appropriate and
to facilitate the continuous improvement of the learning opportunities they offer.

Reflective question

¡ How do your systems of monitoring and evaluation feed into course review?

Information and data (Guiding principles 3, 5, 7)

Providers ensure the information and data gathered:
¡ is focused and feasible in relation to available resources; it should support rather than
divert resources
¡ is consistent across all courses, accounting for different records systems and institutional
definitions of data
¡ focuses on what providers ‘need to know’, not on what would be ‘nice to know’
¡ is useful and timely in improving course design, learning and decision-making
¡ is accurate, relevant, credible, valid and reliable.
Quantitative and qualitative data can be drawn from internal and external sources, from as close to the
point of delivery as possible up to faculty, thematic and institutional level. Consideration is given to the
construction of a suitable format for collation of information drawn from multiple sources and how it can
be presented coherently.
Common monitoring and evaluation data sources include:
¡ student feedback
¡ statistical information, including internal data on selected key performance indicators,
such as student progression and achievement, retention, course or modular results,
graduate employment and grade improvement trends
¡ research degree-specific information, for example, completion within expected time frames,
prizes, publications, conference attendance and presentations
¡ external data sets such as the National Student Survey (NSS), Postgraduate Taught Experience
Survey (PTES) and Postgraduate Research Experience Survey (PRES)

6
¡ feedback from course teams and professional services staff
¡ reports and feedback from external examiners, academic reviewers, employers (including those
involved in workplace experience or education provision), external quality reviews (such as the
Scottish Quality Enhancement Arrangements and Quality Enhancement Review in Wales), sector
regulators, professional, statutory and regulatory bodies (PSRBs), and other external experts
¡ feedback from previous monitoring exercises
¡ any conditions and recommendations arising from the course validation or latest revalidation
¡ issues that have arisen related to learning resources, staffing, engagements with employers,
PSRBs, and United Kingdom Visas and Immigration (UKVI).
Where possible, data is disaggregated by protected characteristics to identify any differential impact
on particular groups of students, while meeting the provider’s data protection responsibilities.
Providers also build in any requirements for, and timing of, monitoring and evaluation by PSRBs
where appropriate.

Reflective questions

¡ What key data sources do you use to improve and develop courses?
¡ How do you ensure your use of data is timely and supportive?

Roles and responsibilities (Guiding principles 2, 3, 4, 6)

The respective roles, responsibilities and authority of different individuals and bodies are clearly
defined. Those involved are fully briefed about their role, the hierarchy of procedures, and the location
of ultimate responsibility, including where a degree-awarding body works with a partner or delivery
organisation to offer higher education. The processes to be followed are made clear. Attention is paid to
terminology to aid full understanding. Providers determine responsibility for identifying, disseminating
and embedding good practice through the processes.
Staff involved are drawn from across the provider as required, including academic and professional
services staff. Where a provider works with other organisations, such as partner organisations and
employers, relevant stakeholders contribute.
Providers support staff by putting in place appropriate training and development - for example,
members of staff new to the processes may be given the opportunity to shadow more experienced
colleagues. Providers recognise the value and mutual benefit for their own provision of facilitating staff
involvement in course and departmental monitoring and evaluation at other higher education providers.
The extent to which a degree-awarding body delegates authority and operational roles for course
monitoring and evaluation to a delivery organisation is defined in the agreement between the two
bodies (see also Partnerships Theme).

Reflective questions

¡ What training and development do you have in place for both internal and external staff?
¡ How do you ensure any partners are fully represented in monitoring and evaluation?

7
Student feedback and engagement (Guiding principles 2, 3, 5)
Providers enable students to provide individual as well as collective feedback and ensure they can
engage in their monitoring and evaluation processes. Surveys are a mechanism for enabling this,
as are focus groups, student-staff liaison committees, and student representation on providers’ other
committees. Providers consider the timing, regularity, accessibility and intended outcomes for these
opportunities.
Provider committees may routinely receive reports on survey activity for consideration, and ensure
appropriate action is taken and feedback loops are closed.
Survey reporting can invite comments on:
¡ participation rates
¡ results improvements or downward trends
¡ performance against benchmarks and internal performance indicators
¡ recommendations or actions for enhancement
¡ details on how feedback loops will be closed.
Timely reporting from surveys ensures actions are taken as quickly as possible. Internal and external
surveys can be monitored and evaluated in the same way. The analysis of these surveys allows for
greater student involvement and good practice may be to consider the results with student focus groups
to be able to gain a greater understanding of their meaning.

Reflective question

¡ How do you involve students in the monitoring and evaluation of courses?

Communicating outcomes to staff, students and stakeholders

(Guiding principle 6)
When providers have analysed the data, they have in place appropriate mechanisms for sharing,
discussing and interpreting findings. This may include periodic internal meetings and specific evaluation
workshops. These activities may also facilitate conversation on actions required and in drawing out
learning points for other stakeholders.
Outcomes, including findings and actions taken as a result, are communicated to relevant stakeholders,
for example PSRBs, through formal internal and external processes and/or incorporated into
annual reporting.

Reflective question

¡ How do you communicate the outcomes of monitoring and evaluation to internal

and external stakeholders?

8
 Advisory group

Katie Akerman
Director of Quality and Standards University of Chichester

Simon Bullock
Standards and Frameworks Officer QAA

Lucy Heming
Senior Assistant Registrar Imperial College London

Laura Porter
Deputy Head of Quality Bath Spa University

Cathy Shaw
Quality and Engagement Manager QAA

Dr Andy Smith
Standards and Frameworks Manager QAA

Angela Taylor
Head of Quality Rose Bruford College

Anthony Turjansky
Director of Quality Assurance Edge Hill University

Katy McGowan
Quality Enhancement Manager (Student Engagement) Middlesex University

November 2018
© The Quality Assurance Agency for Higher Education. This material and its content is published
by QAA (registered charity numbers 1062746 and SC037786) on behalf of the UK Standing Committee
for Quality Assessment, in consultation with the higher education sector.
www.qaa.ac.uk/quality-code