0% found this document useful (0 votes)
16 views

UKAIS2024 Rev Final

Uploaded by

Andy Hewitt
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
16 views

UKAIS2024 Rev Final

Uploaded by

Andy Hewitt
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

Teachers' Perceptions of Students' Use of

Generative AI in Summative Assessments at


Higher Education Institutions: An
Exploratory Study
Chekfoung Tan
UCL Centre for Systems Engineering, University College London
[email protected]

Muna M. Alhammad
MIS Department, King Saud University
[email protected]

Marta Stelmaszak
The School of Business, Portland State University
[email protected]

Completed Research

Abstract
The rise of generative AI (GAI) is affecting various sectors, including higher education (HE). In HE,
educators are grappling with students' use of GAI, which might infringe upon academic integrity. Given
the pervasiveness of this technology, particularly through free AI tools, it should be utilised for its
benefits rather than merely blocked. Thus, responsible use of AI in higher education is essential.
However, encouraging students to openly declare their use of AI in summative assessments, for
transparency purposes, has led to fears of negative perception and potential marking down by teachers.
There is currently limited research in this area. Hence, this study aims to explore the boundary of
students using GAI in assessments by gathering views from teachers through an exploratory survey. It
contributes theoretically to responsible AI literature, extending it to HE, and practically by developing
guidelines for AI use in student assessments, potentially informing university policy.

Keywords: Generative AI, Exploratory Research, Responsible AI, Teachers’


Perception, Higher Education

Acknowledgement: This work is inspired by the outcome of the UCL AI


ChangeMakers project back in the summer of 2023. We would like to thank UCL Arena
Centre for the opportunity, as well as the student co-creators, Liam Perez and Kartik
Gupta.
1.0 Introduction
The term “Artificial Intelligence” (AI) was officially coined in 1956 by a group of
computer scientists during the Dartmouth Summer Research Project on Artificial
Intelligence (DSRPAI) at Dartmouth College in New Hampshire (Haenlein & Kaplan,
2019). AI is seen as a system capable of interpreting external data accurately, learning
from this data, and using these learnings to achieve specific goals and tasks through
flexible adaptation (Haenlein & Kaplan, 2019).
The inception of AI use in teaching and learning can be traced back to 1924 when
Sidney Pressey used a machine to assist students in finding correct answers to multiple-
choice questions (Namatherdhala et al., 2022). Since then, AI has been adopted to
personalise learning for students, tailoring to their learning goals and preferences based
on performance, demographics, and behavioural information (Kaplan-Rakowski et al.,
2023). However, OpenAI shocked various sectors in year 2022, including Higher
Education (HE), with the release of ChatGPT, a chatbot driven by Generative AI (GAI)
(OpenAI, 2022). GAI, trained on large language models (LLMs), can generate human-
like text based on given prompts or contexts and is capable of performing natural
language processing tasks such as text completion, conversation generation, and
language translation (Baidoo-Anu & Ansah, 2023). The immediate concern within HE
is that students might use this tool for writing assignments, coursework, or even
answering open-book exams, potentially breaching academic integrity (Michel-
Villarreal et al., 2023). However, a recent report from UNESCO (2023), suggests that
the use of GAI tools is not entirely negative if used responsibly.
Nevertheless, since the emergence of GAI, there has been growing research on its
adoption and how teachers or students perceive it in HE, as seen in Chan and Hu (2023)
and Amani et al. (2023). However, limited research exists on teachers' perceptions of
students using GAI in their summative assessments, such as written coursework, essays,
online open-book exams, presentations, and media productions (e.g., videos, films,
animations). While universities have issued some guidance on declaring the use of GAI
tools, students might be reluctant to disclose their use of GAI if they fear negative
perceptions from teachers. This research aims to address this research question - How
do teachers perceive students' use of GAI in summative assessments at HE institutions?
This paper is organised as follows. Section 2 covers related work on the application of
GAI in HE, incorporating a responsible lens. Section 3 illustrates the research
methodology, guided by an exploratory survey, and Section 4 presents the results. The
paper concludes with discussions, research implications and future work in Section 5.

2.0 Related Work


2.1 GAI in Higher Education
The rise of GAI has been met with intense interest in HE. Universities were quick to
react and develop guidelines for the use of GAI, initiated working groups, and met in
cross-organisational fora to discuss the impact of this technology. Multiple practitioner
publications highlighted the opportunities and challenges of GAI for the sector (e.g.,
Hodges & Ocak, 2023; Schroeder, 2023).
Research soon followed. Existing papers focus on understanding the potential uses of
GAI in HE and conceptualising the challenges. For example, Michel-Villarreal et al.
(2023) outline that GAI can be used to generate answers to questions and ideas for
essays, provide feedback, simulate a tutor. Research into students’ perspectives shows
a recognition of the potential of GAI in personalised learning support, writing,
brainstorming, and research and analysis (Chan & Hu, 2023). From a teacher’s
perspective, it can create lesson plans, develop resources, and even assess written work
(Baidoo-Anu & Ansah, 2023). For example, Kim et al. (2019) show that ChatGPT
trained on human-graded essays could grade high school student essays with a
correlation of 0.86 with human graders. The UNESCO (2023) report outlines ten roles
illustrating how ChatGPT could be useful in the teaching and learning process.
However, less research interest so far has been given to administrators’ perspectives,
and yet GAI has been found to be useful in administrative support with repetitive or
tedious administrative tasks (Chan & Hu, 2023), creating manuals, or developing policy
documents (Yeralan & Lee, 2023). Yet, the use of GAI in HE is a contentious topic,
with multiple responsibility and ethics implications.

2.2 Responsible Use of AI in HE


The growth in AI in general and GAI in particular has been accompanied by increased
interest in responsible AI, that is a growing consensus that the use of AI should follow
principles consistent with user expectations, organisational values, and societal laws
and norms (Mikalef et al., 2022). Table 1 distils these dimensions into the principles of
responsible AI in HE. Responsible AI is often discussed along the principles of fairness,
transparency, and accountability. For example, the principle of fairness as applied to
HE suggests that the use of AI in this sector should enable inclusion and diversity and
not lead to discriminatory outcomes between students. Transparency suggests that the
use of AI in HE should be openly communicated and facilitate traceability.
Accountability entails using AI in line with the set policies and regulations.
Responsible GAI comes into particular focus in HE in discussions around assessment.
While GAI can be used by students in assessed work to help generate ideas, conduct
research, or improve writing (Smolansky et al., 2023), it can also be deployed in ways
that may go against the principles of responsible AI in HE. GAI can be potentially used
by students to submit work prepared by GAI rather than themselves and not disclose it,
which goes against the principle of transparent use. The use of GAI in assessment may
reduce students’ accountability for their own intellectual work and propagate
plagiarism against university policies. Fairness may be at stake as well, as deploying
GAI in assessment may lead to obtaining better marks. Both students and teachers admit
that GAI has an impact on a range of assessment types, from short answer questions,
through essays, to creative work and presentations (Smolansky et al., 2023).
By acknowledging assessments may particularly be impacted by GAI, attempts have
been made to minimise this by implementing the responsible AI principles. For
example, researchers investigate how to develop AI-proof forms of assessment
(Rudolph et al., 2023), and how to develop tools capable of identifying the use of GAI
(Lacey & Smith, 2023). Existing research suggests that teachers are concerned about
the use of GAI in assessment and aim to work out alternatives that either preclude such
a possibility or make it easier to detect it (Smolansky et al., 2023). This indicates that
teachers may be against the GAI use by students, but little research has been conducted
in this area.

3.0 Research Methodology


Following Chan and Hu (2023) and Chan and Lee (2023), this study employed the
exploratory study by using an online survey methodology to investigate teachers’
perceptions on students’ use of GAI tools in their summative assessment within HE.
The survey encompassed a set of closed questions and two open-ended questions, with
the goal of acquiring a holistic comprehension of participants' viewpoints.
Responsible AI Use in HE Sources adapted
AI principles

Transparency Must be disclosed, communicated, and (Clarke, 2019; Jobin et al.,


understood by those involved, with clear 2019; Mezgár & Váncza,
oversight principles established. 2022; Mhlanga, 2023;
Noble & Dubljević, 2022)
Justice and Should be unbiased, promoting justice, (Jobin et al., 2019;
fairness diversity, and inclusion while addressing Mhlanga, 2023; Noble &
inequalities in access to education. Dubljević, 2022)
Non- Must prioritise student safety and (Clarke, 2019; Jobin et al.,
maleficence wellbeing, ensuring protection from 2019; Mezgár & Váncza,
potential threats. 2022; Noble & Dubljević,
2022)
Responsibility Must follow accountability and liability (Clarke, 2019; Jobin et al.,
/ rules, align with policies, and have clear 2019; Mhlanga, 2023;
Accountability regulations and consequences for non- Noble & Dubljević, 2022)
compliance.
Professional Should be used purposefully, in (Noble & Dubljević, 2022)
responsibility collaboration with institutional staff, and
maintain dialogue with relevant bodies.
Privacy / Must protect user privacy, provide consent (Jobin et al., 2019;
Consent options, and allow data use control. Mhlanga, 2023; Noble &
Dubljević, 2022)
Beneficence / Should prioritise wellbeing, the common (Chan & Hu, 2023; Clarke,
Promotion of good, and human values, aiming to benefit 2019; Jobin et al., 2019;
Human society, advance civilisation, and uphold Mezgár & Váncza, 2022;
Values / human rights. Noble & Dubljević, 2022)
Perceived
Benefits /
Education
Values
Freedom and Should be deployed to empower all (Clarke, 2019; Jobin et al.,
autonomy / stakeholders, remaining under human 2019; Mezgár & Váncza,
Human control and open to review. HE institutions 2022; Mhlanga, 2023;
control of must educate students on AI's functions, Noble & Dubljević, 2022)
technology / uses, limitations, and ethical
Dignity / considerations, enabling informed choices
Digital in their academic and future endeavours.
Literacy
Trust Stakeholders can trust AI to unlock (Jobin et al., 2019; Mezgár
potential and add significant value to & Váncza, 2022)
pedagogy and teaching.
Sustainability HE institutions should embed AI (Jobin et al., 2019; Noble &
and sustainably without compromising core Dubljević, 2022)
Inclusivity values, using it to foster sustainable
societies and champion inclusive
education, including tailored support for
disabled students and top-tier education to
students in remote learning.
Solidarity AI benefits in teaching and learning must (Jobin et al., 2019)
be equitably shared among all
stakeholders, ensuring no undue advantage
for any group.
Quality AI deployment must be underpinned by (Clarke, 2019)
Assurance rigorous regulations and standards, subject
to regular review.
Data It's vital that data from GAI tools is (Mhlanga, 2023)
Accuracy accurate, and both teachers and learners
should critically assess information and
cross-check with trustworthy sources.
Robustness All stakeholders in AI in HE must ensure (Clarke, 2019)
and Resilience its robust and resilient, with responsibility
proportional to benefits, data sensitivity,
and potential risks in education.
General Should evaluate the positive and negative (Clarke, 2019)
Impact consequences and implications of
employing AI.
Human- AI systems should be customised for (Mezgár & Váncza, 2022)
centred students and teachers, involving them in
Design development, regularly gathering their
feedback, and adapting to their academic
experiences.
Continuous AI systems must continually learn and (Chan & Hu, 2023)
Learning and adapt, adjusting to student feedback,
Adaptability / evolving educational standards, and AI
Learning advancements.
Skills
Ethics AI research tools must uphold research (Amani et al., 2023)
ethics, including minimising harm,
ensuring informed consent, and
maintaining data confidentiality.
Table 1. Responsible AI principles in HE

A convenience sampling approach from Edgar and Manz (2017) was adopted to enlist
participants for the study. The survey link was distributed to teachers or teaching
academics in HE institutions across diverse international social media groups.
Participants were presented with an informed consent form on the online platform,
ensuring their awareness of the study's objectives and their rights as participants. The
survey questions were adapted from the roles of how GAI tools could be applied in HE
issued by UNESCO (2023) and responsible AI principles in Table 1. A five-point Likert
scale (1-Strongly Disagree; 2-Disagree; 3-Neutral; 4-Agree; 5-Strongly Agree) was
used for these questions. The collected data underwent rigorous descriptive analysis,
examining the quantitative responses to closed questions. Thematic analysis was
employed for analysing the data collected from the open-ended questions. Section 4
discusses the results.
4.0 Results
4.1 Demographics
Table 2 shows the full demographic information. Through the survey, 66 responses
were collected, with most respondents falling into the 40-49 age group (39.4%) and
being male (60.6%). Most respondents teach in the STEM field (56.1%) and have more
than ten years of teaching experience (57.6%). Geographically, most respondents
primarily teach in the United Kingdom (50.0%), followed by Saudi Arabia (16.7%).
Other countries represented include the United States of America, the United Arab
Emirates, Malaysia, China, Singapore, Ireland, and Portugal.

Characteristics Count (n) Percentage (%)


Age Group
20 and below 1 1.5%
30-39 19 28.8%
40-49 26 39.4%
50-59 14 21.2%
60-69 5 7.6%
70 and older 1 1.5%
Gender
Female 26 39.4%
Male 40 60.6%
Teaching Domain
STEM 37 56.1%
Non-STEM 29 43.9%
Level of teaching experience
Less than two years 2 3.0%
Two to five years 9 13.6%
Five to ten years 17 25.8%
More than ten years 38 57.6%
Country where the participant primarily teaches
China 3 4.5%
Ireland 1 1.5%
Malaysia 4 6.1%
Portugal 1 1.5%
Saudi Arabia 11 16.7%
Singapore 2 3.0%
United Arab Emirates 6 9.1%
United Kingdom 33 50.0%
United States of America 5 7.6%
Table 2. Demographic Information
4.2 Familiarity with the GAI Tools and Frequency of Use
As shown in Table 3, most respondents have a moderate familiarity with GAI tools
(42%), and 79% of them have created an account and used GAI tools for either personal
or educational purposes. This indicates a high level of engagement with GAI tools
among the respondents.

Characteristics Count (n) Percentage (%)


Familiarity with GAI tools
Not familiar at all 1 1.5%
Slightly familiar 12 18.2%
Moderately familiar 28 42.4%
Very familiar 21 31.8%
Extremely familiar 4 6.1%
GAI tools account creation for any purpose (either personal or educational)
Yes 55 83.3%
No 11 16.7%
Table 3. Respondents' Familiarity and Engagement with General AI Tools

Based on the 55 respondents who have signed up for a GAI tool account, Table 4 shows
that the majority have used GAI tools for a period ranging from 1 to 6 months, with the
highest percentages observed at 4 and 6 months, both at 14.5%. Adoption appears to
decrease as the duration increases, with only 3.6% of respondents using GAI tools for
more than 12 months. These findings suggest that most respondents are relatively new
to using GAI tools, with a significant drop in usage beyond 6 months. This may be also
because the most popular GAI tools have only emerged approximately 11 to 12 months
ago, thereby providing a limited timeframe for the respondents to integrate these tools
into their daily activities.
According to Table 5, ChatGPT is the most frequently used tool, with 40.7% of
respondents using it once a week, 24.1% using it twice a week, 7.4% using it three times
a week, and 25.9% using it more than three times a week. Only 1.9% of respondents
never use ChatGPT. Google Bard, Bing Chat, and Microsoft 365 Copilot are used much
less frequently, with more than 70% of respondents never using these tools Snapchat
AI also has a high percentage of respondents who never use it (88.6%), with small
percentages using it at varying frequencies.
Months of using GAI tools Count (n) Percentage (%)
1 month 6 10.9%
2 months 5 9.1%
3 months 7 12.7%
4 months 8 14.5%
5 months 6 10.9%
6 months 8 14.5%
7 months 5 9.1%
8 months 1 1.8%
9 months 1 1.8%
10 months 5 9.1%
12 months 1 1.8%
11 months 0 0.0%
More than 12 months 2 3.6%
Table 4. Months of Using GAI Tools

The 'Others' category has a more even distribution across different frequencies, totalling
18.9%. The GAI tools included in this category are GrammarlyGo, Ernie Bot,
Wordtune, Perplexity, Midjourney, Gamma, Notion, Hypotenuse AI, Writesonic,
Invideo.io, and customised GAI tools utilising the GPT-4 API.

GAI tools Frequency Total


Never Once a Twice Three More
week a week times than
a three
week times a
week
ChatGPT Count (n) 1 22 13 4 14 54
Percentage (%) 1.9% 40.7% 24.1% 7.4% 25.9% 100.0%
Google Count (n) 34 8 2 2 2 48
Bard Percentage (%) 70.8% 16.7% 4.2% 4.2% 4.2% 100.0%
Bing Chat Count (n) 34 6 3 2 3 48
Percentage (%) 70.8% 12.5% 6.3% 4.2% 6.3% 100.0%
Microsoft Count (n) 41 2 1 0 1 45
365 Percentage (%) 91.1% 4.4% 2.2% 0.0% 2.2% 100.0%
Copilot
Snapchat Count (n) 39 3 1 1 0 44
AI Percentage (%) 88.6% 6.8% 2.3% 2.3% 0.0% 100.0%
Others Count (n) 30 1 3 0 3 37
Percentage (%) 81.1% 2.7% 8.1% 0.0% 8.1% 100.0%
Table 5. Frequency of Usage of Various GAI Tools
4.3 Teachers' Perceptions of Students' Use of Generative AI in Their Summative
Assessments

4.3.1 Roles of GAI Tools in Students' Summative Assessments

Based on the findings as shown in Table 6, the respondents generally have a positive
perception of students using GAI tools in their summative assessments, as evidenced
by the median scores of 4 for all statements. This suggests that most respondents are
open to, or accepting of, the idea of students leveraging GAI tools for various aspects
of their assessments. However, the mean scores reveal some differences in the level of
acceptance across different uses of GAI tools.

No Statement Median Mean Standard


Deviation
1 I can accept students using GAI tools to write 4.00 3.32 1.10
queries and examine alternative responses for
their assessments.
2 I can accept students entering prompts into 4.00 3.29 1.08
various GAI tools, following the structure of a
conversation or debate, to produce critical
arguments for their assessments.
3 I can accept that working in groups, students use 4.00 3.58 1.10
any GAI tools to find out information to
complete tasks and assignments.
4 I can accept that students use various GAI tools 4.00 3.44 1.22
to provide personalised feedback to them, based
on information provided by students or teachers
(e.g., formative feedback).
5 I can accept that students explain their current 4.00 3.65 1.10
level of understanding related to the assessments
to various GAI tools, and ask for ways to help
them study the related material and prepare for
other tasks in the assessment.
6 I can accept that students ask various GAI tools 4.00 3.76 1.08
for ideas about how to extend their learning after
receiving the scores for their assessment.
7 I can accept that students interact with various 4.00 3.56 1.12
GAI tools in a tutorial-type dialogue, and then
ask the tool to produce a summary of their
current state of knowledge for their assessment.
8 I can accept that students ask various GAI tools 4.00 3.79 0.98
to proofread the language of their assessment.
Table 6. Roles of GAI Tools
Respondents are most accepting of students using GAI tools for proofreading the
language of their assessments (Statement 8, Mean: 3.79) and seeking ideas for
extending their learning post-assessment (Statement 6, Mean: 3.76). This indicates a
recognition of the value of GAI tools as supportive resources for enhancing learning
experiences and improving work quality. However, respondents are less accepting of
students using GAI tools to generate critical arguments for their assessments (Statement
2, Mean, 3.29). This suggests a preference for students to develop their critical thinking
and produce arguments independently, rather than relying on GAI tools for such
complex cognitive tasks. Statements 3 and 7 both have Mean scores above 3.5, but
Standard Deviations above 1.10, indicating a moderate level of acceptance but with
some variability in responses.

4.3.2 Responsible Use of GAI Tools

Table 7 presents the results pertaining to students' responsible use of GAI tools in their
summative assessments from various perspectives. Most respondents express a general
acceptance towards the responsible utilisation of GAI tools by students in their
summative assessments. This is reflected in the median values of 4.00 for Statements
1, 2, 3, 4, 6, 7, 8, 9, 11, 13, 14, 15, 17, 18, and 20, indicating agreement with the
statements. The Mean values for these statements range from 3.45 to 3.97, further
cementing this stance.

No Statement Median Mean Standard


Deviation
1 I can accept students using GAI tools to support 4.00 3.79 1.14
the production of their assessments, such as
proofreading and idea creation, as long as they
declare the use of it, but they must not use it to
produce or write the entire assessment, such as
content generation.
2 If I refrain from judging their use of GAI tools, 4.00 3.71 0.91
my students will likely feel more at ease both
using it and declaring its use.

3 If students are using GAI tools ethically, it will 4.00 3.83 1.10
not affect the way I mark their assessment.
4 I can accept students’ use of GAI tools in their 4.00 3.68 1.15
assessments as long as they are aware of the tools
are not substitute for human tutors.

5 I can trust my students to use GAI tools in their 2.00 2.64 1.05
assessments responsibly and ethically.
6 I can accept students citing information from GAI 4.00 3.52 1.11
tools in their assessment by justifying its
relevance and confirming the accuracy of the
information by comparing it with other sources or
using personal judgement
7 I can accept that students use GAI tools, 4.00 3.53 1.06
employing various plug-ins or functions, for idea
creation when producing their assessments.

8 I believe that all students should be given access 4.00 3.59 1.20
to the same GAI tools for their assessments to
ensure inclusivity and equity.

9 I believe that students' use of GAI tools in their 4.00 3.45 1.10
assessments will help them learn in an efficient
way, as the tools can provide preliminary
feedback.
10 I think GAI tools is a great tool for supporting 3.00 2.98 1.22
students’ assessment due to anonymity.
11 I believe that GAI tools could contribute to 4.00 3.70 1.16
academic dishonesty behaviours.
12 I believe that using GAI tools to complete 3.00 3.00 1.15
assignments undermines the value of university
education.
13 GAI tools may reduce students' opportunities to 4.00 3.41 1.08
interact with peers and socialise during
coursework completion.
14 GAI tools may impede students' cultivation of 4.00 3.44 1.08
generic or transferable skills, including
teamwork, problem-solving, and leadership skills.

15 There's a risk that students might become overly 4.00 3.97 1.12
dependent on GAI tools.
16 I believe that using GAI tools, to write essays or 3.00 2.89 1.10
generate answers can enhance originality and
creativity in students' work.
17 I believe that GAI tools can bolster students' 4.00 3.62 1.00
digital competence.
18 I believe that GAI tools can help students save 4.00 3.67 1.09
time.
19 I think that GAI tools can help students in 3.00 3.23 1.17
becoming better writers.
20 In the long run, the integration of GAI tools in 4.00 3.68 1.03
higher education is likely to have a positive
impact on teaching and learning, influencing how
students are taught and assessed.

Table 7. Responsible Use of GAI Tools


As per Statement 3, respondents generally concur that if students employ GAI tools
responsibly, it will not impinge the way they mark assessments (Mean: 3.83, SD: 1.10).
Statements 1 (Mean: 3.79; SD: 1.14), 4 (Mean: 3.68; SD: 1.15), 6 (Mean: 3.52; SD:
1.11), and 7 (Mean: 3.53; SD: 1.06) imply a conditional acceptance of students using
GAI tools in their summative assessments for tasks such as proofreading, idea
generation, acknowledging that GAI tools are not substitutes for human judgement, and
critically evaluating the information generated by GAI tools.
A significant concern from the respondents is the lack of trust in students to use GAI
tools responsibly, as shown in Statement 5 (Median: 2.00; Mean: 2.64), although the
Standard Deviation indicates a moderate range of opinions among respondents on this
issue. There are concerns about the potential for academic dishonesty, as indicated in
Statement 11, although the Standard Deviation suggests moderate agreement among
respondents on this matter (Mean: 3.76; SD: 1.16). Similarly, Statement 15 shows that
respondents are concerned about students becoming overly dependent on GAI tools
(Mean: 3.97; SD: 1.12). The role of anonymity in assessments, as in Statement 10, is a
contentious issue (Median: 3.00; Mean: 2.98, SD: 1.22), and the highest SD indicates a
diverse range of respondents’ opinions.
Despite these concerns, respondents see potential benefits in GAI tools, such as
enhancing originality and creativity, as indicated in Statement 16 (Median: 3.00, Mean:
2.89, SD: 1.10); bolstering digital competence, as in Statement 17 (Median: 4.00, Mean:
3.62, SD: 1.00); and saving time, as in Statement 18 (Median: 4.00, Mean: 3.67, SD:
1.09). In the long run, as indicated in Statement 20, respondents believe that the
integration of GAI tools in higher education is likely to have a positive impact on
teaching and learning (Median: 4.00; Mean: 3.68, SD: 1.03).

4.4 Governance of the Use of GAI Tools in Higher Education


Based on the open-ended questions where respondents were asked for additional
comments about their perception of students’ use of GAI tools in their summative
assessments, they acknowledged the inevitability of students utilising GAI tools. Table
8 presents the results of a thematic analysis on the governance of GAI tools in HE
institutions, which include the codes clustered into four main themes (Guidelines for
Responsible Use of GAI Tools, Regulations, Assessment and Academic Integrity,
Training and Education) along with the relevant sample quotes from the respondents.
Themes Codes Sample Quotes

Guidelines HE Institutions Respondent 52 - "The HEI bodies should introduce, not only Russel group, which has produced a guidelines, but
for Guideline other at national or even international level should have the ethical guidelines, and acceptable use of GAI tools in
Responsible HEIs."
Use of GAI Transparency Respondent 12 - "A full transcript of interaction between student and AI plus a summary of used inputs should be
Tools submitted with any thesis."
Respondent 19 - "Student must provide a log, with timestamps when they use GAI for particular assignments"
Respondent 40 - "Ensure transparency in AI systems and algorithms, including disclosing the sources of data, the
methods used, and potential biases."
Accountability Respondent 32 - "...student takes ownership of their work and use GAI as a supplementary method to enhance to
quality of their work."
Respondent 23 - "Lecturers should be in control and aware of the sentence pattern created using GAI."
Respondent 40 - "Hold individuals and departments accountable for their use of AI."
Bias Mitigation Respondent 40 - "Address bias in AI algorithms and data to ensure fairness and inclusivity."
Data Privacy and Respondent 40 - "Implement strong data privacy and security measures to protect sensitive information, ensuring
Security compliance with relevant data protection laws and regulations such as GDPR."
Acceptable Use Respondent 45 - " The content generated should be treated as a framework/ broad guideline as against treating it
as the final product."
Respondent 52 - "Students should learn about prompt engineering to better use GAI tools"
Respondent 62 - "Use it wisely, especially in the process of idea generation would be good."
Respondent 64 - "Some students may find AI great for supporting their learning (e.g. proofreading or for
systematising information)."
Regulations Academic Respondent 21 - "Academic regulations need to be updated."
Regulations
Legal Compliance Respondent 40 - "Ensure compliance with all relevant laws and regulations related to AI, including intellectual
property, data protection, and non-discrimination laws."
Review Respondent 40 - "Regularly review and update AI governance policies to adapt to evolving technologies, societal
norms, and emerging ethical considerations."
Assessment Adjusting Respondent 64 - "Institutions need to start/continue debating the use of AI but I still think we should have one last
and Assessments point of assessment where students don't have access to any AI tool... students should be able to demonstrate part
Academic of their learning path without the help of these tools."
Integrity
Plagiarism Prevention Respondent 56 - "It can be integrated with Turnitin to identify plagiarism."
AI Referencing Respondent 32 - "Made mandatory for the students to acknowledge the use of GAI in their assignments and
highlight the aspects/elements of the assignments supported by GAI"
Formative Respondent 63 - "GAI should be widely used for formative assessment rather than summative assessment."
Assessment
Summative Respondent 26 - "For some assessments, maybe it will be necessary to conduct them as invigilated exams again."
Assessment
Training and Skill Development Respondent 17 - "...it also produce the wrong outcome, if they do not know how to ask. So, they at least need to
Education understand the topic of what they are learning, and being able to assess if or not the generated contents are
correct or not...to do that, it requires the deep knowledge of the topic and it is why the education is still important
for us. "
Teacher's Role Respondent 6 - "Teachers must admit the change and do not force the students to lie. Faculties have to be more
open for the change."
Staff Training Respondent 21 - "Universities need to provide clear leadership and guidance to staff, including training in the use
of GAI."
Awareness Respondent 40 - "Educate stakeholders, including researchers, faculty, students, and administrators, about AI
principles, potential risks, and ethical considerations to promote responsible AI use."
Digital Competence Respondent 41 - "Higher educational institutions should provide a curated GPT service to enhance the digital
competence of staff and students."
Preparing Students for Respondent 26 - "In HE, we will have to teach the use of GAI tools as preparation for the workplace."
the Future Workplace

Table 9. Thematic Analysis Results


The Guidelines for Responsible Use of GAI Tools theme in HE encompasses several
principles, as explained by the codes. Transparency in AI systems and algorithms is
crucial to ensure accountability and ethical use. Students should be encouraged to use
GAI as a supplementary method, taking ownership of their work, while faculty should
maintain control and awareness of GAI-generated content. Bias mitigation in AI
algorithms and data is essential to uphold fairness and inclusivity. Furthermore, robust
data privacy and security measures must be in place to protect sensitive information.
There should be a standardised HE Institution Guideline across the sector on the use of
GAI tools. Acceptable uses generally include proofreading and idea generation.
Regarding the Regulations theme, it is imperative that academic regulations are
updated and adapted to accommodate the use of GAI tools within HE institutions. This
requires a comprehensive review of existing policies, alongside the introduction of new
regulations specifically addressing the unique challenges and opportunities presented
by GAI technology. Ensuring legal compliance with all relevant laws and regulations
related to AI is a critical aspect of this process. Such measures will ensure that
institutions can effectively manage and mitigate any risks associated with the use of
GAI tools, while also maximising the potential benefits for both students and teachers.
The Assessment and Academic Integrity theme reveals the importance of adjusting
assessment methods to seamlessly integrate GAI tools. This adaptation is crucial to
navigate the evolving landscape of academic integrity in the age of AI, particularly
concerning AI referencing and citation practices. Incorporating AI text detectors could
serve as a valuable component of plagiarism prevention strategies. GAI tools can be
effectively employed for formative assessments, providing students with continuous
and constructive feedback from tutors or lecturers. However, to uphold the academic
integrity of summative assessments, it is advisable to conduct these evaluations without
the reliance on AI tools, such as through invigilated exams. This approach ensures a
comprehensive and fair assessment of a student's knowledge and capabilities.
Under the theme of Training and Education, as part of skills development, it is crucial
to equip students with the necessary skills to effectively utilise GAI tools. This includes
the ability to validate information obtained from these tools, which is a critical thinking
skill in itself. Currently, there is a gap in students' proficiency in employing GAI tools
effectively. Teachers play an integral role in guiding students on the appropriate use of
GAI tools. In tandem, HE institutions should also provide staff training on the
utilisation of GAI tools, as well as offering guidance on how to incorporate these tools
into teaching and learning processes. Moreover, it is important to raise awareness and
educate all stakeholders about the principles of AI, potential risks, and ethical
considerations associated with its use. Enhancing the digital competence of both staff
and students is imperative, as is teaching the use of GAI tools as a means of preparing
students for the future workplace.
Echoing the suggestions proposed by respondent 28 and 32, in governing the use of
GAI tools in HE institutions, it might be useful to model the 'holistic life cycle of AI use
in academic settings'. HE institutions should play a proactive role in adapting academic
practices by providing guidance to students on how to select a reliable tool, how to use
them, how to apply critical thinking when analysing the AI output, and how AI has
helped them in achieving the learning outcomes.

5.0 Discussions and Conclusion


5.1 Research Implications
This research explores the teachers’ perceptions of students’ use of GAI tools in their
summative assessments. The findings indicate that teachers are inclined to accept the
use of GAI tools in summative assessment by students, as long as that students utilise
such tools responsibly. The findings from sections 4.3 and 4.4 significantly contribute
to informing HE policy regarding the responsible use of GAI tools. Through these
results, the research unveils two pivotal concepts – responsibility and trust – both of
which are integral in shaping future HE policies.
This research recognises that, amid the current paucity of guidelines on the responsible
use of GAI tools in summative assessments, it is challenging to clearly define
responsible usage and subsequently hold students accountable. These two elements are
essential to establish responsibility: without clear duties and obligations, agents (the
students) cannot be held accountable for their actions (McGrath & Whitty, 2018).
Similarly, without clear consequences for irresponsible actions, accountability is
unattainable (Dunn et al., 2021). Therefore, a responsible use of GAI tools in HE
requires the development of clear guidelines and policies that set standards for fairness,
transparency, and acceptable use, as well as the implementation of penalties for
irresponsible usage. However, as responsibility is an evolving and dynamic concept
(Rowe et al., 2023), such ramifications cannot simply be imposed on students, teachers,
or administrators. Rather, they should be co-created by all stakeholders. The findings
reveal a wide range of perceptions among teachers, and similarly, diverse views among
other involved parties could be stipulated. Therefore, responsible use of GAI tools in
HE is not a straightforward concept, but rather depends on a multitude of perspectives
that need to be negotiated as stakeholders arrive at shared notions of responsibility.
Secondly, this study contributes to the discourse of trust. Even with clear guidelines
and policies for the responsible use of GAI tools, the issue of trust remains. Namely,
teachers in this study implied that they did not trust students to use GAI tools
responsibly. This raises an important question regarding the role of trust in using GAI
tools responsibly. As trust is usually founded on non-codified notions, such as norms,
habits, culture, and expectations (Faulkner, 2010), building trust differs from
developing and implementing rules and policies. With the novelty of GAI tools and its
potential significant impact on HE, a lack of trust is evident. This lack of trust is further
exacerbated by the fact that GAI tools evolve more rapidly than the ramifications for
responsible usage. This situation presents opportunities for research and practice to
develop sufficiently elastic and dynamic methods to foster the necessary trust to support
responsibility in GAI tools usage. Such attempts require further investigation into
teachers' perceptions and, specifically, into the ways in which trustworthiness can be
increased.

5.2 Limitations and Future Work


One limitation of this study is the limited number of respondents who participated in
the survey, despite the exploratory nature of the research. In future research, more
participants will be recruited to ensure a comprehensive representation of the
demographic variations in teachers' perceptions of the responsible use of GAI tools in
HE. This includes aspects such as familiarity with and proficiency in using these tools.
Moreover, given the rapid advancements in GAI technology, it is crucial to consider
the potential risks associated with its usage. The survey could be expanded to address
psychological risks, for instance. One potential concern is that the use of GAI tools
could lead to a detachment from reality, which may be challenging to detect and
mitigate.
Future research will delve deeper into the concepts of responsibility and trust,
particularly from a sociotechnical perspective. This could inform the development of
hypotheses that can be tested to enrich understanding of these complex and multifaceted
concepts. By increasing the sample size, more analyses can be conducted, comparing
teachers' perceptions across a range of demographics such as age, gender, and teaching
domain. Moreover, from a cross-cultural perspective, further research could investigate
how cultural differences affect teachers' expectations regarding students' learning and
behaviours. This could shed light on the ways in which cultural norms and values shape
teachers' perceptions of responsibility and trust in the context of GAI tool usage. Such
insights would be invaluable in developing culturally sensitive guidelines and policies
for the responsible use of GAI tools in HE.
References
Amani, S., White, L., Balart, T., Arora, L., Shryock, D. K. J., Brumbelow, D. K., &
Watson, D. K. L. (2023). Generative AI Perceptions: A Survey to Measure the
Perceptions of Faculty, Staff, and Students on Generative AI Tools in
Academia. arXiv preprint arXiv:2304.14415.
Baidoo-Anu, D., & Ansah, L. O. (2023). Education in the era of generative artificial
intelligence (AI): Understanding the potential benefits of ChatGPT in
promoting teaching and learning. Journal of AI, 7(1), 52-62.
Chan, C. K. Y., & Hu, W. (2023). Students' Voices on Generative AI: Perceptions,
Benefits, and Challenges in Higher Education. arXiv preprint
arXiv:2305.00290.
Chan, C. K. Y., & Lee, K. K. (2023). The AI generation gap: Are Gen Z students more
interested in adopting generative AI such as ChatGPT in teaching and learning
than their Gen X and Millennial Generation teachers? arXiv preprint
arXiv:2305.02878.
Clarke, R. (2019). Principles and business processes for responsible AI. Computer Law
& Security Review, 35(4), 410-422.
Dunn, B., Jensen, M. L., & Ralston, R. (2021). Attribution of responsibility after
failures within platform ecosystems. Journal of Management Information
Systems, 38(2), 546-570.
Faulkner, P. (2010). Norms of trust. OUP Oxford.
Haenlein, M., & Kaplan, A. (2019). A brief history of artificial intelligence: On the
past, present, and future of artificial intelligence. California Management
Review, 61(4), 5-14.
Hodges, C., & Ocak, C. (2023). Integrating Generative AI into Higher Education:
Considerations. EDUCAUSE Review. Retrieved 1 September 2023 from
https://er.educause.edu/articles/2023/8/integrating-generative-ai-into-higher-
education-considerations
Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines.
Nature Machine Intelligence, 1(9), 389-399.
Kaplan-Rakowski, R., Grotewold, K., Hartwick, P., & Papin, K. (2023). Generative AI
and Teachers’ Perspectives on Its Implementation in Education. Journal of
Interactive Learning Research, 34(2), 313-338.
Kim, S., Park, J., & Lee, H. (2019). Automated essay scoring using a deep learning
model. Journal of Educational Technology Development and Exchange, 2(1),
1-17.
Lacey, M. M., & Smith, D. P. (2023). Teaching and assessment of the future today:
higher education and AI. Microbiology Australia, 44(3), 124-126.
McGrath, S. K., & Whitty, S. J. (2018). Accountability and responsibility defined.
International Journal of Managing Projects in Business, 11(3), 687-707.
Mezgár, I., & Váncza, J. (2022). From ethics to standards–A path via responsible AI to
cyber-physical production systems. Annual Reviews in Control.
Mhlanga, D. (2023). Open AI in education, the responsible and ethical use of ChatGPT
towards lifelong learning. Education, the Responsible and Ethical Use of
ChatGPT Towards Lifelong Learning (February 11, 2023).
Michel-Villarreal, R., Vilalta-Perdomo, E., Salinas-Navarro, D. E., Thierry-Aguilera,
R., & Gerardou, F. S. (2023). Challenges and Opportunities of Generative AI
for Higher Education as Explained by ChatGPT. Education Sciences, 13(9),
856.
Mikalef, P., Conboy, K., Lundström, J. E., & Popovič, A. (2022). Thinking responsibly
about responsible AI and ‘the dark side’of AI. In (Vol. 31, pp. 257-268): Taylor
& Francis.
Namatherdhala, B., Mazher, N., & Sriram, G. K. (2022). A comprehensive overview of
artificial intelligence tends in education. International Research Journal of
Modernization in Engineering Technology and Science, 4(7).
Noble, S. M., & Dubljević, V. (2022). Ethics of AI in organizations. In Human-
Centered Artificial Intelligence (pp. 221-239). Elsevier.
OpenAI. (2022). Introducing ChatGPT. Retrieved 01 September 2023 from
https://openai.com/blog/chatgpt
Rowe, F., Jeanneret Medina, M., Journé, B., Coetard, E., & Myers, M. D. (2023).
Understanding responsibility under uncertainty: A critical and scoping review
of autonomous driving systems. Journal of Information Technology,
02683962231207108.
Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of
traditional assessments in higher education? Journal of Applied Learning and
Teaching, 6(1).
Schroeder, R. (2023). Generative AI in College and Departmental Administration.
Inside Higher Ed (IHE). Retrieved 1 September 2023 from
https://www.insidehighered.com/opinion/blogs/online-trending-
now/2023/08/02/generative-ai-college-and-departmental-administration
Smolansky, A., Cram, A., Raduescu, C., Zeivots, S., Huber, E., & Kizilcec, R. F.
(2023). Educator and Student Perspectives on the Impact of Generative AI on
Assessments in Higher Education. Proceedings of the Tenth ACM Conference
on Learning@ Scale,
UNESCO. (2023). ChatGPT and Artificial Intelligence in Higher Education.
https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-and-
Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf
Yeralan, S., & Lee, L. A. (2023). Generative AI: Challenges to higher education.
Sustainable Engineering and Innovation, 5(2), 107-116.

You might also like