UKAIS2024 Rev Final
UKAIS2024 Rev Final
Muna M. Alhammad
MIS Department, King Saud University
[email protected]
Marta Stelmaszak
The School of Business, Portland State University
[email protected]
Completed Research
Abstract
The rise of generative AI (GAI) is affecting various sectors, including higher education (HE). In HE,
educators are grappling with students' use of GAI, which might infringe upon academic integrity. Given
the pervasiveness of this technology, particularly through free AI tools, it should be utilised for its
benefits rather than merely blocked. Thus, responsible use of AI in higher education is essential.
However, encouraging students to openly declare their use of AI in summative assessments, for
transparency purposes, has led to fears of negative perception and potential marking down by teachers.
There is currently limited research in this area. Hence, this study aims to explore the boundary of
students using GAI in assessments by gathering views from teachers through an exploratory survey. It
contributes theoretically to responsible AI literature, extending it to HE, and practically by developing
guidelines for AI use in student assessments, potentially informing university policy.
A convenience sampling approach from Edgar and Manz (2017) was adopted to enlist
participants for the study. The survey link was distributed to teachers or teaching
academics in HE institutions across diverse international social media groups.
Participants were presented with an informed consent form on the online platform,
ensuring their awareness of the study's objectives and their rights as participants. The
survey questions were adapted from the roles of how GAI tools could be applied in HE
issued by UNESCO (2023) and responsible AI principles in Table 1. A five-point Likert
scale (1-Strongly Disagree; 2-Disagree; 3-Neutral; 4-Agree; 5-Strongly Agree) was
used for these questions. The collected data underwent rigorous descriptive analysis,
examining the quantitative responses to closed questions. Thematic analysis was
employed for analysing the data collected from the open-ended questions. Section 4
discusses the results.
4.0 Results
4.1 Demographics
Table 2 shows the full demographic information. Through the survey, 66 responses
were collected, with most respondents falling into the 40-49 age group (39.4%) and
being male (60.6%). Most respondents teach in the STEM field (56.1%) and have more
than ten years of teaching experience (57.6%). Geographically, most respondents
primarily teach in the United Kingdom (50.0%), followed by Saudi Arabia (16.7%).
Other countries represented include the United States of America, the United Arab
Emirates, Malaysia, China, Singapore, Ireland, and Portugal.
Based on the 55 respondents who have signed up for a GAI tool account, Table 4 shows
that the majority have used GAI tools for a period ranging from 1 to 6 months, with the
highest percentages observed at 4 and 6 months, both at 14.5%. Adoption appears to
decrease as the duration increases, with only 3.6% of respondents using GAI tools for
more than 12 months. These findings suggest that most respondents are relatively new
to using GAI tools, with a significant drop in usage beyond 6 months. This may be also
because the most popular GAI tools have only emerged approximately 11 to 12 months
ago, thereby providing a limited timeframe for the respondents to integrate these tools
into their daily activities.
According to Table 5, ChatGPT is the most frequently used tool, with 40.7% of
respondents using it once a week, 24.1% using it twice a week, 7.4% using it three times
a week, and 25.9% using it more than three times a week. Only 1.9% of respondents
never use ChatGPT. Google Bard, Bing Chat, and Microsoft 365 Copilot are used much
less frequently, with more than 70% of respondents never using these tools Snapchat
AI also has a high percentage of respondents who never use it (88.6%), with small
percentages using it at varying frequencies.
Months of using GAI tools Count (n) Percentage (%)
1 month 6 10.9%
2 months 5 9.1%
3 months 7 12.7%
4 months 8 14.5%
5 months 6 10.9%
6 months 8 14.5%
7 months 5 9.1%
8 months 1 1.8%
9 months 1 1.8%
10 months 5 9.1%
12 months 1 1.8%
11 months 0 0.0%
More than 12 months 2 3.6%
Table 4. Months of Using GAI Tools
The 'Others' category has a more even distribution across different frequencies, totalling
18.9%. The GAI tools included in this category are GrammarlyGo, Ernie Bot,
Wordtune, Perplexity, Midjourney, Gamma, Notion, Hypotenuse AI, Writesonic,
Invideo.io, and customised GAI tools utilising the GPT-4 API.
Based on the findings as shown in Table 6, the respondents generally have a positive
perception of students using GAI tools in their summative assessments, as evidenced
by the median scores of 4 for all statements. This suggests that most respondents are
open to, or accepting of, the idea of students leveraging GAI tools for various aspects
of their assessments. However, the mean scores reveal some differences in the level of
acceptance across different uses of GAI tools.
Table 7 presents the results pertaining to students' responsible use of GAI tools in their
summative assessments from various perspectives. Most respondents express a general
acceptance towards the responsible utilisation of GAI tools by students in their
summative assessments. This is reflected in the median values of 4.00 for Statements
1, 2, 3, 4, 6, 7, 8, 9, 11, 13, 14, 15, 17, 18, and 20, indicating agreement with the
statements. The Mean values for these statements range from 3.45 to 3.97, further
cementing this stance.
3 If students are using GAI tools ethically, it will 4.00 3.83 1.10
not affect the way I mark their assessment.
4 I can accept students’ use of GAI tools in their 4.00 3.68 1.15
assessments as long as they are aware of the tools
are not substitute for human tutors.
5 I can trust my students to use GAI tools in their 2.00 2.64 1.05
assessments responsibly and ethically.
6 I can accept students citing information from GAI 4.00 3.52 1.11
tools in their assessment by justifying its
relevance and confirming the accuracy of the
information by comparing it with other sources or
using personal judgement
7 I can accept that students use GAI tools, 4.00 3.53 1.06
employing various plug-ins or functions, for idea
creation when producing their assessments.
8 I believe that all students should be given access 4.00 3.59 1.20
to the same GAI tools for their assessments to
ensure inclusivity and equity.
9 I believe that students' use of GAI tools in their 4.00 3.45 1.10
assessments will help them learn in an efficient
way, as the tools can provide preliminary
feedback.
10 I think GAI tools is a great tool for supporting 3.00 2.98 1.22
students’ assessment due to anonymity.
11 I believe that GAI tools could contribute to 4.00 3.70 1.16
academic dishonesty behaviours.
12 I believe that using GAI tools to complete 3.00 3.00 1.15
assignments undermines the value of university
education.
13 GAI tools may reduce students' opportunities to 4.00 3.41 1.08
interact with peers and socialise during
coursework completion.
14 GAI tools may impede students' cultivation of 4.00 3.44 1.08
generic or transferable skills, including
teamwork, problem-solving, and leadership skills.
15 There's a risk that students might become overly 4.00 3.97 1.12
dependent on GAI tools.
16 I believe that using GAI tools, to write essays or 3.00 2.89 1.10
generate answers can enhance originality and
creativity in students' work.
17 I believe that GAI tools can bolster students' 4.00 3.62 1.00
digital competence.
18 I believe that GAI tools can help students save 4.00 3.67 1.09
time.
19 I think that GAI tools can help students in 3.00 3.23 1.17
becoming better writers.
20 In the long run, the integration of GAI tools in 4.00 3.68 1.03
higher education is likely to have a positive
impact on teaching and learning, influencing how
students are taught and assessed.
Guidelines HE Institutions Respondent 52 - "The HEI bodies should introduce, not only Russel group, which has produced a guidelines, but
for Guideline other at national or even international level should have the ethical guidelines, and acceptable use of GAI tools in
Responsible HEIs."
Use of GAI Transparency Respondent 12 - "A full transcript of interaction between student and AI plus a summary of used inputs should be
Tools submitted with any thesis."
Respondent 19 - "Student must provide a log, with timestamps when they use GAI for particular assignments"
Respondent 40 - "Ensure transparency in AI systems and algorithms, including disclosing the sources of data, the
methods used, and potential biases."
Accountability Respondent 32 - "...student takes ownership of their work and use GAI as a supplementary method to enhance to
quality of their work."
Respondent 23 - "Lecturers should be in control and aware of the sentence pattern created using GAI."
Respondent 40 - "Hold individuals and departments accountable for their use of AI."
Bias Mitigation Respondent 40 - "Address bias in AI algorithms and data to ensure fairness and inclusivity."
Data Privacy and Respondent 40 - "Implement strong data privacy and security measures to protect sensitive information, ensuring
Security compliance with relevant data protection laws and regulations such as GDPR."
Acceptable Use Respondent 45 - " The content generated should be treated as a framework/ broad guideline as against treating it
as the final product."
Respondent 52 - "Students should learn about prompt engineering to better use GAI tools"
Respondent 62 - "Use it wisely, especially in the process of idea generation would be good."
Respondent 64 - "Some students may find AI great for supporting their learning (e.g. proofreading or for
systematising information)."
Regulations Academic Respondent 21 - "Academic regulations need to be updated."
Regulations
Legal Compliance Respondent 40 - "Ensure compliance with all relevant laws and regulations related to AI, including intellectual
property, data protection, and non-discrimination laws."
Review Respondent 40 - "Regularly review and update AI governance policies to adapt to evolving technologies, societal
norms, and emerging ethical considerations."
Assessment Adjusting Respondent 64 - "Institutions need to start/continue debating the use of AI but I still think we should have one last
and Assessments point of assessment where students don't have access to any AI tool... students should be able to demonstrate part
Academic of their learning path without the help of these tools."
Integrity
Plagiarism Prevention Respondent 56 - "It can be integrated with Turnitin to identify plagiarism."
AI Referencing Respondent 32 - "Made mandatory for the students to acknowledge the use of GAI in their assignments and
highlight the aspects/elements of the assignments supported by GAI"
Formative Respondent 63 - "GAI should be widely used for formative assessment rather than summative assessment."
Assessment
Summative Respondent 26 - "For some assessments, maybe it will be necessary to conduct them as invigilated exams again."
Assessment
Training and Skill Development Respondent 17 - "...it also produce the wrong outcome, if they do not know how to ask. So, they at least need to
Education understand the topic of what they are learning, and being able to assess if or not the generated contents are
correct or not...to do that, it requires the deep knowledge of the topic and it is why the education is still important
for us. "
Teacher's Role Respondent 6 - "Teachers must admit the change and do not force the students to lie. Faculties have to be more
open for the change."
Staff Training Respondent 21 - "Universities need to provide clear leadership and guidance to staff, including training in the use
of GAI."
Awareness Respondent 40 - "Educate stakeholders, including researchers, faculty, students, and administrators, about AI
principles, potential risks, and ethical considerations to promote responsible AI use."
Digital Competence Respondent 41 - "Higher educational institutions should provide a curated GPT service to enhance the digital
competence of staff and students."
Preparing Students for Respondent 26 - "In HE, we will have to teach the use of GAI tools as preparation for the workplace."
the Future Workplace