INTEGRATED PROGRAMME IN MANAGEMENT

RESEARCH METHODOLOGY

IMPACT OF AI TOOL UTILIZATION ON ACADEMIC

PERFORMANCE AMONGST COLLEGE STUDENTS

RESEARCH PROPOSAL

GROUP 6
Group Members:
Ajeet Sharma – 2023IPM017
David Anthony Ekka – 2023IPM043
Madhav Goel – 2023IPM074
Rishiraj Nath – 2023IPM100
Shreyash Agarwalla – 2023IPM118
Sahaj Choudhary – 2023IPM154
RESEARCH OBJECTIVE & IMPORTANCE:

The primary aim of this study is to find the extent to which the usage of AI tools impacts the
academic performance of college students and if the students who leverage these tools efficiently
can get a competitive advantage over their counterparts.

The results of this study offer insights into shaping educational practice and policy. Furthermore,
the study establishes a foundation for future research on the scalability and long-term effects of
AI integration in educational institutions.

LITERATURE REVIEW

The era that we live in is one in which revolution has become a constant, especially when it
comes to the learning frameworks existing in the academic world. It is essential that the learners
of the 21st century adapt to this transition to a world where cutting-edge technologies such as
Artificial Intelligence (AI) have changed the global educational landscape.

There has been an extensive study on the effect that AI has on students’ performance, and the
prevailing motif is that the heightened usage of AI tools, for example, Chatbots and Gen AI
tools, has a significant effect on not only the resultant student academic performance but also on
their motivation and inclination to study. It significantly affects their learning patterns, thus
inducing a change in their study habits. AI also has the ability to cater to the personalized needs
of the students whilst maintaining the pace and learning style required by them. This indubitably
leads to an improvement in the student's academic performance, providing compelling evidence
to conduct this study. (Swargiary, 2024)

(Sasikala & Ravichandran, 2024) conducted a comprehensive study examining the impact of
Artificial Intelligence (AI) on student learning outcomes. Their study focuses on how AI
integrates into education, emphasizing its role in personalizing learning experiences, enhancing
student engagement, and, therefore, improving academic performance. Their study reveals that
AI-powered tools, like adaptive learning platforms and intelligent tutoring systems, significantly
enhance learning and comprehension by tailoring content to individual student needs, leading to
better academic performances.

The integration of such AI technologies holds the potential to personalize learning experiences,
increase student engagement, augment subject understanding, and improve academic outcomes.
(Eltahir & Babiker, 2024) conducted a quasi-experimental study examining the influence of AI-
powered personalized learning tools on pre-service teachers' academic performance and
perceptions by making two groups- an experimental group that was instructed via AI-powered
learning techniques, while another control group received traditional pedagogy. Their findings
revealed significant improvements in academic performance and critical thinking skills among
the students, i.e., pre-service teachers who utilized AI tools compared to those who received
traditional instruction.

Numerous research also explore the transformative and disruptive potential of ChatGPT, a
leading AI-based language model developed by OpenAI with 200+ Million monthly users as of
September 2024. By automating routine tasks and providing intelligent support in both teaching,
research activities, and comprehension, ChatGPT enables educators, researchers, and students to
focus more on critical thinking, creativity, and the advancement of knowledge (Abbas, 2023)

While the aforementioned studies contribute significantly to the understanding of AI's impact on
education, they exhibit some methodological limitations. While (Eltahir & Babiker, 2024)
utilized a quasi-experimental design focusing on pre-service teachers in an e-learning
environment, their study may lack generalizability to the broader college student population due
to potential sample limitations and the absence of controls for individual differences such as
intelligence quotient or socioeconomic status. Analogously, other research work, such as
(Swargiary, 2024), also lacks generalizability due to a limited sample size of 300 data points and
a different target group of middle and high school students. (Sasikala & Ravichandran, 2024)
conducted a study on AI's impact on student learning outcomes, yet their methodology does not
offer empirical data or quantify specific academic performance metrics such as Cumulative
Grade Point Average (CGPA) or Batch Percentiles. (Abbas, 2023) discussed the uses and
misuses of ChatGPT, focusing on ethical considerations and providing guidelines, but did not
empirically assess the direct impact of AI tools on academic performance. furthermore, the study
focussed only on one popular tool.

These studies often do not adequately control for confounding variables or account for individual
differences that may influence results. Such limitations hinder the ability to establish a strong
causal relationship between AI tool usage and academic performance.

This research aims to address the aforementioned unexplored gaps by examining the effect of AI
tool usage on the relative percentiles of college students in their respective batches. The research
hypothesis argues that the use of AI tools improves students' academic performance, which is
thus reflected in higher batch percentile scores amongst the students. This study also considers
individual differences among students and controls for confounding variables such as
Intelligence Quotient (IQ), socioeconomic status, and study habits via average study hours per
week. Furthermore, this research study also accounts for the number of AI tools being used and
self-reported prompt efficiency as moderating variables, which, in combination with the duration
of AI usage, can have a significant effect on student academic performance. All these measures
have been taken to establish a sound causal relationship.

In summary, it is evident that the usage of AI tools has a significant impact on the academic
performance of students. The existing scholarly work provides insights into this relationship.
However, there are subtle gaps that exist in these works, and it is paramount to explore them
with more comprehensive research that delves into the nuances of the manner in which AI

THE PROPOSITION & HYPOTHESIS

Concepts & Proposition:

Proposition: A heightened exposure to AI tools leads to an improvement in academic
performance among students studying in colleges.
Independent concept: The manner and duration for which AI tools are utilized.
Dependent concept: The effect on the academic performance of the students in comparison with
their peers.
Although there is a plethora of methods to measure the academic performance of an individual,
such as the Cumulative Grade Point Average, which is the standard criteria by which students are
graded, for our study, we shall consider the percentile obtained by the students in their institute
since the CGPA might differ depending on factors like the course pursued by a student and the
institute they study in. The percentile, on the other hand, represents how one performs with
respect to others pursuing the same course.

Variables & Hypothesis:

Hypothesis: College students who dedicate more time to the usage of AI tools and who use
multiple AI tools have a tendency to perform better academically than their counterparts.
Independent variable: The average number of hours of AI utilization per week for academic
purposes by the students.
Dependent variable: The percentile achieved by the students in the sample.

Unit of analysis: The students across all programs belonging to the IIM Indore campus, as well
the students outside the campus pursuing any university-level program, be it on the postgraduate
or undergraduate level.

Measurement strategy: Participants will be anonymously surveyed to assess the weekly average
number of hours they employ AI for academic commitments. Students will also be enquired
about their institute percentile through the medium of the questionnaire. The collected data will
be analyzed via statistical methods while controlling the confounders to gauge the effectiveness
of AI tools in improving academic performance.

STRUCTURAL MODEL:

The Confounding Variables:

1. Total Study Hours of a Student- It is often observed that pupils who study for more
hours are more inclined to use AI for better comprehension and concept understanding.
Being academically oriented, these students tend to perform better in exams as well.
Therefore, the number of study hours of an individual can act as a confounding variable
in our study.
2. Socioeconomic Status (SES)1 of a student- An individual's social and economic status
plays a vital role in determining both the number of hours and the variety of AI tools
used, as well as the academic performance of an individual. To account for the various
factors that determine students' socio-economic status, we have devised an SES index,
which shall be a combination of various factors as mentioned in the footer.
3. The Intelligence Quotient of the students- The students' IQ level can significantly affect
the efficiency of AI usage and academic performance relative to their peers. Thus, we
consider the student IQ to be a confounding variable in our study.

Socioeconomic Status (SES)1- This shall be a score that will be calculated on the weighted average of factors like- family income,
education level, assets and wealth of the family, as well as the access to the internet resources of a student.
Moderating Variable: Proficiency in using prompts to generate AI responses.
Any student making extensive usage of AI tools needs to give prompts or commands to generate
responses that are tailored specifically to their need. In such a case, a prompt that can extract a
more comprehensive and accurate response from the AI tools is pivotal when it comes to AI
usage. The combination of the level of one’s proficiency in generating prompts and working for
a longer duration of time on a variety of AI tools could imply an even better academic
performance.

Mediating Variable: Study the efficiency of the students.

In the proposed model, the heightened usage of AI tools leads to better academic performance.
However, the mechanism through which AI tools lead to a better academic performance is not
direct. There is a high likelihood that it affects the students’ study efficiency - which includes
their productivity, engagement, and learning strategies, which lead to an improvement in their
grades. The Motivated Strategies for Learning Questionnaire (MSLQ) is a well-established
report that employs various subscales (inclusive of the factors mentioned above) and then
combines their weighted average to create an overall composite score reflecting the study
efficiency one has.
Measurement strategy:

Confounding variables:
• The total number of study hours of a student can be measured by a self-reported
questionnaire wherein they will report the mean number of hours they study in a week.
• The socioeconomic status (SES) of a student can be measured by means of a
questionnaire that will be circulated amongst the students. The different subscales can be
measured on a Likert scale, which can then be combined after being assigned weights to
obtain a composite score.
• The Intelligence Quotient (IQ) of a student can be effectively calculated by means of
any standard IQ test available (such as the Wechsler Adult Intelligence Scale (WAIS)).

Moderating variable: This variable can also be captured by the employment of self-reported
questionnaires. Questions pertaining to criteria like clarity, conciseness, completeness of
prompts, and relevance of responses will be posed to the students under study. Further, a
combined score of these criteria shall be calculated, which will effectively capture the student's
prompt efficiency.

Mediating variable: As mentioned, the MSLQ is an effective way of measuring the study
efficiency of the students.
Primary Statistical model:
P = α + β1H + β2PE + β3T + β4(H×PE) + β5(H×T) + γ1SH + γ2SES + γ3IQ + ϵ - (1)

Variable Definitions
• Dependent Variable:
o P: Academic performance in terms of percentile of a student.
• Independent Variables:
o H: Number of hours of weekly AI usage by the student for academic purposes.
• Moderating Variable:
o PE: Prompt Efficiency of the student.
o T: Number of different AI tools used by the student.
• Confounding Variables:
o SH: Total study hours per week for the student.
o SES: Socioeconomic Status score for the student (Ordinal Variable).
o IQ: Intelligence quotient score of the student.

Explanation of Coefficients:
• β1: Effect of weekly AI usage hours (H) on academic performance percentile (P).
• β2: Main effect of prompt efficiency (PE) on academic performance percentile (P).
• β3: Main effect of the number of AI tools (T) on academic performance percentile (P).
• β4: Interaction effect between AI usage hours (H) and prompt efficiency (H×PE).
• β5: Interaction effect between AI usage hours (H) and the number of AI tools used (T).

γ1, γ2, γ3: Control effects of total study hours (SH), socioeconomic status (SES), and IQ.
ϵ: This captures the error term of the model after the percentile is regressed on the other factors.
Effect of Independent Variable on Mediating Variable:

MV = α0 + α1H + u

Where:

• MV represents the dependent variable (the Mediating Variable),

• α0 is the intercept,
• α1 is the coefficient for the independent variable H (the effect of H on MV),
• H is the independent variable.
• u = Error term.

Effect of Mediating Variable on the Model Dependent Variable:

P = λ0 + λ1MV + C + ϵ

Where:

• P = Percentile of the student (dependent variable),

• λ0 = Intercept (constant term),
• λ1 = Coefficient for the mediating variable MV (independent variable),
• MV = Mediating variable (the independent variable),
• C = The collection of the Confounding variables and interaction effects of the primary
statistical model (1),
• ϵ = Error term or residual.

Schematic Representation Of Structural Model

RESEARCH DESIGN

Collection of data- For data collection, a survey will be conducted with at least 200 students,
including IIM Indore and other reputed institutions. The research objective will be outlined in the
form floated, and the anonymity of the respondents will be maintained. All the questions will be
quantitative for ease of analysis.

The survey will consist of different sections to get specific information regarding each variable.
Details about various factors such as the number of hours AI tools are used, the variety of AI
tools employed, and the student’s batch percentile shall be recorded as part of this study. The
socioeconomic status, confounder, will be measured via SES: A measure that captures the
educational attainment, total family income, wealth, and access to internet resources of the
students under study. After recording these parameters, the SES composite scores are calculated.
This will allow us to quantify the socioeconomic status of the respondents.

The moderating variable will be measured by a Likert scale, evaluating how effectively and
efficiently students can frame prompts to leverage AI tools to the fullest extent possible for their
study sessions. The mediating variable will be measured via the Motivated Strategies for
Learning Questionnaire (MSLQ) to gauge the effectiveness and productivity of the study session
due to AI tool usage.

Analysis of Data- The collected data will be analyzed using appropriate statistical methods,
including the above stated regression model, to evaluate the significance of using AI tools on
academic performance. The coefficients will be tested at a 5% significance level.

CAUSAL ARGUMENTATION

Logical connection- AI tools offer more personalized and in-depth learning opportunities,
leading to better comprehension and application, thus improving study efficiency. These factors
are very likely to impact the academic performance of a student positively.
Precedence of Cause- Some students already have high percentiles, and their academic
performance may not show a significant causal change after using AI tools. To establish
precedence, questions on regularity and frequent usage of AI tools over the past 3 months will be
included to analyze before and after results. Only responses demonstrating 80% consistency will
be used for analysis.

Alternative Explanations- Potential alternate explanations, such as total study hours,

socioeconomic status, and the student IQ level, will be controlled within the statistical model.
This approach minimizes their impact on the dependent variable, ensuring a robust analysis of
the relationship between AI tool usage and academic performance.

OVERALL IMPRESSION

Overall, this research addresses the critical gaps in the existing literature on AI's impact on
college students' academic performance. Furthermore, by focusing on measurable outcomes like
relative batch percentiles and controlling for variables such as intelligence quotient (IQ),
socioeconomic status, and study habits, it seeks to establish a clear causal relationship between
AI tool usage and academic achievement.

The methodological rigor and consideration of individual differences enhance the study's internal
validity. Furthermore, the study is generalizable to all students pursuing higher education
nationwide, thus establishing external validity. This research promises to contribute valuable
insights into the effectiveness of AI integration in the education sector, shaping educational
practices and policies and setting the stage for future studies on the long-term and scalable
effects of AI tools across diverse student populations.
 REFERENCES

1) Sasikala, P., & Ravichandran, R. (2024). Study on the Impact of Artificial intelligence on

Student Learning Outcomes. Journal of Digital Learning and Education, 4(2), 145–155.

https://doi.org/10.52562/jdle.v4i2.1234

2) Abbas, M. (2023). Uses and Misuses of ChatGPT by Academic Community: An

Overview and guidelines. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4402510

3) Eltahir, M. E., & Babiker, F. M. E. (2024). The Influence of Artificial intelligence Tools

on Student Performance in e-Learning Environments: case study. The Electronic Journal

of e-Learning, 22(9), 91–110. https://doi.org/10.34190/ejel.22.9.3639

4) Swargiary, K. (2024, July 1). The impact of AI-Driven personalized learning and

intelligent tutoring systems on student engagement and academic achievement: ethical

implications and the digital divide.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4897241