Computers & Education 71 (2014) 113

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

Experimenting with electromagnetism using augmented reality:

Impact on ow student experience and educational effectiveness
Mara Blanca Ibez a, *, ngela Di Serio b, Diego Villarn a, Carlos Delgado Kloos a
a
b

Departamento de Ingeniera Telemtica, Universidad Carlos III de Madrid, 28911 Legans, Madrid, Spain
Departamento de Computacin y Tecnologa de la Informacin, Universidad Simn Bolvar, Caracas, Venezuela

a r t i c l e i n f o

a b s t r a c t

Article history:
Received 1 June 2013
Received in revised form
9 September 2013
Accepted 12 September 2013

Educational researchers have recognized Augmented Reality (AR) as a technology with great potential to
impact affective and cognitive learning outcomes. However, very little work has been carried out to
substantiate these claims. The purpose of this study was to assess to which extent an AR learning
application affects learners level of enjoyment and learning effectiveness. The study followed an
experimental/control group design using the type of the application (AR-based, web-based) as independent variable. 64 high school students were randomly assigned to the experimental or control group
to learn the basic principles of electromagnetism. The participants knowledge acquisition was evaluated
by comparing pre- and post-tests. The participants level overall-state perception on ow was measured
with the Flow State Scale and their ow states were monitored throughout the learning activity. Finally,
participants perceptions of benets and difculties of using the augmented reality application in this
study were qualitatively identied. The results showed that the augmented reality approach was more
effective in promoting students knowledge of electromagnetic concepts and phenomena. The analysis
also indicated that the augmented reality application led participants to reach higher ow experience
levels than those achieved by users of the web-based application. However, not all the factors seem to
have inuence on learners ow state, this study found that they were limited to: concentration, distorted sense of time, sense of control, clearer direct feedback, and autotelic experience. A deeper analysis
of the ow process showed that neither of the groups reported being in ow in those tasks that were
very easy or too difcult. However, for those tasks that were not perceived as difcult and included
visualization clues, the experimental group showed higher levels of ow that the control group. The
study suggests that augmented reality can be exploited as an effective learning environment for learning
the basic principles of electromagnetism at high school provided that learning designers strike a careful
balance between AR support and task difculty.
2013 Elsevier Ltd. All rights reserved.

Keywords:
Secondary education
Simulations
Interactive learning environments
Applications in subject areas
Augmented reality

1. Introduction
Interactive technologies such as 3D virtual worlds (3DVWs) and augmented reality (AR) are potentially effective in promoting learning
benets (Conole & Dyke, 2004); they act through the mediation of usability and psychological states on learning outcomes (Choi & Baek,
2011; Dalgarno & Lee, 2010; Dickey, 2005; Dunleavy, Dede, & Mitchell, 2009; Kye & Kim, 2008; Lee, Wong, & Fung, 2010; Wu, Lee,
Chang, & Liang, 2013). Therefore, an active area of research is the exploration of learning affordances offered by these emerging technologies in different knowledge areas (Dalgarno & Lee, 2010; Mikropoulos & Natsis, 2011).
Augmented reality refers to technologies that enhance the sense of reality allowing the coexistence of digital information and real
environments (Azuma, 1997). These technologies enable learners to interact with real objects in ways still to be discovered (Bujak et al.,
2013; Cuendet, Bonnard, Do-Lenh, & Dillenbourg, 2013). However, high-quality user experiences are difcult to achieve and interaction
with the learning environment should help, not hinder the teaching-learning process (Dunleavy et al., 2009; Zaharias, 2003). Many studies

* Corresponding author. Tel.: 34 916248744; fax: 34 916248749.

E-mail addresses: [email protected], [email protected] (M.B. Ibez), [email protected] (. Di Serio), [email protected] (D. Villarn), [email protected]
(C. Delgado Kloos).
0360-1315/$ see front matter 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.compedu.2013.09.004

M.B. Ibez et al. / Computers & Education 71 (2014) 113

have reported that once high-quality interaction with the learning environment is achieved, students emotional states contribute to
improved learning effects (Billinghurst, 2003; Dalgarno & Lee, 2010; Kye & Kim, 2008; Lee et al., 2010). Furthermore, researchers have shown
that AR features might increase students motivation, engagement and their satisfaction on performing learning activities. In this regard, B.
Kye and Y. Kims conceptual model (2008) states that ARs media characteristics namely sensory immersion, navigation and manipulation
might foster feelings of presence, ow state and satisfaction. Their model is similar to B. Dalgarno and M. Lees model of learning in 3d virtual
learning environments (2010) and both conclude that a positive impact of AR on emotions would improve students cognitive processes and
performance. What is lacking, however, is empirical studies to support or refute these theoretical claims (Cheng & Tsai, 2012; Dalgarno &
Lee, 2010; Kye & Kim, 2008).
A full history of emotional states that emerging technologies might foster on learning activities is beyond the scope of this work. We
concentrate our efforts on the observation the optimal experience or ow (Csikszentmihalyi, 1990). Flow describes a state of complete
absorption or engagement in an activity that acts as a motivating factor in daily activities such as work, sport, and education (Chan & Ahern,
1999; Choi & Baek, 2011; Kye & Kim, 2008; Pearce, Ainley, & Howard, 2005). The motivation promoted by the ow state enables learners to
engage in activities without receiving any apparent reinforcement (Davis, Bagozzi, & Warshaw, 1992). As this self-motivated learning is
considered the best way to learn (Ghani & Deshpande, 1994), a main challenge in education is to foster learners ow state. In this regard,
research in web-based learning environments has shown that there is a positive correlation between the ow state that students might
reach when using these environments and their learning outcomes (Liao, 2006; Shin, 2006; Webster, Trevino, & Ryan, 1993). Positive results
have also been highlighted by studies that analyzed the performance of students within multi-user virtual environments and game-based
learning environments (Faiola, Newlon, Pfaff, & Smyslova, 2013; Papastergiou, 2009). Therefore, it is reasonable to expect that an emergent
technology such as AR may also promote learners ow state and consequently help them to achieve better learning outcomes. Based on
these concerns, this study aimed to research on the impact of AR on learners ow state in the context of electromagnetism, a domain area
that underpins the operation of much of todays technologies.
Electromagnetism is abstract and cognitively demanding, thus it is one of the most difcult subjects for students to master (Dori &
Belcher, 2005). To understand abstract scientic concepts, students are required to build mental models where they internalize and
organize knowledge structures (Dede, Salzman, Loftin, & Sprague, 1999). Unlike what happens in other Physics conceptual areas, when
dealing with electromagnetism, students mental models should include abstractions and invisible factors for which students have no reallife references (Maloney, OKuma, Hieggelke, & Van Heuvelen, 2001). The relevance of presenting learning materials not only through words
but also through visual assets to fully understand the nature of scientic phenomena and processes was reported by Dori, Hult, Breslow, and
Belcher (2007). Indeed, these concerns were already addressed in the MIT Technology Enabled Active Learning (TEAL)/Studio Project where
students developed much better intuition about, and conceptual models of, physical phenomena through the use of visualization in an
electricity and magnetism course using web-based technologies (Belcher & Bessette, 2001; Dori et al., 2007). Consequently, developing
effective pedagogical strategies and using emergent technologies for helping students in this endeavor will be a step ahead to validate TEAL
Project ndings when using AR technology. In this regard, augmented reality has been recognized as a technology with great potential for
science learning (Bujak et al., 2013; Cheng & Tsai, 2012; Wu et al., 2013) as it provides new ways of tactile and visual interactions which could
be useful to improve learning outcomes (Cheng & Tsai, 2012; Gilbert, 2005; Rapp, 2005). Visualization features of AR have been successfully
used to improve spatial abilities in science and engineering (Dnser, Steinbgl, Kaufmann, & Glck, 2006; Martn-Gutirrez et al., 2010).
However, few studies have explored the visualization benets of AR in science in general and physics in particular (Cuendet et al., 2013; Wu
et al., 2013).
In response to the aforementioned issues, the aim of this study was to assess the learning effectiveness of an augmented reality
experimental lesson for learning the basic principles of electromagnetism and the level of enjoyment of high school students. The lesson was
designed according to the curricular objectives and subject matter of the Spanish high school Physics curriculum, and compared to a similar
web-based lesson which encompasses identical learning objectives and content. Both AR and web based lessons guided the students
through the building of an electromagnetic circuit representing a problem to solve which involved basic principles of electromagnetism.
Each building stage offered students the possibility to review (or learn) concepts related to the proposed learning task. The AR-based lesson
guided the learning workow by leading students to build a circuit while allowing them the visualization of the forces involved and the
exploration of their circuit behavior. The following research questions shaped this study:
1. Do students who use an augmented reality based lesson develop deeper understanding of the embedded basic principles of electromagnetism compared to peers who use a similar web-based lesson?
2. Do students who use an augmented reality based lesson experiment reach a ow experience higher than the one achieved by peers
using a similar web-based lesson?
The study is unique in that it investigates the use of AR technology within real school settings for teaching electromagnetism at high
school level, while also comparing an AR-based application with a web-based application. The study can help us to learn whether AR
technology can be effective in promoting student ow, and to gain understanding on which activities maintain students engagement in ARbased learning environments. Furthermore, this empirical research might contribute to a better understanding of the impact of AR on
learning outcomes, mainly those requiring the understanding of electromagnetic invisible forces. In addition, it can provide insight into
what benets and difculties students found when interacting with an AR-based learning environment.
2. Learning affordances of augmented reality
Each interactive technology has a set of features that facilitates particular approaches to educational practice (Choi & Baek, 2011; Conole
& Dyke, 2004; Dalgarno & Lee, 2010; Dickey, 2005; Dunleavy et al., 2009; Kye & Kim, 2008; Lee et al., 2010; Wu et al., 2013). For instance,
three-dimensional virtual world (3DVW), an interactive technology that shares relevant characteristics with augmented reality, offers
representational delity and learning interaction as technical features (Chittaro & Ranon, 2007). These features potentially promote
learners psychological states such as the construction of identity, sense of presence, and co-presence, and foster the ow state (Choi & Baek,

M.B. Ibez et al. / Computers & Education 71 (2014) 113

2011; Dalgarno & Lee, 2010). As result, enhanced spatial knowledge representation, experimental learning, and collaborative learning are
among the learning affordances that can be found in 3DVWs and AR learning environments (Dalgarno & Lee, 2010; Kye & Kim, 2008).
Initial attempts for analyzing the potential benets of AR as applied to learning have been carried out. Cheng and Tsai (2012) differentiate
between image-based AR and location-based AR in the context of science learning, whereas Wu et al. (2013) argue that it is better to
consider AR as a concept rather than a type of technology and classied AR instructional approaches that emphasize roles, tasks and locations. A psychological perspective is presented by Bujak et al. (2013) exploring physical, cognitive and contextual dimensions of these
technologies. Although these studies examine AR learning affordances from different perspectives, they concur on the necessity to
determine whether AR increases students motivation, engagement and their satisfaction on performing learning activities. In this regard, B.
Kye and Y. Kims conceptual model (2008) states that ARs media characteristics namely sensory immersion, navigation and manipulation
might foster feelings of presence, ow state and satisfaction once a satisfactory levels of interaction with the learning environment is
achieved. Their model is similar to B. Dalgarno and M. Lees model of learning in 3d virtual learning environments (2010). A positive impact
of AR on emotions would improve students cognitive processes and performance (Csikszentmihalyi, 1990; Efklides, Kuhl, & Sorrentino,
2001; Keller, 1979; Linnenbrink & Pintrich, 2002). Psychological states such as motivation, ow, cognitive benets, reection, and sense
of presence are positively related to learning outcomes (Antonietti, Rasi, Imperio, & Sacco, 2000; Conole & Dyke, 2004; Dalgarno & Lee, 2010;
Dunleavy et al., 2009; Lee et al., 2010). Due to the limited scope of this work, we concentrate our efforts on the observation of the psychological state in which an individual feels cognitively efcient, motivated, and happy, known as ow (Csikszentmihalyi, 1990), that has
been widely recognized by many researchers as supportive of students learning (Choi & Baek, 2011; Kye & Kim, 2008; Pearce et al., 2005).
Flow has been conceptualized as an optimal experience characterized by the perceived balance between challenge and skills. Nine factors
have been related to this psychological state: clear goals, immediate feedback, perceived skills versus challenges, merger of action and
awareness, concentration on the task, control, a loss of self-consciousness, an altered sense of time, and experience which becomes autotelic
(Csikszentmihalyi, 1990). The ow concept has been applied to disciplines such as marketing, sports, work, and education (Hoffman &
Novak, 2009; Novak, Hoffman, & Yung, 2000) and research conducted in web environments suggests that interactive media might foster
users ow states (Chen, Wigand, & Nilan, 1999; Cowley, Charles, Black, & Hickey, 2008). These results have encouraged researchers to study
the effect of augmented reality on students ow states (Choi & Baek, 2011; Kye & Kim, 2008; Pearce et al., 2005). Consequently, several
approaches have been used to model and measure this feeling of enjoyment. The original model considers the ow state on a plot of
challenge versus skill, separating the states of anxiety and boredom by the ow channel. This simple model which captures the essential
principles of ow was used by Pearce et al. (2005) to measure ow as a process rather than just an overall state, as most ow measurement
instruments (Chen, 2006; Jackson & Marsh, 1996; Martin et al., 2011). This two-channel model has been enhanced to include more dynamic
relationships between challenges and skills. For instance, the four-channel model (Csikszentmihalyi, 1990) incorporates the apathy state to
emphasize the possibility of not reaching the ow when the participants skill level increases faster than the challenges, and the eightchannel model (Massimini & Carli, 1988) adds the states of worry, arousal, control, and relaxation to the four-channel model.
Regarding possible learning effects promoted by augmented reality technologies in education, Kye and Kim (2008) structural equation
model study revealed that interaction had a direct effect on students satisfaction, which is considered a learning affordance. Furthermore,
two other psychological factors sense of presence and ow proved to have an inuence on the application abilities of learners but to a
lesser extent in knowledge and understanding factors. A similar study that outlines technological, pedagogical, and learning issues related to
the implementation of AR in education focuses on students interactions with their peers, the physical environment, and learning objects
respectively (Wu et al., 2013). They claim that (1) the role-based approaches enhance the sense of presence, immediacy, and immersion
which can foster engagement; (2) location-based approaches help to contextualize learning experiences; and (3) task-based approaches
promote students self-direct learning, self-motivation, problem-solving skills, and knowledge-application skills. When Cheng and Tsai
(2012) analyzed augmented reality affordances for science education, they identied two major approaches for utilizing AR technology:
the rst one is image-based AR, and the second one is location-based AR. They concluded that image-based AR applications are useful to
foster students spatial ability, practical skills, and conceptual understanding whereas location-based AR applications tend to support
scientic inquiring learning. On the other hand, an analysis by Dunleavy et al. (2009) which focused on location-based AR found that
interactive, situated, and collaborative AR affordances produce highly engaging learning experiences.
In summary, ARs unique technical feature mentioned in the literature is interaction which involves view control and object manipulation. Good interactive experiences with augmented objects and eventually with peers have proved to be useful in promoting learners
emotional states, such as ow, which in turn might have positive implications for learning. Finally, among the learning effects that researches claim can be promoted by AR are improvement of knowledge, understanding, problem solving, and spatial ability.

3. Design of experiment
The study compared two educational applications for learning basic principles of electromagnetism to identify the effects of AR and web
technologies on learning outcomes and students task involvement. The study followed an experimental/control group design using the type
of the application (AR-based, web-based) as independent variable. Students were distributed among the experimental and control groups
using a random assignment.
To explore the effect of the application on students task involvement, two dependent construct were measured: the overall-state
perceived ow and the step-by-step perceived ow. Overall-state perceived ow measured the nine factors mentioned as Section 2 as
affective attributes indicating a sense of ow and it was useful to examine the elements that allowed students to carry out activities in ideal
conditions, optimizing satisfaction and performance. Furthermore, a closer view of the perceived tradeoff between challenge/skill was used
to understand the engagement of students along the tasks (Csikszentmihalyi, 1990; Pearce et al., 2005). A pretest/posttest design was used
to explore the effect of application used on students achievement as measured by a knowledge test on basic electromagnetic concepts. To
counteract order effects taking the pretest and posttest questionnaires, (a) two knowledge tests: test1 and tests2 with similar questions
were designed; (b) control and experiential groups of each section were divided in two parts, half of participants completed test1 as pretest
and test2 as posttest and the other half completed test2 as pretest and test1 as posttest. Finally, to explore the perceived benets and

M.B. Ibez et al. / Computers & Education 71 (2014) 113

difculties of using the AR application to learn the basic principles of electromagnetism, an open-ended questionnaire with two open-ended
questions was administered to the experimental group at the end of the learning exercise.
Based on the overview of the research literature, the specic research questions were formulated as follows:
1.Is there any difference in students overall state of ow depending on which of the learning applications they used?
2.Are there any differences in the students challenge-skill perception depending on which of the learning applications they used?
3. Is there any difference in students learning outcomes depending on which of the two learning applications they used?
4. Is there any difference in students learning outcomes for the theoretical questions and the understanding of spatial relationships
among forces and electromagnetic elds depending on which of the two learning applications they used?
5. According to the participants of the experimental group, what were the students perceived benets and difculties of using an AR
application to learn the basic principles of electromagnetism?

3.1. Participants
In this study participated students enrolled in the 12th grade (science specialization area) at four Spanish schools which had in total ve
sections. The research was conducted through extraordinary sessions carried out during school hours and students participation in the
learning activity was mandatory. The sample were 64 high school students (age 1719, M 17.4, SD 0.66). Four students did not complete
some of the tests and thus were not considered for the purposes of this study. Among the 60 respondents, 15 were female and 45 were male.
In each section, participants were randomly assigned to either the experimental group which tried the AR-based application, or to the
control group which used the web-based application. Table 1 shows the distribution of participants according to school and intervention
group.
Students had basic computer skills (e.g. elemental computer operation, the ability to use a web browser), which form part of their
Computer Science curriculum. However, they never had used a tablet before.
3.2. Procedure
The study was set in ve physics sections from four high schools in Spain. The sections were taught the basic principles of electromagnetism by their respective teachers who followed the same physics curriculum. All participants completed the pretest questionnaire.
Two weeks after, in each of the selected schools, students were assigned to the experimental (AR-based application) or control (web-based
application) groups randomly, received the corresponding application and brief oral instructions on its use. After the tutorial session,
students completed the ve stages of the learning activity interacting with their respective application. At the end of each stage, students
completed the Pearce et al. survey (2005). Throughout this time, students received technical and procedural help by the researchers and
their physical teachers respectively. Upon completion of students intervention, students completed the posttest questionnaire, followed by
the Flow State Scale. Finally, students who interacted with the AR-based application completed an open-ended questionnaire to provide
feedback about perceived benets and difculties of using the AR-based application for learning principles of electromagnetism.
The learning activity lasted 40 min. The maximum amount of time given for the completion of the pretest and posttest questionnaires
was 20 min for each one of them. The time given to complete the Flow State Scale and open-ended questionnaires was 10 min for each one of
them. All the questionnaires were completed by students anonymously, in their respective classrooms, in presence of their teachers and the
researchers. Neither teachers nor researchers were actively involved in the students work. For matching of the questionnaires, pseudonyms
that researchers had asked the students to adopt and write down in their questionnaires were used (see Fig. 1).
3.3. Materials
Authors designed a learning activity that allowed students to explore the effect of a magnetic eld on a current-carrying wire. To this end,
students were asked to solve a problem that involved the knowledge of basic concepts of electricity, magnetism, and Lorentzs law. The
designed learning activity allowed students to review relevant electricity topics to the subject taught but its main focus was on helping
students to understand and apply basic magnetism concepts and Lorentzs law.
Students were guided through the resolution of the problem which involved ve sequential stages. Each stage was associated with one
element of the electromagnetic circuit together with the problem which students had to solve: Cable, Battery, Magnet_1, Magnet_2, and
Magnet_3. The narrative of the activity consisted of the construction of the circuit that solved the proposed problem by adding a new
element in each stage. In order to be allowed to add a new circuit element, students were inquired about the main concepts related to that

Table 1
Amount of students evaluated and surveyed for school sites.
School name

Section

Centro Escolar Amanecer

S1
S2
S1
S1
S1
5

IES Enrique Tierno Galvn

IES Montes Obarenes
IES Pedro Duque
Total

Number of students
Experimental group

Control group

Total

6
9
5
6
6
32

6
8
4
7
7
28

12
17
9
13
13
64

M.B. Ibez et al. / Computers & Education 71 (2014) 113

Fig. 1. Procedure of the experiment.

circuit element; then they had the option to read about the concepts involved with the new element included or to continue with the
learning activity. Finally, students had to solve sub-parts of the proposed problem.
Two different applications were developed to support the learning activity: one application was web-based; the other application used
image-based augmented reality technology. Both applications offered the same educational content and followed the same workow according to participants preferences and their answers to assessment items (see Table 2). Educational content included learning material
(e.g. text, images) related to stages and problems to solve which were presented in the form of multiple choice or ll-in-the-blank short
answer questions. However, the AR-based application allowed students to visualize electromagnetic forces upon the real environment, the
web-base application did not provide this possibility.
3.3.1. AR-based learning application
The AR-based learning application was structured around students manipulation of 3D shapes that mimicked circuit elements. Each
element was tagged with a ducial marker that enabled its recognition (see Fig. 2). The manipulation of each element had an associated
specic learning material, problem(s) to solve, and for some of them simulation activities which helped students either to visualize the
electromagnetic forces or to explore the circuit behavior. AR-based learning application users constructed step by step a physical paper
circuit. They were allowed to visualize the electric elds, the magnetic elds and the movement of the current-carrying wire at stages
battery, magnet_2 and magnet_3 respectively through the display of their tablets. Fig. 3 shows the ve stages of the learning application as
followed by users of the AR application and Fig. 4 shows students interacting with the AR-learning application.
The AR-based application was developed with the Apple Integrated Development Environment Xcode (https://developer.apple.com)
using the Vuforia Augmented Reality Software Development Kit (https://developer.vuforia.com). The simulations were programmed using
OpenGL ES 2.0 (http://www.khronos.org/opengles/2_X/) and then integrated to Vuforia.
3.3.2. Web-based learning application
The web-based learning application was an educational web-site structured around the same ve stages of the AR-based application.
Specically, each stage contained the same learning objectives and educational content as the respective stage of the AR-based learning
application. The learning material was accessible through navigation hyperlinks.

Table 2
Content of AR-based and web-based applications associated with each stage.
Stage

Concepts

Problems to solve

Cable

Charge and Coulombs Law

Current intensity denition, units, formula
Electrical resistance denition, units, formula
Electric eld and electric potential
Ohms law
Background on the physics of magnetism and magnets
Forces on moving charges in magnetic elds
Lorentzs Law
Forces on current-carrying wires in magnetic elds

Battery
Magnet_1
Magnet_2
Magnet_3

2
0
2
1

M.B. Ibez et al. / Computers & Education 71 (2014) 113

Fig. 2. Tangible elements of the electromagnetic circuit.

AR-based and web-based applications differed in two aspects. First, users of the web-based application arrived to stages through
navigation hyperlinks, whereas users of the AR-based applications required to manipulate the physical object that mimicked an element of
the circuit to begin the stage corresponding to such element. Second, the web-based application did not provide any dynamic visualization
of circuit behavior.
3.4. Measurement instruments
Research instruments designed and selected for this experiment aimed to understand the impact of augmented reality in learning the
basic concepts of electromagnetism. In this study, ow state and learning outcomes were measured with the aim of discovering analogies
and differences in ow state and learning outcomes between the experimental and the control group.
The impact of the AR application in a ow learning experience was measured in two different ways. First, as an overall state with the Flow
State Scale (FSS) developed (Jackson & Marsh, 1996), adapted, and validated to Spanish language by Garca Calvo, Jimnez Castuera, SantosRosa Ruano, Reina Vallo, and CervellGimeno (2008). Second, ow was measured as a process with the survey proposed by Pearce et al.
(2005). The FSS contains 36 questions with 5-point Likert-scale items and provides information about nine factors which are directly
related to the ow state: balance between ability level and challenge (AC); merging of action and awareness (AA); clear goals (CG); direct
and clear feedback (CF); concentration on task being performed (CT); sense of control (SC); loss of self-consciousness or inhibition (LS);
distortion of sense of time (DT); and autotelic experience (AE) (Csikszentmihalyi, 1990). The FSS includes as one of its factors the balance
between ability level and challenge which constitutes the basic denition of ow, thus Pearce et al.s (2005) survey was also used to monitor
students ow state throughout the learning task. Pearce et al.s survey measured students perception of their skills and abilities in relation
to the learning tasks that they had to perform, a theoretical meaningful reference for the presence or absence of ow (Csikszentmihalyi,
1990). The survey has two question items with a ve-point Likert scale that asks the level of perceived skill and perceived challenge of a
task: (FQ1) How challenging did you nd this last task? (FQ2) Were your skills appropriate for understanding this last task? Within the
model, balance of perceived challenge and skill is considered as ow; higher perceived challenge with lower perceived skill is indicative of
anxiety whereas lower perceived challenge with higher perceived skill is classied as boredom.
To assess the effectiveness of the interventions on students electromagnetism basic concepts knowledge pretest and posttests were
conducted and analyzed. The pretest and posttest comprised eight short-answer and problem solving questions. Given the difculties to

Fig. 3. Stages of the learning activity within the AR application.

M.B. Ibez et al. / Computers & Education 71 (2014) 113

Fig. 4. Students experimenting with their circuits.

understand invisible factors involved in electromagnetism (Dori & Belcher, 2005; Maloney et al., 2001), the tests included not only theoretical but also visualization questions which assessed the degree of understanding of concrete and abstract concepts involved respectively.
Theoretical questions were related to the use of formulas to calculate the magnitude of physical quantities. Visualization questions assessed
students knowledge of spatial relationships among forces and electromagnetic elds. Each question was worth one point. Tests were
designed by researchers and examined as to its content validity by teachers of the high schools taking part in the study. The structure of the
questionnaires is presented in Table 3.
4. Data analysis and results
4.1. Research question 1
Is there any difference in students overall state of ow depending on which of the learning applications they used?
The minimum and maximum scores of the Flow State Scale are 36 and 180 as the response scale ranges from 1 to 5. The total scores of
students that used web-based and AR-based applications ranged from 104 to 161 (M 136.14, SD 13.37) and 96 to 156 (M 124.81,
SD 12.81), respectively. These results indicate that overall, control and experimental group students were experimenting ow while using
their respective learning applications, with results slightly more favorable for the experimental group.
The ShapiroWilk test of normality distribution was used to examine the distribution of the factors when the two learning applications
were used. For the web-based application, all factors but LS may come from a normal distribution whereas for the AR-based application, AA,
CT, SC and DT may come from a normal distribution. For those factors that followed a normal distribution in both learning applications, an
unpaired-sample t-test was applied; for the others a MannWhitney Wilcoxon test was used. The Cohens effect size index d (Cohen, 1988)
was used to estimate the practical signicant difference between groups. The Cohens d values of 0.20, 0.50, 0.80, and 1.0 are considered as
small, medium, large, and very large effect sizes, respectively.
Unpaired t-tests were applied to compare the students ow state in the control and the experimental group over AA, CT, SC, and DT
factors. Results indicate that ow differences between the two groups were statistically signicant for the factors concentration on task (CT),
sense of control (SC), and distorted sense of time (DT). The experimental group experienced higher levels of concentration on tasks
(M 4.02, SD 0.60) than the control group (M 3.49, SD 0.83) [t 2.771, df 58, p-value 0.008, d 0.87]. Furthermore, AR-based
application users had a higher sense of control (M 3.79, SD 0.53) than web-based application users (M 3.42, SD 0.59) [t 2.505,
df 57.923, p-value 0.015, d 0.68]. Finally, the experimental group reported a higher distorted sense of time (M 3.37, SD 0.92) than
Table 3
Structure of the pretest and posttest questionnaires.
Theoretical questions
(T1) Determine the magnitude of the electric force of repulsion (or attraction) between two electrically charged particles using of the Coulombs force law.
(T2) Denition of an electrical concept (e.g. intensity, resistance).
(T3) Determine the magnetic force of a particle in a uniform magnetic eld.
(T4) Determine the resistance of a conductor and its relationship to the electric current in a circuit using Ohms Law.
(T5) Determine the magnetic force on a current carrying wire.
Visualization questions
(V1) Determine the direction of charged particles in a uniform magnetic eld.
(V2) Determine the direction of the electric current on a circuit.
(V3) Determine the direction of the magnetic force on a current carrying wire.
Finally, an open-ended questionnaire with two open-ended questions was issued to document students perception of the impact that visualization and manipulation had on their learning experience: (O1) What benets did you perceive from using the AR-based application to learn the basic principles of electromagnetism?
(O2) What difculties did you have working with the AR-based application?

M.B. Ibez et al. / Computers & Education 71 (2014) 113

the control group (M 2.85, SD 0.71) [t 2.392, df 50.366, p-value 0.02, d 0.56] (see Fig. 5). Therefore, the Cohens d value also
suggested large, medium and medium practical effect of the difference between the two groups for the factors CT, SC and DT respectively.
Since multiple tests of signicance were performed, the Bonferroni correction procedure was applied to avoid making a type I error. In all
factors, the level of signicance was conrmed.
For the remaining factors (AC, CG, CF, LS, and AE), MannWhitney Wilcoxon tests were used. Results indicate that the difference between
the two learning applications was statistically signicant for clear and direct feedback (CF) and autotelic experience (AE). Results indicate
that the experimental group had higher levels of clear and direct feedback factor (M 3.64, SD 0.48) than the control group (M 3.27,
SD 0.64) [U 303.5, p-value 0.03, d 0.58]. Similarly, the experimental group had higher values of autotelic experience (M 4.20,
SD 0.71) than the control group (M 3.63, SD 0.69) [U 243.5, p-value 0.002, d 0.86] (see Fig. 6). Therefore, the Cohens d value also
suggested medium and large practical effect of the difference between the two groups for the factors CF and AE respectively.
4.2. Research question 2
Are there any differences in the students challenge-skill perception depending on which of the learning applications they used?
The analysis of the balance between ability level and challenge factor using the Flow State Scale [W 491.5, p-value 0.519] did not
show statistical signicant difference between the control (M 3.67, SD 0.52) and experimental group (M 3.49, SD 0.61). However,
according to Pearce et al. (2005), representing students ow by a single value loses valuable information about the variable experiences
within a learning session. Therefore, a deeper analysis of ow experiences by learning stages could be useful to understand possible ow
differences between both groups of students. In this regard, a chi-square test was performed to test the null hypothesis of no association
between the ow state and the learning stage. Subsequently, a chi-square test was performed to test the null hypothesis of no association
between each of the ow states and the learning application used at each learning stage; the null hypothesis was only rejected in the
Magnet_2 learning stage.
An association between ow state and application stage was found when the AR learning application was used, c2 18.585, df 8, pvalue 0.01. The adjusted residuals associated with the ow states Boredom and Anxiety, and the application stages Cable and Magnet_3
respectively, showed that most of the students that used the AR learning application experienced boredom at the rst stage of the activity
(2.62) and anxiety at the last activity (3.25). The frequency measures showed that there were few students experiencing ow along the
learning stages but this number increased through the evolution of the experiment (see Table 4).
For the web-based application, an association between ow state and application phase was found, c2 26.41, df 8, p-value < 0.001.
The adjusted residuals associated with the ow states and the application phase data included next to the observed frequencies in Table 5
show similar results to those found with the AR-based application. Indeed, for the control group, Cable stage was also an easy challenge for
students (2.53) whereas Magnet_3 was a difcult one (3.69). In the case of ow state, there were uctuations: at some stages their values
were above the expected values and in others, they were below.
The ow levels of the students were different in the experimental and control groups in stage Magnet_2. A Chi Square test showed that
there was some association between students ow states and the learning application they used during the Magnet_2 stage, c2 6.587,
df 2, p-value 0.03. The adjusted residuals associated with the ow states and the learning applications were included next to the
observed frequencies in Table 6. The observed frequency of students who experienced boredom when using the AR-based application was
2.04 standard errors lower than would be expected if there was no association between ow state and the learning application. Similarly, the
adjusted frequency of students in ow when using the AR-based application was 2.56 standard errors higher than we would expect if there
was no association between ow state and the learning application. Results showed that students were less bored and more in ow state
when the AR-based application was used during the Magnet_2 stage.
4.3. Research question 3
Is there any difference in students learning outcomes depending on which of the two learning applications they used?
An analysis of covariance (ANCOVA) was carried out to compare the impact of the two learning applications used. To investigate potential
initial differences between groups an analysis of the pretest scores was performed. Result showed that there was no statistically signicant
difference between the students using the web-based application (M 3.375, SD 1.10) and the students using the AR-based application

Fig. 5. Boxplot for ow factors: AA, CT, SC, and DT in web- and AR-based applications.

M.B. Ibez et al. / Computers & Education 71 (2014) 113

Fig. 6. Boxplot for ow factors: AC, CG, CF, LS, and AE in web- and AR-based applications.

(M 3.25, SD 1.17), F(1,58) 0.181, p-value 0.672, which indicates that the two groups had similar background knowledge of the basic
concepts of electricity, magnetism, and Lorentzs law.
Before conducting the analysis of covariance (ANCOVA) on posttest scores to evaluate the effectiveness of the two learning applications,
preliminary verications were performed to conrm that there was no violation of the assumptions of normality, linearity, homogeneity of
variances, and homogeneity of regression slopes. The sample satised the requirements for analysis of covariance (web-based group 32,
AR-based group 28). The skewness and kurtosis was between 1.0 and 1.0 for pretest and posttest scores, thus the assumption of
normality is satised (see Table 7). Based on the test of linearity, the relationship between pretest scores and posttest scores was linear, at a
statistically signicant level, F(1,58) 57.696, p < .001. The univariate general linear model procedure was used to test the signicance of an
interaction term in the model, made up of the covariate (pretest scores) and the factor (learning applications). The result indicated that the
assumption of homogeneous regression slopes is satised (F(1,56) 0.2126, p-value 0.6465).
Results showed that there are differences in students learning outcomes depending on which of the two learning applications they used.
Indeed, students using the AR-based application performed signicantly higher in the posttest than those that had used the web-based
application.
4.4. Research question 4
Is there any difference in students learning outcomes for the theoretical questions and the understanding of spatial relationships among
forces and electromagnetic elds depending on which of the two learning applications they used?
Pre- and posttests had two types of questions, namely theoretical and visualization questions. In order to determine whether there was
any difference in students learning outcomes caused by the type of questions depending on which of the learning application they used, the
pre- and posttest scores were analyzed to distinguish between the two types of questions. The sample size satised the requirements for
analysis of covariance (web-based group 32, AR-based Group 28), but the assumption of normality was not satised since the skewness
and kurtosis was not between 1.0 and 1.0 for some of the categories (Table 8). Therefore, a Mann-Whitney-Wilcoxon non-parametric test
was used to investigate potential initial differences between the control and the experimental group for the Theoretical and the visualization
questions in the pretest. The statistical results indicate that the control group (M 2.88, SD 0.839) and the experimental group (M 2.50,
SD 0.84) did not differ signicantly on pretest theory scores (U 350, p-value 0.146). Similarly, there were no statistically signicant
difference between the control group (M 0.44, SD 0.67) and the experimental group (M 0.75, SD 0.84) on pretest visualization scores
(U 355.5, p-value 0.125). These results indicate that the two groups had similar background knowledge of the theory and visualization of
the basic concepts of electromagnetism before starting the experiment.
The paired Wilcoxon test was used to compare pretest and posttest scores within each group both for the theoretical and visualization
questions. Regarding the pre- and post-test scores of theoretical questions of the control group, results indicate a statistically signicant
difference in their pretest scores (M 2.88, SD 0.87) and posttest scores (MD 3.66, SD 1.23), V 53, p-value 0.004. For the
experimental group, results also indicate a statistically signicant difference in their pretest scores (M 2.50, SD 0.84) and posttest scores
(MD 3.89, SD 0.79), V 6, p-value < 0.001. Thus, both groups improved their scores in theoretical questions. In order to analyze learning
benets in the use of formulas to calculate the magnitude of physical quantities when using the AR-based application (M 3.89, SD 0.79)
compared to the web-based application (M 3.66, SD 1.23), a MannWhitney Wilcoxon test was used. Results indicate that the control
and experimental groups did not differ signicantly on posttest scores related to the theoretical questions (U 422.5, p-value 0.6961).
Regarding the pre- and posttest scores of visualization questions, results indicate a statistically signicant difference in pretest scores
(M 0.44, SD 0.67) and posttest scores (MD 1.34, SD 1.04) when the web-based application was used (V 14, p-value < 0.001). For the
Table 4
Observed frequencies and adjusted residuals for students using the AR application.

Boredom
Flow
Anxiety

Cable

Battery

Magnet_1

Magnet_2

Magnet_3

19(2.62)
6(2.08)
3(081)

13(0.08)
11(0.09)
4(0.23)

14(0.51)
12(0.52)
2(1.39)

11(-0.76)
14(1.39)
3(0.81)

7(2.46)
11(0.09)
10(3.25)

10

M.B. Ibez et al. / Computers & Education 71 (2014) 113

Table 5
Observed frequencies and adjusted residuals for web application.

Boredom
Flow
Anxiety

Cable

Battery

Magnet_1

Magnet_2

Magnet_3

23(2.53)
8(0.69)
1(2.46)

14(1.02)
13(1.47)
5(0.41)

18(0.55)
9(0.26)
5(0.41)

21(1.74)
6(1.55)
5(0.41)

7(3.79)
12(1.03)
13(3.69)

AR-based group, results also indicate a statistically signicant difference in pretest scores (M 0.75, SD 0.84) and posttest scores
(MD 2.21, SD 0.96), V 12, p-value< 0.001. Thus, both groups improved their scores in the visualization questions. To analyze learning
benets for questions related to spatial relationships among forces and electromagnetic elds when the AR-based application (M 2.21,
SD 0.96) compared to the web-based application (M 1.34, SD 1.04), a MannWhitney Wilcoxon test was used. The AR-based group
(M 2.21, SD 0.96) reported signicantly higher posttest scores for visualization than the web-based group (M 1.34, SD 1.04), U 248,
p-value 0.001.
Thus, there were differences in students learning outcomes depending on which of the two learning applications they used.
4.5. Research question 5
According to the participants of the experimental group, what were the students perceived benets and difculties of using an AR
application to learn the basic principles of electromagnetism?
The open-ended questionnaire included two questions to inquire about the benets and difculties perceived by students when using
the AR application. The data gathered (42 responses from 14 students) was examined using an open coding scheme (Strauss & Corbin, 1998).
Comments related to benets revealed two major themes: academic (11) and affective (19); however, the comments related to difculties
referred to usability (8) and cognitive (4) issues.
The majority of affective benets perceived by users of the AR application pertained to the use of the tablet and the real objects (8
responses). Affective benets mentioned were related to a sense of well-being (14) and factors associated with motivation through the
learning activity (5). Benets reported were linked to different features of the application. The visual capabilities of the application were
mentioned in six responses. Five students highlighted the affective benets of learning and practicing with the AR-application when
comparing it with the traditional way of learning.
Examples of responses relating to affective benets:
I liked it; it was like a game to build the circuit in my tablet.
It is stimulating to add new elements to the circuit; you have the feeling of success.
It is an amazing use of tablets. We can practice these lab exercises also at home.
It felt rewarding when I saw how my solution worked properly.
I enjoyed building the circuit, experimenting with its parts and observing the changes produced.
This is a fun way to learn, it is better than study with my book.
Regarding the cognitive benets perceived by users of the AR application, four students highlighted that the application helped them to
better understand the circuit operation; three students remarked that the application was useful for keeping concentration; two comments
were related to advantages of having real-time feedback to evaluate their progress and other students claimed that similar learning activities
would help them to learn with less effort.
Examples of responses relating to academic benets:
Visualizing forces helped me to understand the behavior of the circuit.
I could understand better how to solve the problem thanks to the possibility to experiment with the circuit elements.
Building the circuit by myself helped me to focus my attention on the problem to solve.
It helped me to see whether or not I was doing well, if I applied correctly the formulas.
I could learn Lorentzs Law with less effort than studying at home.
Some students reported they had difculties while completing the learning activity. Two main usability problems arose: (1) the most
common difculty for students was to make the system recognize the markers (seven students), and (2) three students experienced
problems manipulating the tablet and the physical objects representing the components of the circuit at the same time. Both problems were
easily solved with the help of instructors. Finally, two students reported that they lost concentration when they had the option to read
review information.

Table 6
Frequency and adjusted residuals for Magnet_2 learning stage.

AR-based application
Web-based application

Boredom

Flow

Anxiety

11(2.04)
21(2.04)

14(2.56)
6(2.56)

3(0.55)
5(0.55)

M.B. Ibez et al. / Computers & Education 71 (2014) 113

11

Table 7
Descriptive statistics of pretest and posttest scores for control and experimental groups.
Test

Learning application

Mean

SD

Median

Trimm

Skew

Kurtosis

Pretest

Web-based
AR-based
Web-based
AR-based

32
28
32
28

3375
3250
5000
6110

1.10
1.17
1.87
1.40

3
3
5
6

3.35
3.21
5.12
6.17

0.24
0.32
0.35
0.50

0.30
0.56
0.62
0.81

Posttest

After adjusting the posttest scores in the pretest (covariate), the following results were obtained. A statistically signicant main effect was found depending on the learning
application used on the posttest scores, F(1,57) 6.645, p-value 0.012, in favor of the group using the AR-based application. Partial eta squared values were obtained from
the ANCOVA test in order to determine the effect size of the learning tool used (AR- or web-based) on the posttests scores. The partial eta squared values of 0.01, 0.06, and 0.14
are considered as small, medium, and large effect sizes, respectively. Partial h2 0.1044 obtained suggested a nearly large practical effect of the difference between the two
groups.

Examples of responses relating to the difculties encountered:

I had problems to make the system recognize the battery.
I was confused sometimes when the system failed to recognize elements previously recognized.
I wish I had a stand for the tablet, I was tiring to keep it in one hand while I was manipulating the elements of the circuit.
Going back and forth from reading information, to touch the tablet and manipulate objects, made me lose concentration sometimes.
Students were exposed to and required to quickly apply multiple skills: handheld and object manipulation, solving physics problems
presented within a narrative and with no training. Most of them were able to do it easily, in a very natural way. However, a short training
session would have helped to reduce the frequency of problems encountered by the others.

5. Discussion and conclusions

This study presented and evaluated an augmented reality learning application to learn the basic concepts of electromagnetism. The study
compared the AR-based application with its equivalent web-based application in order to study its learning effectiveness and students level
of enjoyment. Both applications were designed to provide the same information and workow capabilities. The main ndings and their
implications are discussed below.
With respect to the overall ow state achieved by both groups of learners, the statistical results of this study indicate that participants
who used the AR-based application were more likely to experience positive moods following the learning activity than those using the webbased application. From the analysis of the nine factors underlying users ow experiences, the experimental group showed higher levels of
concentration on the task (CT) and distorted sense of time (DT) than the control group, which might have led to the students achieving deep
learning (Liu, Cheng, & Huang, 2011). This result is consistent with the comments provided by AR-based application users who claimed that
one of the main advantages of using AR was that it helped them to concentrate in the tasks while they were building the circuit step by step.
Further studies are necessary to determine whether the manipulation features of AR in this learning context promote students concentration. In general terms, discovering the impact of each AR feature in ow factors will be useful to guide the design of better AR-based
learning applications. Statistical results also show that the experimental group had a better sense of control (SC) and clearer direct feedback (CF) than the control group, which can be benecial for the recall of information, according to Bujak et al.s (2013) study. Indeed,
students said that interacting with real objects in the AR application was a natural activity which allowed them to build a circuit step by step
and verify its physical properties in real time. However, a usability problem could have a negative inuence on these results as some
participants mentioned they had difculties handling the tablet and the real objects simultaneously. This drawback could be solved by
introducing stands for the tablets, thereby reducing extraneous cognitive load (Zaharias, 2003). Finally, participants in the AR-based
learning activity reported the highest levels of intrinsic satisfaction (AE) compared to those who used the web-based application; this
may have helped the experimental group to perform the task more easily that the control group. Those empirical ndings corroborate
theoretical claims about ow affordances of augmented reality on learning (Bujak et al., 2013; Cheng & Tsai, 2012; Dunleavy et al. 2009; Kye
& Kim, 2008). However, not all the factors seem to have inuence on learners ow state, this study found that were limited to: concentration, distorted sense of time, sense of control, clearer direct feedback, and autotelic experience. Therefore, the design of AR-learning
application should consider the strengths and limitations of affective learning affordances of this emergent technology to deploy effective learning environments.

Table 8
Descriptive statistics of pretest and posttest scores by intervention learning application and question type.
Test

Learning application

Question type

Mean

SD

Median

Trimm

Skew

Kurtosis

Pretest

Web-based

32

AR-based

28

Web-based

32

AR-based

28

Theoretical
Visualization
Theoretical
Visualization
Theoretical
Visualization
Theoretical
Visualization

2.88
0.44
2.50
0.75
3.66
1.34
3.89
2.21

0.87
0.67
0.84
0.84
1.23
1.04
0.79
0.96

3
0
3
1
4
1
4
3

2.88
0.31
2.50
0.67
3.81
1.31
3.92
2.29

0.05
1.16
0.36
0.82
0.75
0.32
0.27
0.66

0.11
0.03
0.71
0.22
0.37
1.11
0.54
1.08

Posttest

12

M.B. Ibez et al. / Computers & Education 71 (2014) 113

From the analysis of the factor that measured the balance between ability level and challenge within the Flow State Scale, no statistically
signicant difference between the two groups was found. However, the study specically collected the perception of students regarding skills
and challenge with a two question survey applied at the end of each learning stage (Pearce et al., 2005). Flow imbalances among the learning
stages were found to be mainly associated with the perceived difculty of the content and the form of interaction with the application. The
rst stage was perceived as boring and it was devoted to reviewing content through the touching interface provided by the tablet; no
experimentation was included in the AR version of this rst stage, thus both groups were exposed to similar conditions and similar results
were found. On the other end of the spectrum was the last stage which required applying the knowledge acquired in previous stages of the
learning activity and caused anxiety to participants. The nal simulation activity included in the AR application where students could observe
how the circuit behaved according to their solution was not perceived as sufcient to help them to overcome the nal challenge. Finally, the
ow state was achieved in both groups at the Magnet_2 stage, with statistically signicant differences in favor of the experimental group.
Magnet_2 presented educational content through textual information and participants had to solve a problem related to that content; its AR
version offered the possibility to experiment with the magnetic elds. The main differences between the Magnet_2 stage with the rst and the
last stages are described as follows: (1) Magnet_2 introduced new content; (2) it dealt with a moderate challenge; and (3) it helped students
by providing interactive activities. Those ndings, support the Pearce at al.s work (2005) where they remark the importance of analyze the
ow as a process rather than a just an overall state, useful information can be gather from this measure approach. In this regard, from the
results observed we conclude that students enjoyed interacting with objects and visualizing digital information associated with the real
objects in tasks that where achievable for them but not extremely easy. Thus, learning designers should strike a careful balance between AR
support and task challenge. The textual information, necessary to guide and provide educational content, sometimes acted as an intrusive
component in the visual and tactile media used. Therefore, it is necessary to nd out in which situations it is benecial to integrate textual
information in AR-based learning activities and identify effective ways to incorporate the information in this new media.
Regarding the learning effectiveness of both applications, after conducting a statistical analysis on the pre- and posttest scores, it was
found that students who used the AR application performed signicantly better on knowledge than those who were taught using the webbased application. Those ndings seem to support the outcomes of research studies (Andujar, Mejias, & Marquez, 2011; Chen, Chi, Hung, &
Kang, 2011; Kamarainen et al., 2013; Kaufmann & Schmalstieg, 2003; Lin, Duh, Li, Wang, & Tsai, 2013), which showed that AR technology
contributed to improve academic achievement compared to traditional teaching methods. Moreover, it was found that the experimental
group showed better results on visualization-related questions than students in the control group. The AR-based application enabled
students to experiment interactively with electric and magnetic elds as well as to observe the effect of magnetic forces on their circuit. It
provided instant and reliable feedback. Therefore, the AR-based application gave students the opportunity to try and observe different
options instantly, whereas the web-based application did not offer students any possibility to experiment. A possible explanation of the
effectiveness of the AR-application over the web-based application might be associated with either the concentration, control, autotelic
levels achieved, or the differences in cognitive effort required to handle the abstract concepts involved. The control group participants
required a higher cognitive effort to learn and solve problems that required visualization whereas the experimental group had better
opportunities to understand the combined behavior of electromagnetic forces. The ndings of the current study extend those of Dori et al.
(2007) who found that students improved learning outcomes through the use of visualization in a web-based course. Further studies with a
larger data sample are necessary to identify the causes of learning effectiveness of the AR-based application over the web-based application.
Some limitations of the study need to be pointed out. First, it involved short-term retention of the basic principles of electromagnetism; it
is likely that a long-term retention evaluation would have provided more insight into the effectiveness of the AR application in high school
students. Restrictions in the time tables of the high schools involved compelled us to conduct these short-term interventions. Second,
although both groups used tablets to interact with the learning environment, the novelty factor of augmented reality could act as a disturbing factor. Thus, further studies are necessary to diminish the novelty factor introduced by augmented reality technology. Third, data
collected were self reported.
Our ndings seem to support the outcomes of prior studies (Bujak et al., 2013; Dunleavy et al., 2009; Kye & Kim, 2008) which showed that
AR-based applications contributed to increased academic achievement and promoted positive emotional experiences compared to traditional
teaching in STEM elds (science, technology, engineering, and mathematics). However, the ndings of this study went further, suggesting
stronger indicators in favor of the use of AR applications for learning. Indeed, the AR-based application was not compared to traditional teaching
but to another appealing form of ICT-based learning. Furthermore, both the control and experimental groups used the same kind of handheld
device, making both groups similarly vulnerable to the novelty effect caused by the use of tablets for a learning activity. Finally, it is worth
highlighting that participants were high school students, who are typically harder to engage and motivate than children (Martin, 2012).
Based on the results of this study, it can be concluded that the AR-based application was more effective than the web-based application in
both promoting students knowledge of the basic principles of electromagnetism and in fostering ow experiences. Therefore, augmented
reality applications can be used as effective learning tools within high school electromagnetic courses.
Acknowledgments
The authors are deeply grateful to professors and students from Centro Escolar Amanecer, I.E.S. Enrique Tierno Galvn, I.E.S. Montes
Obarenes, and I.E.S. Pedro Duque (Spain) who gently collaborated on this research study. Special thanks to Prof. Mara Gregoria Casares who
contacted most of the Institutes. The last author wishes to acknowledge support from Fundacin CajaMadrid to visit Harvard University and
MIT in the academic year 2012/13.
This research has been partially supported by the Spanish project EEE (TIN2011-28308-C03-01, Plan Nacional de IDi, Ministerio de
Economa y Competitividad) and the eMadrid network (S2009/TIC-1650, Comunidad de Madrid).
References
Andujar, J. M., Mejias, A., & Marquez, M. A. (2011). Augmented reality for the improvement of remote laboratories: an augmented remote laboratory. Education IEEE
Transactions on, 54(3), 492500.

M.B. Ibez et al. / Computers & Education 71 (2014) 113

13

Antonietti, A., Rasi, C., Imperio, E., & Sacco, M. (2000). The representation of virtual reality in education. Education and Information Technologies, 5(4), 317327.
Azuma, R. T. (1997). A survey of augmented reality. In Presence: Teleoperators and Virtual Environments, 6, 355385.
Belcher, J. W., & Bessette, R. M. (2001). MIT educators share success. SIGGRAPH Computer Graphics, 35(1), 1821.
Billinghurst, M. (2003). Augmented reality in education. Retrieved from. New Horizons for Learning http://www.newhorizons.org/strategies/technology/billinghurst.htm.
Bujak, K. R., Radu, I., MacIntyre, B., Catrambone, R., Zheng, R., & Golubski, G. (2013). A psychological perspective on augmented reality in the mathematics classroom.
Computers & Education. http://dx.doi.org/10.1016/j.compedu.2013.02.017.
Chan, Y. S., & Ahern, T. C. (1999). Targeting motivation: adapting ow theory to instructional design. Journal of Educational Computing Research, 21(2), 151163.
Chen, H. (2006). Flow on the net-detecting web users positive affects and their ow states. Computers in Human Behavior, 22, 221233.
Chen, Y.-C., Chi, H.-L., Hung, W.-H., & Kang, S.-C. (2011). Use of tangible and augmented reality models in engineering graphics courses. Journal of Professional Issues in
Engineering Education and Practice, 137(4), 267276. American Society of Civil Engineers (ASCE).
Cheng, K. H., & Tsai, C. C. (2012). Affordances of augmented reality in science learning: suggestions for future research. Journal of Science Education and Technology. http://
dx.doi.org/10.1007/s10956-012-9405-9.
Chen, H., Wigand, R. T., & Nilan, M. S. (1999). Optimal experience of web activities. Computers in Human Behavior, 15(5), 585608.
Chittaro, L., & Ranon, R. (2007). Web3D technologies in learning, education and training: motivations, issues, opportunities. Computers & Education, 49(1), 318.
Choi, B., & Baek, Y. (2011). Exploring factors of media characteristic inuencing ow in learning through virtual worlds. Computers & Education, 57(4), 23822394.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaurn.
Conole, G., & Dyke, M. (2004). What are the affordances of information and communication technologies? ALT-J, Research in Learning Technology, 12(2), 113124.
Cowley, B., Charles, D., Black, M., & Hickey, R. (2008). Toward an understanding of ow in video games. Computers in Entertainment, 6(2), 127.
Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York, NY: HarperCollins Publishers.
Cuendet, S., Bonnard, Q., Do-Lenh, S., & Dillenbourg, P. (2013). Designing augmented reality for the classroom. Computers & Education, 68, 557569 (Elsevier Ltd).
Dalgarno, B., & Lee, M. J. W. (2010). What are the learning affordances of 3-D virtual environments? British Journal of Educational Technology, 41(1), 1032.
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1992). Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Psychology, 22(14), 11111132.
Dede, C., Salzman, M. C., Loftin, R. B., & Sprague, D. (1999). Multisensory immersion as a modeling environment for learning complex scientic concepts. In W. Feurzeig, &
N. Roberts (Eds.), Computer modeling and simulation in science education (pp. 282319). New York: Springer-Verlag.
Dickey, M. D. (2005). Brave new (interactive) worlds: a review of the design affordances and constraints of two 3D virtual worlds as interactive learning environments.
Interactive Learning Environments, 13(12), 121137.
Dori, Y. J., & Belcher, J. (2005). Learning electromagnetism with visualizations and active learning. In J. K. Gilbert (Ed.), Visualization in science education (pp. 187216).
Netherlands: Springer.
Dori, Y. J., Hult, E., Breslow, L., & Belcher, J. W. (2007). How much have they retained? Making unseen concepts seen in a freshman electromagnetism course at MIT. Journal of
Science Education and Technology, 16(4), 299323.
Dunleavy, M., Dede, C., & Mitchell, R. (2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of
Science Education and Technology, 18, 722.
Dnser, A., Steinbgl, K., Kaufmann, H., & Glck, J. (2006). Virtual and augmented reality as spatial ability training tools. In Proceedings of the seventh ACM SIGCHI New Zealand
chapters international conference on computerhuman interaction (pp. 125132).
Efklides, A., Kuhl, J., & Sorrentino, R. M. (2001). Trends and prospects in motivation research. Dordrecht; Boston; London: Kluwer Academia.
Faiola, A., Newlon, C., Pfaff, M., & Smyslova, O. (2013). Correlating the effects of ow and telepresence in virtual worlds: enhancing our understanding of user behaviour in
game-based learning. Computers in Human Behavior, 29, 11131121.
Garca Calvo, T., Jimnez Castuera, R., Santos-Rosa Ruano, F. J., Reina Vallo, R., & Cervell Gimeno, E. (2008). Psychometric properties of the Spanish version of the Flow State
Scale. The Spanish Journal of Psychology, 11(2), 660669.
Ghani, J. A., & Deshpande, S. P. (1994). Task characteristics and the experience of optimal ow in humancomputer interaction. The Journal of Psychology, 128(4), 381389.
Gilbert, J. K. (2005). Visualization: a metacognitive skill in science and science education. In J. K. Gilbert (Ed.), Visualization in science education (pp. 927). Netherlands:
Springer.
Hoffman, D. L., & Novak, T. P. (2009). Flow online: lessons learned and future prospects. Journal of Interactive Marketing, 23(1), 2334.
Jackson, S. A., & Marsh, H. W. (1996). Development and validation of a scale to measure optimal experience: the Flow State Scale. Journal of Sport & Exercise Psychology, 18, 17
35.
Kamarainen, A. M., Metcalf, S., Grotzer, T., Browne, A., Mazzuca, D., Tutwiler, M. S., et al. (2013). EcoMOBILE: Integrating augmented reality and probeware with environmental
education eld trips. Computers & Education, 68, 545556.
Kaufmann, H., & Schmalstieg, D. (2003). Mathematics and geometry education with collaborative augmented reality. Computers & Graphics, 27(3), 339345 (ACM Press).
Keller, J. M. (1979). Motivation and instructional design: a theoretical perspective. Journal of Instructional Development, 2, 2634.
Kye, B., & Kim, Y. (2008). Investigation of the relationships between media characteristics, presence, ow, and learning effects in augmented reality based learning. International Journal for Education Media and Technology, 2(1), 414.
Lee, E. A.-L., Wong, K. W., & Fung, C. C. (2010). How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Computers & Education,
55, 14241442.
Liao, L.-F. (2006). A ow theory perspective on learner motivation and behavior in distance education. Distance Education, 27(1), 4562.
Lin, T.-J., Duh, H. B.-L., Li, N., Wang, H.-Y., & Tsai, C.-C. (2013). An investigation of learners collaborative knowledge construction performances and behavior patterns in an
augmented reality simulation system. Computers & Education, 68, 314321.
Linnenbrink, E. A., & Pintrich, P. R. (2002). Achievement goal theory and affect: an asymmetrical bidirectional model. Educational Psychologist, 37(2), 6978.
Liu, C.-C., Cheng, Y.-B., & Huang, C.-W. (2011). The effect of simulation games on the learning of computational problem solving. Computers & Education, 57(3), 19071918.
Maloney, D. P., OKuma, T. L., Hieggelke, C. J., & Van Heuvelen, A. (2001). Surveying students conceptual knowledge of electricity and magnetism. American Journal of Physics,
69(7 Suppl. 1), S12S23.
Martin, A. (2012). High school motivation and engagement: gender and age effects. Available from http://www.eric.ed.gov/contentdelivery/servlet/ERICServlet?
accnoED532692.
Martn-Gutirrez, J., Lus Saorn, J., Contero, M., Alcaiz, M., Prez-Lpez, D. C., & Ortega, M. (2010). Design and validation of an augmented book for spatial abilities
development in engineering students. Computers & Graphics, 34(1), 7791.
Martin, S., Diaz, G., Sancristobal, E., Gil, R., Castro, M., & Peire, J. (2011). New technology trends in education: seven years of forecasts and convergence. Computers & Education,
57(3), 18931906.
Massimini, F., & Carli, M. (1988). The systematic assessment of ow in daily experience. In M. Csikszentmihalyi, & I. S. Csikszentmihalyi (Eds.), Optimal experience: Psychological studies of ow in consciousness (pp. 266287). Cambridge, New York: Cambridge University Press.
Mikropoulos, T. A., & Natsis, A. (2011). Educational virtual environments: a ten-year review of empirical research (19992009). Computers & Education, 56(3), 769780
(Elsevier Ltd).
Novak, T. P., Hoffman, D. L., & Yung, Y.-F. (2000). Measuring the customer experience in online environments: a structural modeling approach. Marketing Science, 19(1), 2242.
Papastergiou, M. (2009). Digital game-based learning in high school computer science education: impact on educational effectiveness and student motivation. Computers &
Education, 52(1), 112.
Pearce, J. M., Ainley, M., & Howard, S. (2005). The ebb and ow of online learning. Computers in Human Behavior, 21(2005), 745771.
Rapp, D. N. (2005). Mental models: theoretical issues for visualizations in science education. In J. K. Gilbert (Ed.), Visualization in science education (pp. 4360). Netherlands:
Springer.
Shin, N. (2006). Online learners ow experience: an empirical study. British Journal of Educational Technology, 37(5), 705720.
Strauss, A. L., & Corbin, J. M. (1998). Basics of qualitative research: techniques and procedures for developing grounded theory. In P. Labella (Ed.), Sage, 2; pp. 312). Sage
Publications.
Webster, J., Trevino, L. K., & Ryan, L. (1993). The dimensionality and correlations of ow in humancomputer interaction. Computers in Human Behavior, 9, 411426.
Wu, H.-K., Lee, S. W.-Y., Chang, H.-Y., & Liang, J.-C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 4149.
Zaharias, P. (2003). Developing a usability evaluation method for e-learning applications: from functional usability to motivation to learn. International Journal of HumanComputer Interaction, 25(1), 112.