Journal

 of  Interactive  Online  Learning   Volume  13,  Number  1,  Spring  2014  
www.ncolr.org/jiol   ISSN:  1541-­‐4914  
 

Mathematics Achievement: Traditional Instruction and

Technology-Assisted Course Delivery Methods

Robert Vilardi & Margaret L. Rice

Troy University & The University of Alabama

Abstract

The purpose of this study was to analyze technology-assisted course delivery methods to
determine their overall effectiveness as it pertains to mathematics courses. This study analyzed
both current and historical data in the areas of achievement, retention, and grade distribution for
mathematics classes. The study included 14,562 students enrolled in Pre-Calculus Algebra at a
Southeastern University. Significant differences were found in student achievement as
determined by course grade point average with students in the traditional course scoring higher
in average course grade point average. Students in the traditional courses also had a significantly
higher number of A’s and a significantly lower number of F’s in the grade distribution. There
was no significant difference in the overall retention rate between the 18-week traditional
courses and the technology-assisted courses, but there was a significant difference between the 9-
week traditional and the technology-assisted courses. In general, the students enrolled in the
technology-assisted courses did not perform at the same level as the students in the traditional
setting.
______________________________________________________________________________

Distance education has grown at an amazing rate throughout the last 40 years (Ashby,
2002). One area where distance education has become a widely used method is in mathematics.
While previous distance learning mediums were not conducive to success in mathematics, the
introduction of online services, multimedia tools, and tutorials have proven invaluable (Juan,
Huertas, Steegmann, Corcoles, & Serrat, 2008). Currently there are a significant number of
universities offering portions of their mathematics instruction via distance learning methods. In
addition to courses taught entirely through distance education, there are new methods of
instruction including traditional/internet hybrid courses and computer-assisted courses.
One area of mathematics where the trend has been significantly pro-distance education
and pro-internet instruction is the area of developmental studies (Ashby, Sadera, & McNary,
2011). Recently many universities have found it necessary to increase remedial mathematics
course offerings. While students have the necessary criteria for admittance into the college, their
mathematics skills are frequently substandard and must be enhanced prior to taking a credit
course. The rationale behind the use of these instructional methods are varied, including student
needs for self-paced instruction, lack of available faculty, need for quick response on assessment,
and significantly varied student educational backgrounds. The tool utilized most frequently in
these mathematics courses is the course management system, which allows mathematics
instructors to identify skills that students must learn, provide quality instruction, assign problems

  16  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

for practice, and gather assessment information. Because of the length of time needed to develop
these materials, teachers often use materials developed by textbook companies (Kennett-Hensel,
Sneath, & Pressley, 2007).
Textbook companies provide management systems complete with question banks,
homework sets, test/quiz generators, multimedia instructional tools, discussion boards, etc.
Instructors choose what they would like the students to learn out of the course modules and
assign grades based on student performance on the given tasks. Currently many schools have
incorporated courses with either distance or lab components into their regular course offerings
(Twigg, 2005). While online or web-assisted courses have many positive aspects, it is
imperative that they are analyzed for overall impact on achievement to determine what gains or
losses can be attributed to the implementation of these technologies.
The questions posed on the quality of technology-assisted courses are not dissimilar to
those posed about distance education throughout the last 50 years; however, because the primary
medium has changed, they bear revisiting. These questions range from fundamental (i.e.,
mastery) to more qualitative (i.e., student perception). It is important to determine if students
participating in this new medium reach the necessary mastery level for their course (Cotton &
Gresty, 2007). Perhaps equally important to mastery of skills is the question of whether students
in these technology-assisted courses connect concepts presented in a manner that will facilitate
success in subsequent courses. If students master skills and make connections between concepts,
it is important that the technology-assisted courses are doing this at a level that is at least on par
with that of a traditional classroom. This is particularly important in the case of prerequisite
courses. Students who take an online prerequisite for a particular course must have the same
level of preparation as students who took the course in residence (Alkharusi, Kazem, & Al-
Musawai, 2010). If students are learning the material at the same level, is that similarity carried
over into assessments and performance (Yates & Beaudrie, 2009)? Additionally, has the change
in medium had a positive or negative effect on student retention rates (Rossett & Schafer, 2003)?
After these fundamental questions have been addressed, more qualitative questions concerning
student experience should also be considered. Students in a technology-assisted course do not
have the same interactions with other students or faculty. Because of this different style of
interaction, students may perceive their learning experience differently and it is important to
gauge the impact of this change (Jackson & Helms, 2008).

Theoretical Framework

Using technology to enhance education is a topic that touches on several educational

theories. The two theories most closely related to the motivation behind this study are those of
the Cognitive Theory of Multimedia Learning and Transactional Distance Theory. These
theories play a central role in the use of technology-assisted education and offer a lens through
which to analyze the methods of instruction in the study. The Cognitive Theory of Multimedia
Learning argues that the brain processes audio and visual information in different manners
(Moreno & Mayer, 1999). If a particular concept can be demonstrated through the use of
multimedia tools, a student will have the potential to gain a better understanding than through the
use of a single media. Because the use of multimedia tools is so prevalent in technology-assisted
courses, there should be some difference in the achievement of those students who take a class in
a traditional manner versus those who take one that is technologically assisted.

  17  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

Transactional Distance Theory has been an important theory in distance education since
the 1970s. The key constructs to this theory are the educator, the learner, and communication
(Kang & Gyorke, 2008). In a traditional classroom, there is a considerable amount of interaction
between the student and the teacher. In a technology-assisted course, there is significantly less
interaction. The interplay between the instructor, the student, and the communication between
these parties in the technology-assisted courses are what constitutes an educational “transaction.”
Both of these theories offer insight into why the current embodiment of technology-assisted
courses should be preferable and offer some tangible academic difference over the traditional
classroom.
Purpose

The purpose of this study was to analyze technology-assisted course delivery

methods to determine their overall effectiveness as it pertains to mathematics courses.

Research Questions

1. Will there be a difference in achievement between students in technology-assisted

courses and students in traditional courses?
2. Will there be a difference in grade distribution between a technology-assisted course
and a traditional course?
3. Will there be a difference in retention rate between a technology-assisted course and a
traditional course?
4. Is the amount of time spent in the course a significant factor in student performance
and retention?

Methodology

Setting
The setting for this study was a major university in a southeastern state. While data for
this study could very well be ascertained from any university with technology-assisted
instruction in their mathematics departments, this public university in this southeastern state had
recently moved to this style of instruction and thus still had the historical data needed for the
analysis of the new methods. The data in this study were grouped by the method of instruction
and the semester or term when the course was taught. Courses were taught in a traditional
format and in a technology-assisted format via online delivery method. All of the technology-
assisted courses were taught on a 9-week term while the traditional courses were taught on either
an 18-week semester or a 9-week term depending on the course.

Participants
Participants were mathematics students enrolled in PreCalculus Algebra at a southeastern
university, designated as University A, during the semesters analyzed. The student grade data
for the technology-assisted and traditional courses were requested from the registrar’s office and
were anonymized for privacy issues. The PreCalculus Algebra courses were taught as
completely online, web-enhanced or traditional courses. The online and technology-assisted
courses had common formative and summative assessments, although there may have been
variation in quizzes. Some of the traditional course assessments varied from the online and

  18  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

technology-assisted courses, but the courses all used the same formative assessments, syllabus
and textbook. In the traditional courses, at most a small portion of the student work required
technology such as online homework sets.

Data Collection
Data collected included the final grades for students in several sections of PreCalculus
Algebra and retention information. The data were from all semesters from Fall 2007 to Fall
2012. To facilitate the analysis of technology-assisted course delivery methods, the university
was petitioned to provide historical data concerning the mathematics achievement of the students
in the PreCalculus courses using either online or web-enhanced components. In order to get an
accurate depiction of the effect of technology-assisted mathematics instruction, the data included
traditional classes and technology-assisted courses.

Data Analysis and Results

The data in this study were comprised of end-of-course grades and retention information.
The letter grades assigned for the course being analyzed were as follows: A (traditionally a
numerical grade 90-100), B (traditionally a numerical grade 80-89), C (traditionally a numerical
grade 70-79), D (traditionally a numerical grade 60-69), and F (traditionally a grade 59 or
below). The data also included grades of FA (failure due to absence), I (incomplete), NG (no
grade), P (pass), AU (audit), W (withdrawal), WP (withdrawal passing), WF (withdrawal
failing), DR (dropped course), DP (dropped course passing), DF (dropped course with academic
penalty), IP (in progress), and FI (course requirements not completed by end of time limit on an
incomplete).
Achievement was the first area analyzed. Achievement can be defined in multiple ways,
and is often difficult to quantify. It can mean successfully completing a course (passing) or
could be measured by student performance on a standardized test. In this case, achievement was
measured as the overall performance of a student in the course. Because achievement is
generally connected to performance, the use of student grades to determine achievement level is
appropriate. To determine the level of student achievement, the grades assigned in each course
were totaled, giving a total number of A’s, B’s, C’s, D’s, and F’s assigned for a given semester.
By taking the proportion of these grades to the total number of students in the course and using
standard weights of A = 4.0, B = 3.0, C = 2.0, D = 1.0, and F = 0.0, the overall grade point
average for a given semester/term was obtained. These course grades were grouped into
traditional courses and into technology-assisted courses. The grade point averages were
calculated for each semester/term and were then analyzed for average grade point averages and
standard deviation.
The average course grade point average for traditional courses was 2.14 and 1.70 for
technology-assisted courses. The null hypothesis of “There is no significant difference in the
overall achievement of students in a traditional course and those in a technology-assisted course”
was tested using a t test for means. The t test yielded a t value of 4.28 with 30 degrees of
freedom. The critical t value at the 0.05 level of confidence with 30 degrees of freedom is t =
2.04, therefore, the null hypothesis was rejected. The traditional courses had a significantly
higher overall course grade point average than the technology-assisted courses.
The second area analyzed in this study was grade distribution using the null hypothesis
“There will be no significant difference in the grade distributions of traditional courses and

  19  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

technology-assisted courses.” The grade distribution for a given course was the number of each
grade that was assigned to the students. To determine the effect the course delivery method has
on grade distribution, the standard grades of A, B, C, D, and F were analyzed in the traditional
courses and the technology-assisted courses. The proportions that these grades appeared in the
courses was computed and a proportion test was used to determine if there was a significant
difference in the number of A’s, B’s, C’s, D’s, and F’s given in the courses. When grouping all
of the traditional courses together and all of the technology-assisted courses together, the
proportion test yielded the following z-scores for the grades: A - z = 12.42; B - z = 1.62; C - z =
1.71; D - z = -0.85; and F - z = -14.20. At the 0.05 level of significance, the critical z-score is
1.96 and for A and F, the null hypothesis was rejected. This indicates a significant difference in
the proportion of A’s and the proportion of F’s, indicating a difference in grade distribution
between traditional courses and technology-assisted courses. The traditional courses had a
significantly higher proportion of A’s, and the technology-assisted courses had a significantly
higher proportion of F’s.
The third area of study was student retention, analyzed using the proportion test with the
null hypothesis “There is no significant difference in the retention rates of students in traditional
courses and those in technology-assisted courses.” In order to determine if a student completed a
course using only end-of-course grades, the grades that indicate failure to finish the course were
analyzed. In this study, those grades were FA, W, WF, WP, DR, DP, and DF. Although there
are other grades possible (such as I-incomplete), they are not well defined and a student’s
retention may not be apparent. For each of the FA, W, WF, WP, DR, DP, and DF grades, the
student did not complete the course, and therefore, was not retained. The differences in the
categories of these grades is primarily due to when the student decides to leave the course and
what the student’s grade is at that time. These grades were divided into traditional courses and
technology-assisted courses, and a proportion test was performed to determine if there was a
significant difference in the retention rates in the courses. The proportion test yielded the
following z-scores: FA - z = 1.39; W - z = -5.47; WF - z = -4.89; WP - z = -2.80; DR - z = -0.61;
DP - z = 1.97; and DF - z = -2.80. At the 0.05 level of significance, the critical z-score was 1.96.
For W, WF, WP, DP, and DF, the null hypothesis was rejected and for FA and DR, the null
hypothesis was not rejected. This indicated a significant difference in the proportion of several of
the non-completion grades. There were higher proportions of the grades of DP in the traditional
courses and higher proportions of W, WF, WP, and DF in the technology-assisted courses.
The study data were further analyzed to compare the length of the course to student
achievement, grade distribution, and retention rates. The length of the traditional courses was
either 18-week semesters or 9-week terms; however, the length of each of the technology-
assisted courses was 9 weeks. This change in the length of the course is an important factor and
was studied to ensure that the differences observed were due to the course delivery method and
not the length of the course. In order to determine what effect the length of course had on
student achievement, the traditional courses were divided into two categories: the 18-week
courses and the 9-week courses. The technology-assisted courses were grouped as previously.
Upon regrouping the data, the average course grade point average of the 18-week traditional
students was 2.02, and was 2.35 for the 9-week traditional students. These averages were
compared to the technology-assisted course average of 1.70 using a t test. In both the 18-week
traditional course and the 9-week traditional course there were significant differences, indicating
that the change in student achievement was not simply a result of the length of the course. In the

  20  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

18-week traditional courses and the 9-week traditional courses, the overall course grade point
average was significantly higher than the technology-assisted courses.
The proportions of each of the grades were analyzed to determine what effect course
length had on grade distribution, and in the case of the 18-week traditional courses, the grades of
A, C, and F had proportions that differed by significant amounts. The z-scores for these grades
were 10.22, 1.99, and -12.54, respectively. In the case of the 9-week traditional courses, the
grades of A, B, D, and F had proportions that differed by significant amounts. The z-scores for
these grades were 13.80, 5.52, -5.93, and -10.72, respectively. In both the 18-week traditional
courses and the 9-week traditional courses, there were significant differences in the proportions
of the grades assigned. In the 18-week traditional courses, there were significantly higher
proportions of A’s and C’s. In the technology-assisted courses there were a significantly higher
number of F’s. When comparing the 9-week traditional and the technology-assisted courses, the
traditional courses had a higher proportion of A’s and B’s while the technology-assisted courses
had a higher proportion of D’s and F’s. Because of these differences, the length of course is not
the only factor causing the change.
The proportion tests were run to determine if length of course was a cause of the change
for FA, W, WF, WP, DR, DP, and DF. In the 18-week traditional courses, FA, W, WF, WP, DP,
and DF all were determined to have significant proportions with z-scores of 2.61, -6.24, -4.49, -
3.34, 2.49, and -3.34, respectively. The 9-week traditional course was determined to have
significant differences in the proportions of FA, WF, and DR with z-scores of -2.99, -2.49, and -
4.65, respectively. This indicates significant differences in the proportions of the grades
associated with retention rate. There were higher proportions of FA, WP, and DP in the 18-week
traditional courses and higher proportions of W, WF, and DF in the technology-assisted courses.
In the 9-week traditional courses, there were higher proportions of FA, WF, and DR. There were
higher proportions of FA in the 9-week traditional courses, but higher proportions of WF and DR
in the technology-assisted courses.
In addition to each individual non-completing grade distribution, the number of non-
completing grades for students was totaled and the proportion of non-completion grades was
analyzed for the traditional and the technology-assisted courses. Because there are multiple
ways in which a student can fail to complete a course, the total number of students who did not
complete a course was analyzed. The result of this computation is valid regardless of when the
withdrawal was processed or if the student was passing at that moment they left the course. The
first test performed was on the traditional courses and the technology-assisted courses and tested
the null hypothesis “there will be no significant difference between the proportion of students
who fail to complete the traditional course and those who fail to complete the technology-
assisted course.” The z-score of -3.09 was beyond the critical z-value and thus the null
hypothesis was rejected. The 18-week traditional courses did not have a significant difference in
course completion. The 9-week traditional courses had a significant difference in the proportion
of non-completion grades to the technology-assisted courses with the technology-assisted
courses having a higher proportion of non-completion grades.
The course-retention results also were analyzed to determine if course length was a factor
in the proportions of student non-completion. The traditional group of students was divided into
18-week traditional and 9-week traditional. The technology-assisted courses were grouped as
previously. A proportion test was run on the 18-week traditional course compared with the
technology-assisted course to determine if there were significant differences in the non-
completion grades. The z-score was z = -1.51, which is lower than the critical z-score at the 0.05

  21  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

level of confidence; therefore, the null hypothesis was not rejected. A proportion test was run on
the 9-week traditional course compared with the technology-assisted course to determine if there
were significant differences in the non-completion grades. The z-score was z = -5.80, which is
greater than the critical z-score at the 0.05 level of confidence; thus, the null hypothesis was
rejected. Because the null hypothesis was not rejected for the 18-week traditional courses and
was rejected for the 9-week traditional courses, there may be a connection between course length
and retention rates between traditional courses and technology-assisted courses.
The biggest surprise of this study was a change in the grade distribution in the
technology-assisted courses in the Fall of 2010. Prior to Fall 2010, the average percentage of
A’s in the technology-assisted courses was 0.19. Courses taught during Fall 2010 and after had a
percentage of A’s in technology-assisted courses of 0.05. This difference of 0.14 is significant
and is seen in each term after Fall 2010. The causes of the shift in grade distribution is not
covered in the scope of this study, but due to the markedly different values, the courses before
and after the change were analyzed to determine the effect on the results of the study. The
courses were divided into courses taught prior to Fall 2010 and courses taught Fall 2010 and
after. Once again, because course length can be a factor, the courses were grouped into three
categories: 18-week traditional, 9-week traditional, and technology-assisted. For grade
distribution the grades of A, B, C, D, and F were analyzed using the proportion test. The null
hypotheses are that “There will be no significant difference in the grade distributions of 18-week
traditional courses and technology-assisted courses taught prior to Fall 2010,” and “There will be
no significant difference in the grade distributions of 9-week traditional courses and technology-
assisted courses taught prior to Fall 2010.”
The data were grouped into 18-week traditional, 9-week traditional, and technology-
assisted over the semesters from Fall 2007 to Summer 2010 and each assigned grade was
analyzed to determine if there was a significant difference in the proportion of the grades
assigned. The proportion test yielded the following z-scores for the 18-week traditional course
compared to the technology-assisted course: A - z = 3.21; B - z = -0.47; C - z = 3.30; D - z =
1.34; and F - z = -9.88. At the 0.05 level of significance, the critical z-score was 1.96 and for A,
C, and F, the null hypothesis was rejected. This indicates that the proportions of A, C, and F are
significantly different, with the 18-week traditional courses having a higher proportion of A’s
and C’s. The technology-assisted courses had a higher level of F’s. For the grades of B, and D,
the null hypothesis was not rejected.
The proportion test yielded the following z-scores for the 9-week traditional course taught
prior to Fall 2010 compared to the technology-assisted course: A - z = 6.92; B - z = 4.71; C - z =
1.96; D - z = -3.45; and F - z = -10.10. At the 0.05 level of significance, the critical z-score was
1.96 and for A, B, D, and F, the null hypothesis was rejected. This indicates that the proportions
of the grades A, B, D and F are significantly different, with the 9-week traditional courses having
a higher proportion of A’s and B’s. The technology-assisted courses had a higher level of D’s
and F’s. For C, the null hypothesis was not rejected.
The grade distributions for courses taught Fall 2010 and after were also analyzed and the
grades of A, B, C, D, and F were analyzed using the proportion test. The null hypotheses are that
“There will be no significant difference in the grade distributions of 18-week traditional courses
and technology-assisted courses taught Fall 2010 and after,” and “There will be no significant
difference in the grade distributions of 9-week traditional courses and technology-assisted
courses taught Fall 2010 and after.”

  22  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

The data were grouped into 18-week traditional, 9-week traditional, and technology-
assisted groups over the semesters from Fall 2010 to Fall 2012 and each assigned grade was
analyzed to determine if there was a significant difference in the proportion of the grades
assigned. The proportion test yielded the following z-scores for the 18-week traditional course
compared to the technology-assisted course: A - z = 13.54; B - z = 0.79; C - z = -0.67; D - z = -
0.08; and F - z = -9.11. At the 0.05 level of significance, the critical z-score was 1.96 and for A
and F, the null hypothesis was rejected. For B, C, and D, the null hypothesis was not rejected.
The proportion test yielded the following z-scores for the 9-week traditional course taught Fall
2010 and after compared to the technology-assisted course: A - z = 14.92; B - z = 2.41; C - z = -
1.66; D - z = -4.77; and F - z = -4.32. At the 0.05 level of significance, the critical z-score was
that of 1.96 and for A, B, D, and F, the null hypothesis was rejected. For the grade of C, the null
hypothesis was not rejected. The apparent change in the distribution of A’s in the Fall 2010 is an
area where more study is warranted; however, when considering the distributions before and
after this change, there are still significant differences in grade distributions between the
traditional courses and the technology-assisted courses.

Discussion

The finding that there are significant differences in achievement for students enrolled in a
traditional course versus a technology-assisted course supports Larreamendy-Joerns and
Leinhardt (2006). The authors contended that although there is academic value to courses taught
online, the level at which the students learn is inferior to other methods. While there are other
studies that have shown differing results (Barbour & Mulcahy, 2008), the data gathered in the
current study would support the position that students are learning in the technology-assisted
classroom, but not at the same level as their traditional counterparts.
There are many studies in which students in an online classroom have higher
achievement levels than their traditional counterparts, and there are other studies where the
opposite relationship is present. One reason behind this difference may be the support of the
students in the classroom. If students have a difficult time navigating the online environment,
then their academic performance may suffer as well (Muilenburg & Berge, 2005). The
technology-assisted courses in this study utilized MyMathLab, which is a proprietary course
shell created by Pearson Education. While having a course shell is beneficial, it may have been
an obstacle in this study. Additionally, the students in the technology-assisted courses in this
study had all of their assessments online and these assessments were automatically graded.
There is a significant challenge in grading mathematics problems due to syntax and format.
Although computers have come a long way, there are still issues when it comes to mathematical
symbols (Loch & McDonald, 2007). Instructors for the traditional courses had the option of
awarding partial credit for problems, which did not happen with the technology-assisted courses.
The change in grade distribution between students in the traditional courses and those in
the technology-assisted courses could be attributed to course design issues, grading obstacles,
time management, and course readiness. While students enrolled in the traditional course would
need to have the same prerequisite courses or placement as students in the technology-assisted
courses, the students may not become aware of their deficiencies as quickly and may not be able
to remediate themselves in the time allotted. While online remediation tools exist and are often
successful (Biesinger & Crippen, 2008), students may not know what areas need to be addressed
and may have trouble locating the resources online.

  23  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

The students in the technology-assisted courses in this study showed a significant change
in grade distribution. Because the grades in the study were end-of-course grades, these reflect
the average of the student performance for the entire course. The grades in each of the courses
studied were an average of the formative and summative assessments taken throughout the
course. The difference in grade distribution, as well as student achievement, may stem from a
problem noted by the U.S. Department of Education (2005). In this report, the National
Assessment of Educational Progress (NAEP) noted that the computer grading of the constructed-
response questions did not agree with the human grading. Additionally, it is difficult once a
question has been developed to produce benchmarks for partial credit. Finally, the NAEP found
that the electronic device used to take the assessments, whether school computer or NAEP
provided laptop, affected the outcomes of the assessment. Because students in technology-
assisted courses are assessed in an all-online format, differences in the assessments, grading, and
the device the students use to access the course may all attribute to the change in grade
distribution.
In addition to changes in achievement and grade distribution, there were significant
differences in retention rates among the traditional courses and the technology-assisted courses.
Students in both traditional courses and technology-assisted courses have trouble with time
management, motivation, course design, and communication, but these issues are more distinct
in an online setting (Aragon & Johnson, 2008). Among the grades that indicate non-completion
of a course, there were several that showed significant differences, and between the 9-week
traditional and the technology-assisted courses, there was a significantly higher number of total
non-completers in the technology-assisted courses. The magnification of the issues that lead to
student non-completion could have caused the changes in the proportion among non-completion
grades and the change in totals among these courses.
The Cognitive Theory of Multimedia Learning and Transactional Distance theory are the
theories on which this study on technology-assisted mathematics education was centered. The
Cognitive Theory of Multimedia Learning states that if a particular concept is delivered using
multiple mediums a student should have better understanding. This increase in comprehension is
due to the brain processing this information in multiple manners (Moreno & Mayer, 1999). The
achievement and grade distributions that were seen in this study are not consistent with an
increase in student comprehension. The lack of an increase in this study, however, does not
contradict the theory. Although there were many multimedia tools available to the students in
the technology-assisted courses, their use was not required. Students could take the course and
finish the assessments without having to access the multimedia resources. Because students
could choose not to utilize the multimedia tools, they may not have experienced the potential
benefits. Students in the technology-assisted courses needed to seek out the multimedia tools.
They were not required as part of the course.
Transactional Distance Theory is relevant to this study because it is possible that the
technology-assisted courses did not provide for all three of the interactive components that
provide a meaningful learning experience: dialog, structure and autonomy or self-directedness
of learners (Kang & Gyorke, 2008). In a traditional classroom, there is a considerable amount of
interaction between the student and the teacher. In a technology-assisted course, there may be
significantly less interaction. The technology-assisted courses in the study had no requirement
for interaction between the instructors and the students, so it is possible that students could
complete the course without communicating with the instructor. This would mean that the
students would not experience the dialog or self-directedness of Transactional Distance Theory,

  24  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

which could have affected their achievement. Because the course did not require any type of
educational “transaction,” students in the technology-assisted classroom may have missed
valuable learning opportunities.

Implications

Much of the research conducted prior to this study indicates that students in a technology-
assisted course can perform at a level consistent with or even higher than the students in a
traditional course. This study, however, showed considerable differences between the two
groups with the traditional courses having a higher level of achievement and a better grade
distribution. Because of the differences between prior research and this study, the root cause of
the differences must be analyzed. Prior research indicates that in many cases when students in a
technology-assisted course delivery method fail to perform as well as students in a traditional
course, the course design or assessment methods may be the cause. Students in the technology-
assisted courses in this study are not performing as well as the students in the traditional courses
and because of the scope of this study both in sample size and time, the cause is most likely not
due to some external factor such as time management. The use of computerized testing methods
for assessments and policies regarding student engagement in technology-assisted courses may
need to be examined and changed, in order for achievement levels in technology-assisted courses
to rise and grade distributions to more accurately mirror populations.

Conclusion

This study revealed that students in the technology-assisted courses did not perform at the
same level as students in the traditional courses. Significant differences were found in student
achievement as determined by course grade point average, with students in the traditional course
scoring higher in average course grade point average. Students in the traditional courses also had
a significantly higher number of A’s and a significantly lower number of F’s in the grade
distribution. There was no significant difference in the overall retention rate between the 18-
week traditional courses and the technology-assisted courses, but there was a significant
difference between the 9-week traditional and the technology-assisted courses. These results
should be viewed in the context of this study and may not be generalizable to other types of
online courses or subject areas.
Because the data in this study yielded results that were significant and challenging to
some of the other research that has been conducted in this area, it is important that further
research is conducted into the grading practices both in the traditional courses and the
technology-assisted courses to determine if any incongruities exist. Since the technology-
assisted courses in this study did not have multimedia learning module requirements or any
requirement for communication between the students and the instructor, other technology-
assisted PreCalculus courses should be analyzed where these are part of the course requirement.
Additionally, it is important to look at not only end-of-course grades, but also student grades
throughout the course to determine if the changes are localized or consistent. This research
should be repeated on other courses in the mathematics curriculum including courses that are
prerequisites and subsequent courses to determine if the differences noted in this study are
evident in other courses as well. Finally, research should be conducted with students who take

  25  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

more than one mathematics course to determine how traditional and technology-assisted courses
prepare students for subsequent courses.

  26  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

References

Alkharusi, H., Kazem, A., & Al-Musawai, A. (2010). Traditional versus computer-mediated
approaches of teaching educational measurement. Journal of Instructional Psychology,
37(2), 99-111.
Aragon, S. R., & Johnson, E. S. (2008). Factors influencing completion and noncompletion of
community college online courses. The American Journal of Distance Education, 22(3),
146-158.
Ashby, C. M. (2002). Distance education: Growth in distance education programs and
implications for federal education policy: GAO-02-1125T. Washington, DC: GAO
Reports.
Ashby, J., Sadera, W. A., & McNary, S. W. (2011). Comparing student success between
developmental math courses offered online, blended, and face-to-face. Journal of
Interactive Online Learning, 10(3), 128-140.
Barbour, M. K., & Mulcahy, D. (2008). How are they doing? Examining student achievement in
virtual schooling. Education in Rural Australia, 18(2), 63-74.
Biesinger, K., & Crippen, K. (2008). The impact of an online remediation site of performance
related to high school mathematics proficiency. Journal of Computers in Mathematics
and Science Teaching, 27(1), 5-17.
Bondeson, W. (1977). Open learning: Curricula, courses, and credibility. The Journal of Higher
Education, 48(1), 96-103.
Cotton, D. R., & Gresty, K. A. (2007, October). The rhetoric and reality of e-learning: using the
think-aloud method to evaluate an online resource. Assessment & Evaluation in Higher
Education, 32(5), 583-600.
Jackson, M. J., & Helms, M. M. (2008). Student perceptions of hybrid courses measuring and
interpreting quality. Journal of Education for Business, 7-12.
Juan, A., Huertas, A., Steegmann, C., Corcoles, C., & Serrat, C. (2008, June). Mathematical e-
learning: State of the art and experiences at the Open University of Catalonia.
International Journal of Mathematical Education in Science and Technology, 39(4), 455-
471.
Kang, H., & Gyorke, A. S. (2008). Rethinking distance learning activities: A comparison of
transactional distance theory and activity theory. Open Learning, 23(3), 203-214.
Kennett-Hensel, P. A., Sneath, J. Z., & Pressley, M. M. (2007). PowerPoint and other publisher-
provided supplemental materials: Oh lord. What have we done? Journal for Advancement
of Marketing Education, 10(Summer), 1-11.
Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education.
Review of Educational Research, 76(4), 567-605.
Loch, B., & McDonald, C. (2007). Synchronous chat and electronic ink for distance support in
mathematics. Innovate: Journal of Online Education, 3(Feb-Mar), 6.
Mastin, D. F., Peszka, J., & Lilly, D. (2009). Online academic integrity. Teaching of Psychology,
36, 174-178.
Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of
modality and contiguity. Journal of Educational Psychology, 91(2), 358.
Muilenburg, L. Y., & Berge, Z. L. (2005, May). Student barriers to online learning: A factor
analytic study. Distance Education, 26(1), 29-48.

  27  
 
 Journal  of  Interactive  Online  Learning   Vilardi  &  Rice  
 

Naismith, L., & Sangwin, C. J. (2004). Computer algebra based assessment of mathematics
online. Proceeding of the 8th Annual CAA Conference (pp. 235-242). Loughborough:
Loughborough University.
Rossett, A., & Schafer, L. (2003). What to do about e-dropouts. T+D, 57(6), 40-46.
Twigg, C. A. (2005). Increasing success for underserved students: Redesigning introductory
courses. Saratoga Springs, NY: National Center for Academic Transformation.
U.S. Department of Education. (2005). The national assessment of educational progress online
assessment in mathematics and writing: Reports from the NAEP technology-based
assessment project, research and development series. Washington DC: National Center
for Education Statistics.
Yates, R. W., & Beaudrie, B. (2009). The impact of online assessment on grades in community
college distance education mathematics courses. The American Journal of Distance
Education, 23(2), 62-70.

  28