Computers & Education 99 (2016) 68e80

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

Closing the gaps e Improving literacy and mathematics by

ict-enhanced collaboration

lii Genlott*, Åke Gro
Annika Age € nlund
€
Orebro €
University, Swedish Business School, Informatics SE-70182, Orebro, Sweden

a r t i c l e i n f o a b s t r a c t

Article history: Literacy and mathematics are necessary skills that for different reasons unfortunately not
Received 5 June 2015 everybody acquires sufficiently. In OECD countries there is also a gender gap; boys lag
Received in revised form 29 February 2016 behind girls in literacy but often outperform girls in mathematics (OECD., 2012). ICT (In-
Accepted 7 April 2016
formation and communication technologies) may contribute useful tools to address both
Available online 11 April 2016
these problems but in order to effectively create better educational conditions there is yet
a need to develop effective methods that combine ICT with key factors for learning. This
Keywords:
research contributes to this by measuring effects of the “Write to Learn” (WTL) method.
Literacy/mathematics
Collaborative learning
WTL lets children from 1st grade use several ICT tools to write texts and subsequently
ICT discuss and refine them together with classmates and teachers using digital real-time
Formative feedback/assessment formative feedback and assessment. The central learning factor addressed, in mathe-
Gender gap matics as well as in literacy, is the written communication allowing the learners to interact
with peers and teachers. WTL draws on methods from socio-cultural theory, including
continuous social interaction and written real-time formative feedback among peers, using
shared electronic forums for collaboration, thereby providing social meaning and
increased learning of literacy and mathematics, among both boys and girls.
The study uses quantitative methods and two control groups, one using traditional method
(no ICT) and one using technology individually (without integrated social interaction and
formative feedback), to compare results from 502 students in grade 3 national tests in
mathematics and literacy. WTL yields by far best results; higher average score both in
literacy and mathematics, smaller gender gap, and significantly better results for the
under-achievers. The ITU method performs worst, which shows that ICT use must be well
integrated into the pedagogy to be useful.
© 2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

Improving literacy is a challenge across the world, for many reasons. The information society requires increased ability to
handle information and interact collaboratively than the industrial society, and certainly than the agrarian one from which
many people today take a step directly to the information society. Therefore, more people in the world need enhanced literacy
skills to be able to find, select, interpret, analyze, and produce information relevant to them. This development also requires
that educational systems equip young students with new skills and competencies, sometimes called 21st century skills

* Corresponding author.
E-mail address: [email protected] (A.A. Genlott).

http://dx.doi.org/10.1016/j.compedu.2016.04.004
0360-1315/© 2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/
licenses/by-nc-nd/4.0/).
 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro 69

(European Commission, 2007; European Commisson, 2010), which allow them to collaborate and contribute actively in ways
more related to needs of social development than suited to an industrial mode of production (Ananiadou & Claro, 2009).
This leaves us with a serious challenge as to pedagogical methods for employing ICT in literacy development. While the
amount of computers and other relevant technologies is increasing in schools there is considerable uncertainty among
teachers concerning how to use new technology and digital tools for enhanced learning among students. There is generally a
scarcity of tested such methods. Most research is very small-scale e one or a few classes e and there is no compelling evi-
dence of direct positive influence of technology use (Means, Toyama, Murphy, Bakia, & Jones, 2009). While technology in itself
does not lead to improvement of student results, it may well be used to reinforce pedagogic factors that have been shown to
have positive impact.
Two such key success factors for learning in focus of this study are formative feedback and formative assessment.
According to Cohen (1985) feedback “… is one of the more instructionally powerful and least understood features in
instructional design” (p 33). Feedback, when combined with effective instructions in classrooms is found to be very powerful
in enhancing learning (Hattie & Timperley, 2007). It is in fact identified as one of the most important factors for learning
(Hattie, 2009). Black and Wiliam (2009) argue that the quality of interactive feedback is crucial when determining the quality
of a learning activity; feedback is hence a central feature of pedagogy. Mueller and Dweck (1998) discuss the importance of
how feedback is delivered, showing that (teacher) praise for (student) effort strengthens the learning process while praise for
intelligence may rather be counterproductive.
Another success factor is for students to give formative feedback to their peers. According to the literature review by Shute
(2008), most research conducted in the area of formative feedback shows that good feedback can significantly improve
learning processes. Formative feedback may benefit the learner through several cognitive mechanisms, both to identify a gap
between a desired goal and the current level of performance and to bridge that gap, which motivates greater effort (Locke &
Latham, 1990; Song & Keller, 2001). Formative feedback can also reduce students' cognitive load and is particularly important
for those who are novices or struggle (Paas, Renkl, & Sweller, 2003).
Another key factor for learning is formative assessment. Unlike summative assessment, formative assessment involves “…
the process of curriculum construction, teaching, and learning for the purpose of improving any of these three processes”
(Bloom, Hastings, & Madaus, 1971 p 117). What is well suited to summative assessments does not necessarily serve its
purpose for formative assessment on a daily basis in order to improve learning (Black & Wiliam, 2003). In particular, aspects
of learning that are not easily quantitatively measurable may be crucial for formative assessment.
According to Black and Wiliam (2009), “Practice in a classroom is formative to the extent that evidence about student
achievement is elicited, interpreted and used by teachers, learners or their peers, to make decisions about the next steps in
instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the
evidence that was elicited” (p 9).
In today's information society there is a need to develop effective methods for implementing key factors for learning
making best possible use of ICT innovations. The method presented and assessed in the present study, Write to Learn (WTL), is
based on a model for combining social and formative interaction between younger peers and teachers with innovative use of
ICT for increasing the effects of those learning factors. Drawing on socio-cultural theory the method includes a social view of
learning combining collaboration, formative feedback, and formative assessment with strategic ICT use.
There is also a gender gap in education. According to the OECD report, “Closing the Gender Gap Act Now” (OECD., 2012),
boys in OECD countries lag behind girls in reading skills and are less likely to spend time reading for pleasure, but perform
better in mathematics, even though that gap is narrower than the one in literacy. According to the national tests in Sweden in
mathematics grade 3, there is a mixed picture and no systematic gender gap (Skolverket, 2014). In higher grades boys in
Sweden perform slightly better than girls in the national mathematics tests, although girls still overall receive higher grades
in both mathematics and literacy (Skolverket, 2014). Boys in high-income countries are also more likely to drop out of
secondary education and young women are increasingly better educated than young men in many OECD countries. In order to
reap the highest economic and social return on education investment it is important to find ways to reverse the imbalance
and close the gender gap (OECD., 2012).
The WTL method was designed to make use of ICT in order to facilitate writing and improve feedback and formative
assessment. This paper reports results from a test of the method on a total sample of 502 students from a Swedish city. The
results of the intervention group are compared to two control groups, one using no ICT and one using ICT “spontaneously”, i.e.
without this specific method of structured, formative feedback and social interaction.
This research draws on an earlier pilot study (n ¼ 87), in which we found that the test group outperformed the control
group (Age
lii Genlott & Gro
€nlund, 2013). The present study goes on to test the effects of the same method on a larger sample
of students (n ¼ 502). We also include mathematics as observations during the pilot study suggested that the method would
yield positive results also in that field of knowledge. From these indications, and from the knowledge of how written
communication allows learners to engage and interact with peers in “… the cultivation of mathematical knowledge”
(Brendefur & Frykholm, 2000 p 125) we hypothesized that the social context and formative feedback and assessment the
method implements were conducive also for learning mathematics e in fact the most important factor as there were no
math-specific ICT tools used. In this extended study, therefore, we include results in mathematics and compare WTL with not
only traditional training but also with training using ICT without the WTL method.
70 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro

2. Theoretical perspective

There is comprehensive research on reading and writing skills, and there are various theoretical perspectives. This
research draws on socio-cultural theory developed from the work of Vygotsky (1962); Vygotsky and Eccles (1978) proposing
that learning is an active process which also is not just an individual matter but develops within a social context. Hence,
context is important for learning, (Hall, 2007). The role of the human mediator is defined by Vygotsky and Eccles (1978)
through the notion that “… each psychological function appears twice in development, once in the form of actual interac-
tion between people and the second time as an inner internalized form of this function” (Chaiklin, 2003). Socio-cultural
theories emphasize the social, cultural, and historic situation of language (Street, 1984); (Barton, 2001). In this perspective,
learning must be anchored in the learners' experiences, creativity, and interests (Frederickson & Cline, 2002; Liberg, 2006).
Learning takes place in formal as well as in informal contexts, and important aspects of learning include use of tools and
development of artifacts (Sa€ljo
€ , 2000; 2002). Luke and Peter (1997) add the dimension of critical thinking; reading is about
engaging in, reshaping, and critically examining the ways in which the world is described. Reading and writing are culturally
defined social activities and gaining literacy is about learning through communicating in social relations with parents,
teachers, friends, and media. Texts always represent values and views. Literacy hence includes social use of language. In this
perspective literacy activities in the classroom become crucial. Following a social-cultural perspective, the “Four literacy
resources model” Luke and Peter (1997) specifies a set of practices that children need to participate in so as to develop into
good readers;

 Breaking the code of written texts;

 Participating in understanding and composing meaningful written, visual and spoken texts;
 Using texts functionally;
 Critically analyzing and transforming texts.

Drawing on this theoretical perspective, and very practically using the four literacies model, the WTL method developed in
this research includes teaching and learning activities designed for all four practices. WTL also extends the fourth practice,
critically analyzing texts, by the use of formative feedback- and assessment. It also uses ICT tools for all stages, which dis-
tinguishes it from many other ICT-supported methods, even those using similar names.

3. Purpose and hypotheses

The purpose of the research is to test if the ICT-supported method WTL, Write To Learn e which implements some ele-
ments from socio-cultural learning theory, notably formative feedback and formative assessment e yields better student
results than two other forms of literacy training; (1) “traditional”, without ICT use and (2) using ICT but not the WTL method.
This study extends our previous pilot study (Age
lii Genlott & Gro€ nlund, 2013) both by size and by scope. It employs two
control groups, a “traditional” (not using ICT at all) and one where ICT was used but in an individual manner without any
specific method for learning through social interaction and formative feedback among classmates. In summary the study
extends the previous one in the following ways:

1. A larger sample (n ¼ 502);

2. Use of the standard national tests for Grade 3;
3. Separating the technology and the method factors by including also a third method where ICT is used but without the WTL
method (hereafter called the “individual technology use” (ITU);
4. A longer time period. This study measured results after three school years as compared to one year in the previous study;
5. Including tests in mathematics.

The overall hypothesis of this study is that ICT use in combination with a pedagogical method including formative feedback
and assessment as a digital social interaction among peers improves results in literacy development and in mathematics. This
overall hypothesis is operationalized by the following more specific ones.
H1. WTL leads to higher student achievement in literacy compared to the traditional method.
H2. WTL leads to higher student achievement in literacy compared to the ITU method.
H3. WTL leads to higher student achievement in mathematics compared to the traditional method.
H4. WTL leads to higher student achievement in mathematics compared to the ITU method.
H5. WTL leads to better performance among low performers than the traditional method.
H6. WTL leads to better performance among low performers than the ITU method.
Since girls' overall better performance in school is often attributed to their better ability to adapt to the school environ-
ment and organize their work we also hypothesized that a more structured method and technical tools supporting a social
 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro 71

learning-oriented peer culture (Legewie & DiPrete, 2012) would improve the boys' results even more than the girls'. Hence
the following additional hypotheses:
H7. WTL leads to a lower gender gap in performance than the traditional method.
H8. : WTL leads to a lower gender gap in performance than the ITU method.
The study provides empirical evidence that the WTL method is effective in both literacy and mathematics development
and helps closing the gender gap.
More generally the study contributes by showing that ICT use in schools without structured integration with pedagogy is
not effective but may in fact prove detrimental. It also shows that ICT can effectively be used for extended social learning
activities and it provides evidence that social learning theory is a good base for teaching and learning.

4. The WTL method

The WTL (write to learn, first named iWTR for “integrated write to read”) method was initially developed in 2011 with
inspiration from Trageton (2003). The aim was to structure crucial aspects concerning how to integrate literacy education of
young students with ICT so as to make the best use of the opportunities offered by technology. The overall aim was to increase
literacy skills and close the gender gap among young students (Age
lii Genlott & Gro
€ nlund, 2013). For the present study the
method was further elaborated by improving certain key factors for learning. In particular, written social interaction within
mathematics teaching was added, hence the name change to WTL. This extension was made due to indications of better
student performance also in mathematics during the pilot study (ibid). WTL emphasizes collaborative learning and has been
influenced by several effective pedagogical factors, in particular the ones mentioned above, feedback (Hattie, 2009; Hattie &
Timperley, 2007), formative feedback, and formative assessment (Black & Wiliam, 2003; 2009; Shute, 2008). Other important
pedagogical factors included in WTL model include visible learning (Hattie, 2009) and early learning of different literacy
genres (Rothery, 1996).

4.1. The WTL cycle

WTL prescribes social use of ICT involving communication and written, formative feedback and assessment among stu-
dents as well as between students and teachers. The method lets children from 1st grade use computers or tablets and a set of
software tools to collaboratively write texts and subsequently discuss and refine them together with classmates and teachers.
“Texts” in literacy training are various stories, in mathematics they are discussions about how to understand and solve
mathematical problems. Children work in pairs writing texts, taking turns typing and reading. The child who is not writing
provides both inspiration to formulating the text and a check that it is properly written. If needed the speech synthesis app
ClaroRead is used to let the children check how their writing sounds. For interaction Google Apps for Education (GAFE) were
used, mainly Google Drive and Google Sites. The students give each other written formative feedback through shared digital
documents iteratively during the writing process (production of longer texts extends over several days or even weeks).
Finally, they publish their written work on Google class sites where they get feedback on their final product from other
readers. The contrast to other ICT-supported training is that formative feedback and assessment are used systematically and
that ICT is used as an important medium for increasing the effects of the formative feedback. Hence WTL extends previous
methods i.e. ASL (Trageton, 2003), by including ICT-based social work methods, using digital learning spaces and formative
peer commenting for providing social meaning and formative feedback (Age
lii Genlott & Gro
€nlund, 2013).
The WTL method is based on a socio-cultural perspective integrating social interaction and collaborative learning as main
features for learning. The method includes collaborative writing, formative feedback, formative assessment and publishing as
main distinctive components. Supporting components include inspiration and pre-understanding (“Preview”, Fig. 1), early
learning of text genres, scaffolding, and different writing strategies. The general work process by which WTL lessons are
carried out is designed as follows (illustrated in Fig. 1). In each phase of the WTL method the students use computers or
tablets. All students have their own GAFE account and use different GAFE applications such as Google Sites (class web-sites) or
Google Drive (shared documents) in accordance with the tasks defined by the teacher and with the different phases in the
method. WTL is implemented in mathematics by letting the students create and solve mathematical problems through
written collaboration, again communicating, solving problems, and publishing their solutions using the same GAFE appli-
cations. The WTL steps (Fig. 1) are:

1. With the aim of making learning goals visible and understandable also for the youngest students WTL takes its starting
point in explaining, and in simple words writing down the goals and knowledge requirements of the Swedish national
curriculum (“Lgr 11”; Skolverket, 2011). These goals are published on the same digital Google class site used for the student
assignments, enabling the students to go back and read if needed.
2. “Inspiration/Preview” aims at arousing interest among the students, but also at providing them with the pre-
understanding of the task that is necessary to create the very best conditions for each and every student to succeed.
This can be done in different ways depending on the task as well as on different student needs. Technology used in this
72 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro

lii Genlott & Gro

Fig. 1. Steps in the WTL method (Age € nlund, 2013).

phase differs but previews are often videos used as a form of flipped classroom, meaning that tutorial or inspirational
videos are published on the Google class site for the students to take part of and learn from before the lesson.
3. “Text type/writing strategies” refers to which text type/literacy genre the task concerns and consequently which kind of
writing strategy to use. Technology used in this phase is mainly shared documents through Google Drive, where students
learn from written examples given from the teacher.
4. “Collaborative Writing” represents social interaction and collaborative writing. The students are now starting to write
either in pairs or individually but always sharing their texts with peers and teachers digitally thereby continuously both
giving and receiving formative feedback during the writing process. The technology used here is mainly Google Drive.
5. “Formative Feedback & Formative Assessment/Publishing” refers to the continuous work of the students learning how to
give and take formative assessment to each other from early years. As giving formative feedback is complex and requires
certain skills, the students are continuously taught by means of instructions and practice in order to further develop skills
of reviewing others' work in a formative way. Finally, the students publish the final version of their work on a web site,
which gives them more readers and more final feedback. Technology used in this phase is Google Drive, and the final
versions of the texts are published in Google Sites.
6. “Final assessment” is done by the teacher. Due to the consistent use of ICT throughout the process, all student texts and
formative feedback produced and received by the students are available on the digital collaborative spaces, and are
therefore all considered in the assessment. This step is often formative as well, aiming to help the students see what they
have achieved so far in terms of the Lgr11goals and what the next steps are. Technology used in this phase includes all the
student drafts and final work saved on the Google site. The teacher can also through the documents of Google Drive review
the history of the drafts, this is important so as to understand the writing process of the students.

The above activities integrate technology use and physical oral communication throughout the process. An important part
of WTL is that all students are given criteria (item 5 above) for writing formative feedback to their classmates with the aim of
contributing to others' texts as well as increasing their own learning by cognitively reflecting and formulating written
formative feedback. Publication is done to make sure texts are read by even more readers, primarily other students and
teachers, but also by parents. This serves to increase the students' awareness of an audience and their knowledge concerning
writing for recipients, and to make sure there are final responses to the published texts.
WTL is used in all subjects where linguistic abilities such as reading, writing, collaboration, and reasoning are important. It
is already well known that linguistic abilities affect performance in mathematics, and that mathematics in itself is a
specialized language. Brendefur and Frykholm (2000) categorize the various forms of mathematical communication as uni-
directional, contributive, reflective and instructional. Uni-directional communication is often used in schools where teachers
dominate the discussions by mostly lecturing knowledge passively received by the students. Contributive communication
instead focuses on interactions between teachers and students but mostly by sharing and often without much deeper
thought. Reflective communication is when students share their ideas, strategies, and solutions, thereby engaging in math-
ematical conversations. Central to instructive communication is modification of the students' mathematical understanding,
where the written communication serves to expose the students' thinking. This helps the teachers to gain insights of the
students mathematical thinking but also make them shape subsequent instruction. The use of WTL in mathematics education
focuses mainly on four abilities specified in the Swedish mathematics curriculum; communication, reasoning, solving
 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro 73

problems, and learning different mathematical concepts. In WTL, students publish their written mathematical reasoning,
thinking, and solutions in writing through shared documents or on the class site for their classmates to read, give feedback to,
and/or to solve. In this way WTL serves to increase the linguistic foundation for reasoning and mathematical thinking, the
logic being that improved mathematical thinking will lead to improved ability to deepen the mathematical understanding.

5. Method

The study included 502 students who had been exposed to different teaching methods over a period of three years during
grades 1e3.
Student test scores in the National Standard Tests (NST) of literacy and mathematics were used to examine if the WTL
method produced better results than other methods. Two control groups were used, one “traditional” without ICT and one
using ICT but not the WTL method.
Potential bias caused by socio-economic factors was tested using the model for calculation of such bias that was used by
the municipality in which the study was conducted; the test used the Pearson correlation index.
The Grade 3 NST is the only nationwide test carried out by all grade 3 students in Sweden. The NST tests a number of
abilities related to reading, writing and mathematics with a total of 15 tests, 7 in mathematics and 8 in literacy. The NST also
includes grading guidelines and rules for teachers, aiming at standardizing assessment and minimizing bias as far as possible.
In addition, so as maximize conformance in assessment, the local school authority organizes collaborative assessment across
schools. The NST is carried out in several steps during the same period (February-May 2013) in all schools. The NST is the most
standardized test available and hence most appropriate to use for this study.
The three groups involved differ as follows.

1. The WTL is a structured way of integrating ICT from grade 1 when learning how to write and read. Teachers in the WTL
group have attended a one-year community-wide course, theoretically and practically learning how to integrate ICT into
the pedagogy of literacy and mathematics. Students learning how to read and write using the WTL, in addition to reading
books as usual, continuously use class websites and other web arenas for collaboration publishing and formative feedback.
2. The traditional method (no or sporadic technology use) is learning how to read and write without ICT use. The first focus is on
the reading, thereby identifying different letters, using the phonics method, sometimes combined with the whole word
method. Handwriting often starts by learning one letter at a time to train the fine motor skills at the same time as working
to break the code of learning how to read.
3. The ITU (Individual technology use) include teachers and students continuously using digital devices though without using
(or having been trained in) the specific WTL method. The teachers in the ITU use ICT but not shared documents or web
arenas for structured formative feedback between peers and teachers.

The study hence tested WTL in two ways: regarding the use of ICT tools and regarding the use of a specific social peda-
gogical method. The traditional method was used as control group for the ICT use, and the ITU method was used as control for
the social interaction arrangements - such as the formative feedback supported by ICT - that were a crucial part of the
intervention.
The distribution of classes into groups was done using both the documentation concerning which of the teachers who had
attended the WTL course as well as information from school principals and course leaders with the knowledge of which
method of the three included in this study that most appropriately described the pedagogy of the teachers. The latter test was
done to (1) control that teachers who had attended the course actually used the method they had been taught, and (2) find
any teacher who had not attended the course but used the method based on knowledge acquired from other sources. As the
classification into different groups was not decided beforehand, this cannot have affected the teachers or the students in any
way.
Test results from the Spring 2013 national tests were collected from the city's database (public schools only). The database
contained all 696 students who started school in Fall 2011 and took the tests in Spring 2013. Due to incomplete data 194
students were excluded from the study. One major reason for missing test data is migration over the three years of primary
school. One school was excluded because of inconclusive information concerning which teaching method was used. Other
reasons are not known, but may include students being absent due to illness, travel, or other. In total, the sample includes data
from all 502 students for which complete data was available. The total sample includes the results for students from 21 classes
taught by 23 teachers in 11 public schools.
For privacy reasons student names and personal identity numbers were not extracted from the database. Data was
analyzed by means of hypothesis tests using mean comparison and t-tests as well as Pearson's Chi-Square tests of inde-
pendence. All analyses and tests in this study were done using the software STATA.
Of the 502 students, 247 used the WTL method, 128 the traditional method, and 127 the ITU method. Gender distribution
was equal in the WTL and traditional groups (48% girls in both groups) while boys were in majority in the ITU group (61%)
(Table 1).
74 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro

Table 1
Total sample included in the study.

Method Girls Boys % Girls % Boys Total

WTL 118 129 48% 52% 247
Traditional 61 67 48% 52% 128
ITU 50 77 39% 61% 127
Total 229 273 46% 54% 502

6. Results

This section presents the results for literacy, mathematics, and combined for the 15 test that together make up the national
tests. Each test is graded by pass or fail, so all the numbers below refer to those students who have passed the tests.

6.1. Literacy

The results of this study show that students using the WTL method performed better than students in both other groups.
In the WTL group 92 percent of the students passed all eight tests, which is better than the traditional group (79%) and the ITU
group (77%). Concerning the gender perspective, while both boys and girls in the WTL group performed better than those in
the other groups, the difference was greatest for the boys. WTL boys scored 18 percentage points better than the traditional
group (7 for the girls) and 21 (5) compared to the ITU group. This means the gender gap is shrunk to 5 percentage units for the
WTL method compared to 15 for the traditional method and 21 for ITU (Fig. 2). This supports hypotheses 7 and 8.
In the WTL group there was also less spread with no WTL student passing less than five of the eight tests. In both the other
methods a lower share of students passed all tests and the spread was wider with lower minimum results (Table 2).
The statistical test, a two-sample t-test (Table 3), shows that the differences in means between WTL-Traditional and WTL-
ITU methods are significant at the 0.05 level. The difference between the traditional method and ITU is not significant. Hy-
potheses H1 and H2 are hence supported; WTL students score significantly better than both other methods.

6.2. Mathematics

In the WTL group, 81 percent of the students passed all seven tests (Fig. 3). This is better than the traditional group (67%)
and the ITU group (56%).
As regards the gender perspective, in the WTL group boys and girls performed equally well; 81 percent of both boys and
girls passed all tests. This was considerably better than both control groups; the boys scored 52 percent and the girls 60 in the
ITU group, for the traditional group the shares were 62 and 75 percent. This means the gender gap has shrunk from 15
percentage points for the traditional method and 10 for the ITU method to zero. The result supports hypotheses 7 and 8.
Table 4 shows that the mean was higher and the standard deviation was smaller in the WTL group than in both the others.
The statistical test, a two-sample t-test, shows that the differences in means between WTL-Traditional and WTL-ITU are
significant at the 0.05 level (Table 5). Hypotheses H3 and H4 are hence supported.

6.3. Literacy and mathematics combined

Altogether, 78 percent of the WTL students passed all tests in both literacy and mathematic, to be compared to 59 percent
for the traditional method and 50 for ITU (Fig. 4).
While both boys and girls in the WTL group performed better than those in the other groups, the difference was greatest
for the boys. WTL boys scored 29 percentage points better than the traditional group (8 for the girls) and 31 (22) compared to
the ITU group. The gender gap is shrunk to 3 percentage units for the WTL method compared to 24 for the traditional method
and 12 for ITU (Fig. 4). This supports hypotheses 7 and 8. The numbers may make the ITU method seem better than the
traditional one (as the gap is smaller) but it is not really; the smaller gap is not achieved by boys performing better but by girls
performing worse overall using ITU due to the lower results in mathematics.
Also concerning the students who passed all 15 tests in both literacy (8) and mathematics (7) the WTL group scored better
than both other methods. Table 6 shows that the mean was higher and the standard deviation lower in the WTL group
compared to both the others. Further, no student in the WTL group passed in less than 9 out of the 15 tests while the lowest
result in the traditional group was 7/15 and in the ITU 5/15. The better performance among low performers supports hy-
potheses H5 and H6.
The statistical test, a two-sample t-test, shows that the differences in means between WTL-Traditional and WTL-ITU are
significant at the 0.05 level (Table 7).
 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro 75

Fig. 2. Student results (%) in the national test of literacy, school year 3.

Table 2
Mean values of number of literacy NS-tests passed (max result ¼ 8).

Method Population Mean Std.dev Min passed Max passed

WTL 247 7.89 0.423154 5 8
Traditional 128 7.67 0.743575 4 8
ITU 127 7.57 0.996755 3 8

Table 3
Literacy. Two-sample t-test.

Methods Differences (mean) Std. Err. t-value Pr (T > t)

WTL-Traditional 0.13 0.035 3.81 0.000
WTL-ITU 0.15 0.036 4.23 0.000
Traditional - ITU 0.02 0.052 0.33 0.369

Fig. 3. Student results (%) in the national test of mathematics, school year 3.

Table 4
Mean values of number of mathematics NS-tests passed (max result ¼ 7).

Method Population Mean Std. dev Min passed Max passed

WTL 247 7.89 0.423154 5 8
Traditional 128 7.67 0.743575 4 8
ITU 127 7.57 0.996755 3 8

Table 5
Mathematics. Two-sample t-test.

Methods Differences in means Std. Err. t-value Pr (T > t)

WTL-Traditional 0.14 0.046 3.11 0.001
WTL-ITU 0.25 0.047 5.43 0.000
Traditional - ITU 0.11 0.061 1.86 0.032
76 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro

Fig. 4. Student results (%) in the national test of literacy and mathematics, school year 3.

Table 6
Mean values of literacy and mathematics in number of NS-tests passed (Max ¼ 15).

Method Number of students Mean Std.dev. Min tests passed Max tests passed
WTL 247 14.56 1.029597 9 15
Traditional 128 14.08 1.585413 7 15
ITU 127 13.65 2.010005 5 15

Table 7
Literacy and mathematics. Two-sample t-test.

Differences in means Std. Err. t-value Pr (T > t)

WTL 0.48 0.065 3.57 0.0002
Traditional 0.92 0.157 5.84 0.0000
ITU 0.43 0.227 1.91 0.0287

6.4. Test for gender gap

As shown above, boys overall improved more than girls by using the WTL method (Figs. 2e4). The differences are sum-
marized in Tables 8 and 9. We tested the relation between gender and student results using Pearson's Chi2 test of inde-
pendence. The result shows that gender does not significantly affect student results in literacy in the WTL group (Chi2 ¼ 2.16,
p-value ¼ 0.141) whereas there are significant gender differences in both the traditional (Chi2 ¼ 4.46, p-value ¼ 0.035) and
the ITU groups (Chi2 ¼ 7.71, p-value ¼ 0.005). This implies that the WTL method decreases the gender gap in literacy.
In mathematics the same pattern concerning gender dependence in WTL was found, that is, the dependence was much
lower in the WTL group than in the others; in fact e as Table 9 shows e boys and girls performed equally well. However, even
though the differences were considerably bigger in both the other groups they were not statistically significant in the Pearson
test.

6.5. Test for bias

Potential bias in the results presented above might result from different preconditions at the schools. Therefore a few tests
were made to control for such variables.
Potential bias caused by different teacher capability: All teachers taking part in the study had formal teacher education. On
average the teachers had 11.5 years of teaching experience. The youngest two had 4 years of experience, the most experienced
39 years. This means all teachers fulfilled reasonable requirements of education and experience. There is no way to objectively

Table 8
Achievements & gender differences in literacy.

Method Literacy Difference

Girls Boys
WTL 94.92 89.92 5.00
Traditional 86.89 71.64 15.25
ITU 90.00 68.83 21.17
 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro 77

Table 9
Achievements & gender differences in mathematics.

Method Mathematics Difference

Girls Boys
WTL 81.36 81.40 0.04
Traditional 75.41 59.70 15.71
ITU 62.00 51.95 10.05

determine individual teacher skills in literacy teaching as there are no quantified criteria for that and hence no standard test.
Also, in all classes more than one teacher took part in teaching so there was good availability of teaching expertise.
The potential bias caused by socio-economic factors was tested using the model for calculation of such bias used by the city
in which the test was done. The model, labeled the SWECO model after the company who developed it, is also used by some
other cities. It is similar but more detailed than the one used by the Swedish national school authorities (national scores were
not available as the national school authority does not make such calculations for primary schools). It measures the socio-
economic status at school levels on the following factors; Parents education level, years since immigration, students' sex,
parents' reception of government economic support, share of students living with both parents.
The SWECO index varied widely across the schools included in this test (Fig. 5). Pearson correlation index was 0.4006,
which means there is no significant correlation between a school's SWECO index and its performance in the national tests.
Fig. 5 illustrates the (lack of) correlation between SWECO index and school performance visually.
In view of these tests it is fair to say that neither teacher skill nor socio-economic factors influenced the results
significantly.

7. Discussion

The purpose of this study was to see if student performance in literacy and mathematics was improved by the use of a
specific method (WTL) which uses ICT support for a number of elements which earlier research has shown to be conducive for
learning such as social interaction, formative feedback and formative assessment (Black and Wiliam (2009); Hattie, 2009).
The study compared student achievements in the Swedish national standard tests of literacy and mathematics in grade 3
for three teaching methods, the WTL, a traditional method (no ICT) and one with only individual ICT use (ITU). The overall
hypothesis was that ICT use in combination with a pedagogical method including formative feedback and assessment as a digital
social interaction among peers improves results in literacy development and in mathematics.
The study found that students using the WTL method achieved better results than both control groups, and most
significantly compared to the ITU method. This is interesting since the ITU method includes students using the same digital
devices as the WTL students, only without the WTL method with its formative feedback and assessment and the specific
structure for integrating social interaction into the education designed based on the socio-cultural perspective. What mainly
differentiates the WTL method from the ITU method therefore is the implementation of socio-cultural theory. In terms of
technology use this includes the specific way of using the class website with the aim of integrating ICT as a way of creating

Fig. 5. SWECO index (blue line under) and schools' performance (red line over). (For interpretation of the references to colour in this figure legend, the reader is
referred to the web version of this article).
78 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro

effective arenas for social interaction, collaboration, formative feedback and formative assessment integrated into the
pedagogy.
The WTL method also scored by far best in reducing the gender gap in performance. The gender gaps in both literacy and
mathematics were much smaller in the WTL group than in the control groups. The most conspicuous result on this point was
that in mathematics boys and girls performed equally well. In accordance with previous research on ICT use in various
processes, this shows that a structured method integrating social interaction is beneficial for both boys and girls, and in both
literacy and mathematics. The gain is the biggest and the most important for boys concerning literacy, in which subject they
traditionally perform worse than girls. This supports the argument of Legewie and DiPrete (2012) that in particular boys
benefit from school resources that “create a learning-oriented peer culture”. This can in this case be attributed to the WTL
method and the supporting use of collaborative ICT tools.
We find the good results in mathematics particularly interesting, as the WTL method does not include anything particular
about numbers; it is mainly about communication. The findings imply that learning mathematics has a lot to do with un-
derstanding the problems to be solved and ways to solve them. This is learned through use of language. At least at this
beginner level, learning mathematic is just as much, or more, about thinking and reasoning as about the numbers. WTL
prompts teachers and students to discuss mathematical issues by formulating written mathematical problems, publishing
them for others to solve, and thereafter reasoning together through formative feedback in written words. In comparison to the
traditional method where the class communicates about these things orally, WTL provides more communication (more time
spent on it as they both talk and write), durable communication (the texts stay for students to go back and recall), and equal
communication (all participate, all have equal time for communication, no competition for attention). According to our test
results these factors are crucial.
Today the WTL method is used more or less in all subjects that are based on reading and writing of some kind in the
studied city. It is therefore possible that the WTL method is also affecting the achievements in other subjects in a positive way.
Unfortunately we could not test that as the national tests in grade 3 only test literacy and mathematics.
One implication of the results is that method development is important in conjunction with ICT use. It is also important to
organize dissemination of innovation so that effective methods do not stay undetected by the rest of the world in the
classroom in which they were invented and first tested. This is a challenge in several ways. First, methods have to be
developed and tested. Second, dissemination is difficult as, according to a 2013 Swedish study, 20% of the teachers are
negative to ICT use and only 1/3 positive (SOU, 2014:13). This points to a leadership challenge, but also to the need to invest
considerable time in dissemination of innovation. In our case study, the City had invested in more than one year of initial
method development and pilot test and subsequently in three years of educating teachers. Even so, not all teachers have yet
been educated and/or convinced of the benefits of making the effort to move to a new teaching method integrating ICT into
the pedagogy. Although some teachers by themselves are early adopters of new digital technology, many others need much
support in how to optimize the use of ICT in the pedagogy. It is therefore probably crucial that teachers are supported in this
matter so as to avoid inequality amongst both students within schools as between schools (as pointed out by Samuelsson,
2014; Beckman, Bennett, & Lockyer, 2014).
The fact that the lowest performers e which include students with difficulties in reading and writing due to e.g. dyslexia,
some kind of language disorder or problems with fine motor skills, or just slower development e scored so much better in the
WTL group is important. While technology can compensate for e.g. lack of fine motor skills the study shows that such factors
alone do not explain the results. As the factor differentiating the WTL and ITU methods is the social interaction using written
language, we believe that the fact that students who suffer from such reduced abilities have been interacting on the same
arena as their classmates in the WTL group, on the class website, thereby being able to take part in the social written
interaction has increased their self-confidence and their interest in communicating by written language. This in turn might
have had an impact on the WTL students' incentives to read and write even more.
The study also showed that the ITU method yielded the lowest student performance of all methods. This shows that ICT
use in itself does not lead to better results but may indeed lead to worse. It is only when ICT is used to support other pedagogic
factors that have been shown to have positive impact more efficient and effective, and does so in a systematic way, that the
positive results occur. These results are in conformance with previous studies in the education field (Age
lii Genlott &
Gro€ nlund, 2013; OECD, 2010), as well as in other fields (Brynjolfsson, 1993); ICT improves operations only in combination
with effective process change.

8. Conclusion

This paper has reported a study of the effectiveness of an innovative pedagogical method including ICT in combination
with social interaction, WTL (Write To Learn), as measured by student results on national standard tests in literacy and
mathematics (n ¼ 502). There were two control groups, one using traditional method (no ICT) and one using ICT individually
(no ICT-supported social interaction).
The study finds that.

 WTL leads to higher student achievement in literacy as well and in mathematics than both other methods e the difference
(combined) is 19 percentage points compared to the traditional method and 28 compared to the ITU method.
 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro 79

 WTL leads to better performance among low performers than both other methodse the results of the lowest performers
are 80% better in WTL than in the worst performing method.
 WTL leads to a lower gender gap in performance than both other methods e in mathematics boys improved their per-
formance up to the level of girls, in literacy the girls did only 3% better. Overall boys performed almost 30 percentage
points better using WTL compared to both other methods.

The contribution to research is to quantitatively compare a model that integrates ICT and theory-based pedagogy into the
education of literacy and mathematics with two other commonly used methods concerning its effectiveness in producing
better student results in the national standard tests in literacy and mathematics in school year 3. The study showed that use of
ICT based on a socio-cultural perspective and including effective arenas for collaborative learning, feedback and formative
assessments are key factors for learning.
The contribution to practice is empirical evidence that the method is effective in both literacy and mathematics, and that
ICT use without integration in social learning activities is not effective but may in fact prove detrimental.

Acknowledgements

€
This research was funded by Orebro University. The authors are grateful to City of Sollentuna for providing the necessary
data; to all the principals and teachers for providing useful information; to Jonas Bergvall, Statistician in Sollentuna, for
handling the data collection and contributing with expert statistics knowledge; to Sara Penje, teacher and development
strategist at the City of Sollentuna for contributing in developing the method towards including mathematics; and to Sune
€
Karlsson, Professor of Statistics at Orebro University for examining the methods used.

Appendix A. Supplementary data

Supplementary data related to this article can be found at http://dx.doi.org/10.1016/j.compedu.2016.04.004.

References

Age
lii Genlott, A., & Gro

€nlund, Å. (2013). Improving literacy skills through learning reading by writing: the iWTR method presented and tested. Computers &
Education, 67, 8e104. http://dx.doi.org/10.1080/17439884.2012.756518.
Ananiadou, K., & Claro, M. (2009). 21st century skills and competences for new millennium learners in OECD countries, OECD education working papers, no. 41.
http://dx.doi.org/10.1787/218525261154.
Barton, D. (2001). Directions for literacy research: analysing language and social practices in at textually mediated world. Language and Education, 15(2e3),
92e104.
Beckman, K., Bennett, S., & Lockyer, L. (2014). Understanding students' use and value of technology for learning. Learning, Media and Technology, 39(3),
346e367. http://dx.doi.org/10.1080/17439884.2013.878353.
Black, P., & Wiliam, D. (2003). In praise of educational research: formative assessment. British Educational Research Journal, 29(5), 623e637.
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational assessment. Evaluation and Accountability (Formerly: Journal of
Personnel Evaluation in Education), 21(1), 5e31. http://dx.doi.org/10.1007/s11092-008-9068-5.
Bloom, B. S., Hastings, J. T., & Madaus, G. F. (Eds.). (1971). Handbook on the formative and summative evaluation of student learning. New York: McGraw-Hill.
Brendefur, J., & Frykholm, J. (2000). Promoting mathematical communication in the classroom: two preservice teachers conceptions and practices. Journal of
Mathematics Teacher Education, 3(2), 125e153.
Brynjolfsson, E. (1993). The productivity paradox of information technology. Communications of the ACM, 36(12), 66e77. http://dx.doi.org/10.1145/163298.
163309.
Chaiklin, S. (2003). The zone of proximal development in Vygotsky's analysis of learning and instruction. In A. B. Kozulin, B. Gindis, V. S. Ageyev, & S. M.
Miller (Eds.), Vygotsky's educational theory and practice in cultural context, 19. Cambridge: Cambridge University Press.
Cohen, V. B. (1985). A reexamination of feedback in computer-based instruction: implications for instructional design. Educational Technology, 25(1), 33e37.
European Commission. (2007). E-skills for the 21st century: Fostering competitiveness, growth and jobs. Communication from the commission to the council, the
European parliament, the European economic and social committee and the committee of the regions. http://ec.europa.eu/enterprise/sectors/ict/e-skills/
index_en.htm. retrieved 25 April 2015.
European Commisson. (2010). Digital Agenda for Europe. (DAE). http://ec.europa.eu/digital-agenda/en/digital-agenda-europe. retrieved 12 August, 2014.
Frederickson, N., & Cline, T. (2002). Special education needs, inclusion and diversity: A textbook. Buckingham. Open University Press.
Hall, A. (2007). Vygotsky goes online: learning design from a socio-cultural perspective, learning and socio-cultural theory: exploring modern vygotskian
perspectives international workshop 2007, 1(1). Research on Line, 1(6), 93e107. http://ro.uow.edu.au/llrg/vol1/iss1/6. retrieved 25 September, 2014.
Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London & New York: Routledge, Taylor & Francis Group.
Hattie, J., & Timperley, H. (2007). The power of Feedback. University of Auckland Review of Educational Research, 77(1), 81e112. http://dx.doi.org/10.3102/
003465430298487. March 2007.
Legewie, J., & DiPrete, T. (2012). School context and the gender gap in educational achievement. American Sociological Review 2012, 77(463). http://dx.doi.
org/10.1177/0003122412440802. originally published online 4 April 2012.
Liberg, C. (2006). Hur barn la €r sig la
€sa (How children learn to read) (2nd ed.). Lund: Studentlitteratur.
Locke, E. A., & Latham, G. P. (1990). A theory of goal setting & task performance. Englewood Cliffs, NJ: Prentice Hall.
Luke, A. F., & Peter. (1997). The social practices of reading. In P. F. Sandra Muspratt, & Allan Luke (Eds.), Constructing critical literacies (pp. 185e225). New
Jersey: Hampton Press.
Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online
learning studies. Center for Technology in Learning. US Department of Education, Policy and Program Studies Service. http://www2.ed.gov/rschstat/eval/
tech/evidence-based-practices/finalreport.pdf. retrieved August 25, 2015.
Mueller, C. M., & Dweck, C. S. (1998). Praise for intelligence can undermine children's motivation and performance. Journal of Personality and Social Psy-
chology, 75(1), 33e52.
OECD. (2010). Joint research centre- european commission. In F. Sheuermann, & F. Pedro
(Eds.), Assessing the effects of ICT in education indicators, criteria and
benchmarks for international Comparisons: Indicators, criteria and benchmarks for international comparisons. Centre for educational research and inno-
vation, 2010 pp. 21e38). OECD Publishing. ISBN: 9264079785, 9789264079786.
80 €nlund / Computers & Education 99 (2016) 68e80
A.A. Genlott, Å. Gro

OECD. (2012). Closing the gender gap: Act now OECD 2012. Part 2. Gender equality in education. http://www.oecd.org/gender/closingthegap.htm. retrieved 15
March, 2015.
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: recent developments. Educational Psychologist, 38, 1e4.
Rothery, J. (1996). Making changes. Developing an educational linguistics. In Hasan, Ruqaiya, & Geoff Williams (Eds.), Literacy in society (pp. 86e123).
London & New York: Longman.
€ljo
Sa €, R. (2000). La€rande i praktiken. Ett sociokulturellt perspektiv. Stockholm: Prisma.
€ljo
Sa €, R. (2002). La€rande i det 21:a århundradet. Utm@ningar och e-frestelser e It och skolans la €rkultur. Stockholm: Prisma.
Samuelsson, U. (2014). Digital (o)ja €mlikhet e IKT anva€ndning i skolan och elevers tekniska kapital (Doctoral thesis). Jo
€ nko
€ping University, School of Education
and Communication.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research March 2008, 78(1), 153e189. http://dx.doi.org/10.3102/0034654307313795
(Florida State University).
Skolverket. (2011). Curriculum for the compulsory school, preschool class and the recreation centre. http://www.skolverket.se/publikationer?id¼2687.
retrieved 17August, 2015.
Skolverket. (2014). SIRIS. http://siris.skolverket.se/siris/f?p¼Siris:1:0. retrieved 16 March, 2015.
Song, S. H., & Keller, J. M. (2001). Effectiveness of motivationally adaptive computer- assisted instruction on the dynamic aspects of motivation. Educational
Technology Research and Development, 49(2), 5e22.
SOU. (2014). Digitaliseringskommissionen. En digital agenda i ma €nniskans tja€nst e En ljusnande framtid kan bli vår (SOU 2014:13). http://www.regeringen.se/
sb/d/18259/a/236035. retrieved 4 August, 2014.
Street, B. (1984). Literacy in theory and practice. Cambridge: Cambridge University Press.
Trageton, A. (2003). Å skrive seg til lesing (IKT i småskolen (Writing to read: ICT in primary school).
Vygotsky, L. S. (1962). Thought and language. Cambridge MA: MIT Press.
Vygotsky, L. S., & Eccles, J. S. (1978). Mind and society: The development of higher mental process. Cambridge MA: Harvard University Press.