Problem-based learning in secondary education:

Evaluation by a randomized experiment

Kristof De Witte
Nicky Rogge

HUB RESEARCH PAPERS 2012/11

ECONOMICS & MANAGEMENT
FEBRUARI 2012
 Problem-based learning in secondary education: Evaluation by a
randomized experiment

Kristof De Witte‡δ1 and Nicky Rogge‡*

(‡): Katholieke Universiteit Leuven (KULeuven)

Faculty of Business and Economics
Naamsestraat 69, 3000 Leuven (Belgium);
(δ): Maastricht University
TIER, Faculty of Economics and Business
Kapoenstraat 2, 6200 MD Maastricht (the Netherlands)
[email protected]
and
(*): Hogeschool-Universiteit Brussel (HUBrussel)
Centre for Economics & Management (CEM)
Stormstraat 2, 1000 Brussels (Belgium)

Abstract

The effectiveness of problem based learning (PBL) in terms of increasing student knowledge and
skills has been extensively studied for higher education students and in non-experimental settings.
This paper tests the effectiveness of PBL as an alternative instruction method in secondary
education. In a controlled randomized experiment, we estimate its effect on tested student
attainments, on perceived student attainments, on autonomous and controlled motivation and on
class atmosphere. The outcomes indicate a non-significant negative effect on student achievements,
a non-significant effect on motivation and a significant positive effect on class atmosphere.

Keywords: Problem-based learning; Secondary education; Student achievements; Student motivation;

Classroom social climate; Randomized experiment.

JEL-classification: I21; C93

1
Corresponding author. Tel.: 003216326566
E-mail address: [email protected]

1
 1. Introduction

As the literature is realizing that active learning is the key for improved educational attainments, old
fashioned instruction methods are replaced for more activating counterparts. Along this line, problem-
based learning (PBL hereafter) as an activating instruction method is increasingly gaining ground in
education. What started as a more practical approach to teach health sciences at the McCaster
University in Canada in the late 1960s and subsequently predominantly has been applied in the
medical school programs, is recently more and more being promoted by both educational experts and
practitioners as an interesting tool to address the major and long-lasting challenges of education
(irrespective of the level). It aims at developing skills and habits of life-long, self-directed learning.
PBL has predominantly been adopted in colleges and universities - particularly in the medical
education programs, but also, though to a lesser extent, in other fields and disciplines such as
economics (e.g., Son & VanSickle, 2000), management (e.g., Joham & Clarke, 2011), chemistry (e.g.,
Black & Deci, 2000) and psychology (e.g., Abrandt Dahlgren & Dahlgren, 2002). Recently, PBL
gained also ground in secondary education (e.g., Gallagher et al.,1992; Mergendoller et al., 2000;
Maxwell et al., 2001, 2005). This paper contributes to this expanding literature. In particular, it
examines by a randomized experiment the effect of PBL as an alternative instruction method in
secondary education on tested educational achievements, perceived educational achievements (i.e.,
what students think they learnt), motivation and class atmosphere.

Despite its increasing use and popularity, the PBL approach remains controversial. The main reason
for this controversy is that, in spite of being the subject of extensive research, several aspects and
influences of PBL remain unclear. First of all, there is continuing ambiguity about what PBL exactly
stands for. Surely, broadly speaking, we know that it concerns an alternative teaching approach in
which students are subdivided into small groups and challenged by a carefully designed problem to
start the learning process and to find out what they really know and what knowledge and skills they
should know so as to be able to solve the problem. However, in practice, several variations of PBL
have been implemented and studied. This makes it a contentious task to define what exactly
constitutes PBL. In this paper, we define PBL as “an active learning method that starts from a specific
problem. Through group discussion, individual study and collaboration in small groups, students
discover their own knowledge, try to understand the underlying mechanisms of the problem and solve
the problem together. The teacher acts as a tutor that guides the students and supports the students‟
initiatives”.

Second, it is unclear whether PBL enhances knowledge and skills of students, and whether it increases
student motivation and satisfaction. A concern formulated by some researchers, practitioners and
teachers is that PBL might not be able to provide students with a sound base of theoretical facts and
knowledge. The effectiveness of PBL in terms of increasing student knowledge and skills has been
extensively studied in the academic literature for higher education students (particularly in medicinal
education programs). The existing evidence for higher education students on the impact of PBL on
student test scores is documented in a number of meta-analyses and review studies (e.g., Albanese and
2
Mitchell, 1993; Colliver, 2000; Dochy et al., 2003; Gijbels et al., 2005; etc.). Overall, results seem to
be rather mixed. Whereas some of the studies reported positive effects of PBL on student test scores,
other studies found that PBL-teaching offers no considerable advantages to students in terms of higher
gain in knowledge and/or skills. With respect to the effect of PBL on student satisfaction, results show
that students in a problem-based learning track are overall more satisfied with their teaching than
students in the conventional track. In particular, students who are in a problem-based learning track
consider their courses as more nurturing and enjoyable. Recent studies on the association between
PBL and student motivation found rather mixed results, with some interviewed students indicating
that they believe that PBL has improved their motivation to study and others indicating that the effect
of PBL in terms of increasing their studying motivation was non-existent. The paper at hand considers
the effects of PBL in secondary education. In particular, it uses PBL as an alternative instruction
method during a short time span. To our best knowledge, the literature is silent about the influence on
educational attainments, motivation and class atmosphere of PBL as a didactic tool in a short time
span (i.e., one teaching hour).

As a third issue, among others, Colliver (2000) noted that most previous studies on the effects of PBL
suffer from methodological shortcomings. One of these shortcomings is that in most studies students
were non-randomly assigned to the groups of PBL students and the groups in which student were
taught according to the conventional manner. Frequently the students had to indicate their preference
for the PBL or conventional course. As another shortcoming, schools may have their own criteria to
decide on student admission. Both limitations may cause a selection bias in the outcomes. Colliver
(2000), for instance, remarked that the effects found in some previous studies might be largely
attributable selection bias. Using a randomized experiment, this paper tries to avoid selection effects
at school, class and student level.

In sum, this paper contributes to the literature by studying the effects of a PBL instruction tool on
student test outcomes, student motivation and class atmosphere in secondary education. Contrary to
most previous papers, this study occurs in an experimental setting in which random secondary
education schools are participating. In particular, 15 teachers were asked to teach the same course in
both PBL- and conventional style in 15 different secondary schools. Within one school, the
experiment is evaluated for students in an educational track which is too large to constitute one class-
group (i.e., larger than 30 students) but had to be split (by law) in two class groups. As a priori the
two classes are similar, this allows for randomization at pupil level such that selection effects are
avoided both at the school, class and student level. Consequently, 2*15 similar classes with a total of
531 students were assigned randomly to the PBL-course and the conventional course. To allow for a
meaningful comparison of the PBL-effects over institutions, the 15 teachers were first trained in PBL
and provided with guidelines (concerning their role in the teaching process, the format of the problem,
etc.) that needed to be strictly followed in the teaching of the PBL-courses.

The paper unfolds as follows. The next section briefly describes the literature on PBL. The focus is
mainly on the definition of PBL and the previous findings. A section then follows which introduces

3
the experiment as well as the collected data on student test scores, student motivation and student
appreciation of the class climate. The ensuing section reports the findings on the relations between
PBL and student test scores, motivation, and experience of classroom climate. The paper ends with a
summary of the conclusion reached and a discussion of some limitation that could be addressed in
future research.

2. Literature

2.1 Key features of problem-based learning

There is no consensus on a definition of PBL. One reason is that several variations of PBL have been
implemented and studied.2 Schmidt (1995) made an attempt at providing a general characterization of
PBL: “A collection of carefully constructed and engaging „problems‟ is presented to small groups of
students. These problems usually consist of a description of a set of observable phenomena, situations,
or events. … . The task of the group is to discuss these problems and elaborate on tentative
explanations for the phenomena in terms of some underlying process, principle, or mechanisms”
(Schmidt, 1995 p. 247).

This definition highlights some key features of PBL. First, PBL is an educational model in which the
problem in itself provides a suitable context for subsequent learning. In particular, the core idea is that
the problem (e.g., real-life problem tasks, practical case studies, team-based assignments, etc.), when
being well-defined and resembling very much the actuality, will suffice to activate the students‟
interest in the subject-matter. In this respect, PBL associates with relatively new developments in the
cognitive psychology, a subdiscipline of the psychology which focuses on exploring the mental
processes of people (in terms of how people actively process information: i.e. how they think, solve
problems, interpret, etc.) and states that information will be better understood and recalled when it
was learned in a situation (i.e., the created problem-setting) which closely resembles the situation in
which it will be applied (Norman & Schmidt, 1992; Schmidt, 1993; Glaser, 1981). Though not
mentioned in the definition, Schmidt (1995) emphasized that it is also crucial for PBL that not all
information needed to solve the problem is provided to the students.

Second and contrary to conventional methods of teaching in which the teacher directly instructs the
students in the content of the course, PBL is student-centred in the sense that the students themselves
have a greater input and responsibility in deciding what and how they will learn. In other words, in a
PBL setting, students are expected to use a self-directed learning approach to search and gain new
relevant information (information which is typically related to existing knowledge). Specifically,
students are expected to study the problem, construct an idea of the phenomenon/phenomena that are

2
The complexity of defining PBL is discussed by, among others, Barrows (1996, 1986). Barrows remarked that
many schools implemented own variations of PBL which typically consist of a mix of conventional and PBL-
teaching. To help clarify the picture, Barrows (1986) felt it necessary to develop a taxonomy of PBL types.
4
related with the problem, link the problem under study to prior knowledge, identify the lacunas in
their current knowledge and look for new and available sources of knowledge (e.g., in the library, on
the internet, etc.).

As a third key feature, all the above occurs on an individual basis as well as in small-group
discussions. Among others, Antepohl and Herzig (1999) argued that small groups of students are an
essential part of PBL. In the group discussions, students have the opportunity to exchange findings
and ideas, look for additional insights and information that enables them to acquire a deeper and better
understanding of the problem at hand. Interactions with other students also enable students to acquire
more knowledge and skills, learn from each other, and develop social and co-working skills.

Nevertheless, while the definition of Schmidt (1995) is a good attempt at providing a (more or less)
standard characterization of PBL, it is incomplete. Several features of PBL are not covered in this
definition. A more comprehensive description of the PBL-approach is provided by Barrows (1986,
1996) who listed six basic characteristics of PBL. An example of a characteristic that is less treated in
the definition of Schmidt (1995) but that is discussed in the characterization of PBL made by Barrows
(1996) is the role of the teacher in PBL. Barrow (1996) described that the teachers play the role of
facilitators and guides. In the PBL literature this role is often referred to as “tutor”. Typically, the task
of the tutor consists in stimulating the group-discussions, monitoring the group interactions (e.g., to
avoid free riding of group members), and evaluating the progress.

Before concluding this section, it is important to note that whereas providing a comprehensive
definition of PBL is a difficult and complex matter, the same holds for the definition of conventional
teaching approaches (as remarked by, among others, Albanese and Mitchell (1994) and Dochy et al.
(2003)). That is, even though traditional teaching is typically featured by large-group lectures and
teacher-provided learning objectives and assignments, different versions of conventional teaching are
applied in practice.

2.2 PBL and student outcomes

A major question that inevitably arises in the study of the effects of PBL is whether the PBL-approach
offers merits to students in terms of high attainments in knowledge and skills. Unsurprisingly, this is
one of the questions which has received most attention in the PBL literature. Some studies indicate
that PBL has positive impacts on the students‟ knowledge and skills, other studies finding more
complex results, indicating that PBL students do better in terms of problem solving skills while they
do equally good in terms of acquiring knowledge. The results of these studies are analyzed in a
number of review studies and meta-analyses. Most of these studies focus on higher education.
Examples include Albanese and Mitchell (1993), Vernon and Blake (1993), Berkson (1993), Colliver
(2000), Dochy et al. (2003), Gijbels et al. (2005), and Dignath et al. (2008).

5
The review study of Colliver (2000) scrutinizes, among other things, the main findings of Albanese
and Mitchell (1993), Vernon and Blake (1993), Berkson (1993). Colliver (2000) discussed that
whereas these three studies differ greatly in their approach of studying the previous PBL literature
(Albanese and Mitchell (1993) listed and categorized the results of quantitative studies, Vernon and
Blake (1993) perform a meta-analysis on previous studies, and Berkson (1993) followed a traditional,
narrative approach), they all came to a similar conclusion: that is, whereas some of these studies
reported some positive effects of PBL on student test scores (and, thus, on students knowledge and/or
skills), at best these effects are only small. Moreover, Colliver (2000) remarked that the positive
effects of PBL on test scores may be attributable to selection bias (as most reviewed studies are non-
randomized).

More recent studies analyze the effects of PBL on several categories of student outcomes. Dochy et
al. (2003), for instance, performed a meta-analysis (using both a vote counting and a combined effect
size method) to look upon the link between PBL and students‟ knowledge and knowledge application.
They observed (a) a negative impact of PBL on students‟ knowledge base and (b) a positive and
statistically significant effect of PBL on students‟ knowledge application.3 With respect to the former
result, Dochy et al. (2003) indicated that this should be treated with caution as it is strongly influenced
by outliers.4 Gijbels et al. (2005) looked at the impact of PBL on three levels of knowledge (i.e.,
understandings of the concept, understanding of the principles that link concepts, and linking of
concepts and principles to conditions and procedures for application). They found the most positive
effects at the level of knowledge which concerns understanding principles that link concepts. Smits et
al. (2002) focused particularly on the effects of PBL on student knowledge as found in studies in
medical education and concluded that overall the evidence that PBL has positive effects is rather
weak.

In sum, previous literature on the effectiveness of PBL in terms of increasing student knowledge (as
compared to conventional teaching methods) shows mixed results. With all of this in mind, we
therefore concur with Colliver‟s (2000) conclusion that the empirical results on the effects of PBL on
student knowledge are rather disappointing, “providing no convincing evidence for the effectiveness
of PBL, at least not the magnitude of effectiveness that would be hoped for with a major curriculum
intervention” (Colliver, 2000, p. 264). In his opinion, the education effectiveness of PBL compared
with conventional methods therefore remains to be seen.

2.3 PBL and classroom learning environment

Previous literature indicated that students in a PBL-track are more satisfied with their teaching and the
classroom as a learning environment than students in the conventional track. In particular, students

3
Note that the meta-analysis of Dochy et al. (2003) looked at studies both within and beyond the medical
domain.
4
The Appendix A and B in Dochy et al. (2003) give a detailed overview of the effects of PBL found in the
studies considered in the meta-analysis.
6
which are in a PBL-track consider their lectures/teaching as more nurturing and enjoyable. For
instance, Moore-West et al. (1989) looked upon how students perceive and appreciate their learning
environment and courses. They found that PBL students rated their experiences higher in terms of
flexibility, emotional climate, meaningfulness, nurturance, student interactions, and hospitability.
Dochy et al. (2005) reported that students in the PBL tracks were satisfied with the problem solving,
applicability, and group discussions aspects of the PBL-based courses. Other studies confirm the latter
result (e.g., Percac and Armstrong, 1998; McNaughton and Krentz, 2000).

Nevertheless, PBL students are also identifying aspects of PBL that are less positive. In particular,
Struyven et al. (2008a,b) showed that while the perceptions of students in the conventional courses
were almost uniformly positive, the experiences of students that followed courses with student-
activating teaching approaches were more disperse with both extreme positive and extreme negative
perceptions. Students were generally positive about, e.g., the challenging and activating nature of the
assignments. They were more negative about the time pressure and workload associated with the
assignments, the increased responsibility, and the possibility of group difficulties and lacking co-
operation with some group members. Another issue is that PBL-courses tend to encourage
competition between students (Moore et al., 1990). On the other hand, Moore-West and O‟Donnell
(1985) found that conventional students were more stressed than the counterparts in the PBL tracks
(even though both the PBL and conventional students were initially found to be equally stressed, that
is, before the start of the courses).

To sum up, the findings of previous studies on the impact of PBL on student satisfaction with the
teaching approach and the classroom as a learning environment are not consistent. Generally students
appreciate very much the group interaction and the challenging and active nature of the problem
assignment. However, students also indicate that there are some aspects of PBL which may cause
problems (for instance, group difficulties and free-riding, difficulties in time management, etc.).

2.4 PBL and student motivation

Self-directed and active learning is believed to engage students in developing a higher level of
autonomous study motivation, a larger variety of learning skills, and better learning outcomes. Based
on the self-determination theory (Black & Deci, 2000) and the concepts of autonomous motivation
and controlled motivation, Vansteenkiste et al. (2009) distinguished between four motivational
profiles of students and found that the „good quality motivation‟ group of students (i.e., the students
with high autonomous and low controlled motivation) displayed the most optimal learning pattern
relative to all other groups of students. It also seems that an active learning environment stimulates
students to adopt deeper learning strategies in the study of the course content (Wilson & Fowler,
2005; Pintrich, 1999). That is, whereas conventional students are typically more studying the course
for reproduction, PBL students are more likely to study the course for meaning (with more focus on
understanding the material rather than on just being able to memorize the material for the short term).
This result was also found at different levels of education (that is, not only in higher education). For
7
instance, using a meta-analysis, Dignath et al. (2008) examined the effects of self-regulated learning
interventions (e.g., PBL) on students‟ motivation in primary schools. The results of this analysis
showed a considerably positive effect.

However, there are also studies who found more mixed results. Phipps et al. (2001) examined the
students‟ perceptions of cooperative learning (like PBL) and, in particular, the impact of such learning
approaches on student motivation. In their study, about half of the students believed that techniques of
cooperative learning were rather ineffective in terms of increasing student motivation. That is,
approximately one out of two questioned students perceived the active learning approaches to have a
positive influence on the student motivation, the other half was not at all convinced from this. So, as
with the effects of PBL on student outcomes and student perception and satisfaction with the
classroom learning environment, the findings of previous studies on the association PBL-student
motivation is not conclusive.

Due to selection bias and the non-random assignments of schools, previous findings should be
interpreted with caution. Moreover, most studies focus on higher education from which the results can
not straightforwardly be generalized to other education levels. To account for these limitations, the
next section outlines a randomized experiment in which PBL is employed as an instruction method in
secondary education.

3. Data

The experiment consists of 15 teachers teaching a course both in PBL and in traditional style at a
secondary school in Flanders (a region in Belgium). In particular, each teacher was randomly assigned
with a random secondary school in which he or she had to teach a course in PBL and in the traditional
format. As mentioned previously, several variations of PBL have been implemented and studied. To
avoid that the teachers involved in the experimental study would have different ideas of what exactly
constitutes PBL, teachers participated a brainstorm meeting on a definition of PBL before the start of
the experiment. The discussion (which took about 4 hours) aimed for a common understanding and
wide support on the definition. This resulted in the following characterization of PBL: “PBL is an
active learning method that starts from a concrete problem. Through group discussion, individual
study and collaboration in small groups, students discover their own knowledge, try to understand the
underlying mechanisms of the problem and solve the problem together. The teacher acts as a tutor that
guides the students and supports the students‟ initiatives”. Note that in the definition, the role of the
teacher in the PBL-courses is rather clearly explained to the 15 teachers involved. The role was
defined as one of facilitating the problem-solving process. It was strongly emphasized to the teachers
that they should not provide any solutions. Nevertheless, teachers were informed that guiding students
to (more) interesting sources of information was allowed. The teachers were strongly encouraged to
teach the PBL-version of the course completely consistent with this definition. Moreover, the
experiment leaders (i.e., the authors of the paper) checked whether teachers were teaching the PBL-
version of the course in a completely consistent way with this definition. The check was based on the
8
course preparations (of both instruction methods).

The students in the PBL-groups were presented with a problem that refers to an existing situation or
event in the real world and that asks for one or multiple concrete solution(s). They were told that they
were expected to study the problem, construct an idea of the phenomenon/phenomena that are related
with the problem, link the problem under study to prior knowledge, identify the lacunas in their
current knowledge, and look for new and available sources of knowledge (e.g., in the library, on the
internet, etc.). All of this occurs in small-group discussions.

The experimental and control group classes are classes of the same size and in which students are
randomly assigned by the school management. In particular, the experiment is evaluated for students
in an educational track which is too large to constitute one class-group (i.e., larger than 30 students)
but had to be split (by law) in two class groups. It is clear that the students are randomly assigned to
one of the two class groups such that all background variables, including student ability and
motivation, should be comparable across both groups. Therefore, selection effects are avoided both at
the school, class and student level.

In total, over the 15 secondary schools, PBL-based courses grouped 260 students whereas the
traditional lecture-based courses consisted of 271 students. This means that in total 531 students
participated in the experiment. The students in the PBL-based classes were further subdivided into
small groups of 4 to 5 students. This is completely according to the PBL-approach which states that
students should co-operate in small groups. Respectively 66.1% and 33.9% of the students are girls
and boys. The average age of the participating students is 17.21 years (stdv. = 1.46). The majority of
the participating students (77.6%) are in the final two years of secondary school. The remaining
22.4% of the students are predominantly in the third and fourth year of secondary school. Some
descriptive statistics on the student characteristics for both the PBL- and conventional groups of
students are listed in Table 1. The results (by a t-test) indicate that the control and experimental group
are well comparable. That is, the composition of the experimental and control class groups are,
overall, rather similar (in terms of gender, age, class size, etc.).

[Table 1 about here]

Note that Table 1 also displays summary information on the student test scores, student motivation
and student appreciation of the class atmosphere in both the control and experimental group. To
measure the gain in student knowledge, both before and after the course, students were asked to
complete multiple choice questions to test for their factual knowledge on the subject-matter. Besides
this formal test of added value in educational attainments, we asked by a questionnaire to which
extent the students thought they have learnt a lot during the PBL or traditional course. It is measured
by a likert-type variable which could range from „I learnt nothing‟ till „I learnt very much‟. This
provides us with an indication on the perceived educational attainments.

9
Students were also asked about their motivation for studying and, in particular, whether he/she is
autonomously (cf. intrinsic) or controlled (cf. extrinsic) motivated for studying. To measure the
student‟s motivation on a scale, the Academic Self-Regulation Scale after Vansteenkiste et al. (2009)
was used. This questionnaire consists of 16 items which measure the students‟ motivation for
studying. In particular, each item provided students with a possible motivation for studying. The
students were asked to indicate to what level this motivation applies to their study behaviour (i.e.,
each item was measured using a 5-point Likert scale with rates varying from “1: completely not
important” to “5: very important”). Of these 16 items, 8 items are used to measure the students‟
autonomous motivation for studying. These items provide reasons for studying that are characterized
by a feeling of choice and an own will to study. Examples of possible reasons for studying include:
„because it is personally important to me‟ and „because I want to learn new things‟. The other 8 items
are employed to measure the level of controlled motivation for studying. These items provide students
with reasons for studying that are characterized by a feeling of obligation and/or pressure. Examples
of statements are „because I‟m supposed to do so‟ and „because that‟s something others (parents,
friends, etc.) force me to do‟. Two principal component analyses are performed to check whether the
items that are used to measure respectively autonomous and controlled motivation effectively measure
those two types of motivation. The results are presented in Appendix 1 and indicate that for each type
of motivation, the resulting principal component explains a satisfactory level of variance. In addition,
for both the factor autonomous and controlled motivation, each of the selected items loads sufficiently
to the principal component. The summary statistics of the autonomous and controlled motivation
components are presented in Table 1.

As a final outcome variable, a questionnaire was also administered to measure the students‟
perception and appreciation of the class climate during the teaching of the course. In particular, the
questionnaire contains 8 items that measure, among other things, the atmosphere in the class during
the course, the students‟ perception of the participation in class, etc. The questionnaire is presented in
Appendix 2. The student ratings on the 8 items are aggregated by a simple average into an overall
appreciation score.

4. Findings

In each of the secondary education schools, students in the PBL-course are compared to students in
traditional lecture-based course in terms of their achievements (i.e., student test scores), their
perceived learning, autonomous (intrinsic) motivation, controlled (extrinsic) motivation and the
atmosphere in the class. As classes and schools were randomly assigned, a causal interpretation can be
given to the variable of interest. Students who received a course by an instruction method which
follows the PBL-idea receive a value 1(denoted below by „experimental group‟), students in the
traditional lecturing receive a value 0 (i.e., the control group). In what follows, we discuss the most
important results.

4.1 Actual and perceived student outcomes

10
First consider the causal relationship between PBL and student achievements. The latter are tested by
uniform (pre and post-test) tests in the control and experimental group. The results are presented
along 4 different model specifications in which we increasingly control for observed heterogeneity.
The results are displayed in the left part of Table 2. A first model specification (i.e., Model 1)
considers the simple regression of belonging to the experimental group (the control group is reference
group) while only controlling for the pre-test on the student achievement. The results indicate that,
once controlled for prior knowledge, students in the experimental group obtained significantly less
educational attainments than students in the control group. On a test score that ranges from 0 to 10
(with a mean of 7.19), the score decreases in the experimental group by 0.33 in comparison to the
control group. In other words, a PBL-inspired lecturing style does not contribute to students
outcomes. Quite the contrary, at first sight, it seems that PBL has a negative impact on student
knowledge (as measured by the test scores).

[Table 2 about here]

However, this estimated effect fades out if we control for sources of observed heterogeneity. Model 2
includes pupil gender (boys are reference group) and class size. The effect of receiving PBL-based
learning stays negative, although non-significant (i.e., non-significant negative estimates of -0.3073,
-0.2372 and -0.1830 for the Models 2-4). Thus, the results from the extended models (accounting for
heterogeneity) indicate that the test scores of the students in the PBL-based courses are not
significantly different from the scores of the students that followed the traditional courses. Students in
our experimental application did not experience any considerable advantage in terms of realizing a
higher factual knowledge gain from following the PBL-course compared to their counterparts that
were in the traditional course track. As indicated in the literature section, a similar result was also
found by several previous studies. In fact, the finding of a non-significant impact of PBL on student
learning confirms Colliver‟s (2000) conclusion, made in the previous section, that the empirical
results on the effects of PBL on student knowledge are rather disappointing.

Note in the left part of Table 2 that, as expected, prior educational attainments have a positive
significant effect on the achievements. In addition, although females are correlated with lower
outcomes, the estimate is not significant. Finally, class size has a significant negative effect on student
achievements (with -0.0527 as significantly negative estimate of class size in Model 2). This is in line
with some of the previous literature (e.g., Akerhielm, 1995).5 The results are robust to further
inclusion of heterogeneity. To capture the location of the school, model 3 includes provincial fixed
effects. We observe 5 provinces, where a different attitude of students might be present. Model 4
includes class atmosphere and the willingness of the student to intervene during the lecture. Neither of
the latter variables have a significant influence on the student achievements, nor do they change
previous outcomes.

5
Although, admittingly, there are also several examples of studies that revealed an insignificant relationship
between class size and student achievements (e.g., Hanushek, 1997, 1999; Hanushek & Luque, 2003).
11
Besides evaluating the impact of problem based learning on what students really have learnt, it is also
interesting to examine the effect on what students think they have learnt. The pre-test corresponds to
the prior knowledge of the student (i.e., the same pre-test as with actual achievements). The results are
listed in the right part of the Table 2. Interestingly, the effect of a PBL inspired instruction method
decreases in the perceived learning in comparison to the actual learning. While the former estimate
equals -0.10, it was -0.33 (and statistically significant at the 10%-level) in the latter. In other words,
despite students did not learn significantly more in the experimental group, the differences between
the control group and experimental group are less pronounced in the perception of the student than in
the actual learning process. This observation is constant in the four model specifications.

4.2. PBL and student motivation

Table 3 presents the estimated effect of the PBL instruction method on autonomous motivation (left
panel, Model 9-12) and controlled motivation (middle panel, Model 13-16). The autonomous
motivation, as measured by 8 questions (see the section “Data” as well as Appendix 1), is lower in the
experimental group, although this outcome is not significantly different from 0. More precisely, the
four negative estimates of -0.200, -0.0125, -0.0221, and -0.0571 are all statistically insignificant. In all
four model specifications (model 9-12), the correlation of the pretest with the posttest is significantly
positive, which is intuitive. In addition, results indicate that females have a lower autonomous
motivation (with in Model 10-12 negative coefficients that are statistically significant at the 10%- or
5%-level), and also class size influences it negatively (all three estimates have coefficients negative
and statistically significant at the 1%-level). As can be observed from model 12, in classes with better
atmosphere, the autonomous motivation of students is higher (i.e., estimate 0.1929 significant at the
1%-level). Note, however, that there is in the paper at hand no evidence on the direction of this effect,
such that reverse causality cannot be excluded. A similar observation can be made for the activity of
the student. In particular, students who do not ask anything if they have a question have in comparison
to students who immediately raise their hand if they have a question a significantly lower autonomous
motivation.

[Table 3 about here]

The results in Table 3 show that PBL seems to have a non-significant positive impact on controlled
motivation. Students in the experimental group increase their controlled motivation in comparison to
students in the control group, although not significantly different from 0 (i.e., the estimates of 0.0180,
0.0267, 0.0206 in the Models 13-15 are all statistically insignificant). This surprisingly contrasts to
the estimates of autonomous motivation where a negative coefficient is estimated. The other
coefficients (i.e., the estimates for pre-test scores, gender, and class size) are similar to before.

12
 4.3. PBL and classroom climate/atmosphere

Finally, we examine the relationship between PBL and the class atmosphere. The results are displayed
in the right panel of Table 3. Students in the experimental group indicate a significant higher class
atmosphere (i.e., significant positive estimates of 0.1385 and 0.1251 in the Models 18 and 19). To be
more precise, a comparison of the satisfaction rates of the students with several aspects of the course
and the class atmosphere shows that, on average, PBL-students are rating the course more positively
on a large majority of the questionnaire items. For instance, students reported higher satisfaction on
working together in group to solve the problem made, one of the core aspects of PBL.

All other variables have the expected sign. For instance, pre-test scores are positively related to the
class atmosphere and class size is found to be negatively related to the class atmosphere. The larger
the class, the lower the class atmosphere. In model 18, there is also a negative coefficient for the
gender of the students (the estimate of -0.1642, significant at the 5%-level). This result seems to
suggest that females feel a significant lower class atmosphere in comparison to boys. Also observed in
Table 3 is the positive coefficient for the atmosphere in class (i.e., coefficient estimate of 0.2981). A
better class atmosphere before the start of the PBL-based lecture results in a better atmosphere
afterwards. However, as the coefficient is not statistically significant, this interpretation should be
treated with caution.

In sum, despite the finding that a PBL-instruction method does not result in higher test scores, it
improves the class atmosphere. It can therefore be considered as an interesting alternative didactic
tool for secondary education students, rather than an alternative instruction way. Obviously one
should take care in generalizing the results. We come back to this issue in the concluding section.

5. Conclusion and discussion

Though problem-based learning is increasingly popular in higher education (particularly in medicine

education), it is only rarely applied in a secondary education context (although lately, there are more
and more applications of PBL in secondary education). This paper tests the effectiveness of PBL as an
alternative instruction method in secondary education. In a controlled randomized experiment, we
estimate its effect on tested student attainments, on perceived student attainments, on autonomous (cf.
intrinsic) and controlled (cf. extrinsic) motivation and on class atmosphere. The experiment covers 15
random secondary schools in Flanders. 2*15 parallel classes with a total of 531 students were
randomly assigned to either a control and experimental group. In an experimental group the teacher
was teaching the subject using principles of PBL; in a control group he/she was teaching the subject
using traditional lecturing.

13
The results indicated that students in a PBL course do not obtain higher test results than students in a
traditional taught course. On the contrary, the estimated effect turned out to be negative, although for
most model specifications not significantly different from 0. Stated otherwise, on average, students in
the PBL-courses did not realize significantly higher or lower test scores compared to the counterparts
in the non-PBL courses. PBL does not result in advantages for the students in PBL-courses in terms of
acquiring factual knowledge, however, neither does it lead to disadvantages. We also did not observe
a difference between the actual test scores and the perceived attainments (i.e., what student think they
have learnt). Second, if PBL does not have an effect on educational attainments, it might have on
student motivation. In order to examine the association between PBL and student motivation, we
distinguished between autonomous (intrinsic) motivation and controlled (extrinsic) motivation. For
both types of motivation, we observed non-significant effects of PBL. Controlling for various sources
of heterogeneity does not alter the results. Finally, we find a significant positive influence of PBL on
class atmosphere. PBL as an instruction method is considered as a welcome change on the traditional
lecturing.

Of course, caution is warranted in the interpretation of the results of this study, and this for several
reasons. One possible reason for the inconclusive results may be transitional problems associated with
the change from conventional courses to PBL-based courses. Among others, O‟Hanlon et al. (1995)
pointed to this issue. According to this study, it may take some time for both students and teachers to
get accustomed with a group-based learning environment. In the current experiment, students had
only little time to adjust to the idea and the process of PBL. In this sense, our results can be
interpreted as a lower bound estimate as it can be expected that repetition of the instruction method
increases the attainments and motivation. As a second issue, the results provided by the multiple
choice tests may only provide a partial picture of the gain in student knowledge and skills. One could
question whether a multiple choice test is sufficient for assessing the change in student knowledge
during the course. The answer is probably not. In order to get a more reliable picture of the student
knowledge, it would also be interesting, for instance, to ask students to complete one or more essay
questions. Perhaps an examination of more complex knowledge or of other types of knowledge should
have been more appropriate. A third reason for why one should be cautious in interpreting the results
of this study is that the number of students (531) and the number school institutions (15) participating
in this study is rather low. To increase the power of the analysis a larger data set is required. This third
issue also results in lower bound estimates. Despite these limitations, the lower bound estimates point
to some favourable but often insignificant effects of problem based learning as an instruction method
in secondary education.

14
 Literature

Albanese, M.A., and Mitchell, S.M. (1993), “Problem-based learning: A review of literature on its
outcomes and implementation issues”, Academic Medicine, 68(1), pp. 52-81.
Abrandt Dahlgren, M., and Dahlgren, L.O. (2002), “Portraits of PBL: Students‟ experiences of the
characteristics of problem-based learning in physiotherapy, computer engineering and
psychology”, Instructional Science, 30(2), pp. 111-127.
Akerhielm, K. (1995), “Does class size matter?”, Economics of Education Review, 14(3), pp. 229-
241.Antepohl, W., and Herzig, S. (1980), “Problem-based learning versus lecture-based
learning in a course of basic pharmacology: A controlled, randomized study”, Medical
Education, 33, pp. 106-113.
Barrows, H.S. (1986), “A taxonomy of problem-based learning methods”, Medical Education, 20, pp.
481-486.
Barrows, H.S. (1996), “Problem-based learning in medicine and beyond: A brief overview”, New
Directions for Teaching and Learning, 68, pp. 3-12.
Berkson, L. (1993), “Problem-based learning: Have expectations been met?”, Academic Medicine,
68(10), pp. S79-S88.
Black, A.E., and Deci, E.L. (2000), “The effects of instructors' autonomy support and students'
autonomous motivation on learning organic chemistry: A self‐determination theory
perspective”, Science Education, 84(6), pp. 740-756.
Colliver, J.A. (2000), “Effectiveness of problem-based learning curricula: Research and theory”,
Academic Medicine, 74(3), pp. 259-266.
Deci, E.L., and Ryan, R.M. (2000), “The what and why of goal pursuits: Human needs and the self-
determination of behavior”, Psychological Inquiry, 11(4), pp. 227-268.
Dignath, C., Buettner, G., and Langfeldt, H.-P. (2008), “How can primary school students learn self-
regulated learning strategies most effectively? A meta-analysis on self-regulation training
programmes”, Educational Research Review, 3, pp. 101-129.
Dochy, F., Segers, M., Van den Bossche, P., and Gijbels, D. (2003), “Effects of problem-based
learning: A meta-analysis”, Teaching and Learning in Medicine, 13, pp. 533-568.
Dochy, F., Segers, M., Van den Bossche, P., and Struyven, K. (2005), “Students‟ perceptions of a
problem-based learning environment”, Learning Environments Research, 8(1), pp. 41-66.
Gallagher, S.A., Stepien, W.J., and Rosenthal, H. (1992), “The effects of problem-based learning in
problem solving”, Gifted Child Quarterly, 36(4), pp. 195-200.
Gijbels, D., Dochy, F., Van den Bossche, P., and Segers, M. (2005), “Effects of problem-based
learning: A meta-analysis from the angle of assessment”, Review of Educational Research,
75(1), pp. 27-61.
Glaser, R. (1981), “The future of testing: A research agenda for cognitive psychology and
psychomentrics”, American Psychologist, 36(9), pp. 923-936.
Hanushek, E.A. (1997), “Assessing the effects of school resources on student performance: An
update”, Educational Evaluation and Policy Analysis, 19(2), pp. 141-164.

15
Hanushek, E.A. (1999), “Some findings from an independent investigation of the Tennessee STAR
experiment and from other investigations of class size effects”, Educational Evaluation and
Policy Analysis, 21(2), pp. 143-163.
Hanushek, E.A., and Luque, J.A. (2003), “Efficiency and equity in schools around the world”,
Economics of Education Review, 22, pp. 481-502.
Joham, C., and Clarke, M. (2011), “Teaching critical management skills: The role of problem-based
learning”, Teaching in Higher Education, in press.
Maxwell, N.L., Bellisimo, Y., and Mergendoller, J.R. (2001), “Problem-based learning: Modifying
the medical school model for teaching high school economics”, The Social Studies, 92(2), pp.
73-78.
Maxwell, N.L., Mergendoller, J.R., and Bellisimo, Y. (2005), “Problem-based learning and high
school macroeconomics: A comparative study of instructional methods”, The Journal of
Economic Education, 36(4), pp. 315-239.
McNaughton, K., and Krentz, C. (2000), “Reflections on constructivist practices in early childhood
teacher education”, Canadian Journal of Research in Early Childhood Education, 8(2), pp.
7–20.
Mergendoller, J.R., Maxwell, N.L., and Bellisimo, Y. (2000), “Comparing problem-based learning
and traditional instruction in high school economics”, The Journal of Educational Research,
93(6), pp. 374-382.
Moore, G.T., Black, S.D., Style, C.B., and Mitchell, R. (1989), “The influence of the new pathway
curriculum on Harvard medical students”, Academic Medicine, 69, pp. 983-989.
Moore-West, M., Harrington, D.I., Mennin, S.P., Kaufman, A., and Skipper, B.J. (1989), “Distress
and attitudes toward the learning environment. Effects of a curriculum innovation”, Teaching
and Learning in Medicine., 1(3), pp. 151-157.
Norman, G.R., and Schmidt, H.G. (2000), “the psychological basis of problem-based learning: A
review of the evidence”, Academic Medicine, 67(9), pp. 557-565.
O‟Hanlon, A., Winefield, H., Hejka, E., and Chur-Hansen, A. (1995), “Initial responses of first-year
medical students to problem-based learning in a behavioural science course: Role of language
background and course content”, Medical Education, 29, pp. 198-204.
Percac, S., and Armstrong, E.G. (1998), “Introducing a problem-based anatomy course in a traditional
curriculum: A Croatian experience”, Medical Teacher, 20(2), pp. 114-117.
Phipps, M., Phipps, C., Kask, S., and Higgins, S. (2001), “University students‟ perceptions of
cooperative learning: Implications for administrators and instructors”, Journal of Experiential
Education, 24(1), pp. 14-21.
Pintrich, P.R. (1999), “The role of motivation in promoting and sustaining self-regulated learning”,
International Journal of Educational Research, 31, pp. 459-470.
Schmidt, H.G. (1995), “Problem-based learning: An introduction”, Instructional Science, 22, pp. 247-
250.
Schmidt, H.G. (1993), “Foundations of problem-based learning: Some explanatory notes”, Medical
Education, 27, pp. 422-432.

16
Schmidt, H.G., Machiels-Bongaerts, M., Hermans, H., ten Cate, T.J., Venekamp, R., Boshuizen,
H.P.A. (1996), “The development of diagnostic competence: Comparison of a problem-based,
an integrated, and a conventional medical curriculum”, Academic Medicine, 71, pp. 658-664.
Smits, P.B.A., Verbeek, J.H.A.M., and De Buisonje, C.D. (2002), “Problem based learning in
continuing medical education: A review of controlled evaluation studies”, British Medical
Journal, 321, pp. 153-432.
Son, B., and VanSickle, R.L. (2000), “Problem-solving instruction and students‟ acquisition,
retention, and structuring of economics knowledge”, Journal of Research and Development in
Education, 33(2), pp. 95-105.
Struyven, K., Dochy, F., and Janssens, S. (2008a), “Students‟ likes and dislikes regarding student-
activating and lecture-based educational settings: Consequences for students‟ perceptions of
the learning environment, student learning and performance”, European Journal of
Psychology of Education, 23(3), pp. 295-317.
Struyven, K., Dochy, F., Janssens, S., and Gielen, S. (2008b), “Students‟ experiences with contrasting
learning environments: The added value of students‟ perceptions”, Learning Environment
Research, 11, pp. 83-109.
Vansteenkiste, M., Lens, W., and Deci, E.L. (2006), “Intrinsic versus extrinsic goal contents in self-
determination theory: Another look at the quality of academic motivation”, Educational
Psychologist, 41(1), pp. 19-31.
Vansteenkiste, M., Simons, J., Lens, W., Sheldon, K.M., and Deci, E.L. (2004), “Motivating learning,
performance, and persistence: The synergistic effects of intrinsic goal contents and autonomy-
supportive contexts”, Journal of Personality and Social Psychology, 87(2), pp. 246-260.
Vansteenkiste, M., Soenens, B., Sierens, E., Luyckx, K., and Lens, W. (2009), “Motivational profiles
from a self-determination perspective: The quality of motivation matters”, Journal of
Educational Psychology, 101(3), pp. 671-688.
Vernon, D.T.A., and Blake, R.L. (1993), “Does problem-based learning work? A meta-analysis of
evaluative research”, Academic Medicine, 68(7), pp. 550-563.
Walker, A., and Leary, H. (2009), “A problem based learning meta analysis: Differences across
problem types, implementation types, disciplines, and assessment levels”, Interdisciplinary
Journal of Problem-based Learning, 3(1), pp. 6-28.

17
 Table 1: descriptive statistics (n= 531)

t-statistic
Average Stdev. difference
Gender (dummy: 1 = boys) Control 0.6777 0.0283
Experimental 0.6484 0.0290 0.7233
Age Control 17.3300 1.2400
Experimental 17.1000 1.6400 0.1120
Year of school Control 5.1500 0.9000
Experimental 4.8900 1.3800 0.1578
Class size Control 17.6264 0.2897
Experimental 17.9304 0.2887 -0.7433
Asking questions Control 1.4884 0.0495
Experimental 1.4144 0.0460 1.0946
Pre-test educational attainments Control 5.4286 0.1603
Experimental 5.3577 0.1495 0.3236
Post-test educational attainments Control 7.3485 0.1403
Experimental 7.0313 0.1403 1.5974
Post-test 'Perceived' attainments Control 3.4962 0.0498
Experimental 3.3755 0.0487 1.7326 *
Pre-test class atmosphere Control 3.5627 0.0593
Experimental 3.8185 0.0538 -3.1970 ***
Post-test class atmosphere Control 3.4302 0.0562
Experimental 3.5655 0.0576 -1.6806 *
Pre-test autonomous motivation Control 2.6334 0.0461
Experimental 2.7906 0.0432 -2.4914 **
Post-test autonomous motivation Control 3.4617 0.0441
Experimental 3.4721 0.0444 -0.1666
Pre-test controlled motivation Control 2.8073 0.0404
Experimental 2.7625 0.0458 0.7325
Post-test controlled motivation Control 2.7202 0.0346
Experimental 2.7764 0.0344 -1.1502
where ***, ** and * denote a significant different mean, respectively, at 1, 5 and 10%-level.

18
 Table 2: Effects of problem based learning on learning

Outcome variable Student achievement (tested) Student achievement (perceived)

Variable Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 Model 8
Experimental group (pbl = 1) -0.3283* -0.3073 -0.2372 -0.1830 -0.1035 -0.1017 -0.1052 -0.1244*
0.1985 0.1978 0.1852 0.2009 0.0706 0.0707 0.0693 0.0715
Pretest (for respective outcome) 0.0672 0.077* 0.1006** 0.069* 0.0022 0.0029 0.0165 -0.0010
0.0408 0.0408 0.0423 0.0410 0.0143 0.0143 0.0153 0.0143
Gender (female = 1) -0.0864 -0.2601 -0.0367 -0.1062 -0.0837 -0.0921
0.2118 0.2027 0.2142 0.0751 0.0752 0.0758
Class size -0.0527** -0.141*** -0.0629*** -0.0072 -0.0215** -0.0103
0.0206 0.0252 0.0211 0.0074 0.0095 0.0076
Atmosphere in class -0.1632 0.1012**
0.1119 0.0398
If I have a question:
(1) I raise my hand during class -0.0365 0.1629
0.3242 0.1166
(2) I ask teacher afterwards 0.315 -0.2047**
0.2754 0.0979
(3) I don't ask anything 2.1796 -0.2985
2.2076 0.7947
Constant 7.0126*** 7.9363*** 8.6868*** 8.6383*** 3.4561*** 3.6496*** 3.3751*** 3.3777***
0.2613 0.4447 0.5777 0.6351 0.0922 0.1590 0.2185 0.2267
Province fixed effects Yes Yes

Number of observations 495 495 495 485 508 508 508 496
R²-adjusted 0.0071 0.0168 0.1416 0.0211 0.0003 0.0025 0.0437 0.0226
where *, ** and *** denote, respectively significance at the 10, 5 and 1%-level; standard errors below the estimated coefficient

19
 Table 3: Effect of problem based learning on motivation

Outcome variable Autonomous motivation Controlled motivation Atmosphere in class

Model 9 Model 10 Model 11 Model 12 Model 13 Model 14 Model 15 Model 16 Model 17 Model 18 Model 19 Model 20
Experimental group (pbl = 1) -0.0200 -0.0125 -0.0221 -0.0571 0.0180 0.0267 0.0206 -0.0312 0.1278 0.1385* 0.1251* 0.0532
0.0614 0.0600 0.0564 0.0592 0.0623 0.0611 0.0577 0.0597 0.0801 0.0770 0.0719 0.0749
Pretest (for respective outcome) 0.2128*** 0.2237*** 0.2377*** 0.1654*** 0.1017** 0.0998** 0.0942** 0.0864** 0.1691** 0.1816*** 0.1989*** 0.0975
0.0419 0.0410 0.0398 0.0408 0.0445 0.0435 0.0417 0.0424 0.0713 0.0686 0.0658 0.0667
Gender (female = 1) -0.1478** -0.1009* -0.1096* -0.1357** -0.0983 -0.0994 -0.1642** -0.1144 -0.0967
0.0632 0.0605 0.0625 0.0646 0.0622 0.0632 0.0817 0.0778 0.0795
Class size -0.0304*** -0.055*** -0.0295*** -0.0291*** -0.0531*** -0.0289*** -0.0527*** -0.0837*** -0.0505***
0.0063 0.0078 0.0063 0.0065 0.0080 0.0064 0.0082 0.0101 0.0081
Atmosphere in class 0.1929*** 0.2089 0.2981
0.0328 0.0329 0.0415
If I have a question:
(1) I raise my hand during class 0.0868 0.1335 0.0686
0.0979 0.0983 0.1234
(2) I ask teacher afterwards -0.1774** -0.2291*** -0.2857***
0.0787 0.0794 0.0994
(3) I don't ask anything 0.0042 -0.0170 0.2307
0.6699 0.6778 0.8474
Constant 2.8989*** 3.5002*** 3.0628*** 2.9523*** 3.1744*** 3.7775*** 3.4283*** 3.077*** 2.9691*** 3.9675*** 3.3485*** 3.1025***
0.1188 0.1608 0.1952 0.2020 0.1325 0.1770 0.2100 0.2164 0.2020 0.2430 0.2822 0.2812
Province fixed effects Yes Yes Yes

Number of observations 531 531 531 518 531 531 531 518 526 526 526 513
R²-adjusted 0.0429 0.0906 0.2012 0.1499 0.0061 0.0485 0.1545 0.1295 0.0123 0.0898 0.2089 0.1795
where *, ** and *** denote, respectively significance at the 10, 5 and 1%-level; standard errors below the estimated coefficient

20
 Appendix 1: Principal component analyses of items of autonomous and controlled motivation

Factor loading mean Stdev. α (if item removed)

Component „autonomous motivation

- „Because I want to learn new things‟ 0.56 3.36 0.90 0.87

- „Because it is personally important to me‟ 0.75 3.11 0.98 0.85

- „Because this represents a meaningful choice to me‟ 0.67 3.25 1.04 0.86

- „Because this is an important life goal to me‟ 0.60 3.31 1.06 0.87

- „Because I am highly interested in doing this‟ 0.80 2.46 0.88 0.85

- „Because I enjoy doing it‟ 0.80 1.92 0.93 0.85

- „Because it‟s fun‟ 0.82 1.90 0.92 0.85

- „Because it‟s an exciting thing to do‟

Component „controlled motivation‟

- „Because I‟m supposed to do so‟ 0.61 3.41 1.02 0.77

- „Because that‟s something others (parents, friend, 0.73 2.61 1.20 0.75
etc.) force me to do‟

- „Because others (parents, friend, etc.) oblige me to do 0.74 2.70 1.23 0.75
so‟

- „Because that‟s what others (parents, friend, etc.) 0.71 3.19 1.13 0.75
expect me to do‟

- „Because I want others to think I‟m smart‟ 0.52 2.68 1.10 0.77

- „Because I would feel guilty if I didn‟t study‟ 0.51 2.63 1.12 0.78

- „Because I would feel ashamed if I didn‟t study‟ 0.60 2.36 1.11 0.76

- „Because I want others to think I‟m a good student‟ 0.59 2.54 1.06 0.76

21
Appendix 2: Student questionnaire of class climate/atmosphere

1. During the course I found the atmosphere in the classroom:

Very bad 1 2 3 4 5 Very good

2. During the course when I understood something wrong:

1. I raised my hand and asked for more explanation
2. I went to the teacher after class and asked for more explanation
3. I did nothing

3. During the course, time passes most quickly for me when:

1. The teacher is teaching him/herself.
2. We are solving problems in groups.
3. We are doing the exercises individually before discussing the results in class.

4. As a class, we cooperate best when:

1. The teacher is teaching him/herself.
2. We are solving the problems in groups.
3. We are doing the exercises individually before discussing the results in class.

5. If I disagree with the teacher/group or I have a comment then:

1. I say that immediately
2. I wait until after class and tell it to the teacher/person in the group
3. I remain silent

6. Working together in group to solve the problem made that:

1. I got to know my fellow students better and feel better in class.
2. I have more trouble to understand the content of the course.
3. incongruence arose between the group members.

7. If the teacher teaches in the traditional way then :

1. I can easily follow without having fellow students disturbing me.
2. I talk with fellow student about how they experienced the course after class.
3. there is at least no turmoil as we don‟t have to work together in group.

8. Overall, I learnt from this course:

Nothing 1 2 3 4 5 A lot

22