Computer Science Education for Primary and Lower Secondary

School Students: Teaching the Concept of Automata

2
DAIKI ISAYAMA, MASAKI ISHIYAMA, RAISSA RELATOR, and KOICHI YAMAZAKI,
Gunma University, Japan

We explore the feasibility of early introduction to automata theory through gamification. We designed a
puzzle game that players can answer correctly if they understand the fundamental concepts of automata
theory. In our investigation, 90 children played the game, and their actions were recorded in play logs.
An analysis of the play logs shows that approximately 60% of the children achieved correct-answer rates
of at least 70%, which suggests that primary and lower secondary school students can understand the
fundamental concepts of automata theory. Meanwhile, our analysis shows that most of them do not fully
understand automata theory, but some of them have a good understanding of the concept.
Categories and Subject Descriptors: K.3.2 [Computers and Education]: Computer and Information Science
Education—Computer science education
General Terms: Experimentation
Additional Key Words and Phrases: Automata theory, primary and lower secondary school students,
gamification
ACM Reference Format:
Daiki Isayama, Masaki Ishiyama, Raissa Relator, and Koichi Yamazaki. 2016. Computer science education
for primary and lower secondary school students: Teaching the concept of automata. ACM Trans. Comput.
Educ. 17, 1, Article 2 (September 2016), 28 pages.
DOI: http://dx.doi.org/10.1145/2940331

1. INTRODUCTION
1.1. Background and Motivation
As information technology develops, Computer Science Education (CSE) becomes in-
creasingly essential. Indeed, the computer has become such an essential tool in our
lives that it is important to develop “human resources” who are not only familiar with
computers and computer systems but are also able to create new computer technolo-
gies. Against this background, it has been discussed how CSE in K-12 schools should
be conducted [Brown et al. 2014; de France Académie des Sciences 2013; Hubwieser
et al. 2014]. These studies report that CSE in K-12 is in transition and that the cur-
riculum design of CSE in K-12 is still under development in many countries. Because
some general principles in the curriculum have been just announced, its applicability
will become clearer only gradually. In CSE, there is a tendency to focus on digital
literacy skills rather than computational thinking skills. Computational thinking is a

Authors’ addresses: D. Isayama, Faculty of Science and Technology, Gunma University, 1-5-1 Tenjin-cho,
Kiryu, Gunma, 376-8515 Japan; email: [email protected]; M. Ishiyama, R. Relator, and
K. Yamazaki, Graduate School of Science and Technology, Gunma University, 1-5-1 Tenjin-cho, Kiryu,
Gunma, 376-8515 Japan; emails: [email protected], [email protected]
u.ac.jp, [email protected].
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted
without fee provided that copies are not made or distributed for profit or commercial advantage and that
copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for
components of this work owned by others than ACM must be honored. Abstracting with credit is permitted.
To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this
work in other works requires prior specific permission and/or a fee. Permissions may be requested from
Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212)
869-0481, or [email protected].
c 2016 ACM 1946-6226/2016/09-ART2 $15.00
DOI: http://dx.doi.org/10.1145/2940331

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:2 D. Isayama et al.

fundamental skill for everyone that involves logical reasoning, problem-solving, system
design, an understanding of human behavior, and so on Wing [2006]. Actually, computa-
tional thinking has been defined in a number of ways (see NRC [2010] for details). CSE
is now turning to the development of students’ computational thinking. These studies
also address the problem of teacher training in computer science. They say that the
problem is critical and difficult, especially in primary and secondary schools. Several
countries have started teaching computer science at the primary and secondary school
levels. However, because most primary school teachers are generalists, teaching a new
subject like computer science increases their workload. Given these considerations,
it is important to develop learning materials for CSE specialized for primary school
students.
Although computer science is now just as important as the natural sciences and
mathematics, the early introduction of computer science (especially in primary and
lower secondary schools) is not as prevalent as the early introductions to other fields.
One reason for this is the difference in the lengths of these fields’ histories. Both
the natural sciences and mathematics have histories reaching back several centuries,
but the history of computer science goes back less than one century. Another possible
reason is that early introduction to CSE is considered difficult for primary and lower
secondary school (and even high school) children. Is that really true? We are interested
in the following points:
(1) How workable is CSE in primary and lower secondary schools (4th–6th grade)?
(2) Which topics in CSE are easy to practice?
(3) To what extent may children understand the concept of each topic in CSE?

1.2. Purpose of this Research

In this article, we explore the feasibility of the early introduction of automata theory
to primary and lower secondary school children (around 4th–6th grade: ages 9–12).
There are several studies on this topics (e.g., Bischof and Sabitzer [2011], Hamada and
Sato [2010], Korte et al. [2007], and Bell et al. [1998], see also Hartmann et al. [2001]
and Thomas et al. [2006]). For university students, automata theory is usually intro-
duced in an abstract manner using mathematical notions. For children in K-12, such an
approach easily discourages them. Thus, the problem is how to smoothly introduce au-
tomata to them. To address this problem, the preceding studies demonstrated suitable
approaches, most using simulations and/or games in which automata are incorporated
in a natural way. These findings indicate the possibility of the early introduction of not
only automata theory but also CSE to K-12 children.
Intuitively, an automaton can be thought of as a description of how to change the
state of a machine when performing an operation on it. An automaton is described
by states and symbols. Each state represents a state of the machine, and each symbol
corresponds to an operation on it. When performing an operation on the machine, its
state is deterministically changed into a new state according to the previous state and
the operation. (Note that the new state can be the same as the previous state.) There
also exists a “nondeterministic” version of automata. The initial state of the machine is
called the start state, and the desired states are referred to as accept states. When dis-
cussing an automaton, we are usually concerned with a process that leads to an accept
state from the start state. We call such a process (i.e., a sequence of symbols) acceptable.
In general, the term “word” is used instead of “process” because a sequence of symbols
(i.e., a word) corresponds to a process. There can be a large or even infinite number of
acceptable words. The set of all acceptable words is called the language recognized by
the automaton. Hence, an automaton can be also regarded as a recognizer of languages.
Automata theory includes the study of what kind of languages can be recognized by

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:3

automata, characterization of the languages recognizable by the automata, and how to

extend automata in order to recognize more complicated languages.
One of the reasons why we select automata theory from the topics in CSE is that
we use gamification and also puzzle-based learning to create the sense of a “eureka
moment.” It is then relatively easy to implement the learning contents as puzzle games
[Thomas et al. 2013; Falkner et al. 2009]. In general, discrete mathematics has a high
affinity with puzzles. Another reason is that it is one of the most important CSE topics
that should be learned (see, e.g., Holcombe [1981]). Automata theory forms the basis
for many theoretical computer science topics such as algorithms, programming, formal
languages, computational complexity, model theory, logic circuits, and coding theory. In
addition, automata theory is actually the foundation of many technologies that enhance
our lives, such as the World Wide Web [Schwentick 2007], natural language processing
[Venkateshan et al. 2013], and bioinformatics [Roy and Aluru 2016].
The purpose of this research is to clarify the following questions:
(C1). Is it feasible to teach children in primary and lower secondary schools the concept
of automata?
(C2). To what extent may children understand the concept of automata?
(C3). How many children engage in automata theory as implemented in puzzle games?

1.3. Experimental and Data Analysis Methods

Although automata theory includes the concepts of “nondeterminism” and “regular
expression,” we do not cover these topics; that is, in our research, we focused only on
the concept of a “deterministic finite automaton.” For this research, we created a puzzle
game based on automata theory. Then, we had 90 primary and lower secondary school
children from 36 of Japan’s elementary schools play our puzzle game for 60 minutes.
Fifteen were 4th graders, forty-one were 5th graders, and thirty-four were 6th graders
(aged 9–12 years). The game consists of 112 puzzles with state transition diagrams that
we call questions. Out of 112 questions, only 77 questions were used for data analysis;
the first 35 were used for practice (see Section 3.3). It is worth mentioning that our
puzzle game is designed in such a way that it is difficult to solve its questions without
understanding the fundamental concepts of automata, such as “accept,” “reject,” and
“determinism.” The game outputs a history of each player’s actions, which we refer to as
the play log. We analyze the children’s correct-answer rate and play logs to gauge their
understanding. To clarify questions (C1) and (C2), we analyze not only the children’s
correct-answer rate (to the questions) but also their process, which is available from
the play logs. We also analyze questionnaires given to the children after playing the
game. In our analysis of correct-answer rates, we use a 2-means clustering method to
classify children according to their levels: On the basis of these analysis results, we
discuss question (C1) in Section 5. On the basis of the analysis results of the children’s
comprehension levels, we discuss question (C2) in Section 6. For question (C3), in
Section 7, we analyze the questionnaire to assess their satisfaction. We also measure
the play time to evaluate how much time they devoted to the game.

2. TERMINOLOGY
A deterministic finite automaton is a five tuple (Q, , δ, q0 , F), where Q is a finite set
of states,  is a finite set of symbols ( is called the alphabet), δ : Q ×  → Q is the
transition function, q0 is the start state, and F is a set of accept states. The automata
considered in this article are all deterministic, and transition functions are allowed to
be partial functions. An automaton can be represented graphically by a state transition
diagram, which is a directed graph with vertices and edges corresponding to the states
and transitions in the automaton, respectively (see Figure 1).

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:4 D. Isayama et al.

Fig. 1. An example of a state transition diagram.

Fig. 2. The four labels and the empty label used in the game.

Fig. 3. An example of unlabeled automaton and its corresponding labeled automaton.

We refer to a directed edge in a state transition diagram as an arrow. We call an

assignment of symbol to an arrow labeling, and a specified symbol is referred to as the
label of the arrow. Since a transition function may be a partial function, some of the
states may have leaving arrows of less than ||. The terms “automaton” and “state
transition diagram” are used interchangeably in this article.
Let S be a set of sequences that are classified into two types: acceptable and un-
acceptable. An automaton A recognizes S if A accepts the acceptable sequences in S
and rejects the unacceptable sequences in S. The automata that appear in our game
have four symbols: red leaf (R), blue dew (B), yellow gem (Y), and green slime (G) (see
Figure 2).
The unlabeled automaton U is an automaton in which all labels are empty labels.
That is, U is essentially the same as a multiple directed edge graph with start and
accept states. For an appropriate set S of acceptable and unacceptable sequences, an
automaton A recognizing S can be determined from U by assigning proper symbols to
the labels of arrows in U . For instance, consider the unlabeled automaton U illustrated
in Figure 3. By taking the set S = {“YG”, “BR”, “BBG”} of acceptable sequences, the
automaton A in Figure 3 that recognizes S can be determined from U and S. We call
this a labeled automaton (of U ). The labeling question used in our research is detailed
next.

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:5

Labeling question. Given an unlabeled automaton U and a set S of acceptable and

unacceptable sequences, obtain the labeled automaton A that recognizes S.
Let A be either a labeled deterministic automaton or an unlabeled automaton. A
path in A is a path from one vertex to another when regarding A as a directed graph
in which the vertices correspond to the states. A path from the start state to an accept
state in A is called an accepting path in A. Each sequence accepted by A corresponds
one-to-one with an accepting path. A path P is a Unique Accepting Path (UAP) if P is
an accepting path and there is no other accepting path of the same length as P. For
example, the labeled automaton (as well as the unlabeled automaton) in Figure 3 has
three accepting paths corresponding to the three sequences “YG”, “BR”, and “BBG”, and
only the path corresponding to “BGG” can be a UAP. An arrow is entering (respectively,
leaving) a state if the arrow is an incoming (respectively, outgoing) edge of the state. An
arrow is terminal if it is entering a finite state. For instance, the automata in Figure 3
have two terminal arrows. To children, the labeling questions may look like just a logic
puzzle that enables them to understand (after a number of repetitions) roughly but
implicitly what an automaton is.
3. DESIGN
3.1. Question Design
Recall that “questions” here are the puzzles with state transition diagrams (see Sec-
tion 1.3). As mentioned in Section 1.3, to clarify (C1) and (C2), we give questions on
automaton to the children and analyze their answers and play logs. Here, we discuss
what types of questions are suitable for children and for analysis. Since automata have
two aspects (generator and acceptor), it would be natural to consider the following two
types of questions:
Construction question. Given a set S of acceptable and unacceptable sequences,
construct an automaton that recognizes S.
Recognition question. Given a deterministic automaton A and a sequence s, deter-
mine if A accepts s.
However, in this research, we employ the labeling question detailed in Section 2
rather than the construction question detailed here. We do this for two reasons. First,
the construction question creates obstacles for analysis. We would not be able to pro-
vide many acceptable and unacceptable sequences to the children (i.e., players) because
doing so would make the question seem more difficult: However, when a labeling ques-
tion with only a few sequences is used, there may exist two or more automata that
correctly recognize the given sequences. Furthermore, the problem is how to describe
the automaton constructed as the answer, and it is quite difficult to check the cor-
rectness of the constructed automaton (although some good tools and techniques have
been provided and are known [Alur et al. 2013; Rodger et al. 2007]). Second, it would
be hard to construct an automaton from scratch for children who are not familiar with
automata theory at all, wherea, it is easy to assign the labels to the arrows in an
unlabeled automaton.
Let A be an automaton and {r1 , . . . , rn} a set of input sequences. For each recognition
question with A and ri , most children would be able to solve the question correctly
because the correct answer can be obtained by just tracing the states according to
ri . Therefore, just because the children can answer the questions correctly does not
mean they really understand automaton A. To really understand automaton A, it is
very important to figure out the set of sequences that A accepts and to understand the
characteristic features of the set. Since the set and automaton are essentially the same,
the features of the set are also called the features of the automaton. From recognition

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:6 D. Isayama et al.

questions, we study how well the children actually understand the concept of automata.
See Section 6.2 for details.
The questions (automata and input sequences) used in our game are illustrated in
Appendices A and B.

3.2. Game Design

There are two obstacles to teaching children automata, and, for the obstacles, we
employ puzzle-based learning and gamification.
One is that we cannot use the formal definition given in Section 2 because the formal
definition is premised on knowledge of set theory and functions, such as the set of states
Q and transit function δ, which the children, of course, have not studied. To overcome
this obstacle, we employ puzzle-based learning because puzzles do not require any
specific knowledge [Badger et al. 2012]. We designed the game as a puzzle to enable
children to experience the fundamental concept of automata without formal definitions.
The other obstacle is that it is difficult for children to concentrate on learning for
60 minutes. To overcome this obstacle, we employ gamification. Generally, it is known
that gamification increases children’s engagement [Ott and Tavella 2009; Lee and
Hammer 2011]. We designed the game so that children would more easily become
curious and experience a sense of accomplishment; answering questions in the game
gives children a sense of accomplishment, while the animation used in the game arouses
their curiosity (see Section 8).
In our game, an automaton is considered a machine that produces items. The player’s
aim is to collect the (good) items. To produce an item with a machine, some materials (a
multiset) are required, and the materials are given to the machine in a specific order.
A sequence of materials is called a recipe; thus, a recipe can be thought of as an input
sequence to the machine (automaton).
In the machine, a robot transits the states from a start state according to a given
recipe. More precisely, the robot reads materials one-by-one in the order of a given
recipe. As each material from the recipe is read, the robot passes through the arrow
that has the read material as if its label exists. In this case, the robot moves to the
other state if the arrow is not a self-loop; otherwise, the robot stays in the current state.
If there is no arrow whose label is the same as the read material, the poor robot gets
stuck and explodes. Namely, the robot collects the materials by transiting the state
according to the given recipe. After reading all materials in the recipe, the machine
produces an item only if the robot stands on a specific state (the accept state).
A recipe by which the machine can produce an item is referred to as active (to the
machine), and a recipe that does not produce an item is called inactive. There are two
types of recipes: good and bad. A good recipe yields a good item and a bad recipe yields
a bad item. The good and bad recipes can be regarded as acceptable and unacceptable
sequences, respectively. See Figure 4.
In our game, there is a preventer who wants to use the machine to produce bad
items. To do so, the preventer conspires to mix bad recipes with good recipes. Hence,
the machine should be designed so that good items are produced, not bad items.
The game has two modes: a labeling mode and a recognition mode. In the labeling
mode, an incomplete machine U and a set S of recipes are given, where U is just
an unlabeled automaton and S is a set of acceptable and unacceptable sequences in
automaton words. In the mode, it is indicated which recipe is good or bad. To produce
items, the incomplete machine (i.e., the unlabeled automaton) U should be transformed
into a complete machine (i.e., a labeled automaton) Aby giving labels to arrows properly
so that A produces only good items (i.e., A recognizes S). Hence, game players who want
to collect the good items are expected to try to find the proper labeling.

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:7

Fig. 4. Examples of a good recipe(left) and a bad recipe(right).

Table I. Structure of Levels

Practice Stage Challenging Stage Extra Stage
lv.1 ··· lv.5 lv.6 ··· lv.9 lv.10 lv.11 lv.12
lab. mode rec. mode ··· lab. mode rec. mode ··· ···
L11 , L12 , L13 R11 , . . . , R14 ··· L61 , L62 , L63 R61 , . . . , R68 ··· ···

In recognition mode, a machine (i.e., automaton) A and a set S of recipes are given.
The set S may contain inactive recipes, and, in this mode, it is not indicated which recipe
is active or inactive. Since inactive recipes are unnecessary recipes (and explode with
the robot), they should be separated from active recipes. This separation corresponds
to the recognition question in automata theory. Thus, game players are expected to try
to distinguish between active and inactive recipes.
In what follows, we use the terms “sequence” and “recipe,” and “symbol” and “mate-
rial,” interchangeably.
3.3. Levels Design
From an educational point of view, the game is designed to start with an easy level and
progress to more difficult ones. We provided 12 levels that are divided into three stages:
The practice stage has five levels, the challenging stage has four levels, and the extra
stage has three levels. The practice stage is designed to familiarize children with the
concept of automata. The challenging stage is used to estimate how well children grasp
the concept after practice. The extra stage is provided for children who have passed
every practice and challenging level earlier than expected. As mentioned in Section 3.2,
each level has two modes: labeling and recognition. The labeling mode consists of
labeling questions, and the recognition mode consists of recognition questions. Every
labeling mode in level 1 ≤ i ≤ 12 has three labeling questions Lij 1 ≤ j ≤ 3.
The automata in the last few questions of the challenging stage and all of the ques-
tions of the extra stage—namely, Li3 6 ≤ i ≤ 12—come from the famous automata theory
textbook [Hopcroft and Ullman 2000]. Each recognition mode in level i (1 ≤ i ≤ 5) of
the practice stage has four recognition questions Rij , 1 ≤ j ≤ 4, and each recognition
mode in level i (6 ≤ i ≤ 12) of the challenging and extra stages has eight recognition
questions Rij , 1 ≤ j ≤ 8 (see Table I and Figure 5).
We give the children tips (we call it walkthrough; see Section 3.5) on how to solve the
labeling questions after the practice stage; namely, after the children are getting used
to the game.

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:8 D. Isayama et al.

Fig. 5. Title screen.

Fig. 6. An example of a labeling question. Fig. 7. An example of a recognition question (R66 ).

3.4. Examples of Labeling and Recognition Questions

As explained in Section 2 in automaton words and in Section 3.2 in game words, a
labeling question is, given an unlabeled automaton U and a set S of recipes, to obtain
the labeled deterministic automaton A from U by labeling properly so that A recognizes
S. For example, let us consider the unlabeled automaton depicted in Figure 6 and
three good recipes “YG”, “BR”, and “BBG”. In this labeling question, the labels can be
determined uniquely from the three good recipes.
A recognition question is, given a deterministic automaton A and a recipe, to deter-
mine whether the recipe is active or inactive to A; that is, to determine if A accepts
or rejects the recipe. For example, let us consider the automaton in Figure 7 and the
recipe “RYRBRGRBRYR.” In this recognition question, the automaton cannot accept
the recipe since the recipe has material of “G”. In game words, since an inactive recipe
explodes the robot, the inactive recipe is not good for the players.

3.5. Tips Design

Needless to say, it is important to give children tips on how to solve the questions. Before
starting the challenging stage (and after playing the practice stage), we provide the
following tips: Path Matching (PM), Label Matching (LM), and Deterministic Property
(DP). We call the tips walkthroughs because children play the game rather than learn

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:9

Fig. 8. The unlabeled automaton in L92 . Fig. 9. The automaton in R83 .

automata. The walkthroughs are easy for children to understand, reflect the concepts
of automata, and are useful for solving the labeling and recognition questions. It should
be emphasized that understanding the walkthroughs has important implications for
understanding the concepts of automata.

PM. If there exist a UAP P and a good recipe r such that the length of P equals
the length of r, then the labels of P can be determined by r. This walkthrough
can be used in solving labeling questions. We illustrate this tip with the labeling
question shown in Figure 6. For the third recipe, BBG, to be accepted, there must
be an accepting path with a length of three. Indeed, there is exactly one such path.
As stated in Section 2, we call such path a UAP. By using the UAP, the labels on
the path can be determined. We call this tip the path matching walkthrough or
simply PM.
LM. If there is a good recipe ending with material M, then there must exist a terminal
arrow with label M. This walkthrough can be used in solving labeling questions.
For example, let us consider the labeling question L92 illustrated in Figure 8. The
automaton in L92 has the unique terminal arrow, which has label 1. Furthermore,
the first recipe is a good recipe, and the last material of the recipe is “R.” Thus,
we can identify label 1 as “R.” We call this tip the label matching walkthrough or
simply LM.
The contraposition of LM can be described as “If there is no terminal arrow with
a label M, then there is no active recipe ending with M.” The contraposition can
be used in solving recognition questions. For example, consider the recognition
question shown in Figure 9. The recipe in R83 ends with “B”, but all terminal
arrows have material “R” as their labels. Hence, we can determine that the
automaton cannot accept the recipe. We call this tip the contrapositive label
matching walkthrough or simply Contrapositive Label Matching CLM. It should
be mentioned that we sufficiently explained LM to the children but purposely did
not mention CLM.
DP. Since the automata in our game are deterministic, no state is allowed to have
two or more leaving arrows with the same label. Duplicate labeling is labeling in
which a material is assigned to two or more arrows leaving the same state. This
tip is sometimes useful for narrowing down possible labels. For example, consider
the labeling question shown in Figure 10. If a player clearly recognizes this tip, L83
can be solved straightforwardly: First, we identify label 0 and label 3 as “R” from
the first and second recipes, respectively, by applying PM; then the other labels are

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:10 D. Isayama et al.

Fig. 10. The unlabeled automaton in L83 .

fixed by considering DP. Note that || = 4 and any state has four leaving arrows
in L83 . We call this tip the deterministic property walkthrough or simply DP.
It should be noted that PM and LM are useful for finalizing some labels at the early
stages. Table II shows which walkthroughs can be used with which questions. In Ta-
ble II, S is the number of states and A is the number of arrows.
We gave the children a leaflet in which walkthroughs are illustrated through exam-
ples with figures. With the leaflet, we explained the tips using a slide presentation
of about 10 minutes. More specifically, they are succinctly written in imperative form
(because of the typical style of walkthroughs) as follows:
(PM). Utilize the information on the lengths of recipes! “Length of recipe versus the
number of steps required to reach a goal.”
(LM). Focus attention on materials of arrows entering into the goal! “The last material
in a recipe = the material of an arrow entering into the goal.”
(DP). Master the techniques of blocking the preventer from reaching the goal! “Getting
stuck, Diverting roots, and Garbage box.”

4. IMPLEMENTATION
4.1. Labeling Questions
The labeling questions are implemented as follows. An unlabeled automaton in a level
and a set of recipes appear on the game screen when players start the first labeling
question in the level. Initially, each arrow in the unlabeled automaton has an empty
label (see Figure 11). If the empty labels are clicked once, they change to label “R.”
Each further click then changes the label again so that it rotates through “R”, “B”, “Y”,
and “G” in that order. When the first labeling question starts, players should make a
labeled automaton by changing each empty label into “R”, “B”, “Y”, or “G”. Remember,
the purpose of the labeling question is for the labeled automaton to recognize the set
of recipes. Players may check whether each recipe of the set is accepted or rejected by
the labeled automaton with the current labeling by clicking the check button for the
recipe. After completing the labeled automaton and checking its acceptance of each
recipe, players may submit their final answers by clicking the start button, at which
point the labeled automaton is checked for whether it recognizes the set of recipes. If
the labeled automaton recognizes the set of recipe, the “NEXT” button appears, which
then begins the next labeling question, and so on (see Figure 11).

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:11

Table II. Correspondence Table between Questions and Walkthroughs

question PM LM DP S A
L61 1 1 1 1
L62 1 1 1 3
L63 1 1 2 4

L71 1 1 1 1
L72 2 3
L73 3 4

L81 1 1 2 2
L82 2 4
L83 1 1 2 8

L91 1 1 3 3
L92 1 1 3 4
L93 1 3 6

L10
1 1 1 2 2
L10
2 1 3 4
L10
3 1 1 1 3 5

L11
1 1 1 3 4
L11
2 1 1 4 6
L11
3 1 1 4 8

L12
1 1 1 4 5
L12
2 1 1 4 7
L12
3 1 4 8

Fig. 11. Screenshot of a labeling question: recipes and start and check buttons.

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:12 D. Isayama et al.

Fig. 12. Screenshot of a recognition question.

4.2. Recognition Questions

The recognition questions are implemented as follows. An automaton in a level appears
on the game screen as the child prepares to play the first recognition question in the
level. The first recipe, however, is not displayed on the screen yet (see the left side of
Figure 12). Note that the same automaton is used in every recognition question in the
level. Players may attempt the recognition question any time they like by clicking the
door button, which causes the recipe to be displayed (see the right side of Figure 12).
Players decide whether the recipe is accepted or rejected by the automaton within a
certain time frame (one minute, in our study). The acceptance or rejection is indicated
by clicking the “OK” button or the “NG” button, respectively. After clicking OK or NG,
the correct answer is displayed and the “NEXT” button appears. Note that, at this point,
the recipe is still displayed. Once the NEXT button is clicked, the recipe disappears
and the next recognition question begins. By clicking the door button, the second recipe
is displayed, and players determine whether the new recipe is accepted or rejected by
the same automaton, and so forth.

4.3. Play Log

The play logs consist of the lowest level data, which we call the log record (or, simply,
record). Most records are generated when players complete actions, such as a button
click. Since the players are children, the only mouse action allowed in our game is the
click operation. In other words, the user interface of our game is designed not to require
double clicks, drag, drops, or keyboard actions. The play logs are designed so that the
players’ actions can be traced in the logs. Hence, log records consist of information
regarding when and which buttons are selected and the values set for those buttons at
those times.
An example log record is shown in Figure 13. The example log record shows the
actions of a player solving the labeling question depicted in Figure 14. The first line
shows that the third question (in a level) has begun. Lines 2–17 indicate that labels 0,
3, 2, 1, and 4 are set to “B”, “B”, “R”, “Y”, and “G”, respectively. The 18th line indicates
that the start button has been clicked in the following setting: labels 0, 1, 2, 3, and 4
are set to “B”, “Y”, “R”, “B”, and “G”, respectively. That is, the current labeling has been
submitted. That line also tells us that the first and second recipes are accepted, but
the third and fourth recipes are rejected by the automaton with the current labeling.
Lines 19–21 show that the first, second, and third recipes are checked, respectively,
by clicking the check buttons. Probably, by this time, the player has realized that
the current automaton cannot accept the third recipe. Lines 22–32 tell us that the
player sets label 1 and label 4 to “G” and “Y”, respectively, then checks if the third and

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:13

Fig. 13. An example of log record in the play log for a labeling question.

Fig. 14. Labeling question that corresponds to the play log in Figure 13.

fourth recipes can be accepted; finally, the corrected labeling is submitted. Lines 33–43
indicate that the third question is cleared and that it took 63.91 seconds. From this
analysis, we can see that the player begins fixing labels on the UAP of the automaton
depicted in Figure 14. Therefore, we may conclude that the player understands the
concept of UAP.

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:14 D. Isayama et al.

5. ANALYSIS BASED ON CLUSTERING

To answer question (C1), we performed a cluster analysis of the children. In clustering
children, we performed 2-means clustering for the data obtained from the play logs,
and we had three clusters Cluster 1, Cluster 2, and Cluster 3 that corresponded to high,
average, and low performers, respectively (see details later). After explaining how we
performed our cluster analysis of children, we discuss question (C1).
A brute-force attack can be thought of as an extreme form of case analysis that can
be executed without insight. Seen in this light, we can say that children essentially use
case analysis reasoning with or without awareness when solving our puzzle game. For
example, there are several approaches to solving the labeling question illustrated in
Figure 6. If we focus on the upper-right label as a first step, or we try to apply LM, we
have two cases: One of the two right labels must be “G” and the other “R”. Assuming the
upper-right label is “G”, we can easily determine the other labels. Conversely, assuming
the label is “R”, we realize that the recipe “B,B,G” will not be accepted. Alternatively, if
we focus on the center label as a first step or try to apply PM, we immediately realize
that the center label is “B”. We can determine the other labels in a straightforward
manner. The time to solve the puzzle game depends on the approach; that is, more time
is required if more cases must be considered. Furthermore, in the labeling questions,
by performing many checks with the start button, children might solve the puzzle
game without much thought. This simple observation indicates that when classifying
children by their performance, not only the correct answer rate but also the time to
obtain the correct answer and number of trials should be taken into account. We hence
adopted a k-means clustering method, which is a standard method for classification.
Our cluster analysis is based on these perspectives.

5.1. Player Clustering

To cluster players (i.e., children) into groups based on their performance in answering
labeling and recognition questions (based on their play logs), a matrix was generated
for each player for both tasks.
For these matrices, we only considered questions from levels 6 through 12; thus,
we had 21 questions for the labeling task and 56 questions for the recognition task.
In total, we had 90 (the number of players) 21 × 3 matrices for labeling and 56 × 3
matrices for recognition. From these matrices, we generated one feature matrix that
incorporated each labeling and recognition task by performing:

—For each player i and for each question j, check the following conditions:
(1) Is the total number of trials of player i exactly equal to 1?
If yes, then set matrix entry = 1, else 0.
(2) Did the player get the correct answer at least once in every trial?
If yes, then set matrix entry = 1, else 0.
(3) Let T j = the set of total times of all players for answering question j, and T j∗ =
median(T j ).
Is the total time of player i < T j∗ ? If yes, then set matrix entry = 1, else 0.

Using these conditions, we could generate the feature matrix in Figure 15. Then, from
this feature matrix, we could compute the score si for each player i as the sum of the
entries of row i (see Figure 15). To distinguish between players who performed better
than the rest, we performed k-means clustering, where k (the number of clusters) is set
to 2. We then have the following algorithm for 2-means clustering.
The algorithm outputs two clusters: One with high-scoring players and the other
with the (relatively) low-scoring players. We interpreted these as players who were

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:15

Fig. 15. Scoring based on the feature matrix.

good at answering the specific task (labeling or recognition) and those who were not.
We denoted their clusters as follows:
CL1. Good in labeling
CL2. Not good in labeling
CR1. Good in recognition
CR2. Not good in recognition
Last, to integrate every cluster for the two tasks, we defined the final three clusters
as follows:
Cluster 1 = CL1 ∩ CR1. Good in both labeling and recognition tasks
Cluster 2 = ¬(Cluster 1 ∪ Cluster 3). Good in either labeling or recognition, but not
so good in the other
Cluster 3 = CL2 ∩ CR2. Not so good in both labeling and recognition tasks
5.2. Clustering Results
By performing 2-means clustering, we clustered 90 children into three groups: 37 high
performers in Cluster 1, 18 average performers in Cluster 2, and 35 low performers
in Cluster 3. For all labeling questions L61 , . . . , L93 in the challenging stage, the rate
of correct answers was 70% or more for all clusters. For all recognition questions
R61 , . . . , R98 in the challenging stage (except Cluster 3) most correct answer rates were
70% or more.
By combining the labeling and recognition data, approximately 60% of the children
had correct answer rates of 70% or more. Somewhat unexpectedly, there is no great dis-
tinction between the performances of children in the 4th and 6th grades. As mentioned
in Section 1.3, our game is designed so that understanding the fundamental concept
of automata is essential to clear the levels. On this basis, the results suggest that it
is feasible to introduce the fundamental concept of automata to primary and lower
secondary school students through gamification. In this sense, our answer to (C1) is
YES; that is, we may say that it is feasible to teach children the concept of automata.
One question remains, however: Is it possible to make children understand the concept
of automata more deeply? In this light, question (C1) will be controversial. We discuss
it in Section 8.
6. ANALYSIS BASED ON PLAY LOGS
In this section, to consider our question (C2), we first analyze play logs of the labeling
questions and then play logs of the recognition questions.
6.1. Analysis of Labeling Questions
As stated in Section 3.5, understanding the walkthroughs we provide children (PM,
LM, and DP) is very important in order to form the concept of automata. In analyzing
labeling questions, we simply equate those children who use the walkthroughs well
enough to solve questions with those who have a good understanding of the concept
of automata. That is, in order to answer question (C2), we analyze how well children
understand the walkthroughs. To do so, we use their play logs.
ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:16 D. Isayama et al.

Fig. 16. Unlabeled automaton in L93 that has the Fig. 17. Unlabeled automaton in L10 3 that has the
UAP corresponding to the first recipe. UAP corresponding to the first recipe.

6.1.1. Analysis of PM. First, we explain how to analyze the play log for PM automati-
cally. Again, consider the log records shown in Figure 13. As shown in Section 4.3, by
analyzing the log manually, we see that the player of the log recognizes the automa-
ton’s UAP, depicted in Figure 14. The following procedure details how to approach the
analysis automatically. The log records can be simplistically described as
0 3 2 1 4 1 4
            
RB RB R RBYGRBY RBYG c Ac1 c2 c3 G RBYGRBY c3 c4 c A,
where c A means that the current automaton has been submitted by the player’s clicking
the start button, and ci represents that the ith recipe has been checked for acceptance
by the current automaton by the player’s clicking the check button. When we delete
information regarding button clicks and material from the simplified description, we
are left with the label ID sequence “0321414”. Then, by picking up the rightmost label
IDs (the letters with over-dot) from sequence 0̇3̇2̇141̇4̇, we have the label ID order
“03214”, which represents the order in which the labels were finalized. Hence, we refer
to this as the finalized order. In contrast, by picking up the leftmost label IDs from
sequence 0̇3̇2̇1̇4̇14, we have label ID order “03214”, which represents the order in which
the labels were set out. Thus, we call this the set out order. As we have just seen, a
finalized order may coincide with a set out order. Note that the number of labels in the
automaton equal the length of the finalized order and set out order. Also notice that
the UAP corresponding to the first recipe “BB” in the automaton shown in Figure 14
has 0 and 3 as their label IDs. Since the first two label IDs in the order “03214” are
0 and 3, we may conclude that the player recognizes the usefulness of the UAPs. By
implementing this procedure into a script code, we can do this analysis automatically.
To see how well the children recognize the PM, we analyzed the play logs of labeling
questions L93 and L10 3 (see Figures 16 and 17), which have easily detectable UAPs
corresponding to their first recipes (i.e., (label 0, label 1) for L93 and (label 0, label 2), up
to symmetry, for L10 3 ). Note that labels 2, 3, and 4 are transitions from the same state to
the same accept state. We found that 61.0% of the children (50/82) recognized the UAP
for L93 and 83.3% of the children (50/60) recognized it for L10 3 . We also analyzed the play
logs of L11
3 . L11
3 has two accepting paths (i.e., (label 0, label 1) and (label 2, label 3)) and
two good recipes (“RR” and “BB”). Since there is a one-to-one correspondence between
those recipes and the accepting paths, the PM walkthrough is applicable to L11 3 , even
if it is hard to notice (see Figure 18). For L11 3 , 62.2% of the children (28/45) recognized
the PM.

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:17

Fig. 18. Automaton in L11

3 that has the UAPs corresponding the first and second recipes.

Fig. 19. Automaton in L10

2 .

6.1.2. Analysis of LM. Essentially, a method similar to analysis of PM can be applied

to analysis of LM because UAP labels and labels of arrows entering an accept state
can both be finalized in an earlier time. For instance, consider the automaton in L92 as
illustrated in Figure 8. The arrow with label 1 is a unique arrow that goes to an accept
state, and there is a good recipe ending in “R”. Therefore, it is easy to notice that label
1 is “R”. We can fairly say that the label should be correctly determined at the very
beginning if the player is aware of the usefulness of LM. Thus, similar to PM analysis,
we can automatically analyze how many players are aware of the usefulness of LM
from the play logs.
For LM analysis, we analyzed the play logs of labeling questions L92 and L10 2 because
9 10
applying LM to those questions is easy. The automata in L2 and L2 are depicted in
Figures 8 and 19, respectively. Contrary to our expectation, only 6.3% of the children
(5/80) applied LM to L92 , and 5.0% (3/60) did so to L10
2 . This result was quite unexpected
because we provided sufficient LM explanation to players after the practice stage. The
play log also suggests that the children tend to set labels starting from labels leaving
the start state (or from the left side).
6.1.3. Analysis of DP. Since we deduced that children who understand the walk-
throughs can avoid making duplicate labeling, we counted how many of them made
duplicate labeling in order to analyze how well they recognize DP. This can be easily

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:18 D. Isayama et al.

computed from the play log because any duplicate labeling is recorded in the log when
a check button is clicked or when an automaton with duplicate labeling is submitted
by the player clicking the start button. If a player was not aware of how to use DP, she
would make a duplicate labeling, which can be detected and recorded automatically.
We analyzed L83 (shown in Figure 10) as follows. First, label 0 was determined to
be “R” by applying the PM walkthrough. Then, labels 2, 6, and 7 were determined to
be “G”, “B”, and “Y” by applying the DP walkthrough. Labels 1, 3, 4, and 5 could be
determined similarly. Hence, we first counted how many children applied PM to the
UAP (i.e., label 0 and label 3); next, we counted how many of them made duplicate
labels; we found that 83 children tried to solve L83 , 47 children applied PM, and 27
made duplicate labels in L83 . This indicates that, in L83 , at most 24% (i.e., 20/83) of the
children could use DP properly.
We also analyzed L10 10
3 , which is shown in Figure 17. To solve L3 , DP is implicitly used.
One possible approach is to apply PM essentially (i.e., we stretch PM) to determine
labels 0 and 2 from the first recipe; labels 1, 3, and 4 can be finalized by considering DP.
By consideration of the second recipe, note that labels 3 and 4 are not “B”. Therefore, we
counted the number of children who first fixed the labels on the UAP correctly but then
made a duplicate label in L10 10
3 . Sixty children tried to solve L3 and 50 of them applied
PM properly; 19 of them, however, made duplicate labels in L10 8
3 . The analysis of L3 and
10
L3 tells us that about 70% of the children were not aware of the usefulness of DP.

6.2. Analysis of Recognition Questions

An automaton has its own features by which we can distinguish acceptance and re-
jection recipes (see Section 6.2.1). If children recognize the concept of automata, they
also recognize the features of an automaton. To answer question (C2), we analyzed the
play logs of recognition questions to see how well the children recognized the concept of
automata; we also analyzed the CLM walkthrough using recognition questions because
the CLM walkthrough contains the tips applicable to recognition questions.

6.2.1. Analysis of the Children’s Realization. Recall first that, for each level i (6 ≤ i ≤ 12),
the recognition questions Rij (1 ≤ j ≤ 8) have a common automaton Ai (see Section 4.2).
As stated in Section 3.1, there are two possible ways to solve the recognition questions
Rij in a level i. One is to just trace the states one-by-one according to the recipe, but this
takes time if the recipe is long. The other is to figure out the features of the automaton
Ai , which does not take long if children understand the features of Ai . For example,
consider the recognition questions R6j in level 6 for 1 ≤ j ≤ 8. The feature of automaton
Ai is that Ai cannot accept recipes containing material “G” (see Figure 7). As mentioned
in Section 4.2, children had time to think about this feature before trying to solve each
question. By measuring the time between when the door button was clicked and either
the OK (accept) or NG (reject) button was clicked, we can know how long it took a player
to answer. If the time is relatively short, we may conclude that the child understood
the feature—that is, what recipes the automaton generates.
We analyzed the play logs of recognition questions with the following assumption:
Once children have realized the feature of Ai when solving Rik for some 1 ≤ k ≤ 7,
they should be able to correctly and quickly answer the remaining questions Rij for all
k < j as well. For a player and recognition questions Rij (1 ≤ j ≤ 8) in a level i, we
defined realization point k as the smallest k such that the player solved all recognition
questions Rij for k ≤ j correctly and quickly (i.e., in 5 seconds). In recognition questions,
every recipe was designed so that its length was about 10 materials (average= 10.9).
We expected it would take 0.5 seconds for children to trace single material in each
question. Therefore, we can say that 5 seconds is quick for arriving at an answer. On
ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:19

Table III. Counting Results for the Number of Children Whose Realization Point Was k
Level i 6 7 8 9
point k 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
count 5 6 5 1 6 11 8 4 0 8 7 5 3 14 1 2 1 0 2 0 8 1 0 0 1 1 3 3
Level i 10 11 12
point k 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
count 0 2 0 1 1 5 6 0 0 0 0 0 0 0 0 0 0 0 0 1 3

the basis of this assumption, we counted those children whose realization point was k
for each 1 ≤ k ≤ 7 and each level 6 ≤ i ≤ 12. The results are summarized in Table III.
Realization points with a large k would not be so reliable because all  questions can
be solved with 21 probability, even if a player chose  answers at random.
As can be seen from the automata Ai for 6 ≤ i ≤ 12, the recognition questions Rij —
especially R6j , R8j , R9j , and R12
j —can be solved very quickly if the player understands
the features of automata A , A8 , A9 , and A12 , respectively (see Appendix B). Indeed,
6

the sets accepted by the automata share a common feature: We can decide whether or
not a recipe is accepted by just looking at the last few materials in the recipe. Table III
indicates that some children realized the features of A6 and A8 at an early stage, but
most did not realize the features of A9 and A12 at an early stage because the features
of A9 and A12 are hard to figure out. Indeed, as mentioned in Section 3.3, automata A9
and A12 come from exercises in Hopcroft and Ullman [1979, 2000]. From these results,
we conclude that, even with relatively easy recognition questions in the challenging
stage, 80% or more of the children failed to figure out the automata features at early
stages.
6.2.2. Analysis of CLM. Recognition questions were used to analyze CLM. As stated in
Section 3.5, we purposefully did not mention CLM to the children because we wanted
to know how many of them would recognize CLM from LM without our giving any
explicit explanation of CLM. To determine this, we analyzed recognition questions R83 ,
R86 , R94 , and R96 . It can be reasonably assumed that the recognition questions can be
solved correctly and quickly (i.e., within 5 seconds) if the player understands CLM
well. Hence, we may regard a player who fails to answer one of the questions correctly
and quickly as a child who does not fully understand CLM. Our analysis found that
none of the children participating in the study understands CLM fully.

7. QUESTIONNAIRE RESULTS
To find answers to question (C3), children were debriefed after playing the game.
Children played the game with the goal of completing as many of the 112 questions as
possible within 60 minutes. As a result, they solved 74 (labeling + recognition) questions
on average in 60 minutes. We note that most of the children solved almost all the
questions in the practice and challenging stages, and played the game without a break.
Most of the electronic questionnaire used either a 4- (disagree, somewhat disagree,
somewhat agree, agree) or 3-point (disagree, neither agree nor disagree, agree) Likert
scale format. Questionnaire analysis shows that 98% (88/90) of the children said the
game was “enjoyable” or “rather enjoyable,” and 68% (61/90) and 38% (34/90) took
pleasure in the labeling and recognition questions, respectively. In the questionnaire,
exactly 50% (45/90) of the children said that the level of difficulty was “easy” or “rather
easy.” The questionnaire also asked how the characteristic of an automaton (such as the
number of states, arrows, and recipes, the length of recipe, and the presence or absence
of a bad recipe) affected the difficulty of the questions. Contrary to our expectation,
there was no particular correlation between automata features and question difficulty
based on our questionnaire analysis.
ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:20 D. Isayama et al.

Interestingly, despite our analysis (shown in Section 6.2), 37% (33/90) of the children
responded that they understood the features of automata in most recognition questions;
most of those children are in Cluster 1.
8. CONCLUSION
The purpose of this research is to clarify the following questions:
(C1). Is it feasible to teach children in primary and lower secondary schools the concept
of automata?
(C2). To what extent may children understand the concept of automata?
(C3). How many children engage in automata theory as implemented in puzzle games?
Early automata education is a feasible and meaningful approach to familiarizing
children with the concept of automata. Indeed, as shown in Section 5, approximately
60% of the children had correct answer rates of 70% or higher. Thus, many of the
children understood automata concepts well enough to complete the game stages. In
particular, to complete the game stages, they needed to understand “input sequence,”
“accept,” “reject,” “transition,” “state transition diagram,” and “accepting path,” which
are all fundamental automata concepts. For instance, the labeling question L310 (see
Appendix A) has both good and bad recipes. To obtain the correct answer, it is necessary
to be aware of the distinction between “accept” and “reject” and also to understand the
DP.
On the other hand, it would be difficult to make most children better understand
the concept. Although our analysis shows that some children in our study (not many)
are capable of better understanding the concept, most of them do not fully understand.
Indeed, not surprisingly, 80% or more failed to figure out the features of the automata in
the recognition questions. Understanding these features is difficult even for university
students, and 60 minutes is not sufficient for children to understand the concept well.
Conversely, it is surprising that nearly 20% of the children involved in the study
demonstrated an aptitude for deep understanding of the concept.
Because of our logic puzzle approach, most children were interested in the game.
Indeed, as stated in Section 7, 98% of the children said that the game was “enjoyable”
or “rather enjoyable” in the questionnaires. Most children continued to play the game
without rest during their break time. One possible reason for their enjoyment is that we
provided animation: When checking whether any given recipe was accepted or rejected,
a robot passed through an arrow according to that recipe. Almost half of the children
(44/90) specified the animation as an attractive point of the game. We conclude, then,
that early introduction to the fundamental concepts of automata theory is feasible
through puzzle-based and gamification methods.
In the early introduction of automata theory to children around the 4th–6th grades,
it is good if they experience automata theory through play without being aware that
they are studying. It is also nice if they unconsciously learn how to design automata or
discover automata features. We think that, through playing our game, the fundamen-
tal automata concepts are imprinted onto their subconscious minds. Our game also
includes the fun of finding solutions on their own, and these solutions are helpful for
designing and discovering the features of automata. The fun, however, easily turns into
stress if we set the game level difficulty too high or gave them hints (i.e., walkthroughs)
too late. In finding the appropriate level for teaching automata theory to these chil-
dren, the fundamental automata concepts “single accept state” and “deterministic” can
be naturally generalized to the more complex concepts of “multiple accept states” and
“nondeterminism,” respectively. The more complex concepts would be difficult to teach
to children, and it might be better to avoid doing so. In particular, it would be very
difficult to introduce children to the concept of nondeterminism because the concept is

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:21

essentially based on the concepts of universal and existential quantifiers, which are not
easy to understand, even for university students. In this sense, the restriction of deter-
minism in our game is very essential. From another perspective, taking, for example
automaton A9 depicted in Appendix B, it would be difficult for them to find out what
kind of recipes are accepted by the automaton (i.e., the features of the automaton), but
it is not difficult to understand it. For example, most would understand that the middle
state in A9 means that the last material read is “blue dew” and the material before
that material is “red leaf ” (or empty) whenever the robot stands on the middle state.
As a practical example, first, we let them play the puzzle game without giving tips on
how to solve it. Then, at the right moment, which depends on the situation, we give
them tips or explanations. We then let them play the puzzle game again.
When putting the early introduction to CSE into practice, metaphor becomes one
of the most powerful and useful techniques (see, e.g., Bell et al. [1998]; Forišek and
Steinová [2012] for a discussion of metaphor). The design of a metaphor goes hand-in-
hand with game design. Metaphor affects the design of a puzzle game, in particular, the
terminology metaphor plays a fundamental role in its design. We did not necessarily
create the design in the following order; actually, we went back and forth between
designing the game mission and puzzle game using a draft terminology metaphor.
Which concepts/terminologies can be translated into easy words? We avoided using
complicated terminology (such as one describing an abstract notion or technical terms)
for children without the prerequisite knowledge. Thus, we needed to translate termi-
nologies into easier words that naturally appear in the game we are designing. In our
case, for instance, “alphabet” and “transition function” are translated into “materials”
and “arrows with labels,” respectively. We refer to this translation as a terminology
metaphor. Using the terminology metaphor, we realized that it was not easy to trans-
late “transition function” perfectly. Thus, it is not reasonable to require the children to
fully understand what the transition function is.
Which learning tasks can be described using easy words? Game missions should be
derived from the learning tasks. A good terminology metaphor makes the design of
the game missions easy because the learning tasks are described using the terminolo-
gies. Given the implications mentioned earlier, the construction of an automaton from
scratch would be a difficult learning task for children. However, it would be possible
for them to make an incomplete automaton into a complete one. Indeed, the learning
task “Complete an unlabeled automaton U to be a labeled automaton A by defining
the transit function properly so that A recognizes S” can be translated into the game
mission “Make an incomplete machine U into a complete machine A by giving the
correct labels to the arrows so that A produces only good items.”
How can we make learning tasks into puzzle games? In puzzle game-based learning,
game missions naturally correspond to puzzles. In the design of puzzle games, it would
be good to design a game that has some characteristics of the many other popular puzzle
games. The labeling questions can be classified into the same category as crossword
puzzles, arithmetical restorations (or skeletons), minesweepers, sudokus, and jigsaw
puzzles. In this category, the goal of the puzzles is to uncover concealed information such
as numbers, letters, and pieces, and some uncovered information helps to uncover other
concealed information. We refer to this characteristic as the one-after-another property.
This design approach seems applicable to game designs in other CS topics that can
be modeled as graph labeling problems. Graph labeling has many applications, such as
coding theory, logic circuit design, and communication networks (see, e.g., Kuchipudi
et al. [2014]). Thus, as future research, it would be interesting to apply the design
approach to game designs for other CS topics. For instance, logic circuit design would
be a good topic. For example, it would be difficult for children to design logic circuits

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:22 D. Isayama et al.

from scratch. But we could instead provide incomplete circuits whose shapes (i.e.,
graphs) are predetermined but the AND/OR/NOT gates (i.e., labels) are unassigned.
The terminologies “AND/OR/NOT gates” would be rather unfamiliar to the children;
hence, a terminology metaphor would be needed. It would then be easy to embody
the incomplete circuit problem into a puzzle game with the one-after-another property.
Additionally, logic circuits can be thought of as recognizers on sets of sequences of {0, 1}.
Thus, we could also design recognition questions.

APPENDIX
A. LABELING QUESTIONS

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:23

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:24 D. Isayama et al.

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:25

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:26 D. Isayama et al.

B. RECOGNITION QUESTIONS

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
Computer Science Education for Primary and Lower Secondary School Students 2:27

ACKNOWLEDGMENTS
We thank the anonymous referees for their valuable comments and constructive suggestions which led to
improvements in this article.

REFERENCES
2010. Report of a Workshop on the Scope and Nature of Computational Thinking. Retrieved from http://
www8.nationalacademies.org/cp/projectview.aspx?key=48969.
Rajeev Alur, Loris D’Antoni, Sumit Gulwani, Dileep Kini, and Mahesh Viswanathan. 2013. Automated
grading of DFA constructions. In Proceedings of the 23rd International Joint Conference on Artificial
Intelligence (IJCAI’13). AAAI Press, 1976–1982.
Matthew Badger, Christopher J. Sangwin, Trevor O. Hawkes, R. P. Burn, John Mason, and Sue Pope. 2012.
Teaching Problem-Solving in Undergraduate Mathematics. Technical Report, Loughborough National
HE STEM Centre.
Timothy C. Bell, Ian H. Witten, and Mike Fellows. 1998. Computer Science Unplugged: Off-line activities
and games for all ages. http://csunplugged.org/changelog/.
ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.
2:28 D. Isayama et al.

Ernestine Bischof and Barbara Sabitzer. 2011. Computer science in primary schools? Not possible, but
necessary?! In Informatics in Schools. Contributing to 21st Century Education, Ivan Kalaš and
Roland T. Mittermeir (Eds.). Lecture Notes in Computer Science, Vol. 7013. Springer, Berlin, 94–105.
DOI:http://dx.doi.org/10.1007/978-3-642-24722-4_9
Neil C. C. Brown, Sue Sentance, Tom Crick, and Simon Humphreys. 2014. Restart: The resurgence of
computer science in UK schools. Transactions on Computer Education 14, 2, Article 9 (Jun 2014), 22
pages. DOI:http://dx.doi.org/10.1145/2602484
Institut de France Académie des Sciences. 2013. Teaching computer science in France - Tomorrow can’t wait.
(2013). http://www.academie-sciences.fr/activite/rapport/rads_0513gb.pdf.
Nickolas Falkner, Raja Sooriamurthi, and Zbigniew Michalewicz. 2009. Puzzle-based learning: The first ex-
periences. In Proceedings of the 20th Annual Conference for the Australasian Association for Engineering
Education: Engineering the Curriculum. 138.
Michal Forišek and Monika Steinová. 2012. Metaphors and analogies for teaching algorithms. In Proceedings
of the 43rd ACM Technical Symposium on Computer Science Education. ACM, 15–20.
Mohamed Hamada and Sayota Sato. 2010. Simulator and robot-based game for learning automata theory.
In Entertainment for Education. Digital Techniques and Systems, Xiaopeng Zhang, Shaochun Zhong,
Zhigeng Pan, Kevin Wong, and Ruwei Yun (Eds.). Lecture Notes in Computer Science, Vol. 6249. Springer
Berlin Heidelberg, 429–437. DOI:http://dx.doi.org/10.1007/978-3-642-14533-9_44
Werner Hartmann, Jürg Nievergelt, and Raimond Reichert. 2001. Kara, finite state machines, and the case
for programming as part of general education. In Proceedings of the IEEE Symposia on Human-Centric
Computing Languages and Environments, 2001. IEEE, 135–141.
M. Holcombe. 1981. The role of automata and machine theory in school and college mathematics syllabuses.
Educational Studies in Mathematics 12, 2 (1981), 219–234.
John E. Hopcroft and Jefferey D. Ullman. 1979. Introduction to Automata Theory, Languages, and Compu-
tation (1st ed.). Adison-Wesley Publishing Company, Reading, Massachusetts.
John E. Hopcroft and Jefferey D. Ullman. 2000. Introduction to Automata Theory, Languages, and Compu-
tation (2nd ed.). Adison-Wesley Publishing Company, Reading, Massachusetts.
Peter Hubwieser, Michal Armoni, Michail N. Giannakos, and Roland T. Mittermeir. 2014. Perspectives and
visions of computer science education in primary and secondary (k-12) schools. Transactions on Computer
Education 14, 2, Article 7 (Jun 2014), 9 pages. DOI:http://dx.doi.org/10.1145/2602482
Laura Korte, Stuart Anderson, Helen Pain, and Judith Good. 2007. Learning by game-building: A
novel approach to theoretical computer science education. SIGCSE Bulletin 39, 3 (jun 2007), 53–57.
DOI:http://dx.doi.org/10.1145/1269900.1268802
Sravanthi Kuchipudi, Lakshmi Prasanna Nissankara, and Nagalla Sudhakar. 2014. Applications of graph
labeling in major areas of computer science. International Journal of Research in Computer and Com-
munication Technology 3, 8 (2014), 819–823.
Joey J. Lee and Jessica Hammer. 2011. Gamification in education: What, how, why bother? Academic Ex-
change Quarterly 15, 2 (2011), 146.
Michela Ott and Mauro Tavella. 2009. A contribution to the understanding of what makes young students
genuinely engaged in computer-based learning tasks. Procedia-Social and Behavioral Sciences 1, 1
(2009), 184–188.
Susan H. Rodger, Jinghui Lim, and Stephen Reading. 2007. Increasing interaction and support in the formal
languages and automata theory course. In ACM SIGCSE Bulletin, Vol. 39. ACM, 58–62. DOI:http://dx.
doi.org/10.1145/1268784.1268803
Indranil Roy and Srinivas Aluru. 2016. Discovering motifs in biological sequences using the micron automata
processor. IEEE/ACM Transactions on Computational Biology and Bioinformatics 13, 1 (2016), 99–111.
Thomas Schwentick. 2007. Automata for XML?a survey. Journal of Computer System Science 73, 3 (2007),
289–315.
Allen P. Thomas, Linda B. Sherrell, and James B. Greer. 2006. Using software simulations to teach automata.
Journal of Computing Sciences in Colleges 21, 5 (2006), 170–176.
Colin Thomas, Matthew Badger, Chris Sangwin, and Esther Ventura-Medina. 2013. Puzzle-based learn-
ing of mathematics in STEM subjects. https://www.heacademy.ac.uk/resource/puzzle-based-learning-
mathematics-stem-subjects, April, 2013.
Kishore Venkateshan, Pranith R. Naik, and Krupalini N. Swamy. 2013. Switch state mechanism for digital
life assistant. In Proceedings of the 3rd International Conference on Intelligent Computational Systems-
ICICS. 59–62.
Jeannette M. Wing. 2006. Computational thinking. Communications of the ACM 49, 3 (2006), 33–35.

Received March 2015; revised March 2016; accepted May 2016

ACM Transactions on Computing Education, Vol. 17, No. 1, Article 2, Publication date: September 2016.