Procedural terrain generator for platform games using Markov chain

Gleidson Mendes Costa Tiago Bonini Borchartt

Vision and Image Processing Laboratory - VIPLab Vision and Image Processing Laboratory - VIPLab
Universidade Federal do Maranhão - UFMA Universidade Federal do Maranhão - UFMA
São Luis-MA, Brasil São Luis-MA, Brasil
[email protected] [email protected]

Abstract— The game design is a complex and huge task and need a training data instead of domain knowledge to
game companies spend much time to create a beautiful and generate a new map.
exquisite design for a video game. The terrain is the most The objective of this work is present a methodology for
extensive part of a game. Some games need to present procedural terrain generation for 2D platform games. The
different terrains on each level so that the game does not approach developed was based on the use of Markov
become repetitive to the players. Procedural Terrain models, specifically the hidden Markov chains, to produce
Generation is a technique that can streamline much of this new terrain probabilistically.
work using algorithms to create new terrain randomly. This The game designer can define the terrain behavior of the
work presents a methodology for procedural terrain
game by reporting a transition matrix, where each state is a
generation for 2D platform games. The approach developed
was based on the use of Markov models, specifically the
terrain element.
hidden Markov chains, to produce new terrain Finally, a filtering method was applied to verify the
probabilistically. The results show that the technique consistency and viability of the terrain generated. Therefore,
presented here is able to generate a 2D platform game terrain. such tool may aid the game designer during the game
development and improve the overall quality of the game.
Keywords- 2D platform game; procedural terrain In the next section, we present the fundamental concepts
generation; Markov models; hidden Markov chains to understand the developed methodology. In Section III the
PTG methodology developed are presented. Section IV
I. INTRODUCTION discusses the obtained results. The Section V shows the
conclusions and future works.
The game design is a complex and huge task, it requires
putting in a lot of manpower and material resources and
spend some time in order to create a beautiful and exquisite II. BACKGROUND
design of good games.
One of the most essential aspects of most video games A. 2D Platform Game
is the terrain that the player will traverse. Procedural Terrain Platform games originated in the early 1980s, with 3D
Generation (PTG) can streamline much of this work using successors popularized in the mid-1990s. The term
pseudo-random algorithms. The term PTG refers to the describes games where jumping on platforms is an integral
creation of content by an automated system, rather than
part of the gameplay and came into use after the genre had
being produced manually [1].
been established [7].
There are many of effective techniques currently that
can be applied to procedural game content generation. One of the most famous platform game is “Super Mario
However, the main algorithms are based on noise function. World” (SMW). This game was launched in 1990, and now
The Diamond-Square Algorithm proposed by [2] have is a classic and successful game. SMW is still used as a
fractal nature based on recursive subdivision. In [3], several template for many games titles of the same genre. It is a
noise functions are compared to generating terrain texture: platform-based game, where the main character must
Value Noise, Perlin Noise, Simplex Noise and Whorley follow a path picking items and destroying or deviating
Noise. The authors conclude that each technique present a enemies.
weak point, e.g., Simplex Noise is very difficult to The return of 2D games would be related to aspects of
implement in a game context, Value Noise are very simple, nostalgia or aesthetics. The 2D style is not necessarily a
but need a secondary interpolation function based on technical limitation, since just as hardware has evolved and
fractional Brownian motion to generate a good result. transformed the 3D gaming experience, these techniques
2D platform games are on the rise due to recent big hits may well be applied in two-dimensions [8].
in mobile operating systems, e.g., Angry Birds and Super According to [9], mobile games emerge as the favorite
Mario Run, each with more than 100 bi downloads [4][5]. of the public, with 43.6% choosing smartphones as best to
A platform game is a video game genre of action game play. In them, the games need to be very "light", which does
where the player controls a character or avatar to jump not mean that they must be simple, but that they cannot use
between suspended platforms and avoid obstacles. high realism to be accepted in the market. This combination
In [6], the authors presented a hierarchical approach to of factors enabled the return of 2D games, in particular
generating 2D platform video game maps using multi-
platform games.
dimensional Markov chains for learning, sampling and
clustering for finding high-level structures. Their approach
B. Procedural Terrain Generation (PTG) The evolution of the Markov process can be presented by
Player demand for more content is increasing as games a state transition graph that shows the structure of the
grow in complexity and scope. Traditionally developers process according to the following rules [14]:
have hand-crafted content for games, requiring a  States are represented by nodes that belong to the
substantial investment of time and resources. alphabet ‘S’ of states, S = {s1, s2, …, sn);
Procedural content generation methods are used to  The transitions (possibilities of passing from one
create content algorithmically instead of manually [10]. state to the other state) are represented by (directed)
PTG are specific techniques for the generation of terrain. edges, they are weighted by their probabilities. The
Virtual terrains have an important role in computer probabilities are grouped in a transition matrix ‘A’:
graphics, and their applications range from landscape 𝐴 = {𝑎𝑖,𝑗 = 𝑃(𝑠𝑖 |𝑠𝑗 )}, 𝑤ℎ𝑒𝑟𝑒 ∑𝑁 𝑗=1 𝑎𝑖,𝑗 = 1 (1)
design and flight simulators to movies and computer  The starting probabilities are the probabilities of
games. A terrain is the dominant visual element of the starting in one state or another. They are grouped
scene, or it plays a central part in the application. into a startup vector ‘∏’:
Procedural techniques are a popular choice in game ∏ = {𝜋𝑖 = 𝑃(𝑠𝑖 )}, 𝑤ℎ𝑒𝑟𝑒 ∑𝑁 𝑖=1 𝜋𝑖 = 1 (2)
development because of the simple implementation and
wide range of terrains they provide when a few parameters A model λ is observable because states are directly
are changed [11]. observable, and a transition matrix A and an initialization
The desire for automatic terrain generation stems from vector ∏ characterize the model, where:
the goal of provide a new content to player without a large 𝜆 = {∏, 𝐴} (3)
investment of developer resources.
In [12], the authors cite the properties of a PTG 2) Hidden Markov chain
technique: In a Hidden Markov Model (HMM) the states S = {s1,
 Novelty: contains an element of randomness and s2, …, sn), are not observable. However, they emit
unpredictability; observable signals O = {o1, o2, …, ot) that are weighted by
 Structure: is not merely random noise, but contains their probabilities.
larger structures; An HMM is characterized by Eq. (5), a transition matrix
 Interest: has a combination of randomness and A, an initialization vector ∏ and an emission matrix B,
structure that players find engaging; defined by:
 Speed: can be quickly generated; 𝐵 = {𝑏𝑖 (𝑜𝑘 ) = 𝑃(𝑜𝑘 |𝑠𝑖 )}, 𝑤ℎ𝑒𝑟𝑒 ∑𝑇𝑗=1 𝑏𝑖 (𝑜𝑗 ) = 1 (4)
 Controllability: can be generated according to a set where, bi is probability of the state si to emit the signal ok.
of natural designer-centric parameters. 𝜆 = {∏, 𝐴 , 𝐵} (5)

In 2D platform games, PTG techniques can be used to

generate the path that the character will have to pass to III. METHODOLOGY
complete the level. When the player learns the static path The methodology is based on the follow steps:
of a level, the game tends to get less interesting to him. The 1. Define the alphabet states S = {s0, s1, ..., sn};
PTG makes the game dynamic and challenging at every 2. Define startup probabilities vector: ∏ = {𝜋𝑖 =
new level. 𝑃(𝑠𝑖 )};
3. Define the trasition matrix probabilities: 𝐴 =
{𝑎𝑖,𝑗 = 𝑃(𝑠𝑖 |𝑠𝑗 )};
C. Markov chains 4. Sorts the initial state conform ∏;
A stochastic process is a family of random variables 5. Sorts the next state conform 𝐴;
representing the evolution of a value system over time. In 6. Validate generated state, if not valide, repeat step 5;
cases of discrete time, as opposed to continuous time, the 7. Repeat steps 5 and 6 until reach the end of the level.
stochastic process is a sequence of random variables, such
as a Markov chain [13]. The first 3 (three) steps are dependent from the
Define X(t) a random variable evolving as a function of application, where the game designer defines the
time. Let X(0) = 1, X(1) = 6, X(2) = 2, X(3) = 5. This components that compose the terrain.
evolution process is called Markovian if its evolution does The game designer can change the elements of the
not depend on its past, but only on its current state. alphabet in the first step. The elements entered by the
A theoretical model called “Markov Model” or “Markov designer are used to generate terrain.
chain” can model a Markov process. The probability that each terrain of the alphabet has to
Markovian chains modeled a set of states. These states appear as the first element of the terrain is defined in the
are co-related in a matrix, where each combination is a second step. The game designer is free to choose the
probabilistic chance of a state go to another. The set of probability values of vector ∏ and matrix A.
probabilistic values of the output and arrival at a new state The step 4 sets the initial state. The steps 5 and 7 build
become a state transition matrix. There are two types of the terrain of the game, following the probability of the
Markovian models: observable and hidden. Markovian process observable, explained in Section II.
The step 6 is necessary to avoid sequences that escape
1) Observable Markov chain the expected behavior of the terrain, e.g., distances that the
character cannot jump.
 After create the platforms to the current level at the 9 0 0 0.4 0.1 0.1 0 0 0.2 0.2 0 0 0
game, another Markov process are executed to populate the
scenery with decorative elements. This process is based on 10 0 0 0 0 0 0 0 0 0 0 0 1
HMM. An emission matrix is defined based on the alphabet
11 0 0 0.4 0.1 0.1 0 0 0.2 0.2 0 0 0
of states. Each component generated by the previous steps
receive a decorative element predetermined by the game 12 0 0 0 0 0 0 0 0 0.5 0 0.5 0
designer and sorted conform HMM. Table II. Transition matrix A.

IV. RESULTS
The decorative elements selected to experiments are
The methodology was implemented in Unity game showed in Figure 2.
engine [15]. An experiment was conducted with 12
different possible states. The states can be seen in Figure 1.

Figure 2. Decorative elements.

The emission matrix B was defined with the values in

Table III.
1 2 3
1 1 0 0
2 1 0 0
3 0 0.5 0.5
4 0.2 0.4 0.4
5 0.2 0.4 0.4
6 0.2 0.4 0.4
7 0.2 0.4 0.4
8 0.2 0.4 0.4
9 0.2 0.4 0.4
10 0.2 0.4 0.4
Figure 1. Alphabet defined as sample. 11 0.2 0.4 0.4
12 1 0 0
Table III. Emission matrix B.
The follow initialization vector ∏ was used in the
experiments with the values in Table I:
Figures 3, 4 and 5 show the result of applying the
1 2 3 4 5 6 7 8 9 10 11 12 technique to generate platforms for a 2D game, based on
0 0 0 0 0 0 0 0 1 0 0 0 Markov chain. The game was developed in Unity game
Table I. Initialization vector ∏. engine.
The scenes shown in Figures 3 and 4 were produced
The initialization vector probabilities make the level without the step of inserting decorative elements. The scene
always start with the state 9. presented in Figure 5 represents the application of the
The transition matrix A was defined by Table II. complete method developed.

1 2 3 4 5 6 7 8 9 10 11 12

1 0.4 0 0 0 0.1 0 0.1 0 0.4 0 0 0

2 0.2 0.2 0 0 0 0 0 0 0 0 0 0.6

3 0 0 0.2 0.1 0.1 0.1 0.1 0.2 0 0.2 0 0

4 0 0 0.2 0.2 0 0 0 0.2 0 0.2 0 0.2

5 0 0 0.2 0.2 0 0 0 0.2 0 0.2 0 0.2

6 0 0 0.2 0 0 0.4 0.2 0 0 0 0 0.2

7 0 0 0.2 0 0 0 0.4 0.2 0 0.2 0 0

8 0 0.5 0 0 0 0 0 0 0 0 0 0.5
Figure 3. Sample of generated level by Markov chain.
 Thus, advances must be made to improve the use of
hidden Markov chains, because despite obtaining a useful
result, some errors occur, e.g., the instance of elements in
places that should not exist, as in the case of the instance of
a decorative element on top of an empty terrain segment.
REFERENCES

[1] T. J. Rose and A. G. Bakaoukas, “Algorithms and Approaches for

Procedural Terrain Generation - A Brief Review of Current
Techniques,” 8th International Conference on Games and Virtual
Worlds for Serious Applications, Barcelona, 2016.
[2] G. S. P. Miller, “The definition and rendering of terrain maps,” ACM
SIGGRAPH Comput. Graph., vol. 20, no. 4, pp. 39-48, 1986.
Figure 4. Sample of generated level by Markov chain.
[3] T. Archer, “Procedurally Generating Terrain,” 44th Midwest
Instruction and Computing Symposium, Duluth, 2011.
[4] Angry Birds Classic – Apps no Google Play, Available at
https://play.google.com/store/apps/details?id=com.rovio.angrybird
s, Accessed in 03 ago 2018.
[5] Super Mario Run – Apps on Google Play, Available at
https://play.google.com/store/apps/details?id=com.nintendo.zara,
Accessed in 03 ago 2018.
[6] S. Snodgrass and S. Ontañón, “A Hierarchical MdMC Approach to
2D Video Game Map Generation,” Proceedings of The Eleventh
AAAI Conference on Artificial Intelligence and Interactive Digital
Entertainment, 2015.
[7] S. Rabin, “Introduction to Game Development,” Vol. 1, Editora
Cengage Learning, 2012.
[8] A. Santee, “GS2D e C++: 2D da nova geração em alto nível,” In
Ponto V: Programação de Jogos Profissionais, Available at:
Figure 5. Complete level generated by Markov chain. http://www.pontov.com.br/site/cpp/64-gs2/225-gs2d-e-c-2d-da-
nova-geracao-em-alto-nivel, Accessed in 03 ago 2018.
[9] W. Wakka, “Brasileiro joga mais no smartphone e ainda está na
V. CONCLUSION geração passada de consoles,” Available at:
In this paper, we apply a stochastic model for the https://canaltech.com.br/games/brasileiro-joga-mais-no-
smartphone-e-ainda-esta-na-geracao-passada-de-consoles-
creation of terrain for 2D platform games. The PTG method 113590/, Accessed in 03 ago 2018.
is based on the probability of a new state is created from [10] J. Togelius, G. N. Yannakakis, K. O. Stanley and C. Browne,
the current state, where each state represents a different “Search-based procedural content generation: A taxonomy and
segment of terrain. The land tends to connect survey,” IEEE Transactions on Computational Intelligence and AI
probabilistically. in Games, Vol. 3, No. 3, pp. 172–186, 2011.
However, only an application of this probabilistic [11] J.-D. Genevaux, É. Galin, É, Guérin, A. Peytavie and B. Benes,
“Terrain Generation Using Procedural Models Based on
method can render the result unfeasible in a game, as it can Hydrology,” ACM Transactions on Graphics, Vol. 32, No. 4, 2013.
create new states that hurt consistency of the terrain. To [12] J. Doran and I. Parberry, “Controlled Procedural Terrain Generation
avoid inconsistent terrain, a rule filter was applied. Using Software Agents,” IEEE Transactions on Computational
The hidden Markov model was applied as a second step Intelligence and AI in Games, Vol. 2, No. 2, pp. 111-119, 2010.
to populate the terrain created with decorative game entities [13] R. G. Gallanger, “Stochastic Processes: Theory for Applications.”
such as trees, shrubs, rocks, etc. 1st Edition, Hardcover, 2014.
The methodology shows good results, as can be seen in [14] C. Deng and P. Zheng. “A new hidden markov model with
application to classification,” In Intelligent Control and Automation,
Figures 3, 4 and 5. The method applied for filtering the new 2006.
states produced to maintain the consistency are useful and [15] Unity, Available at: https://unity3d.com/pt, Accessed in 03 ago
the use of Markov chains alone was insufficient to obtain a 2018.
productive result.