Artificial Intelligence (AI) and Machine Learning (ML) in The Future of Animation
Artificial Intelligence (AI) and Machine Learning (ML) in The Future of Animation
W HAT IS 01
A RTIFICIAL I NTELLIGENCE ?
H OW D OES AI W ORK ? 01
I T S A LL A BOUT L EARNING , M ACHINE L EARNING 02
M AJOR D RIVERS OF THE AI R EVOLUTION 04
E MERGING AI E CONOMY 09
U SES OF AI IN C OMPUTER 09
A NIMATION AND G AMING
AI IN THE S IMULATION OF THE 10
H UMAN M OTION
AI IN THE S IMULATION OF C ROWDS 12
AI I N THE S IMULATION OF P HYSICAL P ROCESSES 13
P ROCEDURAL C ONTENT G ENERATION (PCG) 14
WITH M ACHINE L EARNING
AI T O C REATE G AME L EVELS AND 15
S TY LES
AI AND N EW G AME P ATTERNS 16
AI I N P LAYER M ANAGEMENT 17
B ENEFITS OF AI- BASED G AME D ESIGN 18
Issue 01
What is A R T I F I C I A L
INTELLIGENCE
From ancient times, scientists dreamed to
create machines that can reason, learn, plan,
? technology. The AI revolution
spilled over into the
market, making AI systems, such
as Apples Siri, Google Assistant,
Facebooks image
consumer
recognition
s o f t w a r e , a n d G o o g l e Vo i c e a n
see, and move in much the same way as inherent part of our daily lives.
humans do naturally. The prospect of working Rather than pursuing an
Artificial Intelligence (AI) became tangible in ambitious agenda of General
the twentieth century with revolutionary Artificial Intelligence aimed to
breakthroughs made in computer technology, reconstruct the human mind in
neuroscience, and mechanical engineering. full scope, modern AI is much
A l t h o u g h A I t e c h n o l o g y i s n o t n e w, i t r e c e i v e d more specialized, which means it
a powerful momentum in the 2010s when is business-oriented and
global digital infrastructure and science consumer-centric. Contemporary
became ripe for incorporations of AI magic into AI no longer needs robots and
business. For many observers, 2015 was a machines to run; it may be
landmark year for AI technology . Funding for embedded into various desktop,
AI projects grew exponentially with major tech mobile, or SaaS
companies, such as Google, IBM, and (Software-as-a-Service)
Facebook, investing vigorously in the new applications.
MAGA ZINE 01
Issue 01
ITS ALL
as a method that gives computers the ability to
learn without being explicitly programmed . Unlike
standard programming that uses a series of if/else
MAGA ZINE 02
Issue 01
Figure # 1
Architecture of a
Neural Network
MAGA ZINE 03
Issue 01
MAJOR DRIVERS OF AI
REVOLUTION
MAGA ZINE 04
Issue 01
MAGAZINE 05
Issue 01
algorithms are fruitfully applied to teach intuition, planning, and decision-making in complex
combinatorial decision spaces with astronomical amounts of solutions. For instance, combining
intuition and calculation has helped AlphaGo AI, a program designed by Deep Mind beat the best
human players in the old Chinese game Go which has been considered an impossible feat for a
computer to master anytime soon.
https://deepmind.com/blog/exploring-mysteries-alphago/
AI is entering a syndication stage with more AI solutions and applications included in B2B
(Business-to-Business) and consumer offerings. Popularization of AI technology is supported by major
tech companies who spread it across wider developer and business communities by open-sourcing
ML libraries and APIs. Such ML libraries as TensorFlow, originally developed by the Google Brain
project, provide basic ML functionality, such as neural networks, classification, and optimization
algorithms out of the box. Open-source ML software with optimized algorithms and standardized
solutions makes the threshold of entering into the booming AI market much lower.
MAGA ZINE 06
Issue 01
09
economic potential becomes almost boundless.
MAGA ZINE
Issue 01
AI IN SIMULATION
of
HUMAN
MOTIONSince the 1970s, motion capture has been the main technique for motion
animation in computer graphics and video games. In motion capture, motion data
is obtained via specialized electromagnetic or electronic devices placed on
human actors to detect motion features and geometry. Motion data retrieved this
way has to be manually edited by animators to create believable 3D models of
movement. This technique has obvious disadvantages, such as high price of
equipment, complicated operating procedures, constraints in actor movement, and
more. Consequently, alternative motion capture methods were designed to address
these difficulties. In video-based motion capture, for example, a single camera
taking a series of shots of human characters and applying automatic feature
tracking to extract movement patterns is used. Similarly, in silhouette analysis a
video space is searched for human gestures and correspondence is built between
extracted image features and a 3D human model. Despite their advantages
vis-a-vis traditional methods, both video-based approach and silhouette analysis
require individual characters to do a motion capture and thorough hand-made
fine-tuning of resultant animations.
Unlike traditional motion capture, AI methods allow for algorithmic learning
of human movement patterns with the end result of more precise, realistic and
flexible human motion. In fact, high-fidelity motion and facial animation may be
created without any motion capture at all. For example, researchers from the
University of Edinburgh and Method Studios created a machine learning system
that trains on a large data set of motion capture clips to learn various types of
locomotion, such as walking, running, jumping, and climbing in virtual environments.
The system uses Phase-Functioned Neural Networks for Character Control that
models motion based on numerous parameters, such as the previous state of the
character and geometry of the scene . As a result, AI is able to produce smooth
motions where the character automatically adapts to various geometric .
MAGA ZINE 10
Issue 01
MAGA ZINE 11
Issue 01
AI in CROWD SIMULATION
In the early days of computer graphics and VFX
(Visual Effects), directors used thousands of extras
in battle and sport event scenes. The world record
belongs to Richard Attenboroughs 1982 film
Gandhi in which 300,000 extras were used in the
f u n e r a l s c e n e . To d a y , w i t h n e w A I c r o w d s i m u l a t i o n
technologies available we cant afford the luxury of
drawing crowds in games, or hiring extras.
On a large scale, crowd simulation AI was for
t h e f i r s t t i m e i n Pe t e r J a c k m a n s L o r d o f t h e R i n g s
premiered in 2001. Crowd simulation software
MASSIVE (Multiple Agent Simulation System in
Virtual Environments) used in the film allows to
generate thousands of extras that act as individuals
without being explicitly programmed, or animated.
The system uses fuzzy logic to make characters
realistically respond to surroundings and other
characters. In MASSIVE, crowd behaviour patterns
are retrieved from pre-recorded animation clips that
are made in motion-capture sessions, or drawn by
hand.
Modern ML techniques push the limits of crowd
simulation technology even further by using learning
to track crowd behavior patterns from real crowds.
In the state-of-the-art crowd simulation, motion of a
real human crowd is first recorded from an aerial
view to form a training set of data. Then,
two-dimensional moving trajectories of each
character in the crowd are learned by powerful
neural networks. By studying real crowd footage, AI
system can assess the impact of environment and
nearby agents more precisely than in hand-made Crowd Simulation. Source:
animations and motion capture. As a result, AI-made http://gamma.cs.unc.edu/research/crowds/)
crowds behave much the same way as real crowds.
With crowd behaviour patterns learned via ML learning algorithms, game
designers can blend existing crowd data into new crowd animations, selecting their
own virtual environments and characters. This makes crowd animation in games
more flexible and creative. Characters in AI-based crowds may also express social
and emotional intelligence reflected in a rich palette of facial expressions,
emotions, and gestures. Using reinforcement learning, crowd agents can learn from
their mistakes and alter behavior in response to rewards and punishments received
from the environment. Thus, incorporation of learning into crowd simulation allows
to combine group and individual realism and introduce fuzziness of behavior one
may find in real crowds. AI crowd as a living and changing organism becomes a
powerful tool of experimentation in social dynamics and creates opportunities for
new interactive methods in gaming.
MAGA ZINE 12
Issue 01
AI In Simulation
of Physical
Processes
Physically based computer
animation aims to simulate physical laws
of a system for graphical purposes.
It is widely used in water, smoke,
explosion, and character animation.
Real-time simulation of physical
substances and processes, such as
fluid or smoke, is one of the hardest
problems in computer graphics.
High-fidelity simulation of physics requires
immense computational
resources, which makes
real-time applications
impractical. With AI, however,
we may use a data-driven
approach that combines deep learning methods with classical fluid and gas equations to obtain
highly realistic and computationally cheap results. For example, Jonathan Tompson from Google and
researchers from New York University proposed a semi-supervised learning approach in which a
Convolutional Network (ConvNet) is trained on a data set of smoke and fluid simulations. At the end
of the training, AI learns to produce relatively fast simulations of smoke and fluid that outperform
standard non-AI simulation. Using statistical data of fluid and smoke dynamics, new AI-based
simulations can create high-resolution animation that accounts for numerous unseen properties of
gases and liquids and complex interrelations between substance particles.
Not just simulations of physical objects and processes, but game characters, as well, should
obey Newtons laws. This problem is addressed by the physically based character animation. Since
1999, AI played a crucial role in emulation of physical properties of characters. For example, a
NeuroAnimator approach used a neural network to observe and interpret physics-made models in
action in order to synthesize motion that suits animation goals. ML used in NeuroAnimator allowed
to produce faster computations than traditional numerical methods . In pair with simulation of
physical processes, physically based animation of characters allows for more realistic rendering of
human response to physical shocks, such as explosions, and simulation of other dynamic
environments that experience the impact of physical forces (wind, earthquakes, floods). Flexibility of
ML approach also allows simulation of motion under new perturbations, different from those in the
training sample, which opens opportunities for experimentation in sci-fi games.
MAGA ZINE 13
Issue 01
PROCEDURAL
One example of such approach is the
WaveNet neural network developed by Deep
Mind, an AI research company bought by Google
GENERATION
systems (TTS) based on large datasets of small
speech fragments. In contrast to TTS, WaveNet
directly models the raw waveform yielding more
(PCG) WITH
natural-sounding speech. Speech generated by
WaveNet is also more natural by virtue of
non-speech sounds, such as breathing or mouth
movements. Similarly, WaveNet is able to
LEARNING
they were written by a human composer. With
such a technique at their disposal, game
designers will be able to generate more
naturally-sounding NPCs (Non-Player Characters)
Procedural Content Generation (PCG) is and create rich sonic experiences for gamers.
widely employed in game design for algorithmic Today, AI solutions may also be fruitfully
generation of content in the runtime. PCG may used in procedural video and animation
be effectively used to increase replay value, generation. In one of experiments concerning this
save storage space, reduce game production idea, researchers from MITs Computer Science
efforts and costs, personalize narrative for and Artificial Intelligence Laboratory (CSAIL)
players, and create infinite diversity of quests used a deep learning algorithm that could learn
and game environments. In the early days of to transform still images from a scene into a brief
video games, memory shortage was the main video that simulates the future of the scene. To
reason behind using PCG. To reduce memory achieve this, researchers trained the neural
usage, conventional PCG employed pseudo network on two million of unlabeled videos that
random number (PRN) generators that initialized contained a years worth of footage. In gaming,
pre-defined parameters of game space size, such approach could be used to simulate
texture, color, or character appearance. diversity and relationships between objects (e.g
These days, with AI and ML offering trees) in complex structures (e.g forest). Its still
innovative methods to create content in games, hard to develop a good script that covers a
conventional PCG approach becomes outdated. variety of tree species (e.g birches, palm trees).
Such ML techniques as generative adversarial It is even harder to generate a realistically
networks are more efficient in generation of looking forest by simply assembling trees,
such game artifacts as music, speech, or because forest trees tend to form complex ties
animation. In the state-of-the-art ML methods, and grow unevenly. AI-based PCG solves this
music and speech in games may be generated problem by learning on large samples of real
with ML algorithms that use raw waveforms trees to identify real-world patterns and relations
instead of sound samples. Increase of between them. As a result, AI-based PCG
computing power allows neural networks to becomes a powerful alternative to hand-made
process thousands of sound frames in animation that falls short of expressing the full
milliseconds and output more realistic speech complexity of our world.
and music content that perfectly simulates
14
physical properties of sound.
MAGA ZINE
Issue 01
AI To Create
Game Levels
and Styles
MAGA ZINE 15
Issue 01
16
interaction between AI and players, and promote autonomy of AI agents in game
design.
AI in PLAYER MANAGEMENT
With rowing competition among video
game companies, it becomes harder to
attract and retain new customers.
Unfortunately, quality of game content is
not enough to create persistent attachment of players to
games. In a new environment driven by multi-player online and social games,
companies emphasize long-term gaming experience by offering products that
evolve over time and support player communities to produce more game buzz.
Married with this approach, AI technology may give another competitive edge for
companies in attraction and retention of gamers. One of the most promising areas
of innovation in this respect is game analytics. AI may be efficiently leveraged to
capture, process, and understand player behaviour, model player responses to new
game features, and manage player experience via customizable and adaptable AI
features. This expertise may be also used to create innovative player assistance
AI. This type of AI will play a role of
lifelong agents who learn about
players over time, recognize and
adapt to their changes, and use
history of interactions to improve
their gaming experience. Lifelong
agents can also assist players in
overcoming game challenges that
would otherwise make them quit playing a game. By knowing the players strengths
and weaknesses, game AI may intervene in a timely manner to provide them with
useful hints, automatically generated tutorials, and gameplay adjustments. In this
way, game AI may serve as a coach to new players along their difficult mission.
Using AI-powered Dynamic Difficult Adjustment (DDA), the AI coach will adapt
game parameters, enemy behaviours, and other game features to the players skills
and mood. Similarly, given the growing popularity of online games, new game AI
may employ user data, such as GPS location, social profile, and user preferences
to design a better gaming experience with more
adaptable gameplay. In this way, big data generated
by modern online and social games can become a
powerful training sample to be used by AI lifelong
agents to improve player experience and create
lasting and pleasant connection of players to
games.
MAGA ZINE 17
Issue 01
Benefits of
AI-based Game
Design
Gaming is at the peak of its popularity. According to the ESA (Entertainment Software
Association) study, the number of gamers in the world has reached 1.8 billion people in 2016
. To attract and retain customers, video game companies should not hesitate to adopt
state-of-the-art AI solutions that enhance and broaden gameplay experiences. Advantages of
AI for gaming and animation mentioned above are too obvious to be ignored.
MAGA ZINE 18
Issue 01
References
Clark, Jack. Why 2015 Was A Breakthrough Year In Artificial Intelligence? Bloomberg.
https://www.bloomberg.com/news/articles/2015-12-08/why-2015-was-a-breakthrough-year-in-artificial-intelligen
ce
Puget, Jean Francois. (2016). What Is Machine Learning?
https://www.ibm.com/developerworks/community/blogs/jfp/entry/What_Is_Machine_Learning?lang=en
Vincent, James (2017). Artificial Intelligence Can Spot Skin Cancer As Well As A Trained Doctor.
https://www.theverge.com/2017/1/26/14396500/ai-skin-cancer-detection-stanford-university
Pardes, Arielle. (2017). The Slickest Things Google Debuted Today At Its Big Event.
https://www.wired.com/2017/05/slickest-things-google-debuted-today-big-event/
ImageNet. http://image-net.org/about-stats
Pres, Gil. (2016). Forrester Predicts Investment In Artificial Intelligence Will Grow 300% in 2017.
https://www.forbes.com/sites/gilpress/2016/11/01/forrester-predicts-investment-in-artificial-intelligence-will-grow-
300-in-2017/#55093b115509.
McCormick, James (2016). Predictions 2017: Artificial Intelligence Will Drive The Insights Revolution. Forrester.
https://go.forrester.com/wp-content/uploads/Forrester_Predictions_2017_-Artificial_Intelligence_Will_Drive_The_I
nsights_Revolution.pdf
IDC (2016). Press Release. http://www.idc.com/getdoc.jsp?containerId=prUS41878616
Pardes, Arielle. (2017). The Slickest Things Google Debuted Today At Its Big Event.
https://www.wired.com/2017/05/slickest-things-google-debuted-today-big-event/
Pres, Gil. (2016). Artificial Intelligence Rapidly Adopted By Enterprises, Survey Says.
https://www.forbes.com/sites/gilpress/2016/07/20/artificial-intelligence-rapidly-adopted-by-enterprises-survey-sa
ys/#b5b578c12dae
Pres, Gil. (2016). Forrester Predicts Investment In Artificial Intelligence Will Grow 300% in 2017.
https://www.forbes.com/sites/gilpress/2016/11/01/forrester-predicts-investment-in-artificial-intelligence-will-grow-
300-in-2017/#55093b115509
Graft, Kris (2015). When Artificial Intelligence In Video Games Becomes ... Artificially Intelligent.
http://www.gamasutra.com/view/news/253974/When_artificial_intelligence_in_video_games_becomesartificially_i
ntelligent.php
Phase-Functioned Neural Networks for Character Control. http://theorangeduck.com
Templeton, Graham (2016). Deep-learning Neural Network Creates Its Own Interpretive Dance.
https://www.extremetech.com/extreme/227287-deep-learning-neural-network-creates-its-own-interpretive-dance
Yu, Jun and Tao, Dacheng (2013). Modern Machine Learning Techniques and Their Applications in Cartoon
Animation Research. Hoboken, New Jersey, John Wiley & Sons, p.3.
Grzeszczuk, Radek, Terzopoulos, Demetri, Hinton, Geoffrey. (1998). NeuroAnimator: Fast Neural Network
Emulation and Control of Physics-Based Models. http://web.cs.ucla.edu/~dt/papers/siggraph98/siggraph98.pdf
Deep Mind. WaveNet: A Generative Model for Raw Audio.
https://deepmind.com/blog/wavenet-generative-model-raw-audio/
Summerville et al. (2017). Procedural Content Generation via Machine Learning (PCGML).
https://arxiv.org/abs/1702.00539
Nightmare Machine. http://nightmare.mit.edu/
Treanor, Mike et al. (2015). AI-Based Game Design Patterns.
http://julian.togelius.com/Treanor2015AIBased.pdf
McKane, Jamie (2016). There Are 1.8 Billion Gamers In the World, And PC Gaming Dominates the Market.
https://mygaming.co.za/news/features/89913-there-are-1-8-billion-gamers-in-the-world-and-pc-gaming-dominates
-the-market.html
MAGA ZINE 19
About Team Absentia:
Founded by three BITS Pilani drop outs, Absentia consists of a small, highly skilled and electric team
partnering with knowledge partners from around the world. The team, with over 500 publications is
developing Norah AI, an artificial intelligence capable of generating premium games in a few
minutes.
Investors:
Exfinity Venture Partners LLP is focused on funding Technology Ventures but with a difference.
They seek to act as a strategic partner to portfolio companies through high impact interventions.
Their impressive portfolio includes Mad Street Den, Uniken systems.
TV Mohandas Pai
Leadership team and ex-board member of Infosys Ltd (1995-2011). Chairman, Manipal Global
Education Services. Board of Director National Stock Exchange of India (NSE). Prolific angel
investor in India & US.
V Balakrishnan
Leadership team and ex board member of Infosys Ltd. Ex-CFO, Infosys Ltd. Head of BPO, Finacle &
India Business Unit at Infosys.