The Mixed Reality Simulation Platform (MIXR)

Douglas D. Hodson1 , David P. Gehl2

Abstract— The Mixed Reality Simulation Platform (MIXR) constructive simulation applications (where interaction dead-
is an open-source software project designed to support lines are non-existent or of less concern), ‘higher-level’ or
the development of robust, scalable, virtual, constructive, more detailed (possibly processor intensive) system models
stand-alone, and distributed simulation applications. Its can be selected. MIXR organizes code so that the developer
most common use case, is to support the development of can make these trade-offs.
executable simulation applications used to assemble real- The software leverages the Model-View-Controller
time, interactive, distributed virtual environments (DVEs). (MVC) pattern by partitioning functional components and
Its core infrastructure is architected to favor the devel- decoupling them into packages. MVC concepts are taken
opment of applications that can be executed in a determin- a step further in this domain by providing other views,
istically manner to meet real-time interaction/response time such an abstract network interface to support specific
requirements, yet provide a degree of configuration flexibility interoperability solutions; examples include the Distributed
associated with scenario definition, I/O, and standard inter- Interactive Simulation (DIS) protocol and the High Level
operability interfaces (e.g., DIS, HLA, etc.). Architecture (HLA).
The MIXR codebase has been used as a basis to develop Specific applications using the software are numerous, and
numerous military-oriented applications that support Live, include current and future, fighter and bomber platform sim-
Virtual and Constructive (LVC) or mixed reality type simu- ulators, Unmanned Aerial Vehicle (UAV) Ground Control
lations [1]. Stations, Integrated Air Defense Systems (IADS), futuristic
This paper describes the software platform itself, and battle managers and more; for additional information see [2]
how it partitions code to take advantage of multi-core/multi- and [3].
CPU PCs to support the development of real-time simulation
applications.
2. Abbreviated History
Keywords: MIXR DVE LVC mixed reality simulation The genesis of MIXR software codebase can be traced to
the late 1980s when is was written in the C programming
1. Introduction language and executed on an Commodore Amiga 1000 (yes,
The Mixed Reality Simulation Platform (MIXR) is an a Commodore Amiga!). Because C doesn’t directly support
open-source software project that originated in the De- the OO programming paradigm, the codebase defined an
partment of Defense (DoD). It has been used to support OO-like infrastructure to support programming from this
the development of numerous distributed applications in perspective. In the early 1990’s, the C-based OO system
multiple domains, including Live, Virtual, and Constructive was replaced with C++, where applications were developed
(LVC) simulations. and executed on Silicon Graphics (i.e., SGI) workstations.
MIXR defines a high-level organizational pattern that The transition away from Silicon Graphics workstations to
provides a structure for simulation applications (sometimes personal computers (PCs) occurred in 1997.
called simulators). In other words, the software provides a Up until this time, the codebase had no official name
blueprint for the developer to customize to ease the creation associated with it; that changed in 2002 when it was named
of simulation applications. It leverages traditional object- the Enhanced Air-to-Air, Air-to-Ground Linked Environment
oriented (OO) software design principles while blending Simulation (EAAGLES); later updated to mean the Extensi-
real-time system concepts to meet human and/or hardware ble Architecture for the Analysis and Generation of Linked
interaction requirements. Simulations. The meaning of the acronym was changed to
By providing abstract representations of system com- divorce specific domain terminology such as ‘air-to-air’ and
ponents (i.e., abstract classes), system models at different ‘air-to-ground’ to the more general purpose modeling and
levels of fidelity can be intermixed in such a way to simulation capabilities the software is designed to support.
optimize runtime performance. The abstract representations In 2003, ‘EAAGLES’ as a useful simulation product,
of systems enables a developer to tune the application to became more visible within the DoD community as an
run efficiently so that both human and/or hardware (in-the- EAAGLES-based fighter cockpit application was demon-
loop) interaction latency deadlines are satisfied. For purely strated at the Interservice/ Industry Training, Simulation
and Education Conference (I/ITSEC); the world’s largest terms, a ‘synthetic battlespace’). In a military domain, the
modeling, simulation, and training conference. Even though means to exchange data between applications is defined by
the ‘fighter cockpit’ simulator was developed in only a few interoperability protocols, of which several are published
months, it performed flawlessly; a testament to both the as IEEE standards; for example, the Distributed Interactive
design of the application, and the underlying framework. Simulation (DIS) and the High-Level Architecture (HLA).
In July of 2006, a significant subset of the original These distributed simulations provide a means to interact
EAAGLES codebase was released into the public domain; it within the shared virtual world by providing interfaces to
became what is known as O PEN E AAGLES. At the same time, humans and/or hardware.
a website was setup to provide information, documentation Because humans and/or hardware are included as part of
and releases. In 2009, the book titled "Design & Imple- the represented system of interest (i.e., ‘in-the-loop’), addi-
mentation of Virtual and Constructive Simulations Using tional requirements in the form of interactive response times
O PEN E AAGLES" was published [4]. Since then a steady arise, which classifies the entire apparatus as a real-time
stream of releases have been posted. system. Trade-offs must be made concerning responsiveness
In 2017, O PEN E AAGLES was renamed to MIXR for a and fidelity when constructing these real-time systems.
number of reasons including: The MIXR codebase was designed from scratch con-
• a better alignment of the project name with domain sidering these requirements, such that trade-offs between
of interest; i.e., the development of mixed reality sim- deterministic execution performance and model fidelity can
ulation applications. See [1] and [5] to understand the be made.
relationship between mixed reality simulations and LVC
simulations, 4. Software Classification
• and an explicit shift away from using the term ‘frame- A framework is a set of cooperating classes that make up
work’ in favor of ‘platform’ to indicate a concerted a reusable design for a specific class of software [6], [7]. A
effort to expose the functionality/capability it provides framework is customized to a particular application by cre-
in different ways, as opposed to traditional OO-style ating application-specific subclasses of abstract classes from
inheritance (i.e., subclassing), the framework [8]. A toolkit is a set of related and reusable
• and more generally, an indication that the project is classes that provide useful, general-purpose functionality.
transitioning away from a ‘traditional C++’ codebase in They are the OO equivalent of subroutine libraries [8].
favor of a ‘modern C++’ codebase to improve quality,
capability, understandability, and reduce complexity.
Today, the MIXR codebase is quite stable in terms of the
features it provides and capabilities it exposes. As a result of
its relatively long development history, it has accumulated
lots of ‘baked-in’ knowledge which resulted from its use in
supporting the development of a large number of simulation
applications. The current focus is on refining and improving
it in ways to make it even more flexible and accessible to a
wider audience.
Fig. 1: Platform Orientation
3. Terminology As Figure 1 shows, an application built using a framework
Over the past few decades a number of equivalent terms is created by extension; which, in C++, usually means
have arisen to describe real-time, distributed simulations that subclassing classes via inheritance. Whereas, an application
create a shared virtual world for humans and/or hardware that simply uses library functionality (i.e., toolkits) does not
to interact within. For example, distributed virtual environ- exhibit the same kinds of dependencies (i.e., specifically
ments (DVEs), networked virtual environments (NVEs) and coupling).
distributed virtual simulation (DVS). Today is no different; While MIXR is strongly organized as a framework in
but the trend is to characterize these systems, more generally, which applications are developed by extending classes, the
as ‘mixed reality’ environments or simulations; which does goal is to position it to feel and act more like a general
more clearly describe them. purpose modeling and simulation platform. In other words,
A common architectural characteristic of these systems expose more of its functionality without having to subclass
include the asynchronous execution of several largely au- classes.
tonomous standalone simulation applications that interact In either case, MIXR is not an executable application like,
with each other by exchanging data through a network say, Microsoft Word or even a typical game engine. It doesn’t
to create a shared illusion of a virtual world (in military define a main() function; it leaves those details up to the
developer which is in full control of how applications are be captured with state machines and state transitions can use
assembled and configured. provided message passing mechanisms.
MIXR is written in C++ and partitioned into packages
that serve as functional toolkits (i.e., libraries) for the de- 6. Real-Time Simulation Platform
veloper to use as needed. For example, it defines a graphics MIXR was written in C++ because:
toolkit which can be used to facilitate the development of • most real-time systems are developed in C for perfor-
operator/vehicle interfaces and displays. mance reasons [9]. Object-oriented languages tend to be
As organized, the software enables the creation of a viewed with skepticism as overall system performance
diverse set of simulation applications. Derived simulation ap- often outweighs flexibility. But for the modeling and
plications can often be run stand-alone or within a distributed simulation domain, we believe the advantages afforded
networked environment. Distributed applications that interact by an object-oriented language (i.e., C++) outweighs
with each other using standard protocols such as DIS and/or this slight performance penalty,
HLA can be setup almost ‘out of the box’. • C++ is portable and compilers exist on virtually every
platform. This allows developers to build MIXR-based
5. Virtual Simulation Requirements applications on any of the major popular operating
Simulations that interact with humans participants (or systems (Windows, Linux, etc),
hardware) must respond within prescribed deadlines (latency • C++ is flexible in terms of supporting multiple pro-
or response times). A simulation that does not respond graming paradigms,
(like the system it is intended to represent) will frustrate • it is desirable to define memory management so it
the operator and may skew the simulation results. Software does not interfere with the overall performance of the
systems faced with this demanding requirement fall into application. Therefore, the use of new and delete
the category of a real-time system. Real-time systems are operators is preferable to uncontrolled garbage collec-
designed and organized so that time-critical (often periodic) tion.
tasks can meet their deadlines. It is beyond the scope of this paper to cover each and
Two standard approaches to scheduling tasks include every class defined, but a few key classes deserve attention
priority-based and foreground/background systems. Priority- in provide insight into the structure of the codebase.
based designs assign a priority to each task in the system. Object : The Object class is the C++ system object
The task with the highest priority that is ready to run is for the MIXR codebase. Unlike other OO languages (for
executed first. The scheduling of the task resides with the example Java or Ruby), the C++ language does not provide
operating system. a system object. C++ also does not provide native garbage
In a foreground/background system the application con- collection. While lacking these two features could be viewed
trols the scheduling of tasks. Foreground tasks are executed as a negative when comparing the native features of various
with the help of a jump-list, or a managed list of pointers languages, it is considered a positive when the application
to functions (tasks). Tasks are executed one after another as domain consists of applications that need to meet real-time
defined by the list order. Aperiodic events and background requirements.
tasks receive processing time after all the ‘highest priority’ C++ provides the flexibility to define how these mecha-
tasks in the list have finished. nisms work for different application domains. For example,
MIXR is organized as a foreground/background system, if the developer is writing an application in which control
but instead of managing a jump-list (or a list of functions over potentially time-consuming memory management op-
to process), thread execution paths are interwoven into the erations is of little concern, the codebase provides smart
design of the class hierarchy. It is specifically designed to pointers to automatically manage the creation and deletion
take advantage of multi-core/multi-CPU PCs which allow of objects. If, on the other hand, the application has time
the creation of a high priority foreground threads. Because constraints to meet (i.e. a real-time system), the uncontrolled
multiple processors are available, reliable execution of high creation and destruction of objects might lead to perfor-
priority time-critical foreground threads is assured with mance problems.
general purpose operating systems such as Windows and One of Object’s capabilities is to provide a simple ref-
Linux. erence counting system for the memory management of all
It should be emphasized that MIXR-based applications objects. Object provides access to this system so that a de-
execute as a cycle or frame-based system; not as a discrete- veloper can manually control and tune performance-oriented
event processor. This approach satisfies the requirements for applications, if they arise; for example, the processing, in
which it is designed; namely, support for models of vary- real-time, of modeled radio frequency (RF) emission packets
ing levels of fidelity including higher level physics-based or infrared radiation (IR) geometry information.
models, digital signal processing models and the ability to The other subtle but important aspect to providing a
meet real-time performance requirements. Model state can system object appears in the form of typechecking. The
presence of a system object, and the derivation of all classes These rules mimic understood guidelines associated with
from it, enables the dynamic casting of objects. It also avoids real-time system development.
the pitfalls associated with untyped functions and classes.
MIXR’s coding standards explicitly prohibits the use of void 7. Application Structure
pointers for this very reason. A developer using MIXR as a basis for a simulation typ-
Component : In OO programming, a container class is ically builds an application by either using existing classes
a class of objects that contain other objects. The MIXR (i.e., models) or extends them to add detail. Finally, main()
component class is that and much more. Component is a is defined for the application.
container for other components. Component also defines a The mainline usually performs these tasks:
basic messaging system that is used throughout the codebase. • read an configuration file that defines and creates an
From the outset, MIXR is designed to facilitate the object hierarchy. MIXR provides a parser to do this;
creation of simulation applications that execute in real-time it defines a simple context-free scheme-like input lan-
and/or interact with a human participant. Applications with guage, that is easy to extend,
time constraints and latency/response deadlines typically • create and setup threads to be executed. For non-real-
separate time-critical tasks and non-time-critical tasks; for time applications (e.g., purely constructive applications)
example, the execution of an aerodynamic model at a specific a single thread maybe all that is needed. For a virtual
frequency as opposed to writing data to a hard disk, or simulation with time-critical code, both foreground and
printing a document. background threads are usually created,
This separation is facilitated by two methods in the com- • execute the simulation by calling updateTC() and
ponent class. When designing a model, code that needs to updateData() as required. If the application is a
execute in a time-critical manner (usually mathematical cal- virtual simulation high-priority thread(s) are assigned
culations) is placed in an overridden virtual updateTC() to process foreground tasks.
(update time-critical) method. Code that can be run in a The developer fully defines the mainline to allow for a
non-time-critical manner is placed in the overridden virtual variety of execution scenarios to be defined; MIXR does
updateData() method. not define a main() function! Furthermore, application
This organization of code has a number of advantages: mainlines tend to be short and sweet. Most of the work is
• since time-critical code is clearly separated from back- in the design and extension of new classes.
ground code, applications can be designed to meet Simulation applications are typically organized like the
performance requirements, structure as shown in Figure 3. Thinking in terms of a tree
• all the code (time-critical and background) associated of components, the class Station resides at the top, or the root
with a model is logically located within the same class. node. Every other component is a subcomponent of Station.
Station connects models to views (e.g., graphical displays,
As in Figure 2, one can view an instance of a simulation I/O devices and controls, interoperability networks). A Sta-
application as nothing more than a tree of Components. tion owns an instance of a Simulation which manages a
A call to the top (or root) of the tree’s updateTC() list of players (i.e., entities or platforms), keeps track of
method, will automatically execute every subcomponent’s simulation time, which includes the cycle, frame and phase
updateTC(). In other words, every component will exe- that is currently being processed.
cute the code of its children. This process continues until Being a frame-based system, delta time is passed as an
the entire tree has been processed. The same process takes argument to updateTC() so proper calculations involving
place for the background code. time can be performed. Having models rely on delta time
for calculation means the frequency of the entire system
can change without having to change each and every model
(so long as Nyquist rates are met). Additional time related
information is recorded in terms of cycles (16 frames or
sometimes called a major frame) and phases. Phases se-
quence the flow of data throughout a model. Four phases
are currently defined:
• Dynamics – update player or system dynamics includ-
ing aerodynamic, propulsion, and sensor positions (e.g.,
antennas, IR seekers).
Fig. 2: Component Tree • Transmit – R/F emissions, which may contain datalink
messages, are sent during this phase. The parameters
The MIXR coding standard spells out rules to follow when for the R/F range equation, which include transmitter
writing code in updateTC() (e.g., no blocking I/O calls). power, antenna pattern, gains and losses, are computed.
 Fig. 3: Application Structure

• Receive – Incoming emissions are processed and fil- objects such as bitmaps, input/output fields, fonts, polygons,
tered, and the detection reports or datalink messages readouts, textures, and others.
are queued for processing. As Figure 4 shows, the graphics architecture defines
• Process – Used to process datalink messages, sensor de- hierarchical relationships between the Graphic, Page and
tection reports and tracks, and to update state machines, Display classes.
on-board computers, shoot lists, guidance computers, The Graphic class encapsulates attributes associated with
autopilots or any other player or system decision logic. a graphic such as color, line width, flash rate (for graphics
A Player is a component that adds dynamics and other that flash), coordinate transformations, vertices’s and texture
unique behaviors. Some components that can be attached coordinates, select names and scissor box information. Since
include signatures, antennas, sensors and stores. Derived air Graphic is a component, it can contain other graphics.
and ground players are included within the codebase. The Page class defines a ‘page’ of graphics that can
An abstract interoperability network interface is defined facilitate the creation of Multi-Function Displays (MFD)
so specific protocols can be incorporated, such as DIS to where specific page transition events need to be defined. The
support interoperability with other distributed simulation Display class defines all the resources available for drawing
applications. This network interface automatically creates such as fonts, the color table and both the physical and
new players in the player list. As far as the simulation is logical dimensions of the display viewport. Finally, open
concerned, these players are like any other. source GUI toolkits (such as Glut, Fox, FLTK and Qt) can
be used to provide a rich interface.
8. Graphics MIXR graphic classes ease the development of opera-
The platform defines several graphic toolkits for the devel- tor/vehicle displays and leverage open source GUI toolkits,
opment of operator/vehicle interface displays. The graphic but they are not intended to replace visual scenegraph
toolkits are based on OpenGL for all primitive drawing, thus oriented displays (such as a heads up display or image
making it compatible with virtually any platform. generator). The overarching philosophy of MIXR is to avoid
The foundation for graphics drawing is contained in the reinventing what is already well done by other applications
graphics package. It contains classes for drawing graphic and/or packages.
 Higher level MIXR toolkits that use this structure include threads to meet response time requirements. For the example
the instrument library which includes dials, buttons, gauges, just presented, a thread is set up to execute time-critical
meters, pointers, and countless other fully functional instru- mathematical calculations associated with worm movement
ments, along with simple maps. in real-time, and in a non-time-critical manner the operating
All of the graphical toolkits are independent of the sim- system (or Glut in this case) draws the worm during idle
ulation modeling environment. Models don’t include any times.
special knowledge of graphics, and graphics include no
special knowledge of models. The code that connects the 9. Device I/O & Linkage
two resides within the Station class. MIXR abstracts I/O devices and complete linkage systems
Through an ownship pointer in the Station class, the con- so that a hardware interface appears to the application as
trols and displays of any player can be switched at anytime. a single (unified) device that presents a number of analog
Switching from player to player is useful for observing (axis) and digital (button) values as shown in Figure 5. This
simulation interactions from different perspectives. linkage package has interface code for several I/O devices
All of the graphics classes are derived from Graphic including joysticks and USB-based devices. It has also been
which is derived from Component. Being a component, all extended to interact with other commercial I/O cards and
time-critical code can be written into the updateTC() data acquisition devices.
method and background processing can be written into the Once the device is initialized, a call to the virtual
updateData() method. Sometimes, in real-time system receive() method, defined in the IODevice class, obtains
development, it is desirable to set graphic drawing to an even the latest values from the device. Information about button
lower priority than other background processing. Therefore, transitions can be determined as well as setting deadbands
another method within the Graphic class is defined that on analog inputs.
serves as a placeholder to do actual OpenGL graphics
drawing.
Object

Object

IODevice

Component

Joystick USB
Graphic
Fig. 5: Device Class Hierarchy

The Station class defines how axes and buttons are con-
Page
nected to the models and views of the simulation application.

Display
10. Example Applications
A MIXR software distribution consists of both code to
build new applications, and examples that demonstrates how
GlutDisplay FoxDisplay
to use some of its inherent functional capabilities. For an
extended description of some early military products, see [2].
It details an early version of a fighter cockpit, a Ground
Fig. 4: Graphics Class Hierarchy
Control Station and a Group Command Post.
A sample application included in with the MIXR distri-
bution illustrates basic graphics by drawing a worm that 11. Opportunities
moves around the screen and bounces off the walls. Code For almost four decades, the MIXR codebase has im-
for this example is organized as follows. All mathematical proved and evolved in terms of both structure and provided
calculations for the position, speed and direction of the worm functionality. From its earliest beginning when it was written
are performed in updateTC(). All the work to setup what in C with a developer created OO-like system to mainstream
to draw is done in updateData(). The actual drawing of C++; from a Commodore Amiga 1000 to current generation
the graphic is performed by Graphic’s draw() method. PCs, it has proven itself to be a useful and capable software
Organizing code this way enables the application devel- development product in which a wide range of simulation
oper to determine how to execute the code and to define applications have been created.
 Given that, we don’t consider MIXR ‘finished’ - in [5] D. D. Hodson and R. R. Hill, “The art and science of live, virtual,
fact, we believe with the advent of Modern C++ in 2011 and constructive simulation for test and analysis,” Journal of Defense
Modeling and Simulation: Applications, Methodology, Technology,
(which was the first major language update in nearly 13 vol. 11, no. 2, pp. 77–89, 2014.
years), new opportunities for the codebase are in store. For [6] P. L. Deutsch, “Design reuse and frameworks in the smalltalk-80 sys-
example, C++’s standard library is growing and providing tem,” in Software Reusability, Volume II: Applications and Experience,
T. J. Biggerstaff and A. J. Perlis, Eds. Addison-Wesley, 1989, pp.
cross platform solutions for threading, high resolution clocks 57–71.
and atomic operations of which MIXR needs to fulfill [7] R. E. Johnson and B. Foote, “Designing reusable classes,” Journal of
its requirements; all of these aspects can be leveraged to Object-Oriented Programming, no. 2, pp. 22–35, 1988.
[8] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns:
both reduce the size and complexity of the codebase. Also, Elements of Reusable Object-Oriented Software. Boston, MA, USA:
with adoption or mainstreaming of graphics processing units Addison-Wesley Longman Publishing Co., Inc., 1995.
(GPUs) as included capabilities within a typical PC, other [9] P. A. Laplante, Real-Time Systems Design and Analysis. Wiley-
Interscience, 2004.
opportunities present themselves in terms of optimizing
parallel data centric operations.
Favored programming paradigms and idioms have also
changed; excessive inheritance is discouraged for a number Author Biographies
of reasons including: 1) easy misuse, such as use for reuse DOUGLAS D. HODSON is a Associate Professor
(i.e., ‘implementation inheritance’), 2) to reduce (or loosen) of Computer Engineering at the Air Force Institute of
tightly coupled code and 3) the possible refactoring of in- Technology (AFIT), Wright-Patterson AFB, Ohio USA. He
heritance relationships to satisfy new use case requirements. received a B.S. in Physics from Wright State University
Because MIXR has existed for a considerable time, it in 1985, and both an M.S. in Electro-Optics in 1987
could be argued that its inheritance structure is reasonably and an M.B.A. in 1999 from the University of Dayton.
stable and well defined; although the desire to model and He completed his Ph.D. at the AFIT in 2009. His
simulate different, potentially technologies might alter that research interests include computer engineering, software
view. engineering, real-time distributed simulation, and quantum
communications. He is also a DAGSI scholar and a member
12. Finally of Tau Beta Pi.
MIXR is open-source and freely available; it along
with a set of examples can be downloaded from DAVID P. GEHL is employed by L-3 Communications,
www.mixr-platform.org. Link Simulation & Training Division. He has over 45
years of experience in human-in-the-loop simulation and
References training for human factors engineering research including
[1] D. D. Hodson, “Military simulation: A ubiquitous future,” in 2017 extensive knowledge in pilot/operator-vehicle interfaces, air-
Winter Simulation Conference (WSC), 2017, pp. 4024–4025. craft system models (aerodynamics, radars, weapon delivery,
[2] D. D. Hodson, D. P. Gehl, and R. O. Baldwin, “Building distributed navigation, visual systems, etc.), and real-time system devel-
simulations utilizing the EAAGLES framework,” Interservice/Industry
Training, Simulation and Education Conference (I/ITSEC), 2006. opment. Previously, he served as the primary architect for the
[3] “MIXR Platform website,” http://www.mixr-platform.org, accessed: Extensible Architecture for the Analysis and Generation of
2018-06-23. Linked Simulations (EAAGLES) simulation framework. He
[4] D. M. Rao, D. D. Hodson, M. S. Jr, C. B. Johnson, P. Kidambi, and
S. Narayanan, Design & Implementation of Virtual and Constructive received a B.S. in Computer Science in 1979 and a M.S. in
Simulations Using O PEN E AAGLES. Linus Publications, 2009. Systems Engineering in 1986 from Wright State University.