Simulation: Transactions of the Society for

Modeling and Simulation International

Design, Implementation, and Evaluation XX(X):1–15
c The Author(s) 2015
of an XG-PON Module for ns-3 Network Reprints and permission:
sagepub.co.uk/journalsPermissions.nav

Simulator DOI: 10.1177/ToBeAssigned

www.sagepub.com/

Jerome A. Arokkiam1,2 , Pedro Alvarez2 , Xiuchao Wu1 , Kenneth N. Brown1 , Cormac J.

Sreenan1 , Marco Ruffini2 , Nicola Marchetti2 , Linda Doyle2 , David Payne2

Abstract
10-Gigabit-capable Passive Optical Network (XG-PON), one of the latest standards of optical access networks, is
regarded as one of the key technologies for future Internet access networks. This paper presents the design and
evaluation of our XG-PON module for the ns-3 network simulator. This module is designed and implemented with the
aim to provide a standards-compliant, configurable, and extensible module that can simulate XG-PON with reasonable
speed and support a wide range of research topics. These include analysing and improving the performance of XG-
PON, studying the interactions between XG-PON and the upper-layer protocols, and investigating its integration with
various wireless networks. In this paper, we discuss its design principles, describe the implementation details, and
present an extensive evaluation on both functionality and performance.

Keywords
XG-PON, ns-3, simulation, modelling, performance, evaluation

Introduction improving user experience 16 . In addition, XG-PON has been

proposed for Fibre To The Cell, in which XG-PON acts
Passive Optical Networks (PON) offer a highly-efficient and as the backhaul for multiple base stations of a cellular
cost-effective technology for broadband access, based on network 18 . Under this scenario, it is also very valuable to
a model of point-to-multipoint distribution using passive study its integration with various wireless networks (LTE 3 ,
components. WiMAX 15 , etc.) for providing high speed mobile Internet
PONs gained their popularity with the standardisation access.
of gigabit-capable Ethernet PON (EPON) by Ethernet in Due to its very recent standard definition, XG-PON is
the First Mile (EFM) task force 14 of the Institute of still in its early stages of deployment - thus the above
Electrical and Electronics Engineers (IEEE) in 2004; Full research topics can only be studied through simulation as
Service Access Network (FSAN) group of the International it is too expensive and highly complex to set-up a full-
Telecommunications Union (ITU) also introduced its version fledged and fine-grained XG-PON test-bed. In this paper,
of gigabit-capable PON standard with Gigabit-capable PON we present an XG-PON module for the state-of-the-art
(GPON) 17 shortly after, in 2005. Since then, FTTx (Fibre ns-3 1 network simulator. Our XG-PON module is based
To The Home/Building/Curb, etc.) networks based on on a series of G.987 Recommendations from the FSAN
standardised PON technologies, have been widely deployed group of ITU. These recommendations mainly define the
in many countries around the globe, including in the US, specifications of Physical Media Dependent (PMD) and
Korea and Japan. Transmission Convergence (TC) layers of XG-PON. To
10-Gigabit-capable Passive Optical Network (XG-PON) study the above research topics with reasonable simulation
is the new standard released by the FSAN. XG-PON speed, the optical distribution network and the operations
improves GPON in numerous aspects; notable changes of the physical layer are simplified significantly. This XG-
include increasing the default downstream data rate to 10 PON module focuses on the issues of TC layer, such
Gb/s, while increasing the upstream data rate to 2.5 or 10 as frame structure, resource allocation, Quality of Service
Gb/s; the minimum logical split is increased to 256 (from 64 (QoS) management, and Dynamic Bandwidth Assignment
in GPON) and the physical reach extended up to 60 Km. (DBA) algorithms for the upstream wavelength. During the
Since XG-PON could pave the way for many bandwidth-
intensive applications (IPTV, Video On Demand, Video
Conference, etc.), it is very important to study the 1 Dept. of Computer Science, University College Cork (UCC), Ireland
performance issues arising with the deployment of XG- 2 Trinity College Dublin (TCD), Ireland
PON. For instance, it is valuable to study the impacts on
the performance of XG-PON, when the propagation delay Corresponding author:
is much longer than that of the current PON networks 30 . Jerome A. Arokkiam, Room 2.05, CONNECT Centre for Future
It is also important to investigate the interactions between Networks, Dunlop Oriel House, Trinity College Dublin, Dublin 2, Ireland
XG-PON and the upper-layer protocols (TCP 28 , etc.) for Email: [email protected]

Prepared using sagej.cls [Version: 2015/06/09 v1.01]

2 Simulation: Transactions of the Society for Modeling and Simulation International XX(X)

first kind, the active elements, includes the OLT (Optical

Line Terminal) in central office and ONUs at/near customer
premises; the second kind, a passive element, includes the
optical splitters/jointers, which connect OLT and ONUs,
without the need for any other active electronic equipment
in the middle, thereby reducing capex and opex significantly.
In a classical TDMA (Time Division Multiple Access)
based PON network, downstream traffic is broadcast by the
OLT to all ONUs that share the same optical fibre and
encryption is used to prevent eavesdropping. Upstream traffic
from ONUs is interleaved by OLT to use the optical fibre in
a TDMA-like manner. Since ONUs normally have different
Figure 1. An illustration of PON as part of regional network distances to the OLT, the data bursts from these ONUs
must be scheduled carefully to provide a collision-free and
efficient upstream data communication. To accommodate
design and implementation of this module, we have also paid the dynamics in bandwidth demands from users and exploit
a lot of attention to its extensibility and reconfigurability. the gain of statistical multiplexing, dynamic bandwidth
Since this XG-PON module needs to simulate a 10Gb/s allocation (DBA) algorithms are normally used to provision
network and hundreds of Optical Network Units (ONU), its the shared upstream bandwidth. More specifically, ONUs
performance (the speed of simulation and the overhead of will report their buffer occupancy to OLT, which will then
memory) has also been given a high priority when designing allocate the upstream bandwidth to ONUs based on their
and implementing these components. bandwidth demands and their Service Level Agreement
This is the first standards-compliant XG-PON module (SLA).
designed for an open-source simulation platform. It allows Some standards have been developed for PONs by both
reasonable simulation speed, even when the XG-PON EFM of IEEE (EPON) and FSAN of ITU-T (GPON). EPON
supports more than 1000 ONUs and is simulated at is designed for carrying Ethernet frames and GPON can
9.6Gb/s in the downstream (validated by our experiments carry various traffic, such as Ethernet frames and ATM
in ’Evaluation results’ section). This simulation platform cells, through encapsulation. Although EPON and GPON
enables extensive study of the performance issues that would have different frame structures, they share the same network
arise with practical deployment of XG-PON, and allows architecture and employs the same data communication
applications of and modifications to XG-PON, to be explored principles. One important difference between EPON and
in a common and open-source simulation environment. Our GPON is that GPON provides granular QoS definitions,
XG-PON module is built completely in C++ with 72 classes while EPON requires adherence to the simple Multi Point
and approximately 22,000 lines of code. This code is under Control Protocol (MPCP); hence GPON is equipped with a
the GNU General Public License and can be downloaded well-defined QoS framework, a feature highly preferred by
through sourceforge 2 . More related information can be the Internet Service Providers (ISP) for comprehensive traffic
found on our websites (CTVR1 and MISL2 ). management in the access and last-mile networks. XG-PON
Organisation of the paper is such that, firstly PON, ns-3, standard, the successor of GPON and the prime focus of our
and related works are briefly introduced. Then the details work, is given in Section ’XG-PON details’, in detail.
of XG-PON are presented. The design principles and key
decisions are discussed thereafter, followed by the details ns-3 network simulator
on the trade-offs made in terms of simulation accuracy and ns-3 1 is a state of the art open-source network simulator.
speed. After that more specific details of the design and Based on many lessons from the well-known ns-2
implementation of our XG-PON module for ns-3 are given. simulator 23 , ns-3 is written from the scratch to be a
Evaluation results on both functionality and performance are completely new network simulator, with no backward-
then demonstrated before concluding the paper along with compatibility with the former. It is a discrete-event network
directions for future work. simulator with the simulation core and modules of ns-3
implemented in C++;it is built as a library which may be
Background and related work statically or dynamically linked to a C++ main program. ns-3
also exports nearly all of its API to Python, allowing Python
Passive Optical Network (PON) programs to import an ns3 module in much the same way as
Compared to copper, optical fibre can provide higher the ns-3 library is linked by executables in C++.
bandwidth over a longer distance; but deployment of the ns-3 has many attractive features, such as high emphasis
latter in access networks is severely hindered by its heavy on conformance to real networks, good support for
capital (capex) and operational (opex) expenditure. Hence virtualisation and testbeds, a novel attribute system for
PON was specifically designed to reduce overall cost by configuring simulation parameters, automatic memory
sharing fibre and electronic equipment among multiple users. management, a configurable tracing system and the ability
This concept of shared optical medium in PON heavily to incorporate existing open-source modules through simple
improved the economic viability of FTTx. APIs 13 . It has also been reported that ns-3 performs much
As shown in Figure 1, PON is a point-to-multipoint better than other simulators in terms of simulation speed and
fibre network and there are two kinds of equipment: the memory overhead 32 . The first release of ns-3 was made in

Prepared using sagej.cls

Arokkiam, Alvarez 3

June 2008 with support for a number of modules including

"5#
CSMA, Point-to-Point, WiFi (IEEE 802.11), TCP, UDP and 4 "&2 &! 5
IPv4.
In the last few years, many new modules have been
"&2 & "
developed and added into ns-3, such as WiMAX module
&! 5
from Inria 8 and LTE module from CTTC 27 . Thus,
implementing an XG-PON module in ns-3 offers an " " 9&
established research platform for studying the issues that
my occur in a deployment of XG-PON itself as well as in
:26 ! "6
interfacing XG-PON with other network technologies.
& ' +) &!!! ; )

! ":
Related work
Although simulation has been used to study PON in the
past, such work cannot be used directly or extended easily ": ":
to study the performance issues arising with the deployment & ' +
of XG-PON. Song H, et al. developed their own simulator 1
to study dynamic bandwidth assignment (DBA) algorithms
when the physical reach is much longer than the current PON
Figure 2. XG-PON common functions 18
networks 30 . This simulator has limited functions and there ! " #
is no Internet protocol (IP) stack, which is needed to study
many research topics. of XG-PON with a larger number of users, symmetric data
EPON, GPON and XG-PON 4;7;9;24;26 have all been rate (10 Gb/s in both upstream and downstream), and longer
implemented in OPNET 31 . However, these models tend to be reach (100+ km) 25;29 . The aim of our LR-PON research
abstracted away from the relevant standards definitions, and group is to initially build LR-PON from the XG-PON
thus do not represent issues that may occur at lower layers - standard, while identifying the required modifications and
particularly the Physical (PHY) and Medium Access Control improvements.
(MAC) layers - of a large-scale XG-PON deployment. They The initial design and implementation of our XG-PON
were also implemented at bandwidths lower than 1 Gb/s. module had been reported in the 4th Workshop on ns-3 33 .
Finally, as OPNET is not an open-source simulator, there is Since then this module has been significantly redesigned
limited public access to the models, and we cannot adapt the and extensively evaluated. In this paper, we will discuss the
OPNET’s core to simulate a 10Gb/s XG-PON network with design principles of the latest XG-PON module, describe its
a reasonable simulation speed. implementation details and present an extensive evaluation
On the other hand, a simple EPON module has been of the same.
developed for OMNeT++ 6 and the code is available to
the public, thereby providing a better simulation platform
to understand communication in and via a PON network XG-PON details
with conformance for standardisation. However, due to The XG-PON standard has many similarities with GPON,
the differences between EPON and GPON discussed in such as its TDMA scheme used to share the medium, the
’Background’ section, this EPON code will not be very mechanism to provide QoS, and the DBA scheme used for
helpful to implement a GPON or XG-PON module for the upstream bandwidth assignment. However, some changes
OMNeT++, so that communication through (X)G-PON can are required in order to support a larger number of users,
be studied in detail. higher data rate, and extended physical reach. In this section,
we will present the details of XG-PON.
Summary
Hence, there is a requirement to design and implement Overview of XG-PON
a standard-compliant XG-PON module from scratch for A series of recommendations has been released by FSAN
the state-of-the-art and open-source ns-3 simulator. With of ITU-T for XG-PON. ITU-T G.987 explains several
such an XG-PON module, we can simulate XG-PON at its important concepts of XG-PON; ITU-T G.987.1 presents
actual data rates with reasonably swift simulation speeds; the general requirements, services to be supported, hardware
such a scenario would pave the way for researchers to specifications and protocol stack of XG-PON as well as the
identify and validate issues that may occur in a large scale network migration from and coexistence of XG-PON with
XG-PON deployment. With the more realistic IP stack of GPON; ITU-T G.987.2 focuses on issues of the physical
ns-3, we can also study the performance experienced by media dependent (PMD) layer, such as the used wavelength
users/applications in XG-PON networks. With the existing and the supported data rates whereas ITU-T G.987.3 presents
ns-3 modules for various wireless networks (WiFi, WiMAX, the details of transmission convergence (TC) layer for
LTE, etc.), we can study the integration between XG-PON XG-PON. Besides the protocols for data communication,
and wireless networks, which is the trend of the future it also covers QoS management and Dynamic Bandwidth
Internet access networks. Assignment (DBA) scheme for the upstream wavelength.
Besides XG-PON, we can also extend this XG-PON Another related recommendation is ITU-T G.988, which
module to study Long-Reach PON (LR-PON), an evolution specifies ONU management and control interface (OMCI)

Prepared using sagej.cls

4 Simulation: Transactions of the Society for Modeling and Simulation International XX(X)

for both GPON and XG-PON. Figure 2 illustrates XG-PON transmission mechanisms of XG-PON. It does this by map-
common functions and the recommendations in which they ping upper layer traffic to the corresponding connections,
are specified. encapsulating/decapsulating data, segmenting/reassembling
service data units (SDU) when necessary and inserting
Network architecture padding when there is insufficient data to fill an XGTC
frame. If needed, it is also this sublayer’s responsibility to
XG-PON has been proposed for various deployment encrypt/decrypt SDUs.
scenarios to serve different customers, such as residential, When mapping upper layer data to and from the
business, and cell site. To serve these customers, XG- connections of XGTC layer, while the OLT will maintain
PON lists the services to be provided, such as Telephony, information pertaining to all the connections, an ONU
high speed Internet access, mobile backhaul, etc. XG-PON will only maintain the connections that it owns. When the
also introduces many ONU variants that provide different upper layer has something to transmit, it is also the service
functions and interfaces. In summary, XG-PON has been adaptation sublayer’s responsibility to select the connections
well standardized for providing full services to various users to be served according to their QoS parameters. When a
using optical network. As for optical distribution network, connection is scheduled to be served, the service adaptation
XG-PON can be deployed as a classical PON with its sublayer will then get data from its queue and insert an
reach up to 60km. To support longer physical reach, active XGEM header to create an XGEM frame. The XGEM
reach extenders (RE) can be applied before and/or after header will contain an XGEM Port-Id and some other
the passive splitter to connect multiple passive segments information related to segmentation, padding, encryption,
belonging to a single XG-PON network. These REs can be etc. When receiving an XGEM frame, the service adaptation
optical amplifiers or optical- electrical-optical regenerators sublayer will get the XGEM Port-Id from the XGEM header.
that could fulfil the necessary optical link budget. If the corresponding connection exists in the connections
maintained by the OLT/ONU, this sublayer will carry out
PMD Layer reassembly (if necessary) and pass the data to upper layer.
Otherwise, this XGEM frame will be discarded.
There are two flavours of XG-PONs based on the upstream
line rate: XG-PON1, featuring 2.5 Gb/s and XG-PON2,
Framing Sublayer: In XG-PON, the OLT will send
featuring 10 Gb/s in the upstream. The downstream line
downstream XGTC frames every 125 µs, to broadcast
rate is 10 Gb/s in both XG-PON1 and XG-PON2. ITU-
traffic to all ONUs. In the upstream, ONUs send variable
T G.987.2 focuses on the PMD layer for XG-PON1. XG-
length XGTC bursts to the OLT for their upstream traffic.
PON2 has not been standardized yet. In XG-PON1, the
The length and start time of these upstream bursts are
used wavelengths are 1575-1580nm (downstream) and 1260-
determined by the OLT through a DBA algorithm. The
1280nm (upstream). The exact downstream line rate is
framing sublayer is responsible to generate and parse these
9.95328 Gb/s and the upstream one is 2.48832 Gb/s. For
XGTC frames/bursts. When generating a downstream XGTC
line coding, NRZ (Non-Return to Zero) is used for both
frame at the OLT, the framing sublayer gets XGEM frames
directions. ITU-T G.987.2 also specifies the requirements for
from service adaptation sublayer and joins them together into
hardware, such as optical fibre, transmitter/receiver, etc.
an XGTC payload. To create an upstream XGTC burst at
ONU side, the framing sublayer may create multiple XGTC
Transmission Convergence Layer payloads, where each payload will carry XGEM frames
The XG-PON Transmission Convergence (XGTC) layer from a single T-CONT. When parsing an XGTC frame/burst,
contains the definition for the MAC protocol of XG-PON. the framing sublayer will send its payloads to the service
The XGTC layer maintains logical connections between adaptation sublayer for further processing.
OLT and each ONU, in pairs, in order to carry one In the header of the upstream XGTC burst generated
downstream and one upstream traffic between the OLT and by an ONU, there might be queue occupancy reports for
a corresponding ONU. Each connection is identified by a the T-CONTs of this ONU. For each downstream XGTC
unique XG-PON encapsulation method (XGEM) Port-Id, frame, its header contains a BWmap , which instructs ONUs
which, while ensuring that packets are sent to the correct to share the upstream wavelength in a TDMA-like manner.
ONU, associates every connection to a certain Quality of More specifically, BWmap specifies the size of bandwidth
Service (QoS) agreement. Note that one connection can only allocations for T-CONTs, the used burst profile (the length
be configured to carry either a downstream or upstream of preamble, the length of delimiter, forward error correction
traffic. To reduce the overhead of the DBA scheme, upstream or not, etc.), and the time to start to transmit. Since the
bandwidth is allocated to groups of connections belonging to OLT-ONU physical distance could be quite different for
a single ONU. These groups are designated as Transmission ONUs, each ONU should adjust the start time for avoiding
Containers (T-CONT) and each group/T-CONT is identified collision in the upstream direction. Note that when an ONU
by a unique identifier, the Alloc-Id. is activated, the ranging procedure is carried out between the
XGTC comprises of three sublayers, with service OLT and this ONU to determine how to adjust the start time
adaptation sublayer on the top of the protocol stack, followed of its upstream bursts.
by framing sublayer and PHY adaptation sublayer. Figure 3 illustrates the time-lines in XG-PON. The OLT
and ONUs have a common view of the logical one-way
Service Adaptation Sublayer: The service adaptation delay of the optical distribution network (the largest one-way
sublayer is responsible to adapt the upper layer traffic to the propagation delay plus various processing delays) and each

Prepared using sagej.cls

Arokkiam, Alvarez 5

ONU uses its own equalization delay (EqD calculated in we made to realise a simulation-friendly XG-PON module in
ranging procedure) to avoid collisions in upstream direction. ns-3.
In the header of an upstream XGTC burst, the
ONU can send one PLOAM (Physical Layer Operations, Design principles
Administration and Maintenance) message to the OLT. Standard compliance: The ultimate goal of our research
As for a downstream XGTC frame, the OLT can send is to improve the performance issues associated with a
multiple PLOAM messages to multiple ONUs. Through deployed XG-PON network. Hence, when our simulation
exchanging PLOAM messages, many XGTC functions (key model has a close resemblance to a real world XG-
management, ONU power management etc.) can be fulfilled. PON deployment, we will be able to identify more
realistic problems and provide solutions that can directly
PHY Adaptation Sublayer: PHY adaptation sublayer be applied in deployments. Hence, we will follow G.987
interacts with PMD layer directly. Its main functions are Recommendations from the FSAN group of ITU when
forward error correction (FEC), scrambling, and frame designing our XG-PON module.
delineation through a Physical Synchronization Block
(PSB). In the downstream, the PHY adaptation sublayer will Simplicity: Considering that XG-PON is quite a complex
get an XGTC frame to create a PHY frame. These PHY standard, it will take a very long time to simulate a high-
frames are sent continuously every 125 µs. In the upstream, precision PON network, including all the layers, ranging
the PHY adaptation sublayer will get the XGTC burst and from physical to network management. For instance, the
create a PHY burst. These PHY bursts have variable length document for ONU Management and Control Interface
due to the variable-length XGTC bursts. In the PHY burst, (G.988) is more than 500 pages. Hence, only the key
the PSB is determined by a burst profile selected by the components that are necessary to create a simple, yet
OLT (through the BWmap ) among the burst profiles, that are working XG-PON network, are given priority in our module
configured through PLOAM messages. design; less important definitions that require extensive and
complex implementation are added as stub classes for future
extensions. For instance, since we are mainly interested in
Scheduling and DBA XGTC layer and upper layer issues, we can simulate the
To decide the data to be transmitted in a downstream physical layer in a very simple way. We can assume that
XGTC frame, a downstream scheduler is used by the OLT. power budget for the optical distribution network has been
Based on QoS parameters and service history of these satisfied through various techniques. The reach extenders and
downstream connections, the downstream scheduler will passive optical splitters/jointers need not be simulated. The
decide the connections to be served and the amount of data channel, that simulates the optical distribution network of
to be transmitted for each of them. XG-PON, can simply pass downstream frames to all ONUs
As for the upstream scheduling, the OLT uses a DBA and pass upstream bursts to the OLT. As for Forward Error
algorithm to allocate the upstream bandwidth to T-CONTs. Correction (FEC), instead of the algorithm itself, we can
The DBA algorithm makes decisions based on queue simulate only its effect, i.e., the bandwidth overhead and
occupancy reports, QoS parameters, and service history the much lower packet corruption rate. Further details of
of these T-CONTs. The DBA algorithm needs to select omissions of less important features are explained in ’XG-
the T-CONTs to be served, reserve a short gap time PON Details’ section, as we describe our module in detail.
between the consecutive XGTC bursts for tolerating clock Extensibility: When designing the XG-PON module for ns-
synchronization errors, determine the size of each bandwidth 3, we should also consider its extensibility since many other
allocation, and calculate the start time for each bandwidth research topics might also be studied using this module.
allocation. These decisions are broadcast to ONUs through Hence, the extensibility is very important. When designing
BWmap . Since the upstream bandwidth is allocated to T- the class architecture of the XG-PON module, we followed
CONTs and each T-CONT may have multiple upstream the standard principles of Object Oriented Programming
connections, the ONU also needs an upstream scheduler to (OOP) such as using abstract classes to design easily
determine the upstream connections to be served during one extensible interfaces. Such abstraction, while providing full
transmission opportunity assigned to one T-CONT. implementation for the key components of XG-PON, also
These scheduling algorithms, especially the DBA algo- pave way for quick integration of future extensions. For
rithm, are very important to network performance and QoS instance, when designing the class interface for the channel
management. To allow competition and encourage research, that simulates the optical distribution network of XG-PON,
these algorithms were intentionally left out of the standard. we should enable researchers to specify the tree structure of
Indeed, it has been a very hot topic to study DBA algorithms fibres, reach extenders, and splitters. When adding an ONU,
for EPON and GPON 12;22;30 . Hence, there should be many they can also specify the splitter that it will be attached to and
research opportunities for XG-PON too. the physical distance between them. With this interface, it is
possible to simulate the optical signal propagation and the
possible packet corruption. However, for the current phase,
Design principles and key decisions
we can let the channel store a list of ONUs and pass the
When designing and implementing an XG-PON module, downstream frames to all of them (without any error)3 . Since
many issues must be considered and several trade-offs must DBA is a hot research topic, the classes for DBA should be
be made when the goals conflict with each other. This section well designed to allow the easy implementation of various
presents the design principles followed by the key decisions DBA algorithms.

Prepared using sagej.cls

6 Simulation: Transactions of the Society for Modeling and Simulation International XX(X)

Start of the DS PHY frame Start of US PHY frame

in OLT's view in OLT's view

Teqd StartTime

PSBd PHY frame content PSBd PHY frame content PSBd PHY frame content

BW Grant
StartTime = s

PSBu PHY burst content

T1577,i RspTimei EqDi StartTime T1270,i

Start of
DS PHY frame
in ONU's view
G.987.3(10)_F13-5

18
Figure 3. Time-line in XG-PON

Configurability: In a single simulated XG-PON network, When the function is called frequently and some of its
there could be thousands of nodes, such as the OLT, hundreds parameters are smart pointers, we should replace them with
of ONUs, and hundreds of data traffic generators/sinks in the reference of that smart pointer. Second, since C++ allows
core networks. Many nodes will also be attached to ONUs one class to override its new and delete operators, we should
through various networks and act as traffic generators/sinks. exploit this feature for data structures that are created and
Thus, while exporting as many configurable parameters as destroyed dynamically and frequently. By overriding these
possible, we should also provide default parameters for most two operators, a pool of pre-allocated memory is used for
of them. Other methods, such as using helpers, should also small and dynamic memory requirements, thereby avoiding
be considered to ease the researcher’s task of configuring the frequent calls to malloc/free and saving CPU cycles as a
XG-PON network for simulation. result. Third, when we select the data structure for a sequence
of objects, vector should be considered due to its efficiency.
Simulation speed: Since the XG-PON to be simulated is a However, when too many objects are added into one vector,
10Gbps network, simulation speed must be considered at all reallocation may occur and the simulation can be slowed
times. A module, that can simulate XG-PON accurately (but down significantly. Thus, we should reserve enough memory
very slowly), is useless for our research in which extensive if the largest vector size can be pre-determined. Otherwise,
simulations are needed. Saving CPU cycles and memory dequeue should be considered as the container.
are primary focus here. For instance, when XG-PON is Additionally, virtual functions and inheritance are very
fully loaded in downstream and the size of each packet is attractive mechanisms in OOP that is exploited in designing
1KBytes, the simulator needs to process around one million classes for our XG-PON module. However, depending on the
packets per second. Since XG-PON could have hundreds behaviour of entities in XG-PON module, there are instances
of ONUs, the simulator must run the procedure used by when these mechanisms would result in cumbersome parent
ONU to judge whether it is the destination of one XGEM classes, requiring the use of downcast instead. Downcast,
frame one billion times per second. This procedure must though, is a very unsafe mechanism that consumes a lot
be implemented with high-efficiency. Straightforwardly, we of CPU cycles at code execution. At such instances, we
can add one vector at each ONU whose index is XGEM opted for designing classes individually, that is, without
Port-Id. When configuring XGEM Ports for this ONU, this using inheritance, in order to expedite the overall simulation
vector can be marked correspondingly. Consequently, this speed. For example, for each function of XGPON (DBA,
vector can be used to filter out the traffic for this ONU etc.), there should be two classes designed for OLT and
quickly. However, XGEM Port-Id is a 16-bit number and this ONU, respectively, and it is attractive to let them inherit from
vector can consume a lot of memory when the number of the same parent. However, since the logic at OLT is totally
ONUs is large. Hash map, in which GEM Port-Id acts as the different than that at ONU, the amount of reused code is
key, suffers from the same memory issue. In our XG-PON limited, the interface of the parent becomes more complex,
module, we impose some simple relationship among XGEM and simulation speed is slowed down. Thus, these classes are
Port-Id, Alloc-Id, ONU-ID, and IP address of the computer designed independently and the inheritance is not used.
that this XGEM Port belongs to. Consequently, we consume
a small amount of memory in total and achieve O(1) time
complexity when mapping IP address/XGEM Port-Id to the Key decisions
corresponding data structure. Below are several key decisions we made, when designing
During the implementation, many useful features of C++ and implementing our XG-PON module in ns-3. Corre-
language should be exploited and black-holes in CPU sponding implementation details will be presented in section
cycles must be avoided. First, we should pass parameters ’XG-PON module for ns-3’.
by reference whenever it is possible; const reference is
preferred. We should also know that the smart pointer Stand-alone simulation: Since XG-PON is a 10Gbps
provided by ns-3 is fundamentally an object, though small. network with hundreds of ONUs, it would be very

Prepared using sagej.cls

Arokkiam, Alvarez 7

attractive to use distributed simulation to speed up XG- Instead, we can simply add all ONUs to the network before
PON simulation. Yet, it is necessary to have a robust starting the simulation through a helper class. The ranging
cluster or multi-core computer to realise the advantages of procedure that uses PLOAM messages to measure the one-
MPI for a high-speed network such as XG-PON. On the way propagation delay of each ONU, can be simplified by
other hand, although ns-3 supports distributed simulation setting the same delay values to both at the OLT and at
through standard Message Passing Interface (MPI), this the corresponding ONU at the time of configuring (before
feature only works for point-to-point links. But XG-PON is running the simulation of) the XG-PON network. In XG-
fundamentally a point-to-multipoint network and extensive PON, XGEM Port and T-CONT configuration is carried out
works are necessary to enable distributed simulation. These through OMCI (ONU Management and Control Interface:
works include verifying MPI synchronisation against the G.988). For simplifying the dynamic configuration of XGEM
simulation speed of XG-PON, obtaining reasonable speed Port and T-CONT by OMCI, we will configure all XGEM
in simulating an XG-PON with the added overhead of Ports and T-CONTs before starting the simulation through a
(de)packetization at cluster node interfaces, creating the helper class. Relevant stub classes will be designed in future
topology of XG-PON in each core in a cluster as per MPI extensions, for a detailed implementation of PLOAM and
requirement and allocating ONUs to different cores. As OMCI channels.
we would rather prefer an XG-PON module which is both
simple and used extensively among the research community, Simple Optical Distribution Network (ODN) and reliable
in this phase, we decided to model our XG-PON module data transfer: In XG-PON, the optical distribution network
as a stand-alone simulator, without support for MPI. That is quite complex and is comprised of many optical fibres,
is our XG-PON module uses only one core even when one splitters/jointers, and reach extenders. However, we model
computer has multiple processors or cores. In the future, the optical distribution network as a simple channel and
distributed simulation may be considered. we only simulate the propagation delay and line rates. We
assume that the link power budget has been ensured through
Packet-level simulation: Since ns-3 is an event-driven various techniques (reach extenders, etc.) and the laser
simulator, we have the options of simulating our XG-PON receiver can work well. Thus, we will not simulate optical
module at byte, packet or flow level. However, due to the signal propagation (wavelength-dependent) and assume that
high bandwidth of XG-PON (10Gbps) and the moderate all downstream frames and upstream bursts can arrive to their
frequency of state-of-the-art processors (mere several GHz), recipients correctly. In other words, transmission errors are
it is not feasible to simulate the extensive details of data not simulated in our XG-PON module. This is reasonable
transfer in XG-PON at byte (or bit) level. On the other since FEC is normally applied to rectify transmission errors.
hand it’s too complex to model both XG-PON and TCP/IP Based on this decision, Cyclic Redundancy Check (CRC)
protocol stack in a flow-level simulation with the added issue and Header Error Correction (HEC) are not executed in the
of not being able to study the potential subtle interactions simulation.
between TCP/IP and XG-PON. Thus, we decided to In the future, transmission errors may be simulated at
simulate our XG-PON module at packet-level, to simulate the recipient, by dropping an entire downstream frame or
as many details of data transfer as possible, while allowing upstream burst with a distance-dependent probability. That
sufficient interaction between TCP/IP and XG-PON layers. is, an occurrence of transmission error, with no frame
Furthermore, when passing traffic between OLT and ONU, delineation, should be able to prevent a recipient from
all XGEM frames in the downstream frame or upstream decoding an entire frame/burst when FEC fails to identify
burst should be handled together; as a result the number of the frame/burst with sufficient accuracy.
simulation events can be reduced significantly. Due to the
short XG-PON frame size (125µs), the upper layer protocols Serialization avoidance and meta-data in data structures:
won’t be affected if we keep the order of XGEM frames in Since this XG-PON module is designed for stand-alone
the downstream frame or the upstream burst. Based on this simulation,(de)serialization is unnecessary and should be
decision, many physical layer operations, such as line coding avoided4 . Thus we defined our own data structure in this XG-
and Forward Error Correction, will not be implemented PON module, instead of the Packet class from ns-3. This is
in this module. However, the bandwidth overhead of FEC because, though Packet supports easy insertion/extraction of
must be considered. Payload encryption/decryption will not headers, fragmentation and reassembly, when XGEM frame
be implemented as well, though the logic used for key header is added into Packet, the header is serialized into
management will be implemented for future extensions. one byte array. When one XGEM frame is received, the
We also assume that all the sub-modules in our XG-PON recipient needs to extract the XGEM frame header from
module complete their execution at the required point in time the byte array, i.e., create a new data structure and carry
within every downstream/upstream frame, regardless of the out de-serialization. Considering that one XGEM frame in
complexity of the sub-modules. The total run time of the downstream direction will be processed by hundreds of
simulation however depends on the scale and complexity of ONUs, the above operations may consume too much CPU.
the configured XG-PON network. To solve this issue, our data structure is designed to have one
smart pointer to the header and another to the corresponding
XG-PON in operation: Since we are mainly interested in SDU (an instance of Packet). Hence, the header can be
the performance issues of an XG-PON network in operation, directly extracted from our data structure.
many aspects of XG-PON can be simplified. For instance, the Another observation is that some meta-data can be added
activation procedure that uses PLOAM messages to add each into data structures for various purposes since they are
ONU to an operational XG-PON need not be implemented. exchanged between OLT and ONU as objects (instead of a

Prepared using sagej.cls

8 Simulation: Transactions of the Society for Modeling and Simulation International XX(X)

byte array). For instance, all the broadcast XGEM frames in

downstream need be checked by all ONUs, to decide whether
to accept the relevant frames or to drop them. However,
our observations indicate that the traffic in one downstream
might belong only to a few ONUs, due to the small size of
the downstream frames and the bursty nature of bandwidth
allocation. Thus, to speed up the simulation significantly,
we have added a bitmap to each downstream XGEM frame
merely to indicate whether one ONU needs to check the
particular frame, so that not every XGEM is checked by each
ONU in detail for acceptance or dropping.
Extensible DBA, scheduling, and queue schemes: DBA
engines, scheduling algorithms and the queue used by each
XGEM Port at the sender side are very important to the
performance of the whole network and the QoS experienced
by user traffic. Thus corresponding classes were designed
carefully to support future extensions. Creating abstract
classes for these schemes, would allow new algorithms to be
implemented easily, by redefining only a few key functions.
Additional details of DBA classes implemented in our XG-
PON module will be discussed in Section ’XG-PON module Figure 4. Functional block diagram of our XG-PON module
for ns-3’, under ’Major modules’.

fragmentation will be carried out by XGEM Engine if an

XG-PON module for ns-3
SDU is too long for the current transmission opportunity.
Our aim, in developing the XG-PON module for ns-3, is to XGEM Engine is also responsible to encrypt these SDUs to
provide a standard compliant, configurable, and extensible avoid eavesdropping. The keys used for data encryption are
module that can simulate an XG-PON at reasonable negotiated through PLOAM messages and are maintained by
simulation speed with support for a wide range of research Ploam Engine.
topics. Hence, the below subsections explain the functional To construct the XGTC header of the frame, the DBA
blocks as we modelled them in our XG-PON module, Engine is used to generate BWmap that tells ONUs
followed by its implementation details. how to share the upstream wavelength. DBA Engine
makes decisions based on queue occupancy reports, QoS
Functional blocks of our XG-PON module parameters, and service history of T-CONTs. PLOAM
Figure 4 explains the data transmission paths in both messages in the header, are generated by Ploam Engine. The
downstream (green arrow) and upstream (blue arrow) downstream frame is sent to the ODN after passing through
directions, as we modelled in our in our XG-PON module, PHY Adaptation Engine and PMD Engine.
based on the XG-PON standard 18 and our design principles.
Downstream traffic on ONU side: Shown by the green
Downstream traffic on OLT side: As shown by the green arrows in ONU side of Figure 4, when a downstream PHY
arrows in OLT side of Figure 4, when an SDU is received frame arrives at an ONU, it will pass through PMD Engine
from the upper layer (eg: IP), the SDU should first be mapped and PHY Adaptation Engine which will remove the physical-
to the corresponding connection (XGEM Port) based on the layer overhead. The Framing Engine is then responsible to
destination IP address and put into the queue for transmitting parse the resulting downstream XGTC frame.
in the future. Thus, there must be an algorithm for mapping The PLOAM messages from the XGTC header will be
the IP address to an XGEM Port-Id. given to the Ploam Engine, which will process the messages
Since the OLT needs to broadcast the downstream XGTC related with this ONU. The DBA Engine is responsible to
frames every 125 µs, it will periodically ask the OLT’s process BWmap in the header, i.e., schedule its upstream
Framing Engine to generate an XGTC frame. This engine XGTC bursts if required by this BWmap .
will first generate an XGTC header since the available As for the payload, the XGEM frames are passed
space for data in the frame depends on the size of the to XGEM Engine. Based on the list of its connections
XGTC header. For the payload of a downstream XGTC maintained by Downstream Connection Manager, the
frame, the Framing Engine resorts to the XGEM Engine XGEM frames for this ONU are first extracted. XGEM
to get an XGTC payload. This payload is comprised of Engine then carries out decapsulation, decryption, and
concatenated XGEM frames that occupy all the available reassembly (if needed)5 . The received SDUs are then sent
space. As for the SDUs to be encapsulated and transmitted, to the upper layer (eg: IP).
the XGEM Engine lets the Downstream Scheduler decide the
connections to be served. This scheduler makes decisions Upstream traffic on ONU side: As illustrated in Figure 4 by
based on Downstream Connection Manager which knows the blue arrows in ONU side, when a IP packet is received at
queue length, QoS parameters, and service history of each the ONU, it is first mapped, based on the source IP address, to
downstream connection. When carrying out encapsulation, the corresponding upstream connection, which is maintained

Prepared using sagej.cls

Arokkiam, Alvarez 9

by Upstream Connection Manager. The packet is then put

into the corresponding queue for transmitting in the future.
When it is the time to transmit one upstream XGTC
burst (as instructed in the BWmap sent by the OLT in a
previous downstream frame), the Framing Engine in the
ONU assumes responsibility for producing the XGTC burst.
To do this, the Framing Engine in an ONU asks its XGEM
Engine to get an array of XGTC payloads (SDUs), each of
which is a concatenation of several XGEM frames belonging
to an Alloc-Id. To decide the SDUs to be encapsulated,
the Upstream Scheduler and the DBA Engine in the ONU
Figure 5. Sample XG-PON simulation environment
are also needed since the upstream bandwidth is allocated
to each Alloc-Id with the possibility of multiple upstream
connections (or Alloc-Ids) belonging to the same ONU. Both layer-2 frames of various network technologies (Ethernet,
the Upstream Scheduler and the DBA Engine make decisions ATM, etc.), our XG-PON module interacts directly with the
based on several parameters such as the amount of bandwidth IP layer and IP packets are the SDUs. This is reasonable
allocated to each Alloc-Id, queue length, QoS parameters, since we focus on FTTx networks connected to the Internet.
and service history of this T-CONT’s upstream connections.
The OLT and ONUs are attached to XgponChannel
Framing Engine at an ONU also resorts to the DBA Engine
that simulates the optical distribution network (ODN) of
to generate queue occupancy report for the corresponding
XG-PON. As illustrated in Figure 1, the ODN is a very
Alloc-Id, when permitted by the OLT. This report is deduced
complex tree composed by optical fibres, splitters/jointers,
by the Upstream Connection Manager based on the Alloc-
and REs. To produce trustworthy simulation results, it is
Id(s) associated with each ONU. For various purposes,
highly desirable to simulate all details. But, XG-PON is also
PLOAM messages may be generated by Ploam Engine.
a very high-speed network with highly complex standard
When it is allowed by the OLT, one PLOAM message can be
definitions. Hence, our module simplifies several aspects of
put into the header of the XGTC burst, which may contain
XG-PON to reduce the development workload and increase
a range of XGTC SDUs multiplexed when the ONU has
the simulation speed.
several upstream connections/T-CONTs.
Specifically, our XgponChannel merely simulates dmax ,
The upstream XGTC burst is then passed to PHY
i.e., the logical one-way delay of the channel that is
Adaptation Engine with the burst profile to be used. After
determined by the maximum propagation delay of ODN and
going through PMD Engine, this burst is sent to the ODN.
various laser on/off delays at the ONU/OLT. dmax in our
Upstream traffic on OLT side: As shown by the blue arrows XG-PON module can be configured through the attribute
in Figure 4, when the OLT receives an upstream XGTC burst system of ns-3. For a downstream frame from the OLT,
from the XG-PON ODN, this burst first passes through PMD XgponChannel will pass this frame to each ONU after
Engine and PHY Adaptation Engine. The Framing Engine at waiting for dmax . XgponChannel passes the smart-pointer
OLT is then responsible to parse the header and the payloads of this frame to each ONU, which will copy and process
of this burst. The potential queue occupancy report will be the data for itself 6 to avoid unnecessary data copy. As
sent to DBA Engine and the potential PLOAM message is for an upstream PHY burst, XgponChannel will pass the
sent to the PLOAM Engine. As for the XGTC payloads, they corresponding smart-pointer to the OLT, after the appropriate
are sent to XGEM Engine for decapsulation and reassembly dmax . EqD is calculated at the ONU when the upstream
(if needed). Hence, an Upstream T-CONT Manager is needed burst is produced, based on the BWmap from the OLT;
to hold the potential segments for reassembly. EqDi at ONU i is equivalent to the sum of dmax and the
As illustrated in Figure 4, both OLT and ONU should have relative upstream burst delay (with regard to the previously
an OMCI Engine for exchanging OMCI messages that are scheduled ONUs in the same upstream frame) perceived at
used for various purposes (ONU management, XGEM Port the XGTC layer by the Framing Engine in ONU i.
and T-CONT configuration, etc.). This means that although the difference of propagation
delays among ONUs is simulated, the propagation of
Implementation of our XG-PON module optical signals (fibre, splitter, etc.) is not simulated by the
XgponChannel. It is also very CPU-intensive to calculate the
Overview: Figure 5 illustrates a common simulation optical signal strength for each downstream frame when it
set-up that uses our XG-PON module and other ns-3 arrives at each ONU. Thus, our simplification is reasonable
components to study the performance issues that may since the targeted research topics are related with MAC
occur in XG-PON. The OLT is simulated as a node that and upper layers. These simplifications also improve the
has an XgponOltNetDevice and another network device, simulation speeds significantly.
such as PointToPointNetDevice, to connect to an external
network. The ONU is simulated as a node with an Major modules: Table 1 presents the summary of major
XgponOnuNetDevice and other network devices (Ethernet, classes used in our XG-PON module. We have also explained
WiFi, WiMAX, LTE, etc.) for connecting user equipments to some significant sub-modules below, as they involve certain
the ONU. Thanks to ns-3, network devices of a node can be design choices. For information on all the classes our XG-
configured and we can study different deployment scenarios PON module comprises of, the reader may refer to the
of XG-PON easily. Although XG-PON is proposed to carry documentation provided in sourceforge 2 .

Prepared using sagej.cls

10 Simulation: Transactions of the Society for Modeling and Simulation International XX(X)

Table 1. Major classes in XG-PON module in ns-3.

Classes Functionality
XgponChannel Represents the physical medium of ODN
Communicate with both upper layers and PonChannel, implements statistics related
XgponNetDevice
to OLT and ONU, defines various engines representing protocol stack of XG-PON
XgponDsFrame Frame transmitted over XG-PON for downstream data
XgponUsBurst Frame representing upstream burst
XgponXgemFrame Represent XGEM frame and includes the payload and header
XgponXgemHeader Represents the XGEM header defined in XG-PON standard
XgponTcont Represents a T-CONT
Maintains queue occupancy reports from ONU, QoS parameters and service history of
XgponTcontOnu
this T-CONT for DBA algorithm. Also holds the received segments for reassembly
Contains a list of downstream connections and a list of T-CONTs in each ONU
XgponOnuConnManager
and implements both Downstream and Upstream Connection Managers for the ONU.
Downstream Connection Manager and Upstream T-CONT Manager for the OLT. Contains
XgponOltConnManager
broadcast and uni-cast downstream connections and T-CONTs for upstream connections
XgponPhy Implements PMD Engine and PHY layer parameters common to OLT and ONUs
XgponOltFramingEngine Generate the downstream XGTC frames and parse the upstream XGTC bursts in OLT
Implements routines common for both OLT and ONU (eg: XGEM frame creation)
XgponXgemRoutines
It also implements encapsulation, decapsulation, fragmentation, reassembly etc
XgponOltDsScheduler Acts as the OLT Downstream Scheduler shown in Figure 4
XgponOltSimpleDsScheduler A sub-class that follows the round robin (RR) scheme for downstream scheduling
The ONU upstream scheduler shown in Figure 4. Also decides which connection in
XgponOnuUsScheduler
each ONU to be served in the next transmission opportunity, after payload generation
XgponOnuUsSchedulerRoundRobin A subclass to serve the connections of every ONU in a round robin manner
XgponHelper Helper class for configuring an XG-PON network
XgponConfigDb Database that holds the information used by XgponHelper

Connection Managers: For both XgponOnuConnMan- DBA: To study different scheduling and DBA schemes,
ager and XgponOltConnManager, we have implemented two several abstract classes are used in this module for extensi-
subclasses in which these data structures are organized in bility. The actual schedulers can then inherit these abstrac-
different ways for different purposes: (1) XgponOnuCon- tions and implement their specific algorithms. For exam-
nManagerSpeed and XgponOltConnManagerSpeed impose ple, XgponOltDbaEngine is designed for the OLT DBA
some relationships among XGEM Port-Id,d Alloc-Id, ONU- Engine shown in Figure 4. When XgponOltFramingEngine
ID, and IP address of the computer connected to ONU. generates one downstream XGTC frame, it will resort to
They can carry out mapping very quickly, but they also limit XgponOltDbaEngine to generate a BWmap . XgponOltD-
the number of XGEM Ports that an ONU could have; (2) baEngine is also responsible to receive queue occupancy
XgponOnuConnManagerFlexible and XgponOltConnMan- reports from ONUs. Currently, a simple DBA algorithm
agerFlexible do not have such limitations,but they are much is implemented in XgponOltDbaEngineRoundRobin, which
slower. Since millions of packets need to be processed per serves fixed amount of bytes for each T-CONT in a RR
second, we recommend (1) for most cases. manner. Similarly, we could also implement the modified
GigaPON Access Network (GIANT) DBA 22 , which was
initially proposed for GPON and the recent Efficient Band-
PMD and PHY Adaptation: PMD Engine and PHY width Utilization (EBU) DBA 11 in our XG-PON module, for
Adaptation Engine in Figure 4 are simplified significantly supporting different classes of T-CONTs with multiple QoS
for simulating XG-PON with reasonable speed. The most parameters. XgponOnuDbaEngine acts as the ONU DBA
important function of the interface here is to tell other Engine shown in Figure 4. It is responsible for processing
classes about the size of an downstream/upstream frame. BWmap , generating queue occupancy report, and scheduling
XgponOltPhyAdapter and XgponOnuPhyAdapter are used upstream burst.
to implement PHY Adaptation Engine for the OLT and
ONU, respectively. Instead of simulating their functions (line Helper: For facilitating researchers to configure an XG-
coding, FEC, scrambling, etc.) step by step, they just pass PON network with hundreds of ONUs and thousands
frames/bursts between XgponChannel and Framing Engine of connections, XgponHelper is also implemented in
after removing physical layer header. Hence, we implicitly this module. Through XgponHelper, researchers can
assume that all frames/bursts can be received correctly. Since install XgponNetDevice on nodes and attach them to
the network should be well planned and FEC has been XgponChannel. They can also configure XGEM Ports and
adopted, the observed frame corruption rate will be very T-CONTs for carrying user traffic. Researchers can also use
low and this assumption is reasonable. In the future, the XgponHelper to enable Ascii and Pcap tracing.
corruption rate of frames will be simulated based on the
distance between OLT and ONU or empirical measurements Miscellaneous: Further classes of interest in our
of XG-PON networks in real world. implementation are listed here.

Prepared using sagej.cls

Arokkiam, Alvarez 11

Downstream
Traffic Generator 10
ONU1 PC1

Throughput (Gbps)
8
Router OLT
ONU2 PC2
6
downstream
upstream
ONUn PCn 4
Upstream
Traffic Sink
2
Figure 6. Simulated network topology
0
0 50 100 150 200 250 300
• As explained in subsection ’Key decisions’, we used Seconds (s)
XgponXgemFrame to represent XGEM frame in our Figure 7. Effective bandwidth of XG-PON1
XG-PON module, instead of Packet class from ns-3

Avg. Total Throughput (Gbps)

3
• XgponOltPloamEngine and XgponOnuPloamEngine 1
are designed for exchanging PLOAM messages 2.5

Fairness Index
between the OLT and ONU. They also use XgponLink- 0.8
2
Info to maintain per-ONU information, such as keys
0.6 Fairness Index
and burst profiles. 1.5
throughput (Gbps)
• XgponOltOmciEngine and XgponOnuOmciEngine, 0.4 1
are designed for implementing the OMCI channel. For
these classes though, we have only implemented their 0.2 0.5
interactions with other layers. We will simulate their
messages and the related procedures in the future. 0 0
10 20 30 40 50
• XgponOnuUsScheduler is put within XgponTcontOnu
so that T-CONTs of the same ONU may use different No of ONUs
scheduling algorithms for their upstream traffic. Figure 8. Fairness of Round-Robin-based DBA Algorithm
• XgponConfigDb uses one flag to make sure that
XgponOltConnManagerSpeed, XgponOnuConnMan-
point-to-point link between them is used to simulate the core
agerSpeed, and XgponIdAllocatorSpeed are used
network. More specifically, the delay of this link is set to
together.
10ms, which is a practical one-way delay between routers at
ISP and an OLT placed at the ISP edge of an access network.
Evaluation results Downstream Traffic Generator and Upstream Traffic Sink
are connected to Router through point-to-point links whose
In this section, with several typical simulation scenarios, delay is 2ms, which again is a realistic representation of a
we first demonstrate that our XG-PON module, designed one-way delay between an application at the ISP and its core
for simulating a 10Gbps optical network with hundreds of routers. The bandwidth of all the above point-to-point links is
ONUs, can indeed work as expected. Then we evaluate set to 20Gbps so that XG-PON is the only bottleneck link in
our XG-PON module’s simulation speed and memory the entire simulation environment. At the application layer
consumption under different load and ONU numbers using in the Downstream Traffic Generator/PCs, we use traffic
an off-the-shelf server, since simulation performance is one models with uniform inter packet arrival time distribution, to
of the most important metrics in large-scale and high-speed generate the user traffic. More specific details of the network
network modelling. Extensive pressure tests are also carried traffic will be presented in the respective subsections.
out to demonstrate that our XG-PON module can run for a
very long time and work as expected under controlled and
random simulation environments. Functionality validation
Figure 6 shows the network topology used in our Effective bandwidth of XG-PON1: The data rates of
evaluations. We simulate an XG-PON network whose dmax XG-PONs are 10Gbps in downstream and 2.5Gbps in
is 0.4ms, which is more than the one way propagation delay upstream. However, due to the overhead of FEC (Forward
in fibre (= 0.3ms) for a refractive index of 1.5 and a physical Error Correction) and the headers of various layers, the
reach of 60km. For the data rates of XG-PON, we follow effective bandwidth observed by applications is much less.
XG-PON1, which is capable of 10Gbps in downstream and In this experiment, 256 ONUs are simulated, UDP traffic is
2.5Gbps in upstream. There is a total of n ONUs in the generated in both directions, and the overall network load
XG-PON and one PC is connected to each ONU through a is maintained higher than the data rate of XG-PON in each
point-to-point link. These PCs act as the customer of XG- direction.
PON and play the role of generators for upstream traffic and Figure 7 plots the effective bandwidth observed by
that of sinks for downstream traffic. Delay between each applications with time. The red line indicates that the overall
PC and an ONU is set at 2ms, indicating a maximum one- throughput in downstream is around 8.5Gbps while the
way delay between the user application and the ISP terminal blue line indicates that the overall throughput in upstream
near the user. The OLT is connected to a Router and the is around 2.3Gbps; these data rates are equivalent to

Prepared using sagej.cls

12 Simulation: Transactions of the Society for Modeling and Simulation International XX(X)

theoretical values, given a packet size of 1024 Bytes with 10 2.5

the above overheads are accounted for. Thus, these results Average Delay (ms)
Total Throughput (Gbps)

Total Throughput (Gbps)

show that our XG-PON module has properly simulated 8 2.4

Average Delay (ms)

SDU encapsulation (XGEM and XGTC headers) and other
overheads (FEC, inter-burst gap, etc.). 6 2.3
Fairness of Round-Robin-based DBA algorithm: DBA
is one of the most complex functions of XG-PON and it is 4 2.2
necessary to verify that it works as expected. In this group of
experiments, we will demonstrate that our round-robin-based 2 2.1
DBA algorithm can allocate the upstream bandwidth fairly
and efficiently. In each experiment, n ONUs are simulated, 0 2
4000 0
8000 12000 16000
customers (the PCs connected to ONUs) generate the same
Maximum Service Size (Bytes)
amount of upstream UDP traffic, and the overall network Figure 9. Effects of Maximal Service Size used by DBA
load in upstream is higher than the upstream data rate
(=2.5Gbps) of XG-PON1; n is set to 10, 20, 30, 40, and 50. 10
downstream
Figure 8 plots Jain’s fairness index 19 among the customers upstream

Throughput (Gbps)
and the overall throughput in upstream. The red line shows 8
that the fairness index is equal to 1, indicating that the round-
robin-based DBA algorithm fairly allocates the upstream 6
bandwidth to all the customers. The blue line indicates that
our round-robin-based DBA algorithm is also efficient since 4
the overall throughput is 2.3Gbps.
2
Trade-off between throughput and delay: The well-
known trade-off between throughput and delay is also 0
applicable to XG-PON, due to its large bandwidth-delay 0 50 100 150 200 250 300
product (BDP). More specifically, when an ONU gets Seconds (s)
the chance to be served, the larger the maximal service Figure 10. TCP traffic on XG-PON1 (throughput vs. time)
size (MSS) used by the DBA algorithm is, the smaller
the upstream burst overhead is and the higher the overall
upstream throughput is. However, larger MSS also enforces
larger interval between two consecutive services for the
same ONU; as a result the packets wait longer at the ONU
upstream buffer due to larger scheduling delay. In this group
of experiments, 256 ONUs are simulated, customers generate
the same amount of upstream UDP traffic, and the overall
network load in upstream is higher than 2.5Gbps. The MSS is
set to 500B, 1KB, 2KB, 4KB, 8KB, and 16KB. Validation is
considered only for the upstream since the round-robin DBA Figure 11. General CUBIC TCP congestion window growth 10
is employed only for upstream bandwidth allocation.
Figure 9 illustrates the impact of MSS on throughput
and delay; both the overall upstream throughput (blue line) matches well with the congestion window (cwnd) growth
and the scheduling delay (red line) increase with MSS, function of Cubic TCP (Figure 11) Steady State Behaviour
indicating the existence of the well-known trade-off between and Max Probing periods; each CUBIC flow, at the end
throughput and delay in large BDP networks. of every Max Probing period in Figure 10 shows heavy
exponential growth of its cwnd until a congestion is
TCP in downstream and upstream directions: Here, caused by the downstream or upstream capacity XG-PON.
we demonstrate the behaviour of a common congestion We have also performed extensive simulations combining
control algorithm from a realistic TCP stack to validate the our XG-PON module and realistic TCP stacks (Linux
ability of our XG-PON module to work seamlessly with TCP. Kernel from NSC) employing different congestion control
First, we successfully integrated our XG-PON module with algorithms 5 to validate that our XG-PON module is capable
a real-world TCP stack from Linux Kernel(version 2.6.26) of accommodating real-world TCP stacks, under various
packed in Network Simulation Cradle (NSC 20 ). Using scenarios.
this integration, we generated a single CUBIC TCP 10;21
flow across our XG-PON module, both in downstream
and upstream directions, individually. Figure 10 shows the Performance evaluation
throughput vs. time plots for a single TCP connection in the In evaluating performance, one dedicated computer is used
downstream (red line) and the upstream (blue line). When to measure performance of our XG-PON module, to avoid
the data rate at the sender is higher than network bandwidth, interference from other processes. We used Dell PowerEdge
packets are dropped and sending rate (or throughput) is R320 rack server and the processor is Intel(R) Xeon(R) CPU
reduced. We can observe that at each TCP epoch, both in E5-1410 0 @ 2.80GHz with a cache of 10MBytes. Note that
downstream and upstream, throughput curves in Figure 10 although this processor has 4 cores, just one of them is used

Prepared using sagej.cls

Arokkiam, Alvarez 13

increase linearly with the number of packets. We observe,

on the other hand, the consumed time increases at a slower
Scale (seconds)
pace with the number of ONUs. Figure 12 also indicates
Simulation Time (s)
180
160
that our XG-PON module takes around 160s to simulate one
140 second even when there are 1000 ONUs and the downstream
100 120
100 network load is 9.6Gbps with which XG-PON is over-
80
60 loaded.
10 40
20
0
We also used gdb, the standard debugger for GNU
1 software system, to run the simulation of the most difficult
10000 scenario (ONUs: 1000; Network load: 9.6Gbps) in debug
mode. After the simulation enters into steady phase, we break
1000
it at random time, check the call stack, and continue the
100
100 Downstream Network simulation. We observe that our XG-PON code occupies
Number of ONUs 1000 Load (Mbps)
the CPU cycles only at 4 instances of the 126 such breaks.
This clearly indicates that our XG-PON module is not
Figure 12. Simulation speed of our XG-PON module the bottleneck of simulation speed and calls for additional
investigation of other associated ns-3 modules (routing, etc.)
to further improve simulation speed of our XG-PON module.
Besides simulation speed, we also evaluated the memory
Memory Consumed Scale (MBytes) consumption of XG-PON simulation. The same experiments
(MBytes) 5500
5000 are repeated to collect these results. For each experiment,
4500
4000 after starting the simulation, we wait for sufficiently long
3500
3000 time until the simulation enters into its steady phase, where
2500
1000 2000
1500
the amount of consumed memory does not increase any
1000
500
more. The recorded values, plotted in Figure 13, indicate that
0
the consumed memory (again, shown by the vertical axis and
100
the scale in the figure) increases linearly with the network
10000
load while it increases at a smaller pace with the number
1000 of ONUs. When there are 1000 ONUs and the downstream
100 Downstream Network network load is 9.6Gbps, the consumed memory is still less
Number of ONUs 100 Load (Mbps)
1000 than 5GBytes.
Figure 13. Memory consumption of our XG-PON Module Based on the above results, we can conclude that with the
off-the-shelf server, our XG-PON module can simulate an
XG-PON network of 1000 ONUs and 10Gbps (downstream),
by our simulation. As for the main memory, the server is with reasonable speed and moderate memory consumption.
installed with 48GBytes in total.
To study the simulation performance of our XG-PON
module under various scenarios, the number of ONUs
and the amount of network traffic are changed in our Pressure tests
experiments. The evaluated values of n (the number of
ONUs) are 25, 50, 100, 200, 400, 800, and 1000; values To evaluate the robustness of our XG-PON module,
for total amount of network load in the downstream are, we carried out more experiments. We use the same
150Mbps, 300Mbps, 600Mbps, 1.2Gbps, 2.4Gbps, 4.8Gbps, configurations designed for performance evaluation and
and 9.6Gbps. Due to the overhead of XG-PON physical 4000 seconds are simulated in each experiment to
and XGTC layers, we observe packet drop when the demonstrate that our XG-PON module can run for
downstream network load is beyond 9.6Gbps. Furthermore, sufficiently long periods (longer than one hour). Note that
for all experiments, the upstream network load is always one when there are 1000 ONUs and the downstream network
quarter of the downstream network load. Thus, when the load is 9.6Gbps, it takes more than one week to complete
downstream network load is 9.6Gbps, the upstream network the simulation. In another set of 49 experiments, we
load is 2.4Gbps and with observable packet drop in the randomly select the number of ONUs and the amount of
upstream direction. network traffic, and 500 seconds are simulated in each
To evaluate the speed of our XG-PON module, 400 experiment. All these simulations ran successfully without
seconds are simulated in each experiment, the total amount any interruption or memory leakage during the course.
of time used to complete the simulation is recorded; finally
the average time consumed for each simulation second in In summary, these evaluation results indicate that our XG-
each scenario is calculated and plotted in Figure 12. The PON module is sufficiently robust and can simulate XG-PON
results indicate that the average time consumed (shown by accurately with reasonable speed and moderate memory
the vertical axis and the scale in the figure) increases linearly consumption. Thus it can be used as a very good research
with the network load. This is reasonable since ns-3 is a platform for studying performance issues related with XG-
packet-level network simulator and the number of events PON.

Prepared using sagej.cls

14 Simulation: Transactions of the Society for Modeling and Simulation International XX(X)

Summary and future work 8. Farooq J and Turletti T (2009) An IEEE 802.16 WiMAX
Module for the NS-3 Simulator. In: SIMUTools.
In this paper, we introduced an XG-PON module for the ns-
9. Fernando DNV, Milosavljevic M, Kourtessis P and Senior JM
3 network simulator. We described the details of its design
(2014) Cooperative cyclic sleep and doze mode selection for
and implementation, and presented the evaluation results
NG-PONs. In: 2014 16th Int. Conf. Transparent Opt. Networks.
on both functionality and performance. The results indicate
IEEE. ISBN 978-1-4799-5601-2, pp. 1–4.
that our XG-PON module is quite robust and can simulate
10. Ha S, Rhee I and Xu L (2008) Cubic: a new tcp-friendly high-
XG-PON correctly with reasonable speed and moderate
speed tcp variant. Operating Systems Review 42: 64–74.
memory consumption. As the first and a full-fledged XG-
11. Han MS, Yoo H and Lee DS (2013) Development of Efficient
PON module for ns-3, we believe that this work is a
Dynamic Bandwidth Allocation Algorithm for XGPON. ETRI
significant contribution to the scientific community; for any
J. 35(1): 18–26.
interested researcher, our XG-PON module provides the
12. Han MS, Yoo H, Yoon BY, Kim B and Koh JS (2008) Efficient
opportunity to study the performance issues associated with
dynamic bandwidth allocation for FSAN-compliant GPON.
the deployment of XG-PON, using a validated simulation
Journal of Optical Networking 7(8): 783–795.
module.
13. Henderson TR, Lacage M and Riley GF (2008) Network
In the future, we will implement more scheduling
simulations with the ns-3 simulator. In: Sigcomm (Demo).
and DBA algorithms proposed for GPON or XG-PON,
14. IEEE (2004) 802.3ah: Ethernet in the First Mile.
keep improving the simulation speed, add support for
15. IEEE (2004) IEEE Std. 802.16-2004, IEEE Standard for Local
parallel/distributed simulation and investigate how to
and Metropolitan Area Networks - Part 16: Air Interface for
simulate Fibre-to-the-Cell using this XG-PON module and
Fixed Broadband Wireless Access Systems.
WiMAX/LTE modules in ns-3.
16. Ikeda H and Kitayama K (2009) Dynamic Bandwidth
Allocation With Adaptive Polling Cycle for Maximized TCP
Acknowledgements
Throughput in 10G-EPON. Journal of Lightwave Technology
This work is supported in part by the CTVR Grant (SFI 10/CE/I 27(23): 5508–5516.
1853) from Science Foundation Ireland 17. ITU (2008) Gigabit-Capable Passive Optical Networks (G-
PON). Rec. G.984.x.
Notes 18. ITU (2010) 10-Gigabit-Capable Passive Optical Networks
1. http://www.ctvr.ie (XG-PON) Series of Recommendations. G.987.x.
2. http://www.ucc.ie/en/misl/ 19. Jain R, Chiu DMW and Hawe WR (1984) A quantitative
3. Depending on the situation, it may be worthwhile to simulate a measure of fairness and discrimination for resource allocation
likely low packet corruption rate, with the effects of FEC in shared computer systems. DEC Research Report TR-301.
4. All data structures must provide one function to return its 20. Jansen S and McGregor A (2005) Simulation with real world
serialized size to accurately compose downstream frame and network stacks. In: Winter Simulation Conference.
upstream burst. 21. Leith D, RNShorten and GMcCullagh (2007) Experimental
evaluation of cubic-tcp. In: PFLDnet Workshop.
5. For each downstream connection, the Downstream Connection
22. Leligoun HC, Linardakis C, Kanonakis K, Angelopoulos JD
Manager at ONU should hold the initially received segments
and Orphanoudakis T (2006) Efficient medium arbitration
for reassembly while the remaining segments are received.
of FSAN-compliant GPONs. International Journal of
6. In the future, this will be revised to support parallel simulation,
Communication Systems 19: 603–617.
where ONUs will be simulated in different CPUs of a cluster.
23. McCanne S and Floyd S (1997) The LBNL network simulator
(NS-2). Http://www.isi.edu/nsnam/ns/.
References 24. Orphanoudakis TG, Kosmatos EA, Matrakidis C, Stavdas A
1. (2011-15) The NS-3 network simulator. Available at: and Leligou HC (2014) Hybrid resource reservation scheme
http://www.nsnam.org. for transparent integration of access and core optical transport
2. (2014) XG-PON Simulation Module for NS-3. Available networks. In: 2014 16th Int. Conf. Transparent Opt. Networks.
at:http://sourceforge.net/projects/ IEEE. ISBN 978-1-4799-5601-2, pp. 1–4.
xgpon4ns3/. 25. Payne DB and Davey RP (2002) The future of fibre access
3. 3GPP (2008) Evolved Universal Terrestrial Radio Access (E- systems? BT Technology Journal 20(4): 104–114.
UTRA). 26. Peng Z and Radcliffe P (2011) Modeling and Simulation
4. Anthapadmanabhan NP, Dinh N, Walid A and van Wijngaarden of Ethernet Passive Optical Network (EPON) Experiment
AJ (2013) Analysis of a probing-based cyclic sleep mechanism Platform based on OPNET Modeler. In: ICCSN.
for passive optical networks. In: 2013 IEEE Glob. Commun. 27. Piro G, Baldo N and Miozzo M (2011) An LTE module for
Conf. IEEE. ISBN 978-1-4799-1353-4, pp. 2543–2548. the ns-3 network simulator. In: WNS-3 in conjunction with
5. Arokkiam JA, Wu X, Brown KN and Sreenan CJ (2014) SIMUTools.
Experimental evaluation of TCP performance over 10gb/s 28. Postel J (1981) Transmission Control Protocol - DARPA
passive optical networks (XG-PON). In: Globecom 2014 - SAC Internet Program Protocol Specification. RFC 793.
Access Networks and Systems. Austin, USA, pp. 2269–2274. 29. Shea DP and Mitchell JE (2007) A 10-gb/s 1024-way-split 100-
6. Bodozoglou A (2010) EPON for OMNeT++. km long-reach optical-access network. Journal of Lightwave
7. Chang CH (2008) Dynamic Bandwidth Allocation MAC Technology 25(3): 685–693.
Protocols for Gigabit-capable Passive Optical Networks. PhD 30. Song H, Kim BW and Mukherjee B (2009) Multi-Thread
Thesis, University of Hertfordshire. Polling: A Dynamic Bandwidth Distribution Scheme in

Prepared using sagej.cls

Arokkiam, Alvarez 15

Long-Reach PON. IEEE Journal on Selected Areas in

Communications 27(2): 134.
31. Technology R (2015) OPNET Modeler.
32. Weingartner E, vom Lehn H and Wehrle K (2009) A
performance comparison of recent network simulators. In:
ICC.
33. Wu X, Brown KN, Sreenan CJ, Alvarez P, Ruffini M, Marchetti
N, Payne D and Doyle L (2013) An XG-PON module for the
NS-3 network simulator. In: Workshop on NS-3 (held with
SIMUTools).

Prepared using sagej.cls