Conscious Network That Enables Autonomous

Operation and Maintenance

 Takuya Nitta  Yasuhiko Aoki  Kyoko Ohya  Fumiyuki Iizuka

 Toshiki Honda

Today, networks connect computing resources, front devices, and applications that are scat-
tered across many places, including those on-premise and in the cloud, allowing people and
society as a whole to enjoy diverse services. However, a failure or degradation of a network
has the potential to cause considerable problems to connected devices and services. To pre-
vent these risks, a “conscious” network system with autonomous operation and maintenance
capabilities will be required. Such a system will detect the slightest irregularities in the net-
work as early signs of errors and prevent possible failures or degradation from occurring. It will
also ensure that the services remain available without having to modify end users’ systems or
make users aware of the network problems. To realize such systems, Fujitsu has been working
on development of the technologies necessary for integrated network monitoring: detection of
early signs through Internet Protocol (IP) packet behavior analysis and optical transport signal
quality monitoring, optical wavelength reassignment, and self-diagnosis for IP networks. This
paper describes these technologies required for the realization of the conscious network.

1. Introduction broadband access and mobile communications services

In recent years, due to the decrease in main- have expanded, and infrastructure in the form of opti-
tenance and operation man-hours required for ICT cal networks to support them are increasingly in use.
infrastructure and networks, enhanced security, and These optical networks use ultra-high capacity optical
increasing demand for teleworking, cloud services have fiber networks of 100 to 400 Gbps and, should a prob-
come to be equipped with functions conventionally lem occur, an enormous number of services, users, and
provided by devices and on-premise servers. Services IoT devices may be affected. In order not to stop optical
may be offered across multiple clouds. Furthermore, networks, which have become a social infrastructure,
services have come to be provided across virtual and the capability to detect early signs of failures and deal
business-oriented networks, in addition to the wave- with these failures by diverting communications to new
length-division multiplexing (WDM), Internet Protocol paths or by increasing the communication bandwidth
(IP), and wireless networks on which they are placed. is required.
Accordingly, when service response is delayed To quickly identify the failure points as described
or any service becomes unavailable, it is necessary to above and operate services with safety and security,
isolate the problem in order to identify the cause—the Fujitsu attaches importance to integrated monitoring
application or the network. If the problem is in the and operations across communication layers (physical,
network, however, it is difficult to identify the location virtual, and business-oriented) from end to end by de-
in the network where the quality degradation or failure tecting early signs of failures.
has occurred. As a result, problems are increasingly This paper first proposes the operation and moni-
showing a tendency to take a long time to solve, and toring technology for autonomously preventing failures
how to deal with this poses an issue. by detecting early signs of failures with the concept of
Recently, because the IoT has caused the volume conscious networks. It then describes the technology
of data and use of cloud services to increase, fixed for early sign detection required to prevent network

54 FUJITSU SCIENTIFIC & TECHNICAL JOURNAL, Vol. 55, No. 3, pp. 54–60 (2019)
Network
 T. Nitta et al.: Conscious Network That Enables Autonomous Operation and Maintenance

failures and the technology for maintaining service platform where data from multi-layer network such as
quality. IP networks and optical transmission and multi-vendor
devices are collected, integrated, and analyzed.
2. Conventional technologies and The following sections describe technologies
issues with them for realizing the conscious network: IP packet be-
Conventionally, network monitoring was achieved havior analysis technology and early sign detection
by poll monitoring using a combination of Simple technology through optical transmission device signal
Network Management Protocol (SNMP) and Internet quality monitoring analysis, which are needed to pre-
Control Message Protocol (ICMP) or using either SNMP vent network failures; optical signal accommodation
or ICMP and messages output from devices such as trap optimization technology for maintaining service qual-
and syslog messages to detect failures in devices. The ity; and self-diagnosis technology for IP networks.
failures were then handled as needed. For this reason,
problems could be dealt with only after they occurred, 4. Technologies for preventing network
even if service continuity was maintained through failures
network path diversion, and impacts on services were
unavoidable. 4.1 IP packet behavior analysis
In addition, silent failures,note) in which failures are If degradation of network devices or cables or in-
caused by internal failures of network devices but the termittent failures due to network device firmware bugs
devices themselves are unable to detect the failures, cannot be detected by the devices themselves, they
could not be dealt with in the first place. present themselves as events such as packet losses and
Furthermore, network monitoring systems were roundtrip time (RTT) degradation.
set up individually for WDM networks, IP networks FUJITSU Network Proactnes II QM, which sup-
(including core networks, access networks, and data ports network maintenance and optimization, captures
center internal networks) and wireless networks, and packets that flow in a network and makes use of the
monitored and operated independently. This not only characteristics of the Transmission Control Protocol
led to low operational efficiency but also to a large (TCP) to analyze them. In this way, it measures the
amount of time required to identify the failure points, end-to-end communication quality in terms of packet
resulting in a long time taken before services could be loss, RTT, and so on. Detection of early signs of failures
restored. and silent failures is achieved by detecting these packet
losses and instances of RTT degradation. In addition,
3. Conscious networks measurement of end-to-end communication quality
In order to solve the problems mentioned in the allows the communication quality to be grasped for in-
previous section, Fujitsu studied network enhance- dividual sources and destinations. Accordingly, points
ment. Consequently, we have proposed a concept causing degradation of communication quality can be
called conscious network, which maintains end-to-end narrowed down simply by setting up the QM at a point
service quality by making use of our network-related where traffic is concentrated and capturing packets
analysis and visualization technologies to detect early (Figure 1).
signs of failures before they occur and providing au- For detection of failure points, TCP sequence
tonomous control. analysis is first used to isolate suspected points based
To realize this concept, we are developing on the quality of four sections centered around the
technologies to expand scenarios, including early capture point: transmission and reception between
sign detection, failure isolation, cause identifica- the client and the capture point, and transmission and
tion, resource optimization, and path diversion on a reception between the capture point and the server.
Then, the trend in degradation is analyzed based on
the end-to-end communication quality information
note) Failures that cannot be detected by network
and key information such as the network addresses
devices themselves using the autonomous diag-
nosis function. and virtual LAN Identifiers (VLAN-IDs) according to the

FUJITSU Sci. Tech. J., Vol. 55, No. 3 (2019) 55

Network
T. Nitta et al.: Conscious Network That Enables Autonomous Operation and Maintenance

1) Client to Capture point 2) Capture point to Server

Client
4) Capture point to Client 3) Server to Capture point
Site A QM
(probe)

Server
Subnet
VLAN
IP address
specification
Capture point
Suspected points narrowed down

Subnet
...

specification

Subnet
...

VLAN specification

VLAN
Site
specification

Site B VLAN: Virtual LAN

QM: FUJITSU Network Proactnes II QM

Figure 1
End-to-end Isolation of failure points.

target network configuration. This narrows down the practical application. The operation of optical net-
suspected points. works, which was conventionally handled primarily
Methods for narrowing these down include identi- by communications carriers, is expanding to include
fying points using IP addresses and routing information enterprise users as in data center interconnect (DCI)
and identifying points based on V-LAN IDs. In an overlay connecting large-scale data centers.
network that applies Virtual eXtensible LAN (VXLAN), In this way, WDM transmission systems are po-
Ethernet frames are encapsulated in User Datagram sitioned as a platform to support various tasks across
Protocol (UDP)/IP and inner TCP sessions (part of locations on premise and in the cloud. WDM transmis-
Ethernet frames before encapsulation) are used to sion systems are increasingly gaining importance.
evaluate the communication quality. The results can Given this situation, expectations regarding the
be aggregated based on inner IP addresses to identify avoidance of large-scale failures are increasing via
logical degradation points, and based on outer IP ad- the detection of early signs of system failures through
dresses (UDP/IP for encapsulation) to identify physical the visualization and analysis of optical network
degradation points. operations.
In this way, analysis of end-to-end communica- Figure 2 shows early sign detection of failures in
tion quality allows devices and cables with problems to an optical network. The optical transmission nodes are
be identified quickly. At the same time, detailed analy- equipped with a performance monitor (PM) function to
sis such as degradation analysis of optical transmission monitor signal quality such as the intensity of optical
paths as described below can be performed quickly. signals, and these functions are used as sensor devices
in the optical network. PM information, which was
4.2 Optical transmission device signal conventionally used only for monitoring the status of
quality monitoring analysis devices before and after failures and alarms, is continu-
With WDM transmission, a system with a trans- ously stored as log data and analyzed with machine
mission speed of 600 Gbps per wavelength and a learning. This is used to detect early signs of failures
transmission capacity of 76.8 Tbps1) per fiber is nearing in the optical transmission system and identify failure

56 FUJITSU Sci. Tech. J., Vol. 55, No. 3 (2019)

Network
 T. Nitta et al.: Conscious Network That Enables Autonomous Operation and Maintenance

PM data Data Analysis which wavelength fragmentation occurs, hindering

conversion analysis data
unit unit visualization utilization of wavelength resources. In particular, this
phenomenon is conspicuous in optical networks with a
mesh topology composed of ROADM nodes as shown in
Work instructions
Network quality
data collection based on Figure 3 (a).
quality results
To solve this problem, we have developed an
optical wavelength reassignment technology (algo-
rithm) that relocates fragmented optical wavelength
resources in a manner that does not affect the service
PM to regenerate usable wavelength. The effect of the
Optical path developed algorithm in the JPN-48 model2) as shown
Optical fiber
ROADM node transmission path in Figure 3 (b), which simulates a core network in
Optical network Japan, is shown in Figure 3 (c). One indicator of wave-
PM: Performance monitor length utilization efficiency is the maximum occupied
ROADM: Reconfigurable optical add/drop multiplexer
wavelength number, which is the wavelength number
occupied in the network. It has been confirmed to im-
Figure 2
Detection of early signs of failures in optical network. prove the maximum occupied wavelength number by
over 20% by optimizing the use of this indicator and
this algorithm. This shows that it is almost equivalent
points. to the theoretical limit, without interrupting services.3)
The input and output of optical transmission de- The developed algorithm is capable of relocation of
vices, design values of optical signals, and degradation optical wavelength resources in an arbitrary segment in
of optical signals in the optical transmission system are the network. Applying this technology in particular to a
grasped with machine learning and the points of deg- segment in which traffic demand is high and wavelength
radation and their causes are analyzed. In this case, resource depletion is significant is expected to be highly
the connections between the reconfigurable optical effective. That is, a service can be introduced simply by
add/drop multiplexer (ROADM) nodes and optical fiber adding a transponder to an existing system without the
transmission paths as well as the information about need to install additional optical fiber. This allows opti-
the optical paths (start points, end points, and inter- cal network operators to effectively utilize the existing
mediate nodes) are used. equipment and reduce capital investment as well.

5. Technologies for maintaining service 5.2 Self-diagnosis of IP networks

quality This section describes two issues and solutions
for visualizing and monitoring network quality from the
5.1 Optical wavelength assignment perspective of services:
optimization 1) Network delay measurement and factor
Concerning optical networks, expectations are classification
increasing for increased utilization efficiency of wave- Key performance indicators (KPIs) from a network
length resources in order to efficiently accommodate perspective, such as packet loss and RTT, only allow
ever-increasing traffic. Meanwhile, the number of network experts to find out how actual services are
optical wavelength (optical signal bandwidth) that affected.
can be accommodated in optical fiber is restricted by To deal with this issue, we have established a
the system configuration. In addition, to connect technique for classifying the measurement of the actual
optical signals without going through a regenerative latency in responding to a service in the system into
repeater, there was a need to accommodate signals factors arising from network quality and from server
without wavelength conversion. However, the increase and client processing (Figure 4). Specifically, latency
in accommodating services causes a phenomenon in has been classified into the following eight types.

FUJITSU Sci. Tech. J., Vol. 55, No. 3 (2019) 57

Network
T. Nitta et al.: Conscious Network That Enables Autonomous Operation and Maintenance

Usage of optical wavelength Optical wavelength resources

Usage of optical wavelength

Relocation
of optical made available by relocation
wavelength
resources

Optical link Optical link

Fragmented optical wavelength resources

(a) Fragmentation of optical wavelength resources

110 Average occupied

wavelength number
Maximum occupied wavelength number

100
Before optimization: 97.9%

90
Improved
by over 20%
80

Proposed technique: 76.3%

70 Before optimization Theoretical value: 73.3%
Proposed optimization technique
Theoretical value
60
0 50 100 150 200
Traffic pattern

(b) Topology used for simulation (JPN-48 model) (c) Effect of wavelength resource optimization

Figure 3
Optimization of optical wavelength resources in optical network.

• Connection establishment time monitoring of various networks.

• SYN (connection request) retransmission time 2) Establishment of error determination technique
• Client data transfer time The form of use of a network differs depending
• Client data retransmission time on the application, and the latency cannot be assessed
• Client processing time with the same criteria.
• Server data transfer time To deal with this, we classified the latency in
• Server data retransmission time normal times measured in 1) classified for each appli-
• Server processing time cation or subnet, and analyzed the standard deviation
In this way, detailed measurement of the latency mean ± 3σ) of the latency distribution. Then, we
itself that arises from network quality has allowed for established a technique which determines errors deter-
the quantitative evaluation of the impact on actual mination based on the change in the rate of exceeding
services, which could not be grasped previously. Points the threshold covering 97 to 99% of the latency. With
that cause bottlenecks in quality improvement can this technique, however, if the processing details dif-
also be isolated by classifying the factors. This allows fered for each application and variations in latency
problem points to be easily identified by non-experts become large, common distribution models did not
of networks, who can now respond and formulate apply, which caused the error determination accuracy
improvement plans. This technique does not depend to decrease. To address this issue, we have applied a
on specific applications and can be applied to quality robust statistics-like technique.

58 FUJITSU Sci. Tech. J., Vol. 55, No. 3 (2019)

Network
 T. Nitta et al.: Conscious Network That Enables Autonomous Operation and Maintenance

Client User Server

request

Analysis Data

Response time
transfer

User request
1) Server processing time

1) Server
2) Client processing
processing 2) Client processing time
time
time

er
Data transf

time classified
Response
X 3) Server transfer time
4) Retransmission
…
time
3) Server
transfer
time
er
Data transf
4) Retransmission time

Figure 4
Quality analysis by response time.

While the median is often used as the represen- In the future, we intend to have the area of monitor-
tative value for data sequence containing outliers, it ing expanded to include data center fabric and wireless
is not suitable for network KPI data, in which outliers networks. While machine learning is utilized for some
frequently deviate to one side. For this reason, we early sign detection of network failure, we plan to ex-
have adopted the median absolute deviation (MAD) as pand the scope of the application of AI technologies,
an indicator that shows the spread of data using the including machine learning, to provide autonomous
trimmed mean. Then, the deviation from the distribu- control at an earlier stage, thereby offering stable
tion (standard deviation) is used as the threshold value networks.
to compare with the value in normal times. This has Part of this technology includes results from the
eliminated the need to set a threshold value for each “Research and Development on Elastic Optical Networking
application, allowing us to realize practical monitoring Technologies” entrusted by the National Institute of
operations. Information and Communications Technology.

6. Conclusion References
This paper described the early sign detection 1) Fujitsu: Fujitsu Accelerates Path to 5G and Conscious
technology for preventing network failures and tech- Networks with Next-Generation Variable Optical Transport.
nologies for maintaining service quality developed by http://www.fujitsu.com/us/about/resources/news/
press-releases/2018/fnc-20180312.html
Fujitsu based on the concept of conscious network,
2) Technical Committee on Photonic Network, The Institute of
which continuously provide services via networks. Electronics, Information and Communication Engineers:
These technologies are applied to optical and IP JPN Model.
networks to realize integrated monitoring. This makes 3) Y. Takita et al.: Wavelength Defragmentation for Seamless
it possible to see if there is any impact on the packet Service Migration. Journal of Optical Communications
and Networking,Vol.9,Iss.2, pp. A154–A161, (2017).
layer (end-to-end) communication even if the optical
layer has a problem, allowing end users to be notified.

FUJITSU Sci. Tech. J., Vol. 55, No. 3 (2019) 59

Network
T. Nitta et al.: Conscious Network That Enables Autonomous Operation and Maintenance

Takuya Nitta
Fujitsu Ltd.
Mr. Nitta is currently engaged in the develop-
ment of packet analysis and accumulation
products.

Yasuhiko Aoki
Fujitsu Ltd.
Dr. Aoki is currently engaged in research
and development relating to network op-
eration and monitoring technology.

Kyoko Ohya
Fujitsu Ltd.
Ms. Ohya is currently engaged in the devel-
opment of network operation management
products.

Fumiyuki Iizuka
Fujitsu Laboratories Ltd.
Mr. Iizuka is currently engaged in research
and development relating to network ser-
vice quality analysis technology.

Toshiki Honda
Fujitsu Ltd.
Mr. Honda is currently engaged in the
planning and technology development of
network operation enhancement solutions.

60 ©2019 FUJITSU LIMITED FUJITSU Sci. Tech. J., Vol. 55, No. 3 (2019)
Network