By Gabby Nizri

2012 Ayehu Software Technologies Ltd. All rights reserved.

Network Operations Center
Best Practices




April 2012 Edition
Tweet This Guide Share on facebook Share on LinkedIn
Table of Contents
Introduction 1
5 essential NOC tools 4
Knowledge and skills 9
Clearly defined roles 10
NOC employee training 13
Communication & collaboration 15
Escalation processes 18
Prioritization 19
Incident handling 20







2
Tweet This Guide Share on facebook Share on LinkedIn
Introduction
Network Operation Centers (NOCs) are always under a great pressure
to meet both technical and business services demands. In the past
several years NOCs have become a more crucial part of the
organization, taking on more tasks and responsibilities.
In NOCs, technological challenges arrive at unexpected times, and
unforeseen problems always pop up. In fact, the only part you can
count on is shrinking resources.
So whats the best you can do? Make sure youre employing the best
practices, using the most appropriate tools, and optimizing your
processes and knowledge.
This is what this e-book is all about.
3
Tweet This Guide Share on facebook Share on LinkedIn
You know tools are an essential element
in NOC management. But they are also a
key element for improvement. So what
are the tools that will give you the top
return of investment? How do you keep
your team aligned with the most updated
information? And how can tools help your
team gain control over all tasks, while
positioning the NOC as an exclusive IT
resource?

Part 1: Tools
4
Tweet This Guide Share on facebook Share on LinkedIn
5 essential tools
Ticketing system
A ticketing system will enable you to keep track of all open issues,
according to severity, urgency and the person assigned to handle each
task. Knowing all pending issues will help you to prioritize the shifts
tasks and provide the best service to your customers.

Knowledge base
Keep one centralized source for all knowledge and documentation that
is accessible to your entire team. This knowledge base should be a
fluid information source, so make sure you continuously update it with
experiences and lessons learned for future reference and
improvements.


5
Tools Ticketing system
Tweet This Guide Share on facebook Share on LinkedIn
Reporting and measurements
Create reports on a daily and monthly
basis. A daily report should include all
major incidents of the past 24 hours and
a root cause for every resolved incident.
This report is useful and essential for the
shift leaders and NOC managers.
It also keeps the rest of the IT department
informed about the NOC activities and of
major incidents. Compiling the daily
reports into a monthly report will help
measure the teams progress. It will also
show areas where improvements can be
made or indicate any positive or negative
trends in performance.




6
Tools Reporting
Tweet This Guide Share on facebook Share on LinkedIn
Monitoring
There are two types of monitoring processes relevant to NOC:
Infrastructure monitoring, which can consist of the servers,
the network or the data center environment.
User experience monitoring, which simulates user behavior
and activities in order to replicate problems and find the most
effective solutions.
Implementing a service tree model that connects the infrastructure
monitoring with an affected service will allow your team to handle
other areas that may be affected by the problems experienced.
7
Tools Monitoring
Tweet This Guide Share on facebook Share on LinkedIn
IT Process Automation
IT Process Automation can save significant
time on repetitive, daily tasks, and frees up
time for more strategic projects.
It empowers a Level-one team to deal with
tasks that otherwise may require a
Level-two team. Some examples include
password reset, disk space clean-up, restart services etc.
IT Process Automation also helps reducing mean time to recovery
(MTTR) in case of critical incidents. For example, specific workflows
can be triggered during off hours for handling critical system events.
8
Tools IT process automation
Tweet This Guide Share on facebook Share on LinkedIn
By knowledge and skills we dont mean
the obvious technical knowledge and
network know-how your team members
must hold to run day-to-day operations,
but rather
How you can ensure your teams skills
are used to their best potential, and how
to keep those skills up to date over time.
Part 2: Knowledge & skills
9
Tweet This Guide Share on facebook Share on LinkedIn
Clearly defined roles
Definition of roles may vary between data centers and will
depend on team size, the IT environment and tasks. Still, there
should be a clear distinction between the roles and
responsibilities of operators vs. shift supervisors in the NOC.
Why does it matter?
Mostly because it affects decisions making. Without clearly
defined roles and responsibilities, disagreements between
operators lead to late decisions and actions, or to no decisions
taken at all. This can affect customers, critical business services,
and urgent requests during off hours.
It should be clearly defined, therefore, that a shift manager
makes the final decisions.
10
Knowledge & Skills Roles
Tweet This Guide Share on facebook Share on LinkedIn
Tasks & responsibilities
Another potential problem caused by the absence of a clear role
definition is the division of tasks between operators and the shift leader.

A shift manager should be responsible for: prioritizing tasks, assigning
work to operators based on their skills, verifying that tickets are opened
properly and that relevant personnel are notified when required,
escalating problems, communicating with management during
important NOC events, sending notifications to the entire organization,
preparing reports, and making critical decisions that impact many
services, such as shutting down the data center in case of an
emergency.
11
Knowledge & Skills Tasks & responsibilities
Tweet This Guide Share on facebook Share on LinkedIn
Operators, on the other hand, should be
responsible for handling the technical
aspect of incidents either independently
or by escalating to another team member
with the required skills. Operators are also
responsible for following up and keeping
tickets up to date.
While it might sound as if operators lack independence and
responsibility, this is not the case. When faced with technical
challenges, operators input and skills are probably the most critical
for resolution and smooth NOC operation. Operators provide
additional insights into problems, and can provide creative solutions
when the standard procedures fail to work.
12
Knowledge & Skills Tasks & responsibilities
Tweet This Guide Share on facebook Share on LinkedIn
NOC employee training
How often have you started a new job, without receiving any
orientation, mentoring or guided training?
A new NOC operator may not know where to find a procedure or
how to execute it, or in a more severe case - take a decision that
causes equipment damage or results in critical downtime.
Make sure to put in place an extensive training program for new
NOC employees. Ideally, such a program would consist of one
week of classroom training followed by three weeks of hands-on
training under the supervision of a designated trainer.
13
Knowledge & Skills NOC Training
Tweet This Guide Share on facebook Share on LinkedIn
Make sure experienced NOC members, preferably shift leaders, train
new employees. Release trainers from all other duties to ensure that
the training does not gradually fade between all the urgent tasks.

Update the training program on an ongoing basis, and make sure it
includes topics such as required users and permissions, known
problems, troubleshooting, teams and important contacts.

14
Knowledge & Skills NOC Training
Tweet This Guide Share on facebook Share on LinkedIn
Communication &
Collaboration
A good communication flow between NOC members and other
IT teams has many advantages. It propels professional growth,
provides opportunities for advancement in the organization, and
makes it easier to approach other teams when requiring
assistance. But most importantly it allows NOC personnel to
see the larger picture. NOC members that are aware of projects,
services and customers needs, simply provide better service.
How can you improve communication? Here are a few ideas:
Make sure a member of the NOC attends weekly change
management meetings and communicate issues or upcoming
activities, such as planned downtime, to the rest of the team.
15
Knowledge & Skills Communication
Tweet This Guide Share on facebook Share on LinkedIn
Define NOC members as focal points for important IT areas,
such as NT, UNIX, Network, or a specific project. These
members attend the meetings of the relevant teams, deliver
new information and knowledge to the rest of the NOC, and
handle specific professional challenges.

Encourage and promote collaboration between NOC team
members. When your team members are willing to help each
other, information is shared more easily, and problems are
resolved more quickly and efficiently. If the NOC team is
geographically spread out or is located in different countries,
you should spend even more efforts on communication and
collaboration, to overcome cultural and language barriers.
16
Knowledge & Skills Communication
Tweet This Guide Share on facebook Share on LinkedIn
What are the operational, structured
processes that you should implement
for effective and repeatable results?
Here are our top ones.

Part 3: Processes
17
Tweet This Guide Share on facebook Share on LinkedIn
Escalation
You want to establish a clear cut escalation process. One method is an
escalation table, which outlines the protocol, channels for escalating
issues and contact personnel with the appropriate expertise.
For example, see the table below defining the escalation procedures for
DB related problems.






18
Time Frame Escalate To Method
0+15 min DB on call SMS
0+30 min DB on call Phone
0+60 min DB Group Leader Phone
0+90 min
UNIX & DB Project
Manager
SMS
0+120 min UNIX & DB Director SMS
Processes Escalation
Tweet This Guide Share on facebook Share on LinkedIn
Prioritization
The process of prioritizing incidents is different in each NOC, and
therefore should be clearly defined. Incidents should never be handled
on a first come, first served basis. Instead, the shift manager should
prioritize incidents and cases based on the importance and impact on
the business.
The entire team should be familiar with the NOC Top 10 projects, and
have an understanding of what signifies a critical incident whether it is
the temperature rising in the data center, a major network cable
breaking, service x going down, or anything else.










19
Processes Prioritization
Tweet This Guide Share on facebook Share on LinkedIn
Incident Handling
Both NOC operators and shift managers should be familiar with the
specific process of handling incidents.
The incident handling process should cover issues such as:
Full technical solution, if available.
Escalation to appropriate personnel.
Notification of other users who may be directly or indirectly affected.
Quick solution procedures or temporary workarounds for complex
problems that may take longer to be resolved.
Incident reporting.
20
Processes Incident handling
Tweet This Guide Share on facebook Share on LinkedIn
Conclusion
Employing the proper tools, skills and processes in
your NOC will allow you to run more efficient network
operations and ensure smooth day to day operations.
The purpose of these NOC best practices is not only to
enable smooth ongoing operations. It is also to
advance your team with a better understanding
business needs, the IT department and the customers;
it is to encourage progress and growth among team
members; and most important, it is to make a healthy
and supporting work environment that will enable
providing superior service.
21
Tweet This Guide Share on facebook Share on LinkedIn
Learn more about Ayehu
eyeShare for NOC
22