Your future with IBM Tivoli Network Manager

(ITNM)

Rob Clark – [email protected]

Krishna M Kodali – [email protected]
Dominic Heade – [email protected]
Zane Bray – [email protected]
Tom Randles - [email protected]

© 2017 IBM Corporation

 Ø NOI Network Management Solution Enhancements

Agenda: Ø ITNM Migration Best Practices

Ø Efficient Operations with Netcool Analytics

Ø IBM Service Offerings

ØQ & A

1
Netcool Operations Insight v1.4

Detailed overview of Network Management

Tivoli Network Manager v4.2

Tivoli Netcool Configuration Manager v6.4.2
Network Performance Insight v1.2

2
 NOI Network & Event Management
Discovery & Monitoring Event Search Analytics
End-to-end visibility of your Mine hotspots, noisy elements,
physical and virtual network event trends with search engine
infrastructure with automatic style queries and analysis
root cause analysis

Network Performance Seasonal Analytics

SNMP and ICMP based Identifies regularly occurring
performance monitoring of events sorted by confidence
Enterprise and Service level and frequency
Provider environments Netcool Operations
Insight

Config & Compliance Related Event Group

Control the process of Automatic grouping of events,
configuration change and greater efficiency due to fewer
compliance in the operations duplicate tickets being created
environment

Network Flow Related Event Analytics

Analyze network traffic Discover groups of related
usage flow with Network events on-the-fly in each
Performance Insight customers’ unique environment

3
Operator Efficiency
Network Health
Dashboard
Summary status & details for
area of responsibility from all
data sources (NCIM, Events,
NCM UOW+OOBC, historical
KPIs)
Network topology maps
• Applets replaced with
HTML5
• Link bundles and rt click
tooling
• Links showing relative
capacity and status
• Link details in tooltips
• Table view, filtered & sorted
• Right-click Performance
graph
GIS based maps
Real-world context, Open
Streets or Google Maps

A to B Network paths
Visualize and Verify real-time
4 status
 Discovery
Dramatic increase in scalability and performance. 1 million entities per domain
64bit, DNCIM replaces much of OQL, stitcher improvements

SNMP interface filtering to avoid querying and modeling large sets of unmanaged interfaces

Disco profiling for easy comparison of each discovery

New technology support

• Optical, RAN, Mobile 2G/3G/LTE
• New Collector API for Java and
Perl
• Cisco Access Points – Wireless
LAN Mgmt
• Cisco Nexus – VRF, Virtual
Device Context, vPort Context
• MPLS VPN + SAE
• MPLS TE

5
 Monitoring
• Historical Poll data Data
Self-sustained management of data Aggregation
with raw and aggregated tables Raw data – 1 hour
Last 24 hours
Last 7 days
• Increased scale and performance Last 30 days
Multiple pollers Last 365 days

Each table is partitioned

• Adaptive polling (event based) in time slices to maximize
Remove false positives as soon as performance and auto
possible incremental pruning

• Interface filtered polling

6
 Network Performance Insight – NPI v1.2
Integrated Network Performance visibility for Network Operations

Visibility into network performance and

utilization

Rapid identification of bottlenecks and high

utilization areas in your network

Performance root cause

analysis using Flow data to
better identify type and source
of issue

Built on a big-data platform for elastic

scalability and performance

7
 Configuration Manager
Take Control over Network Configuration while ensuring Network Compliance

Reduce manual errors and automate bulk change

Automate bulk configuration changes through reusable templates,
increasing productivity and driving consistency
Track planned and un-planned configuration changes
Update configuration repository with planned and un-planned configuration changes across the
network automatically

Manage compliance of device configurations

Ensure your network device configurations are meeting your regulatory compliance commitments
(e.g. PCI, HIPAA) and industry best practices. Pre-emptive Compliance enables checking impact
of changes BEFORE being sent to devices
Implement a secure change process for device configuration data
Role-based control of who accesses what devices and what changes they
can make within the network, including full workflow approvals

“ITNCM has become an essential part of Bendigo Community Telco’s PCI compliance capabilities as well.
“With the ability to regularly run audit checks we could prove that we were adhering to compliant
configuration templates. The compliance reporting was also very easy to build and rollout.”

Matt Hillman, manager of technical innovation for Bendigo Community Telco.

(1) US Telco experienced an 80% reduction in call center requests, reducing

the demand for help desk staff, and a 95% reduction in MTTR.

8
 Feature Additions in ITNM since ITNM 3.8
Ø Increased discovery performance and scalability to 1M entities per domain
Ø 64 bit support
Ø DNCIM, in-memory database, replacing OQL in discovery 4th phase
Ø Interface filtering available

Ø Netcool Common Deployment via IBM Installation Manager

Ø High Availability of NCIM Database support via resilient native method - DB2 HADR & Oracle RAC
Ø Tivoli Integration Portal (TIP) replacement by Dashboard Application Services Hub (DASH)
Ø Applet Free Network and Hop Views visualization with HTML5 architecture
Ø Faster topology performance with no Java/Browser synchronization
Ø Common portal across Netcool solution easily increased integration with other data and content sources
Ø Geographic Information System (GIS) enriched devices automatically rendered in geographical context.
Ø Network Health Dashboard(NHD) for Available & Unavailable Network sources with top performers widget.
Ø Brings data together from events, configuration changes, and performance KPIs in context with areas of interest (network views)
Ø Topology collection from Element Management Systems (EMS) via Perl & Java based collectors.
Ø Support for Optical Network & Radio Access Networks (2G, 2.5G , 3G) and LTE Networks.
Ø Able to discover and visualise non-IP addressed devices via EMS.
Ø Several vendors EMSs – Alcatel Lucent, Cisco, NSN, Tellabs, Huawei etc.
Ø Cross domain Processing and Visualization of entire core network.
Ø Allows for complete network picture across domain boundaries
Ø Historical Poll data storage within NCIM database and aggregated using Apache Storm (decoupled from Tivoli Data
Warehouse -TDW). No integration to setup and maintain.
Ø Reports migration from BIRT to Cognos
Ø users can have number of formats like PDF, HTML, Excel
Ø schedule to run reports via Cognos scheduler.
Ø Drag and drop report creation

9
 New functionality in ITNM since 3.8
Ø Added support for various technologies
Ø Cisco Wi-Fi Support – Able to discover & monitor Access Points via Controllers.
Ø (Metro/Carrier Ethernet Support , Multicast, RAN, LTE)
Ø Device support for Cisco Nexus and several others.
Ø Integration with ITNCM & TADDM
Ø OOTB integration with ITNCM with 4.2
Ø Network Path Views – e.g. MPLS TE , IPv4.
Ø Right click access to historical performance graphs with MIB Grapher graphing using live poll data.
Ø Scalable and Reliable Monitoring System (multiple Pollers) and able to define Interface level filtering.
Ø Topology Editor to begin monitoring devices quickly, without having to run Discovery.
Ø Search Improvements around Structure Browser, Network & Hop Views.
Ø Deep search across attributes in topology views
Ø Active Discovery Status GUI – provides real time information for:
Ø Current phase of discovery, status of agent stitchers, status of finders.
Ø Aids in diagnosis of discovery issue (Agents doing the most work, hold up discoveries)
Ø Plug-in architecture for ITNM Gateway
Ø Service-Affected Events (SAE) plug-in
Ø Including Event Enrichment, RCA, Disco plugins
Ø Adaptive Polling for monitoring critical devices per SLA.

10
ITNM
Component
Specifics &
Migration
Best Practices

11
 Agenda at Product Level - ITNM

Architecture Root Cause Analysis

Discovery Network Services and Event Correlation

Visualization Reporting & Miscellaneous

Monitoring Upgrade / Migration Options

12
 ITNM IP Architecture

ITNM v3.8 ITNM v4.2

TIP DASH
MySQL DB2
Installation
COI/DE Manager

EMS – 3 Vs 16

v4.2 highlights @ https://ibm.biz/Bdijzg

13
 ITNM v3.8
Architectural Changes in Core Processes

ITNM v4.2

ITNM v4.2

New Apache Storm to manage

historical Polldata

Note worthy:
Ø ncp_auth from v3.8 deprecated (config.settings.m_OQLAuthenticationMode which is configured in
$NCHOME/etc/precision/ConfigSchema.cfg )
Ø RCA daemon i.e. ncp_f_amos is a plug-in for ncp_g_event in v4.2
Ø ncp_ncosae which was optional process in v3.8, now turns into several plugins for ncp_g_event in v4.2
Ø By default, there are two Pollers per domain
14 Ø Tibco Rendezvous message bus replaced by Nano Broker (Really Small Message Broker - RSMB)
 Discovery Architecture

LTE,

ncp_df_dbentry
Database Finder

15
 Discovery Data Flow in v3.8 and v3.9

NCIM
Old Processing - v3.8 & v3.9 ncp_disco 10

2 9 ITNM v3.x
3

4 7 8

GUI

6 ncp_model

1
5
1 – Agent Returns 2 – translations.ipToBaseName 3 workingEntities.finalEntity
4 – workingEntities.containment 5 – Layers ( IP, switch etc ) 6 – fullTopology.entityByNeighbor
7 – scratchTopology.entityByName 8 – master.entityByName 9 – kernel.activeModel
10 – NCIM tables ( entity , connects, collects , etc. )

16
 Discovery Data Flow in v4.2 (v4.x)
NCIM
Current Processing - 4.2 ncp_disco
10
7 ITNM - v4.2
2

4 8

GUI

6 ncp_model

1
5
1 – Agent Returns 2 – translations.ipToBaseName 3 workingEntities.finalEntity
4 – workingEntities.containment 5 – Layers ( IP, switch etc ) 6 – fullTopology.entityByNeighbor
7 – DNCIM tables 8 – Model update table
10 – NCIM tables ( entity , connects, collects , etc. )
Ø DNCIM is a new relational database that replaces the tables scratchTopology.entityByName in ncp_disco and master.entityByName in ncp_model.
Ø DNCIM is populated in the last stage of discovery, then transmitted to ncp_model to populate it's cached topology (ncimCache) which in turn is used to populate the NCIM database.
Ø DNCIM's data model is identical to NCIM.
Ø Note: Like in v3.8 & v3.9, the mapping from the workingEntities.finalEntity table to the NCIM data is defined by the ModelNcimDb schema file.
17
 Discovery - Final Phase Stitching changes

Old New
PostLayerProcessing
PostLayerProcessing Stitcher Deprecations

• CreateScratchTopology ( PopulateDNCIM )
• PostScratchProcessing ( InferDNCIMObjects )
PopulateDNCIM
• Protocol based processing stitchers for
CreateScratchTopology • BGP
• OSPF
RecordToDncimDb
• PIM
• MPLSTE
• JuniperMX
PostScratchProcessing PopulateDNCIM<Rel> • IGMP
• SendTopologyToModel ( BroadcastChanges rule )

InferDNCIMObjects

SendTopologyToModel BroadcastChanges

RecordToDncimDb - Takes the supplied record, or the top one of the record vector if none supplied and passes it through the mappings in ModelNcimDb.cfg and
DbEntityDetails.cfg. Those mappings may populate multiple DNCIM tables, create new objects etc. in a manner very similar to the ModelNcimDb.cfg functionality in 3.9.

18
 More about Discovery Phases

Ø SQLite as DNCIM
Ø DNCIM is only supported on v4.x versions.
Ø Internal database used by discovery (dNCIM) & SQLite is open source embedded in the product.
Ø DbLogins.<domain>.cfg file will now specify 'SQLITE' as the 'm_Server'
Ø Connection can be made using the traditional 'ncp_oql' command:
ü ncp_oql -domain NCOMS -service ncim -dbId DNCIM (v4.11 & v4.2)
Ø A new 'DNCIM' OQL service has been created.
ü Allowing for connections from 'ncp_oql' and Perl (v4.2 only)
Ø Database is now located: $NCHOME/precision/embeddedDb/sqlite/ncp_disco.<domain>
19
 Model OQL updates & ncimCache.entityData format

• Model no longer has a master.entityByName table

• ncimCache tables
• Results can be written to file using–outputFile <fileName>
• Tip: If retrieving a limited set of fields, it can be simpler to grep for results if using –tabular and –outputFile together
• collects
• connectActions
• Topology is stored in the ncimCache database
• Now have multiple cache files in $NCHOME/var/precision/, called Store.Cache.ncimCache.*
• connects
• contains
• Closely tied to the format of NCIM
• dependency
• Already used by ncp_g_event in ITNM 3.9
• domainMembers
• The main table of interest is ncimCache.entityData • entityActions
• All entries have some top-level fields • entityData
• E.g. ENTITYID, ENTITYNAME, ENTITYTYPE, BASENAME
• hostedService
• All entries contain an entityData field, with the row of NCIM entityData table for that entity
• lingerTime
• Nested fields are the fields from NCIM
• managedStatus
• All entries also contain all other relevant rows of entity information from other NCIM tables
• networkPipe
• The top-level CONNECTIONS field provides a quick reference to connected entities
• Details are in the ncimCache.connects table
• pipeComposition
• The connects table may have some connections stored as A->z and some stored as Z->A, so multiple • protocolEndPoint
lookups would be required

20
 An example of ncimCache.entityData & DNCIM Updates

• Updates from ncp_disco arrive on subject Dncim2Ncim

• View topology updates from disco to model using:
• ncp_oql –domain <name> –service Dncim2Ncim –updates
• Updates, insertions and deletions can be made directly against
ncimCache.entityData on subject model. For example:
• ncp_oql –domain <name> –service model
• update ncimCache.entityData set entityData-
>DISPLAYLABEL = ‘Lab’ where ENTITYNAME = ‘RouterA’;
• Do not update top-level fields (e.g. ENTITYNAME, ENTITYID)
• All updates are broadcast from model on subject model
• View topology updates from model using:
• ncp_oql –domain <name> –service model –updates
• All updates are also propagated to NCIM

Updates from Dncim2Ncim

21
 ncimCache.lingerTime & ncimCache.managedStatus

• These are a direct one-to-one mapping with NCIM

• An interface in an unmanged chassis is implicitly unmanaged, but will not have an entry in this table
• Default lingerTime of 3 is still in ModelSchema.cfg
• You can use OQL to do manual/auto updates – e.g.#update ncimCache.lingerTime set lingerTime->LINGERTIME=10 where ENTITYNAME=’RouterA’;

• By default, lingerTime is not applied to entities contained in a main node

• If we discover a device, and the set of discovered interfaces has changed, we immediately linger out any missing interfaces
• If required, lingerTime can be set for individual entities of any type in discovery stitchers – for e.g.
if ( specialCase == 1 )
{
Record lingerTime;
@lingerTime.lingerTime = 10;
DncimInsert( entityId, "lingerTime", lingerTime );
}
lingerTime example: managedStatus example:

22
 RemoveNode.pl, AddNode.pl & Topology Editor

Ø RemoveNode.pl does what the name implies

ü It immediately removes a device rather than lingering it out
ü The device is no longer in the topology
ü The device is no longer visible in network views
ü The device is no longer polled
Ø AddNode.pl continues to add a device using partial discovery
Ø Partial discovery via ncp_g_event ‘disco’ plug-in(event driven) is a direct OQL service
Ø Don’t confuse AddNode.pl Vs ncp_g_event plug-in

Ø Use Topology Editor to add devices instantly and get Polled.

ü Manual devices can still be added and deleted
ü Manual connections can still be added and deleted
ü If a device with the same entityName as a manual device is
subsequently discovered, it will completely replace the manual
entity.

23
 ITNM Cross Domain Visualization

Cross Domain Visualization

Ø Multiple ITNM domains discover network in pieces
Ø RCA is still limited per domain instance
Ø Cross domain visualization is new feature introduced in v3.9-FP4 onwards.
Ø ITNM GUI was enhanced to allow cross-domain topologies to be rendered.
Ø V3.x guidelines for domain size 400k elements
Ø V4.2 guidelines for domain size 1M entities.
Ø Scalability
Ø Discovery Time, Polling etc.
Ø Must run two cycles of discovery before enabling this feature.

24
 Discovery Filters & Topology Enrichment

Make use of filters to discard unwanted devices, interfaces and specific interface data from the devices:

a) IPAddress based Filter at Finder level (using Ping Finder & Discovery Scope)

b) Device Level Filtering using Pre-discovery Filter (against Details agent – Details.returns table)

c) Removal of redundant/backup interfaces, local neighbor information etc using Global Agent Filter

Ø ($NCHOME/precision/disco/agents/DiscoAgentFilter.filter)

d) Post Discovery Filter DiscoScope.cfg – scope.instantiateFilter )

e) Scope Special – for Out of band interfaces (DiscoScope.cfg under scope.special table)

f) SNMP Instance Filter (ncp_dh_snmp - DiscoSnmpHelperFilters.cfg )

Discovery enrichment via File Finder method - https://ibm.biz/BdX777

25
 Discovery Filters & Topology Enrichment
Ø SNMP Interface Filters using SNMP Helper (ncp_dh_snmp) - DiscoSnmpHelperFilters.cfg
Ø The result can be monitored via $NCHOME/log/precision/SnmpHelperDebug.DOMAIN.trace
Ø Enable LogRequests in $NCHOME/etc/precision/DiscoSnmpHelperSchema.cfg (set m_LogRequests = 1).

Columns are:
In short, snmp helper sent 92k requests and got nothing
Target's IP address
due to filter defined in WalkForInstanceFilter
Time taken to process request (hundredths of a second)
Number of objects returned by the request
Whether or not the request succeed. Poll status - See below
Number of SNMP requests sent to satisfy this request
Number of SNMP requests that had to be retried to satisfy this request
Number of requests that were active when this request started being processed
Number of requests that were active when this request finished being processed
Discovery agent on whose behalf this request is being processed
SNMP context to be used for this request (equivalent to SNMPv3 contextName)
The request type. Unfortunately the ITNM nomenclature is inconsistent with more common usage of the same terms, but we have: 0 = Get, 1 =
GetNext (Walk), 2 = GetBulk (Get + Walk in one request), 3 = Get device access credentials, 4 = Get MIB object information, 5 = Get MIB children.
The number of objects requested
A list of the objects being requested (truncated if string > 300 characters)
26
 Discovery – TagManagedEntities.stch(v3.x) Vs PopulateDNCIM_ManagedStatus.stch (New)

Background:
§TagManagedEntities.stch is used to use unmanage entities via Discovery Engine.
ØCertain type of entities you do not want/need to monitor (e.g. Dialer , Async ) & Out of scope Entities etc.

§Location of the stitcher:

ØV3.x - $NCHOME/precision/disco/stitchers/TagManagedEntities.stch
ØV4.x - $NCHOME/precision/disco/stitchers/DNCIM/PopulateDNCIM_ManagedStatus.stch

ManagedStatus (0, 1, 2 & 3) propagation:

From Discovery -> Model -> NCIM

§What’s honored in v3.9 Vs. v4.2 ?
§Ideally only values 2 & 3 should be passed from Disco to Model.
§In v3.9 – you can feed any value from Discovery to Model.
§In v4.x only 2 and 3 are passed onto Model via Disco.

From NCIM -> Model:

§Values 0 and 1 either via direct insert/update(s) to ncim.managedStatus table
§Unmanage/Manage tool execution via GUI (Structure Browser , Hop / Network Views, AEL etc.)

More you can read at - https://www.imwuc.org/p/do/sd/topic=674&sid=3214

27
 Monitoring – Scoping polls..

NCIM Schemas

NCIM
OMNIbus
NCMIB
NCPGUI
ObjectServer
alerts.status
NCMONITOR
activeEvent
NCPOLLDATA
28
 Monitoring – Longer Term Performance Data Collection (Tivoli Data Warehouse -TDW) –v3.8 & v3.9

Ø Data stored in Tivoli Data Warehouse

Ø No migration option to retain Historical data
Ø Better off to recreate policies and definitions in v4.2
29 due to architectural & design changes.
 Monitoring – Longer Term Performance Data Aggregation Using Apache Storm – v4.2

DASH Features that consume this data

NM GUI 4.2 • The new Network Health Dashboard
(NOI entitlement only, not NNM)
WebGUI 8.1 • TCR/Cognos reports
• Right click performance widget from topology views
TCR 3.1

Database server ITNM Core server

Apache Storm
• Aggregates the raw data from pollData table
• Populates the aggregated tables (each with the same
schema)
• Only one instance active
NCIM Apache Storm • Self-sustaining, no setup or maintenance required

Aggregated
tables
Data Aggregation
Spout
• Raw data – 1 hour
• Last 24 hours
• Last 7 days
Raw pollData
• Last 30 days
• Last 365 days
ITNM pollers
Each table is partitioned in time slices to maximize
performance and auto incremental pruning
SNMP
ICMP

Note: We’ve ITM agent for ITNM – but only for process monitoring , more @ https://ibm.biz/Bd4buT

30
 Extend storage of Performance Raw Data Using NPI v1.2 Integration
ITNM ó NPI v1.2– New Microservice Architecture (Data Flow)
Data Ingestion Integrations

Flow Collector Svc. UI Svc. DASH

ITNM 4.2
Poller NM Collector Kafka Event Svc. STDIN Probe OMNIbus
Svc
NM NCIM New storage technology based on
ITNM Svc column-store data warehouse for
NCIM large capacity and fast access

Storage Svc. Fault tolerant and high data

HDFS throughput rates

NOI Integration Flexible performance data retention

Leverages existing network discovery periods of raw and aggregated data
and performance polling,
SNMP and ICMP polling of MIB II and Flow Analytics Svc. Scalability: add or remove nodes as
other standard Enterprise device MIBs needed

Entity Analytics Svc. Analytics engine used for

Aggregation, performance data anomaly detection
Simplified “Ambari” based
Forecasting,
micro-service orchestration,
configuration and PM Anomaly Detection
management

Integration with NPI brings two key new capabilities

1 – Extending storage of Performance metrics for Anomaly Detection.
31 2 – Flow
 Visualization
Ø DASH Integration:
Ø Applet free portlets (HTML5 Architecture)
Ø JRE is a requirement if you want AEL instead of using 'EventViewer’
Ø Pages related to Topology Visualisation are now located under the Incident
navigation toolbar in Dash under the 'Network Availability' heading
Ø Improved Search with-in Network Views
Ø Enhanced Tooltips
Ø ITNM Discovery is isolated
Ø Database access & Network Polling under 'Administration’
Ø Topology Search (feature – don’t confuse with Topology Search part of Analytics)
Ø Canvas Navigator
Ø Status Stage Rendering

32
 Visualization
Migration of NetworkViews:
Rely on using dynamic templates Auto provisioning
For e.g. :
Copy from v3.8 -
$TIPHOME/profiles/TIPProfile/etc/tnm/dynamictemplates/ip_default.xml
To
V4.2 - $NMGUI_HOME/profile/etc/tnm/dynamictemplates/ip_default.xml

Three Steps to get Network Views Post discovery:

1. Call Dynamic Template from Create NetworkView task.
2. Select your custom template
3. Add the Top Container to Bookmark list
Will need to create manually views that are manually created..
If your template relies on custom table, make sure your NCIM db is upto date.

33
 Root Cause Analysis
Event Gateway:
Ø Event enrichment is no longer under NcoGateSchema.cfg (new file is called EventGatewaySchema.cfg)
Ø Event enrichment via $NCHOME/precision/eventGateway/stitchers/StandardEventEnrichmet.stch
Ø e.g. https://www-01.ibm.com/support/docview.wss?uid=swg21595472
Ø ITNM Event Gateway now using IDUCv2 – helps tremendously both scalability & performance.
Ø Migration – Event enrichment is a manual effort from cfg to stitcher.

RCA:
Ø RCA Rules are now stitcher based unlike v3.8 (Amos rules).
Ø RCA Engine is now a plug-in of Gate component
Ø New Cfg File – RCASchema.cfg (replacing old AmosSchema.cfg)
Ø NmosEventMap is supplied via NcKL (replacing NcoGateInserts.cfg)

Ø Migration – no migration necessary as the Event Suppression is same.

Ø Note – the order of the Event suppression changed in v4.2 Vs v3.8

34
 Network Services and Event Correlation

Service Affecting Events - SAEs

More about MPLS VPN Example – https://ibm.biz/BdXGtu
SAE can be configured to work on any Groups – for e.g. Faults for IGMP.

ITNM EMS Integration for Event Enrichment and RCA

– e.g. SAM 5620 (EMS) & , Nokia5529idm (AMS).
Ø In the past ITNM was only used to discover EMS & AMS for
Topology Visualization

Ø Now, we enrich EMS & AMS events and Root Cause

Analysis for complete End to End Solution.

Ø The Probe enrichment rules for EMS & AMS GAed and
available at - https://ibm.biz/BdiiN4

Ø Read more about Probe Extension packages @

https://ibm.biz/Bdiihw

35
 Reporting - Tivoli Network Manager v4.2 - BIRT to Cognos Report migration

The following reports have been migrated from BIRT to Cognos and are part of the Tivoli Network Manager 4.2 report portfolio

Performance Reports
Network Technology Reports
• Bandwidth Top N
• Historical SNMP Top or Bottom N • BGP Details,
• System Availability Summary • BGP Summary
• Device Availability Summarization • MPLS VPN Details
• Interface Availability Summarization • MPLS VPN Summary
• Device Summarization Troubleshooting Reports
• Interface Summarization • Connected Interface Duplex Mismatch
Monitoring Reports • Devices Pending Delete on Next Discovery
• Monitoring Device Details • Devices with Unclassified SNMP Object IDs
• Monitoring Policy Details • Devices with Unknown SNMP Object IDs
• Monitoring Summary report
Asset Reports
• Card Detail by Device Type

High Availability and Miscellaneous

ØITNM HA – no major changes for core components between v3.8 & v4.2
ØNCIM HA – ‘Failover NCIM’ option deprecated and relies on native database HA (e.g. DB2 HADR, Oracle RAC).
ØReference - https://developer.ibm.com/answers/questions/295047/itnm-high-availability.html
ØDASH HA – ITNM GUI (topoviz) is isn’t in the mix unlike WebGUI.
ØThere isn’t any in-place upgrade, its always side-by-side install and migrate.
36
 Upgrade / Migration for 3.8 Customers
Ø There isn’t any migration support from v3.8 to v4.2
Few major areas to focus in-order to migrate with little effort from v3.8 to v4.2:
Discovery:
- Copy Discovery Scope & Seed configuration from from 3.8 to v4.2
- DiscoScope.cfg , DiscoPingFinderSeeds.cfg & DiscoFileFinderParseRules.cfg

- Add manually in Discovery GUI Configuration Snmp community passwords or decrypt via CLI from v3.8 and add them to
SnmpStackSecurityInfo.DOMAIN.cfg & TelnetStackPasswords.cfg
- For e.g. #ncp_oql -domain Version38 -service Config -schema SnmpStackSchema.cfg -username admin -password '’
- ncp_oql#select * from snmpStack.verSecurityTable;
- Kick off full discovery followed by customization per your 3.8 environment (e.g. select correct agents, topology enrichment, filters etc)
Polling:
- Create & Update Poll policies manually
Event Enrichment:
- Migrate manually changes from NcoGateSchema.cfg to StandardEventEnrichmet.stch
Network Views:
- Follow autoprovision process using dynamic templates per slide #32
Others: Require manual effort – for e.g. scripts updating Discovery scope/seed list, decommissioning devices etc..

37
 Upgrade / Migration for 3.9 Customers
Ø Re-written & re-structured Migration Documentation – more @ https://ibm.biz/Bdijqp
Ø More Capable Core & GUI Migration Scripts
Ø nmExport/nmImport (e.g. backend config, stitchers...)
Ø nmGuiExport/nmGuiImport (e.g. gui config, custom pages, roles...)
Ø What to Expect
Ø Standard Configuration: Fully or partially migrated, highlights any manual.
Ø Advanced Customizations: Flagged for manual migration
Ø You can migrate from any of the following versions to Network Manager V4.2:
Ø V3.9 Fix Pack 3 or later
Ø V4.1
Ø V4.1.1
Ø V4.2 (Cloning/copying a V4.2 installation to another server)
What Data is migrated
• The following kinds of data are migrated:
• Configuration files.
• Network views.
• Discovery configuration data.
• Multiple domains. Your domains are automatically recreated, and the network
• Views and monitoring policies are imported.
• Passwords and SNMP community names. This data is unencrypted, imported, and re-encrypted automatically.
• Poll policies and poll definitions.
38 • The SNMP MIBs have been moved from the GUI server to the core components server in Network Manager 4.2.
 Migration: Core Migration Enhancements Overview

• Core (backend) Migration Covers:

$NCHOME/etc/precision/ Configuration Files, AOCs, MIBs
Stitchers, Network Views, Poll Definitions, Poll Policies, Domain Creation
• We do NOT migrate:
Network Topology, Discovery cache files & NCIM cache files, Historical Poll data
• Improvements:
• Standardised Algorithm (for .aoc, .mib, .stch, .agnt, .cfg)
• Improved Summary Files (covers .aoc, .mib, .stch, .agnt, .cfg)
• More Intelligent .cfg Auto Migration
• Improved ITNMDataImport.pl (called by nmImport to perform core migration)

• Introduced new summary files for core

• NMCoreFilesForManualReview.txt
• NMCoreFilesMigrated.txt
• Under the hood
• nmImport calls ITNMAutoMigrateCfg.pl script
• Action Controlled via ConfigSchema.cfg; migration.autoMigration
39 • Unfortunately comments will be lost
 GUI Migration: Core Migration Enhancements Overview

Prepare for migration:

• Install the Network Manager GUI components.
• Ensure that you can log into the Network Manager GUI.
• Recreate all the Network Manager users from the source installation on the target installation. Create the users in the appropriate repository: LDAP or
ObjectServer. If there were any users created in the Tivoli Integrated Portal, create these users again. Network Manager 4.2 does not use the Tivoli
Integrated Portal. Note the user names and passwords. You need these credentials when you migrate user roles.
• Do not start a discovery or begin to work with Network Manager. Begin the upgrade.

Exporting the GUI migration files

• On the target server where the GUI components are installed, run the following command:
• /opt/IBM/netcool/gui/precision_gui/install/scripts/makeExportPackageGUI –j /opt/IBM/JazzSM -n /opt/IBM/netcool/gui/precision_gui
• The script creates a package containing the scripts that you use to export data from the source installation. The package is named
ExportPackageGUI.tar.
• On the source server, decompress the ExportPackageGUI.tar file.
• In the scripts directory that was created by decompressing the file, run the command:
nmGuiExport -u tipadmin|smadmin -p password
• The nmGUIExport script creates a data.zip file in one of the following directories:
• $TIPHOME/profiles/TIPProfile/output/
or
• $JazzSM_HOME/ui/upgrade/data/.

40
 GUI Migration: Core Migration Enhancements Overview
Importing the GUI migration files
• Create the $JazzSM_HOME/ui/input/ directory on the target GUI server and copy the data.zip export file into it.
• Ensure that Dashboard Application Services Hub is running.
• Ensure that the environment variables for Network Manager are set on the target server
• Which files are migrated is controlled by the configuration file $NMGUI_HOME/integration/properties/mergeDefinition.json. You do not need to edit this file but you can:
• Configure which files to add or remove from migration
• Configure which fields
• In the scripts directory on the target server, run the nmGuiImport script using the following command: nmGuiImport -u smadmin -p password [ -f path_to_zip_file ]
• Manually add users to the appropriate user groups on your new system. The import of GUI data does not include user group membership data.
• Review the results of the import in the NMGUIFilesForManualReview.txt file and perform any necessary manual migration steps.

GUI properties file merge tool

1) Simplify migration
2) Merge tool is called automatically by the Import Plugin when a successful migration happens
3) Will help customers because there is no more need to manually merge properties file
4) Speed up the migration process
5) Generate at file NMGUIFilesForManualReview.txt a migration report that includes:
a. Each property that was merged
b. Each property that was not merged and the reason why
c. Any manual action that is needed

41
Efficient
Operations
with Netcool
Analytics

42
Strategies for getting value from NOI – Agenda

Netcool Operations Insight capabilities

How do they work together?
How do I get started?
Some results from the field
Where can I get further information?

43
Netcool Operations Insights (NOI) capabilities

Analytics-based event grouping

Scope-based event grouping
Seasonality
Logfile Analytics
…which one(s) should I use first?

44
NOI capabilities – event grouping goals

Event grouping – goals:

- Organise/group events by incident
- Reduce pages and trouble tickets
- Related alert information in one place
- Aid quicker problem diagnosis
- Reduce MTTR
- Save on operational costs!

45
NOI capabilities – analytics-based event grouping

Analytics-based event grouping

- Analyses event history database
- Finds events that occur together
- Can “deploy” discovered groupings
- Auto-groups future instances of these
- Can be generalised to a pattern

Discover unknown event relationships

3+ examples must be present in history

46
NOI capabilities – scope-based event grouping

Scope-based event grouping

“group events together that occur at
the same place, at the same time.”
- Scope = “same place”
- Quiet period = “same time”

ScopeID = “AUCKLAND”
Quiet Period = 900s

TIME
CLOSE GROUP

47
Use Case#1

CI based event grouping

• Multiple events created for the same outage within the same CI (device,
server, application, database)
• Multiple similar events created for the same CI, that will be worked by the
same technician

48
Use Case#2

Relationship correlation
• Multiple events from the same or different CI’s generated by the same or
different EMS’s related to the same outage

49
NOI capabilities – Seasonality

Seasonality
- Analyses event history database
- Finds events that predictably repeat
- Hour, minute, day of week or month
- Either we care: fix underlying issue
- Or not: create a rule to handle them

50
NOI capabilities – Seasonality

51
NOI capabilities – Logfile Analytics

Provides real-time event search for triage – over current and historical event data

52
 NOI capabilities – Logfile Analytics

Provides pre-canned reports for identifying opportunities for operational efficiencies

53
How do the capabilities work together?

Analytics and scope-based grouping

- groupings created by either can coexist

Analytics-based grouping and Seasonality

- some events are both

Seasonality and Logfile Analysis

- Identify chronic and/or problem areas

54
How do get started? Using NOI…

Scope-based event grouping

Analytics-based event grouping

Seasonality review

Logfile Analysis results review

55
How do get started? Using NOI…

1. What relationships do I know about?

- Apply scope-based event grouping
- Can apply first because data-driven

2. What relationships do I not know about?

- Run the analytics (use “strong” profile)
- Use filter: ScopeID = ’’
- Review and deploy the groupings
- Need at least 1 month worth of data
…the more the better

56
How do get started? Using NOI…

3. What chronic problems do I have?

- Review Seasonality results
Do I care about each case?
- Yes: let’s fix the underlying problem
- No: let’s suppress the (groups of) events

4. What are my main problem areas?

- Review Logfile Analysis hotspot
dashboards

57
How do get started? Using NOI…

► Resulting benefits:
- Event grouping will be maximised
- Chronic problems identified
- Chronic problems fixed or suppressed
- Main trouble areas identified
- Maintenance work can be prioritised
► This will result in:
- Faults can be triaged and fixed faster
- Overall efficiency can be increased
- Overall costs can be reduced

58
How can I measure my effectiveness…?

► Event grouping
- Calculate % row reduction due to grouping

► Reduced ticket counts

- Compare before and after # tickets created
- Can directly calculate savings

► Reduced event counts

- Compare overall number of events

59
Some results from the field
75,134 357 240 Overall
North American
Bank 70% reduction in
Events analysed Event groups identified Average events per group events

Seasonality and analytics-based event grouping enabled an SME to identify that a

Managed Services
Provider single point of failure had been occurring regularly up to that point – highlighting a
previously unknown deficiency in their network architecture.

Reduction in
Middle Eastern The customer leveraged analytics-based grouping and Seasonality to achieve a 21% events
Telco reduction of 21% in events and 11% reduction in tickets. Reduction in
11% tickets

Reduction in
European Auto Identified opportunity to reduce the number of displayed events by up to 34% as 34% events
Manufacturer well as realise a 18% reduction in tickets Reduction in
18% tickets

Applying NOI capabilities to VMware events reduced incidents that generated tickets
Reduction in
VMware events by 33% by leveraging event grouping and by 11% by resolving the underlying issues 44% tickets
highlighted by Seasonality.

Event grouping identified a previously unknown cross-domain correlation between

Asia/Pacific Telco BNG and MSAN networks while Seasonality identified a short-fall in cell capacity at
peak times in the public space network.

60
IBM
Service
Offerings

61
 NOI for Networks Service Offerings
• NOI Solution Workshop – Network Management

Duration: Additional 2 Days to NOI for Operations Workshop (4 days) 6 days in Total.

• The Solution Design Workshop helps get clients up and running quickly with Netcool Operations Insight (NOI) for Network Management. Let
a NOI SME guide you through the process of gathering requirements for an initial deployment and defining the required solution architecture.
The output of the workshop is to be used as input for the NOI Quick Win Pilot for Operations and Network Management

Description: After an initial site visit, IBM will provide services to create a solution design document based on a standard template.
Deliverables: Solution Design Document including:
• Discovery and Polling Requirements captured
• Configuration and Compliance Requirements captured
• Network and EMS Vendor requirements captured
• Details of key use cases and dashboards to be created
• Overview of functional and non-functional requirements including - HA and resilience
• Sizing information based on Number of Network Entities
• Solution Architecture overview including servers and software

62
 NOI for Networks Service Offerings
• NOI for Networks - Upgrade Legacy Network Management to Netcool Operations Insight for Networks

Effort: 10 days + 5 days for High Availability for Network Manager

Effort: 10 days + 5 days for High Availability for Configuration Management

• Used for an existing Network Management Customer.

• Leveraging new capabilities such as Network Health Dashboard and Topology Search
• Upgrade to Netcool Operations Insight for Network Management using IBM’s expertise to ensure rapid time to value. Using an
Asset and Best Practice approach, migrating existing configuration from the existing environment.

Deliverables :
• Installation and configuration of one instance of Netcool Operations Insight for Network Management into a production ready environment
• Prerequisites to be carried out by client prior to deployment– ie Server Deployment, OS libraries and Software Download
• Migration of existing ITNM configuration is completed using standard upgrade methods, including Configuration files, Network Views, Discovery
Configuration, multiple domains, passwords and SNMP community names, Poll Polices, MIBs, new or customized discovery collectors
• Migration of existing ITNCM configuration is included and up to 5 worker servers
• On site knowledge transfer to bring customer SMEs up to speed on the new Network Management features delivered by IBM expert SME
• Guidelines and best practices for moving to environment into production
• Documentation to support the deployment including Run Book procedures for starting and stopping solution

63
 Engage IBM
Send your questions on the NOI for Networks Service Offerings to:
[email protected]

64
 Netcool Operations Insight – Reference Material

Best practices for Operations

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli+Netcool+OMNIbus/page/Best+Practices

Best practices for Network Management

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Tivoli Network Manager/page/Best Practice
Documents

Netcool Operations Insight Deployment Guide Redbooks:

http://www.redbooks.ibm.com/Redbooks.nsf/RedbookAbstracts/sg245348.html?Open

Improving Operations Effectiveness and Efficiency with IBM Netcool Operations Insight: A Scenarios Guide Redbooks:
http://www.redbooks.ibm.com/Redbooks.nsf/RedbookAbstracts/sg248352.html?Open

Delivering Consistency and Automation with Operational Runbooks Redpaper:

http://www.redbooks.ibm.com/Redbooks.nsf/RedbookAbstracts/redp5347.html?Open

NOI Network Management Deployment in 4 hours - https://www.imwuc.org/p/do/sd/sid=3160

Netcool Performance Insight(NPI) Deployment in an hour - https://www.imwuc.org/p/do/sd/sid=3340

 Q&A

© 2016 IBM Corporation Page 66

66