SL Traffic Reporting and Analysis Course 2.0 R9.5.0.0
SL Traffic Reporting and Analysis Course 2.0 R9.5.0.0
Version 9.5.0.0
Unit 1 Sightline Visibility
Overview
Sightline Visibility Course
• Do the introductions
• Instructor introduction
• Student introductions
Unit 2
Unit 1 Unit 3
Sightline
Sightline Visibility Network Visibility
Reporting
Overview and Reporting
Overview
Unit 5
Unit 4 Unit 6
Capacity Planning
External Visibility BGP Reliability
and Congestion
and Reporting and Reporting
Reporting
Unit 8 Unit 9
Unit 7
Traffic MO Multi Dimensional
OTT Reporting
Reporting Reporting
Our goals in providing network visibility and traffic analysis as core value to network
operators
• Achieving Excellence in Nework Operations and Service Availability
• Gaining the knowledge to act with customer analytics and business insights
There are various scenarios and requirements for when to use Sightline traffic reports. It is not
only for DDoS purposes.
• Network Operations
Growth of social media traffic since January by 27%
• Enterprise Security
Abnormalities and threat details
• IT
Traffic increase on systems supporting remote work of 74%
• Network Planning
A new opportunity is discovered for direct peering to save transit cost
• Application
Growth of conferencing apps over the last month by 137%
Network traffic is determined via the network boundary. The network boundary is the border
between your network and the rest of the internet. This boundary is used to determine
when and where traffic enters your network.
The external boundary (interfaces), global boundary, and/or network boundary are all the same
terminologies.
Flow is the main technology Sightline uses to receive data from routers.
BGP neighborship between routers and Sightline is also required to correlate Flow and BGP
data for the traffic reports.
SNMP information provides interface discovery to help to create network boundary.
Solution
✓ New CDN pays for an on-net cache
✓ The MMO game now peers at operator’s IXP (Internet Exchange Point)
Transit traffic reduced to < 35%; gaming and streaming performance improved
An ISP is using Sightline and notices an 18% traffic volume increase in its transit links.
Sightline can provide you a breakdown of the transit links’ utilization historically, so customers
have an idea of which links are being used for which applications.
After discovering which application triggered the traffic increase, more internal discussions
concluded that last month some streaming services changed CDN (Content Delivery
Network).
As a result of the CDN change, new MMO (Massively Multiplayer Online) increased transit
traffic significantly.
MMO Game: A massively multiplayer online game (MMOG, or more commonly, MMO) is an
online game with large numbers of players.
The MMO Game provider welcomes installing cache servers in the ISP network.
Caches both reduces traffic for MMO games on the transit links and increases gaming quality.
Pre-Defined
Reports
Dashboards
Advanced API,
Reporting Scripting
You have multiple ways to access Sightline traffic reports and data. Pre-defined reports are
located in the Sightline main menu that gives you access to most of the Sightline reports.
Sightline also has various tools to discover traffic for special purposes such as Peering
Evaluation, AS Explorer, Peering Traffic Exchange etc.
You can combine multiple traffic reports in customer reports and run these periodically and
even send notifications or emails to related groups.
Managed objects have dashboards that provide you rich information about the traffic they are
carrying.
If you need traffic reports that do not exist by default, these can be created.
Any data you see on the user interface can be accessed via API and you can use it
programmatically for your own purposes.
Sightline provides more than 350 pre-defined reports. These are fixed reports, so Sightline can
provide fast query results.
Besides the pre-defined reports, you can also create custom reports. These can be scheduled
or run on an ad-hoc basis.
Traffic data is stored in a round-robin database at 5 minute intervals and data points get
aggregated over time depending on the report time period chosen.
All data is stored in Coordinated Universal Time, also known as UTC. Reports are
automatically adjusted to User Time Zone Settings.
Custom Reports allow you to create customized reports and to schedule report
generation, either on a recurring basis, or you can run them ad-hoc.
In addition to the predefined reports, users can also create Custom Reports according to their
needs. Reports are divided into:
• User friendly
• Selectable time frame
• Graph elements are user selectable
• Pre-calculated data from collectors
• Provides two facet traffic reporting
• Sightline + Insight allows multi-facet
reporting
- Easy to use
- Provide selectable time frames with predefined and customized settings
- Graph elements can be enabled or disabled
- Data in the table is built on request by appliance
- Sightline provides two facet traffic reporting
- With Insight it allows multi-facet reporting
The report name provides information on the dimension(s) used, for example Network in
relation to Router, or Network in relation to Interfaces. In the stacked graph, the portion above
the center line represents outgoing traffic, and the portion below the center line represents
incoming traffic.
- Graph data
- Sum of selected items
The table provides traffic information for each monitored item. If there are more than 5 items,
Sightline selects the Top 5 items by default. The maximum number of items for which Sightline
can draw graphs is limited to 10. You can select or deselect items in the table to reduce or
enhance visibility in the graph. To apply changes, press the Update button.
If you want to compare your selected item with the Total Traffic, you can enable the Network
Total which provides a line in the graph for the total network traffic IN and OUT.
Timeframe: Today
• Provides a quick view on
the current usage
• Helpful to investigate
high usage within the
last 24 hours
• Not useful for capacity
planning or forecast /
trends
• Data displayed for
“Today” has a 5-minute
granularity
Time Selection: Today (24h ago from now)
You can change the “Time” period which is covered by any of the reports.
By default, Sightline selects “Today” as the displayed period, which provides graph and table
data for the last 24 hours from the time the report is executed.
It is useful to give a quick overview on todays utilization, but not very helpful for capacity
planning or to build forecasts/trends. These should always use a much longer period if time (≥
month).
The “Yesterday” period covers the full 24 hours of the previous day, from 00:00 to 23:59 UTC
(default) or the user’s specified time zone.
This period is useful to spot peaks in your network but as with the “Today” period, it should not
be used to plan your capacity because it provides only a short-term view.
The “Week” period covers 7 days ago from the time the report is executed.
This period provides information on typical weekly usage, and it can spot changes in the traffic
or recurring peaks, eg:
Customer high traffic on Monday to Friday, Saturday and Sunday less or no traffic, normal
office hours traffic etc.
The data granularity is 30 minutes. This means the system creates an average out of six five-
minute samples. Short peaks will not be visible.
Important: 1 week equals 7 days. 1 month has 4 weeks so this is always a fixed value of 28
days.
Useful to see changes over a longer period and to identify hotspots; can be used to plan
capacity.
Data granularity is 120-minutes. Sightline takes four 30-minute samples and builds the
average. Short peaks which can exhaust your capacity could be missed.
The “Yearly” period provides a view on the last 52 weeks, which is helpful to see trends and
major changes over a longer period. Because this view provides a granularity of 24 hours, or
traffic for one day, it may not be helpful for capacity planning as peaks in traffic during the day
will be smoothed over.
• Use the Calendar icon to select • These define exact Start and End times to search for
Start and End Times specific events
• Click the Update Button to apply • Click Update to apply changes.
changes
The “Other” time period allows you to specify a Start and End time from the calendar or use
phrases as shown in the example. You can enter phrases such as 4 hours ago and last 3
hours.
You can use the zoom function to drill into specific areas of your reporting graph. Once you
have selected the zoomed time range in your graph, Sightline will update graph and table
immediately. To narrow down your report, you can deselect the reported items. The table gets
updated after pressing Update button.
Example: If you select Today for the time period, the report includes data for the previous 24 hours
with a 5-minute granularity. If you select a start time of 10 days ago and a stop time of now, Sightline
returns samples with a 30-minute granularity.
Sightline stores all data for traffic reports in a round-robin database. This table shows how
Sightline returns data based on the time period of a report and how long Sightline stores the
data.
The default graph type for traffic reports is a stacked graph, it provides IN and OUT traffic
simultaneously.
As for the bar graph, the pie graph provides a traffic view for IN, OUT and Total. It gives a
percentage overview for each selection. This is useful if you have no more than 5 items you
want to compare. A maximum of 10 items can be selected.
Each report can provide different traffic calculations which take effect on the data. The graph is
updated when you select one of the four calculation options. The Current calculation shows
data based on a the most recent 5-minute sample. The Current calculation is only visible and
selectable if the time period is specified as Today. The Current calculation is not useful for
reporting or capacity planning.
The Average calculation provides the average for all samples for the selected time period.
Average is useful to see typical utilization over a longer period, like 1 week or 1 month. The
network should be able to transport the average traffic without any negative impact.
The Max calculation displays the maximum of all samples for IN and OUT separately. Because
of the nature of traffic flow, the time of the observed maximum incoming traffic can differ from
the time of the observed maximum outgoing traffic. The Max calculation is used to identify
traffic peaks and bottlenecks in the network.
The 95th percentile or PCT95, is a common method used to calculate bandwidth usage for
traffic accounting and service level agreements (SLAs). Use PCT95 for customer and peering
reports. It is a fair method because it closely reflects the required capacity of resources. The
next slide illustrates how the 95th Percentile data is calculated.
140
140
120
120
100
100
This is the 95th Percentile (95PCT)
80 80
60 60
40 40
20
20
0
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Let's assume we have a graph like the one Data values are sorted and ranked and the 95th
above. For the sake of simplicity, we take 100 percentile value is taken. This is a common method
measure points. Traffic is varying and we have used for capacity planning and billing IP transit
some peaks and troughs. utilization.
For 95 Percentile, measured data values are sorted and ranked. Assuming we have 100 data
points, the system draws a notional line at the 95th value. The highest 5% of traffic is not used
for calculation.
The 95th percentile says that 95% of the time, the usage is at or below this amount.
Conversely, 5% of the samples may be bursting above this rate.
• User friendly
• Modern style and technology
• Quick responding
• Reports are built by browser
• Removes load from
appliances
If a pre-defined report
has migrated, you will be
redirected to the Explore Traffic
page with the correct filter(s)
applied
Starting with Sightline 9.3, pre-defined reports began to migrate into latest generation reports.
If a selected report has already been migrated, it will be redirected to the Explore Traffic page.
Latest generation reports are built by the browser, they are much faster to build which gives a
better user experience.
FILTER2 (Optional): depending on FILTER1 selection you can define more precise criteria
Changing any of the Settings on Filter / Time Period / Units are reflected in Graph and Data
values once Update Button is pressed.
The Explore Traffic page has two filters. Depending on which Type is selected in Filter 1, there
are additional options available for Filter 2. In contrast to pre-defined reports, values for each
filter can be selected to narrow down the report.
As with the standard reports, Explore Reports allows the Time Period to be specified by
selecting a predefined period or selecting them from a calendar. Graph and traffic data are
updated with the Update button.
As with pre-defined reports, you can also zoom into any time within the graph. Make sure to
click the Update button to refresh the graph and table details.
Please note that the shortest time frame for which Sightline provides data is 5 minutes.
In the Explore page, hover over the graph with the mouse. Detailed information for that time,
for all selected Items, is displayed.
Changing the calculation method, Last (Current), Average, Max, PCT95, and Total,
immediately updates the table below.
If you select or deselect items in your report, the graph and table will automatically update.
There is no need to press the Update button.
The default graph type for traffic reports is the Stacked Graph. It shows IN and OUT traffic
simultaneously, and the view can be changed between IN, OUT and IN + OUT. For stacked
graphs, note the following information:
• Data above the center line represents outgoing traffic, and data below the center line
represents incoming traffic
• Sightline converts all data to the configured time zone that is selected in the current user
profile
• The Total row, the last row of the data table, displays the total traffic of the target object
The Line Graph draws a line for each reported item and it is related to the monitored traffic.
It provides a better view on traffic per item, compared to a Stacked Graph.
You can also change between IN, OUT and IN + OUT reporting.
As for the bar graph, the Pie Graph provides graphs for IN, OUT and IN + OUT (Total). It gives
a proportional overview on distribution which is useful if you have no more than 5 items to
compare. A maximum of 10 items can be drawn, but the more items selected, the harder the
graph is to read.
The Sankey Diagram gives a view on the traffic relationship between Filter1 on the left side
and Filter2 on the right side. This type of diagram is also used in Insight.
The My Sightline page provides multiple details about the monitored network. It provides a
Network Summary graph which shows the measured traffic from your monitored routers.
N4
N1
Select or deselect the different graphs which will be non-persistent and reset once you leave
the page. Or click on the gear wheel and select or deselect the details to make them persistent
for the current user account.
You can use the various Sightline dashboards to view a summary of the selected report type –
in this example we see the Customer Dashboard. This dashboard provides at-a-glance details
for various reports.
• Displays links to
and descriptions of
other reports that
may be of interest
to you
• Click on a report to
navigate to it
From the More Reports Tab you can jump into other reports which are all in relation to the
report type you have selected.
In this example we have selected the Customer Dashboard and if we use any of the reports
in the More Reports tab, a new report page in relation to Customer will open.
For example, select Cities and the Explore Page with Filter1 = Customer and Filter 2 = Cities
will open.
• Can run on a
scheduled basis and
sent to a
preconfigured
notification destination
on completion
Administration > Reports
The Configure Reports page (Administration > Reports) allows you to search for, configure,
and view custom traffic reports.
You can use the Configure Reports page to create custom wizard reports about different
perspectives of your network’s traffic.
You can configure classic XML reports using the Configure Reports page. These reports allow
you to generate and export raw XML data with your customized DoS information to integrate
with other reporting tools.
• Output formats:
- HTML
- XML
- CSV
- EXCEL-XML
- PDF
The Configure Reports page (Administration > Reports) allows you to search for, configure,
and view custom traffic reports.
You can use the Configure Reports page to create custom wizard reports about different
perspectives of your network’s traffic.
You can configure classic XML reports using the Configure Reports page. These reports allow
you to generate and export raw XML data with your customized DoS information to integrate
with other reporting tools.
N4
N1
This report shows Network Summary traffic (one dimension). We can find this Report at
Reports > Network > Summary.
N4
N1
A
er
om
st
Cu
This report shows Customer Summary traffic for selected customer MSU.
Graph and data values provide information for Time Period Yesterday, the full 24h of the
previous day.
B
P F
IS P
IS
N2
N3
A
P
IS
N4
N1
In this graph we see a Peer Summary report for Peer_ISP_F; Time Period This Week (7 days
ago from now).
We have a gap on the right side because this week graph has a granularity of 30 minutes and
graph information is updated every 30 minutes.
N4
N1
Sending the page as email will create a PDF which contains the
If you click on the icon, you can send the same content as the shown page. This works on any page.
shown report as email.
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 53
You can send the report page as a PDF directly from the page itself. This is useful to provide a
quick report to the requester.
You must have a valid SMTP server configured.
PDF is a fixed format and has the same content as the page view.
XML, CSV and Excel-XML formats allow you to integrate or process the data with your own
tools. e.g. customer report dashboard or billing system etc.
You can use the information icon available on most Sightline pages to get quick information
about that page.
If you want to consult the System User Guide you can use the question mark icon and this will
open the User Guide in a context sensitive manner.
Explain context sensitive manner. ie the help for the particular graph.
Solution: Q1 = A ; Q2 = B + D ; Q3 = C ; Q4 = B
It includes bi-directional
traffic crossing the
network from N4
N1
Customers, Peers,
Services etc.
The Network Summary page provides an overview on the total traffic entering and leaving the
network. Additionally it shows the combined Traffic IN + OUT and multicast traffic.
Most reports provide traffic data for Top 5 Applications, Customer, Interface, Router…
The Applications Report will help you to view traffic for known applications. It references the
Name Mappings List which can be viewed and changed under Administration > User Interface
> Name/Number Mappings.
The Network Routers report shows traffic from every router which is sending or receiving traffic
from/to external networks.
If Sightline sees external traffic from more than 5 routers it will select the top 5 by default. You
can modify the selection to focus on the routers you are most interested in. Once changed, you
have to press Update to refresh the graphs.
Like the Network Router report, Network Interfaces shows each interface on which Sightline
has seen traffic received or sent to/from the network. It preselects the top 5 interfaces by
default.
You can change the selected interfaces you want to focus on. You need to press Update to
refresh the report.
The Network Countries report provides you information on traffic by external country.
Besides countries it can also contain the following Information:
• Anonymous Proxy = Traffic from known Anonymous Proxies
• Satellite Provider = Represents Satellite Providers
• Europe (EU) and Asia-Pacific (AP) = this location appear when the end user location is
unclear e.g:
A corporate proxy that is located in Paris, France could be listed as Europe if the actual users
connect from different parts of Europe. Because the traffic originates from various places in
Europe, “Europe” is used for the country and not France.
Gives an overview on
traffic split by region
and city.
Can help to
optimize network and
service performance.
Traffic is counted for this report if it crosses the global network boundary.
Regions and City data is obtained through network boundary data. If a city is not within your
network boundary, then Sightline cannot obtain data about it.
In this example you want to answer a question on top Origin Networks. This Report is related
to External Networks with which your network exchanges traffic.
Reports > Network > BGP > ASNs Origin
This report shows traffic exchanged between your network and external networks based on the
Origin AS Number.
This report does not require managed object configuration.
Requires a configured
Customer managed object.
As another example, we need a report which shows the top customers using your network.
This predefined report will show you the customers receiving or sending the most traffic. You
must have managed objects configured for each customer.
As a time period you should use at least one week otherwise you can miss changes. Some
customers may have specific peak times or days.
The IPv6 report under Reports > IPv6 Summary > IPv6 Transition shows you which customer
managed objects have IPv6 traffic and compares it to the customer´s total traffic.
The report requires configured managed objects and traffic needs to cross the interface
boundary.
The Traffic tab shows a snapshot of External Resources and Traffic Characteristics. All
information is related to the past 24 hours.
If you use the View All link you get redirected to the predefined Network Reports.
• Top Peers: Reports > Network > Peers
• Top Origin ASNs: Reports > Network > BGP > ASNs Origin
• Top External Countries: Reports > Network > Countries
• Top Applications: Reports > Network > Applications
• Top Fingerprints: Reports > Network > Fingerprints
Network Change shows how network traffic has changed over the last 2 years.
All values are an average.
This screen shows you the available short cuts related to Network Resources reports. Each
short cut redirects you to the predefined report which is located under Reports > Network….
In addition to the Network Reports, we have quick links available for all listed reports.
It also contains
information on the
number of customers
and peers using IPv6.
Besides IPv4 reports and statistics, Sightline provides a view based on IPv6 traffic.
On the Summary Tab you will find the total IPv6 traffic in your network which contains native
and tunneled IPv6 traffic like Teredo.
Current IPv6 traffic and IPv6 share of all network traffic provides only data if the time period is
set to today or yesterday.
6 Month Growth gives you an understanding on how IPv6 traffic is changing compared to your
overall traffic. This can help you to plan your IPv6 capacity in your network like peering or
transit expansions etc.
If you want to see which customer or peers are using IPv6 you can use the Peers Using IPv6
tab. The graph always shows the peak traffic, where the values table has different displays
depending on the selection: Average, Max or PCT95. By default Sightline shows the average
values.
Please note, this report requires that you have configured customer and/or peer managed
objects. Configuring managed objects is not part of the Sightline Visibility Course.
This report compares your total IPv4 with your total IPv6 traffic.
You can change the graph type to get a better visibility. By default, Sightline uses the Stacked
graph. Depending which type you choose, you can also change the direction between In, Out
or In+Out.
It is measured on each
interface which connects
your network to external
networks.
The Network Routers report provides a view of traffic from all routers which have interfaces
connected to external networks (the boundary).
Traffic is measured and reported on all interfaces connected to external networks.
The Explore Traffic Router report shows traffic IN and OUT for all monitored routers, but also
those which only transport internal traffic like backbone or other service routers.
It is measured on each
interface which is part of the
network boundary.
The report shown lists all interfaces which are part of the network boundary.
The network boundary defines where your network connects to other networks.
It includes
backbone
interfaces as well
as peering
interfaces. It shows
100 items.
The Interface report on Explore Traffic shows all router interfaces without taking the network
boundary into consideration. This report shows 100 interfaces in comparison to just 17
interfaces on the previous slide.
Solution: Q1 = D ; Q2 = A ; Q3 = C ; Q4 = D
The Internet is the global system of interconnected networks. The global Internet consists of
tens of thousands of interconnected networks run by service providers, individual companies,
universities and governments. Your network is one of the networks on the internet. Monitoring
your network’s communication with the rest of the internet is called External Visibility.
• Increases efficiency
• Improves security
The Internet is still evolving. And not just its infrastructure. How we use it and where we use it
are also continuously changing. What might the Internet look like in 10 years? What are some
of the biggest challenges we face, and how can we ensure the continued development of an
open Internet for everyone, everywhere?
Internet is also a business. Your organization needs business Internet connectivity that's fast,
reliable, and high-quality, at a cost you can afford.
That is why visibility within or outside of your network is important.
Visibility helps you to drive new investments. You must be able to understand how every bit of
data moves across your network if you hope to make the types of improvements and
investments necessary to improve performance.
Visibility increases efficiency and enhances end-user experience. Once you gain granular
insights into your entire network, it will open an ability to understand how each application,
endpoint, user and service impacts your overall network performance, availability, and
connectivity. This allows you to make intelligent decisions about how to filter traffic, what needs
to be monitored, and where you need to make additional investments to shore up performance
and reduce downtime.
Visibility improves security. You can only secure what you can see. Without the proper
visibility, it can be impossible to identify and patch vulnerabilities in the network to prevent
attacks or quickly respond when attacks do occur.
Peers are beyond your network edge. Peers are how we connect to get traffic to and from the
Internet.
Peer traffic visibility provides you the list of peers and interfaces, where traffic is coming from
for that peer, where it goes, which applications are being used via this peer etc.
Peer traffic visibility also helps you to analyze peer relationships and see which customers or
my other resources are using my peer. You can plan peerings regularly.
You can examine potential peering opportunities by using Peering Reports to save money.
You can do capacity planning of peer interfaces and invest, and plan your network by using
peer traffic usage. New interfaces, expansions, configurations can be concluded.
Peer visibility on your network boundaries (external communication) can also be concluded via
peering reports.
A peer is another BGP domain. Once you’ve been approved to peer with a network, you must
configure your router’s peering settings to talk to a specific ASN using Border Gateway
Protocol (BGP).
A peer has its own ASN. To enter into a peering agreement with most Internet service
providers, you must have at least a publicly routed ASN.
A peer is directly connected. When peering is negotiated, a physical connection is required.
Peers exchang traffic. Both parties directly hand off traffic between each other’s resources.
Peering is of mutual benefit. This 'mutual benefit' is most often the motivation behind peering.
ISPs, social networks, search engine networks and video streaming networks can be
considered as peers in Sightline. These networks share content with you or help you to access
certain resources.
Sightline also has the concept of AIF Managed Objects for OTT resources which are
downloaded dynamically via AIF. An AIF MO is not considered a peer, rather a means to
monitor Over the Top services which are discussed separately in another Unit.
Sightline has managed objects of different types. You need to create managed objects to
monitor resources in Sightline.
To monitor your peers, you must configure peer managed objects accordingly, one for each of
your peers.
To match peer traffic, either an ASN or interface match can be used. Interfaces connect my
network to the peer network and the ASN is the peer’s network ASN.
Peers in the diagram are circled with different colors.
You may also need to focus on your specific peer traffic details. The summary report shows
you the selected peer’s traffic details.
Average, Max and 95th Percentile values for the selected time period for the selected peer is
reported. Backbone shows the traffic passing through backbone interfaces for this peer.
Dropped shows the traffic dropped for this peer.
– Good resource to
understand your peer rtr-jnb-1
Report shows all the traffic for the selected peer broken down by routers. Routers also identify
location and site, so by using Peer By Routers report you can see your peer’s traffic
breakdown by location and site.
This report values are from peer perspective. In Sightline reports, first item (peer in this case)
is called a perspective. Peer by router report can be used for traffic load-balancing purposes to
identify the busiest routers and move traffic from one router to another.
• Identify interface-
wise peer utilization
– Capacity analysis
– Investing plans
– Peak usage time for
the peer Interfaces ae3.22
irb.1922 irb.21
irb.164
Only Layer 3 interfaces are reported because Netflow is only exported from L3 interfaces. Let’s
assume that you have 2 physical interfaces, and they are bundled as 1 logical interface.
Because the IP is assigned to the logical interface, the logical interface is listed on this report
instead physical interfaces.
Let's further investigate peer usage. You have customers paying you for the service you
provide to them. You may also pay for your peering relationship (Transit or Settlement Free
Peer), or at least dedicate resources to manage and operate your peering environment.
Internal usage of your peer traffic makes sense to improve quality, develop resources and for a
better service to your customers.
Peer Traffic by Customer provides you the list of customers that are using your peer’s traffic.
Customer means the manually created Customer managed objects in this case. IN shows the
traffic destined to the selected peer and sourced from the listed customer. OUT shows the
traffic sourced from the selected peer and destined to the listed customer. These calculations
are done by looking at source and destination IP information of the incoming Netflow packets,
interfaces (peer interface) etc.
In addition to customers, you have other resources in your network. These could be DNS
Servers, data centers, downstream ASNs (customers have their own ASN) etc.
These resources are being monitored with Profile managed objects (another type managed
object like Customer and Peer). Monitoring these resources is also important for a better
visibility of your peer’s internal usage.
This report shows the selected peer traffic utilization by Profile managed object. Profile
managed objects in this example are WEB, DNS and CACHE servers in your network.
• Router rtr-ams-1
We have seen peer traffic breakdowns at the network edge (router, interfaces) and internal
resources (customer, profile) so far.
Now we will focus on peer traffic breakdowns by external traffic (the Internet side).
Sightline is BGP aware and each monitored router in your network should establish BGP
sessions with Sightline. Each monitored router sends its entire route table to Sightline.
Sightline correlates incoming Netflow data and BGP table attributes to provide rich reports.
This is one of the most valuable features in Sightline.
The BGP route table contains all BGP information such as as-path, community, nexthop and
prefixes. This is all used by Sightline to make it capable of binning traffic for BGP attributes.
For example, how much traffic for my peer originated from ASN X? As there is a BGP peering
relationship, any changes in the BGP network will be reflected to Sightline. This means reports
for BGP attributes will be always up to date.
Binning is grouping data into chunks or "bins" usually defined by time periods. For example,
traffic for the last 24 hours.
INTERNET
• How the ASN Origin report INTERNET
helps
– Load-balancing of traffic over
peers - Network Operations
– New investments
opportunities - Network
Planning
– Tracking of attacks - Security
One of the commonly used reports in Sightline is the ASN Origin report. ASN Origin means the
ASN from where the traffic initially comes from.
The communication between the origin of the traffic and selected peer is shown in this report.
By knowing which ASN utilized my peer resources, you can do load-balancing between
different peers, routers, interfaces etc.
According to this information you can add new peerings for the originated ASN or add more
capacity to the peer resources.
You can also know which origin ASNs you are communicating with during peace time. This is
very useful in case of any attack that has different origin ASN traffic.
ASNs are derived from the BGP table dynamically, so once a peer starts communicating with a
new ASN it will appear in the reports.
The ASNs in this case are the Internet side ASNs and not customer or downstream ASNs.
17.253.24.0/23
Internet side prefixes (originated from another ASN on the internet) is reported for the selected
peer.
• Sightline has a
Geolocation database
• Country-wise peer
reports can help
– Use for peace time
versus attack time
comparisons
– Traffic distribution by
country
Apart from dynamic BGP reports, Sightline has a GeoIP database. The GeoIP database
provides you country, city, and regional breakdown for the resources.
It is very useful to know which countries you are communicating with during peace time to
understand anomalies when attacks occur.
BGP attributes reporting is one of the most widely used reports in Sightline. We mostly focus
on BGP reports showing us a breakdown of the Internet side traffic.
This is an important requirement for customers because they need to understand where they
are communicating with the Internet.
It is useful to know Internet side reports for operational and security purposes.
Customers are also interested in how traffic is distributed in their networks. Where does traffic
go to? Which of my customers and internal resources are using traffic?
We call these reports “Transit Reports” and we will cover these in the next section.
Your network is neither source nor destination for the traffic. As the wording indicates, traffic is
transiting your network.
Your network is providing Internet or content access to others. It is a service by which
networks have access to the rest of the Internet via BGP.
It is called an IP transit service. In contrast to peering, where networks exchange only their
own customer routes (on a mutual benefit and cost neutral basis), IP transit is a commercial
service whereby one network provides access to the entire Internet routing table (or a subset
thereof), in return for payment.
You can sell your transit service to customers.
• IP transit is a
metered service
• Who is using my
network for
transit?
• Transit traffic
reporting helps
improve the
quality of a
service offering
IP transit is a metered service. Customers pay for the IP transit service. Often metered using
the 95th percentile traffic sampling technique.
You need to know who is using your network and for what purposes so you can improve your
operation, market and quality.
IP transit can include Service Level Agreements (SLAs). With SLAs, the user experience can
be guaranteed. To offer an SLA, the ISP must be able to determine the level of service that
they can consistently deliver to their customers.
For an international IP transit provider, visibility into the network’s traffic is seen as critical in
gaining much-needed details into customer and prospective customer traffic. This invaluable
information could be used to drive investments in additional capacity for existing PoPs, or to
justify adding new PoPs.
OTHER SIDE
Reports > Customers > BGP (Transit) > ASNs Origin ASN B
Sightline also provides transit reports. Transit reports are pre-defined just the same as
standard reports.
Transit reporting uses different logic than standard reports. This logic is only for BGP attributes
reporting.
As previously discussed, standard BGP attribute reports deal with BGP attributes for the traffic
breakdown at the Internet side.
Transit reports BGP attributes provide you with a breakdown for the traffic of the “other side”.
“Other side” can be considered internal resources and customers (not Internet side).
When traffic is destined to a managed object located In your network, standard reports looks at
the source IP to derive Internet side BGP attributes, and transit reports look at destination IP to
derive other side BGP attributes.
When traffic is sourced from a managed object (assuming this managed object is in your
network), standard reports looks at destination IP to derive BGP attributes (Internet side) and
transit reports look at source IP to derive BGP attributes (other side).
Peer
Profile
Interfaces
Customer
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 37
Transit reports are available for managed objects and interfaces in Sightline. Customer, profile
and peer managed objects and interfaces have transit reports enabled by default.
You can see in the diagram what type of managed objects can be used to monitor which
resources in your network.
“Customer” is any of your customers that has a downstream ASN or is located in your network.
“Profile” is any internal or external resource or anything you would like to monitor via Sightline.
“Peer” is your peers – your connection to the external world. “Interfaces” are the only external
interfaces (upstream connections) and are capable of transit reports by default. This is
configurable for other interfaces as well.
• AS • Remote AS Explore
• AS Origin • Remote AS Origin Traffic
• Community • Remote Community
• NextHop • Remote NextHop
Reports
Sightline has access to reports in two places – the Reports Menu (Reports > Customers) and
the Explore Traffic page.
The Explore Traffic page has 2 filters - you select at least 1 filter (2 filter combines filter 1 and
filter 2) and it will provide you the same data as Reports.
The Explore page provides access to all reports.
Understanding the naming convention in the Reports menu and Explore page for transit
reports is useful. There is a naming convention for the Explore page in that any transit report
starts with Remote. It means "other side". Customer > BGP > ASNs Origin (standard) can be
converted to Filter1:Customer Filter2:Remote AS in transit reports.
Both pages fetch data from same resources so they are presenting the same information.
Let's look at Transit Reports use cases before covering Transit Reports examples.
• Is someone using my network as transit?
o Is my network allowing traffic to travel through to its final destination. Regardless of
how your business or product accesses the Internet, you will need to utilize IP
transit in some capacity
• I need details of the traffic using my network as transit (ASN, Community, Prefix etc.)
o Which ASNs and prefixes are using my network as transit and for which purpose?
• Is my customer maintaining the traffic level agreements?
o Meter your customer’s transit usage and have a rich visibility of its transit traffic
• Is my customer providing a transit service?
o Are there any other ASNs behind my customer? And does my customer provide a
transit service to them?
• Monitor the Load-Balance network traffic by using transit reports
o By having various BGP attribute reporting, you have several options to divert,
withdraw, re-announce and update your network announcements
• Am I using my peers efficiently?
o Am I using my peers traffic on purpose? I am reaching to my peer from other peers?
o Are there any new peering possibilities?
• BGP community
– Used to categorize
business, services etc.
• How remote communities
report can help
– Troubleshooting and 237:20940
capacity planning
– Service offerings to
customers
237:12
237:2882
Let's start Transit Reports with peer by BGP communities. As you know BGP communities are
useful for network operation. They can be used to control routing policy or tag, monitor and
track specific traffic.
Because this is Transit Reporting, we are interested in how my network is being used by
others.
Peer X Remote Communities report provides us with a peer traffic breakdown by BGP
communities which is at the other side of my network.
I can get an idea of where traffic goes to after passing my network and sourced from a peer.
Where is the traffic sourced from, passing my network and destined to peer?
BGP communities
This report is useful to understand internal resources and customers utilizing your peers'
traffic. You can understand what type of traffic utilizes a peer.
Use this report to offer new services to these customers and also add more capacity and plan
ahead.
MERIT-AS26
Another transit report we will look at is Customer by Remote ASN. Customer X Remote ASN
report provides us a breakdown of your customer’s traffic by ASN Origins behind your
customer. This gives a clear picture of the origin of traffic my customers send to my network or
the destination of the traffic I am sending to the customer.
Why?
• Is my customer sending me traffic as we agreed before? What are the traffic levels? 95th
percentile calculation etc.
• I can understand if my customer has other ASNs behind and is providing transit service to
others. Does our agreement allow my customer to send traffic to my network for other
ASNs ?
• Reports >
Customers > BGP
(Transit) > ASNs
Origin
– Breakdown of traffic
by ASN origins
behind your
customer
ASN 11164
Another resource we can use in transit reports is the Interfaces (external, upstream) in my
network. Interface X Remote Peer report provides the traffic breakdown by neighbor ASNs
behind my network for the selected interface. Traffic goes to which neighbor ASN (Peer ASN)
for the selected interface when traffic is ingress to the interface and the traffic sourced from
which neighbor ASN when it is egress for the selected interface.
By knowing this, you have ability of reporting on who is using your interface’s resources
• SLA: You can get reports that help maintain your SLAs
• Performance: Invest, expand your interfaces and increase your network’s performance
• Troubleshooting: You have much more visibility over the interface. Let's say the interface is
at 100% utilization and you discovered most of the traffic goes to a customer that has its
own ASN
• Security: During an attack, you can understand which ASNs are destined mostly
– Neighbor AS
numbers utilizing
the selected
interface
We have discovered individual reports so far for each resource (peer, customer etc) for both
standard and transit logic. Sightline also provides you some tools that present you with
information from various reports on the same page.
Peering Traffic Exchange report provides you both in the single page
• Peer's interfaces
• Peer's BGP attributes
It is more data than individual reports and makes troubleshooting easier.
There are two types of Peer Traffic Exchange Reports
• Peer's interfaces utilization breakdown by source ASN – Source Analysis (STANDARD
LOGIC)
• Peer's interfaces utilization breakdown by destination ASN – Destination Analysis
(TRANSIT LOGIC)
opportunities
Peer Interfaces
Let's discuss how Peering Traffic Exchange reporting works. It is pre-defined as a tool. Its main
purpose is to investigate usage details of the peer interfaces. Which Internet side and other
side ASNs are using my peer traffic.
By knowing ASN details on the peering interfaces, you can look for potential other new peering
opportunities. I am seeing most of the traffic for my peer comes from ASN A. Is it possible to
do direct peering with ASN A?
The report only looks at traffic Out of the Peer and destined to my network.
Three report options - Source Analysis, Destination Analysis and Source, Destination Analysis
(provides both source and destination ASNs for your peer interfaces on the same page).
This is what the Peering Traffic Exchange Report looks like. There are two panes. The left
pane lists the selected peer’s interfaces. The right pane lists ASNs according to the report type
(source or destination or both). Selecting or deselecting interfaces on the left pane changes the
results on the right pane.
ATT-INTERNET4
Let's focus on Peering Traffic Exchange Source Analysis. It is like the Interfaces > BGP >
ASNs Origin report.
Its main purpose is to understand where traffic originated from when it comes to my network
via my selected peer’s interfaces. By using this information, you can conclude new peering
opportunities, agreements, traffic steering.
analysis help?
– Understand utilization
on the peering
interfaces
– Provide data to identify
and move loaded irb.164
prefixes to another
peer or other interface irb.1922
My network operation team noticed that my peer interfaces are utilized in the IN direction. I
need to know where traffic goes to.
By selecting and deselecting peer interfaces, you can see traffic goes to which ASNs from
which interfaces and you can prefer that traffic from another peer.
Solution: Q1 = d ; Q2 = b ; Q3 = a ; Q4 = d
• Using peer reporting so you can identify who are your top network peers
• How Sightline correlates BGP and flow information and presents that data in
reports
• How Sightline helps you visualize the traffic transiting your network
• You used Peering Traffic Exchange Reports to view both standard and transit
reports simultaneously
Nowadays service provider business is very competitive. Service providers are required to
offer the best user experience at the lowest available prices. To achieve this goal, it’s crucial to
maintain very efficient resource utilization, which is where the importance of Network Capacity
Planning can be demonstrated.
A well-planned network must take into consideration best practices of resource optimization
which directly affect QoS and the end user experience.
Capacity Planning is also important to provide estimated forecasts of future resource
requirements.
• Identification of potential
congestion points
• Traffic offloading/re-allocation
95%
30%
Capacity Management is another important term. Network operators need to keep network
resources closely monitored so they can identify potential areas of congestion within their
network and take actions to address these congestion points before causing service
degradation or even service impact in some situations.
Sightline traffic reports provide an elaborate capacity management toolkit which enables
network operators to easily monitor traffic utilization within their network, do in depth traffic
analysis which provides help in taking the correct decisions for traffic re-allocation and
optimization, and also to visualize pre and post changes traffic distribution.
- Identify Bottlenecks
Forecast identify
- Manage Available resources
One critical task of a network operator is to monitor bandwidth utilization on critical interfaces.
These can be on the peering edge, they can be backbone interfaces or traffic for a specific
service or for local caching nodes.
This might be manageable in small networks but in the case of huge service provider networks
with hundreds or even thousands of interfaces, a more intelligent way is needed.
Sightline solves this challenge by means of traffic alerts where thresholds can be defined per
interface or service by network administrators, so that traffic alerts are triggered if the traffic
exceeds those thresholds.
So for network operators instead of manually tracking every single interface, they will just need
to address those traffic alerts and take actions ASAP for those interfaces or services.
The first type of traffic alert is an Interface Usage Alerts. Here a threshold can be defined
globally for all interfaces or on a per interface basis and network admins have the option to
configure a high or low threshold as explained below:
Over utilization threshold: an alert is triggered if the 5 Min interface traffic exceeds that value.
Under utilization threshold: an alert is triggered if the 5 Min interface traffic falls below that
value.
• Once the 5 Min interface utilization exceeds the configured threshold, an Interface
Usage alert will be triggered showing the below parameters:
– Interface name
– Router name where the interface belongs
– Traffic rate, utilization percent and interface capacity
Network operators need to monitor their deployment for interface usage alerts. These alerts
raise a flag that an action should be taken for the alerted interface.
The alert provides all the details needed to do further analysis, like interface name, router
name, utilization value…etc.
• Once the 5 Min managed object utilization exceed the configured threshold, a
managed object threshold alert will be triggered showing the below parameters:
– Managed object name
– Traffic rate, utilization percent and the configured threshold
Note: Consult your System Administrator to get the current configured threshold values in your deployment
The second type of traffic alert is the managed object threshold alert. In this case a network
administrator can define a traffic threshold per managed object which is helpful to proactively
monitor the traffic rate for specific customer or service like DNS, Caches, webservers…etc.
The network administrator can then analyze traffic distribution for this service, take
optimization decisions or even request service expansion.
The first step in analyzing interface usage alerts is to check the utilization history of the alerted
interface, using the interface summary report. A network administrator can get a quick view of
interface utilization over time.
By viewing utilization history you can then determine if the alert was valid and if that
interface needs an action to offload some traffic or it was just a temporary traffic spike.
• Comparing interface
utilization of a specific
router helps in identifying
unbalanced traffic
distribution
• Navigate to Reports >
Interfaces > Compare
Interfaces then select a
router to view per
interface traffic
distribution
The network administrator can do further investigation by viewing an overall traffic distribution
at the router level. They can compare traffic utilization / distribution for every interface on a
specific router, and later take some decisions to balance traffic between interfaces or even
decide if that router may need additional interface expansions.
Sightline provides a rich set of interface reports for very low-level traffic analysis, examples of
those reports are application/customers/profiles/peers/Top Talkers/packet
sizes/protocols….etc and much more.
By default these detailed reports are only available for external interfaces, but they can be
manually enabled for other critical network interfaces. But please be aware of your deployment
limits because interfaces with detailed statistics reporting enabled are limited per TRA. Always
follow the best practice guidelines to enable detailed reporting on required interfaces only.
Sightline provides traffic reports based on SNMP data polled from the routers. By default
SNMP reports are available for interfaces where flow records are received via Netflow. This
behavior can be changed but needs a careful consideration. You may consult Arbor ATAC to
evaluate the situation if required.
In some situations, it’s very helpful to have an overview of the top internal hosts utilizing a
specific interface. Those hosts can be internal servers getting content from outside the network
or even internal customers with large data usage i.e bandwidth abusers that could affect
quality for other customers. A network adminstrator can make decisions for those hosts based
on the specific use case.
The top talkers report provides data for both IN and OUT direction for the reported hosts. You
can also filter on specific hosts to view their utilization over time to understand more about their
behavior.
Once a network administrator has identified a highly utilized interface, and in which direction
(inbound or outbound), the next step is to manage the traffic on that interface and direction to
free some capacity.
Traffic management is achieved at the router level by means of route policies, so here
Sightline will provide the tools needed to identify and visualize the routing and traffic changes.
Let’s have a look at both situations, starting with outbound traffic management.
Generally the direction of the traffic is always opposite to the direction of the routes received,
so in order to reroute outbound traffic from your network you will need to manipulate the
incoming BGP routes to prefer one path over another.
The challenge here is how to identify the routes that need manipulation in order to achieve this
goal, and here we can demonstrate the importance of Sightline BGP traffic reports.
The Interface BGP ASNs reports help to segregate traffic of an interface per external AS
number. Here you can see the inbound and outbound traffic from/to those ASNs. Since we are
looking to re-route some outbound traffic from that interface we can use this report to choose
some external AS numbers with a considerable amount of traffic and by means of BGP route
policies we can manipulate the routes originated from those ASNs to make them preferred
over another interface which moves some outbound traffic to that interface.
Not only that, but after the network administrator applies this change from the router side, they
can use the same report for both old and new interfaces to make sure the desired amount of
traffic correctly moved between those interfaces.
Interface AS Paths is another useful report to gain more granular visibility over the full AS
paths used by the traffic on specific interface, so that network administrators have the flexibility
to choose one path over another, perhaps one closer to the content providers. Or prefer a path
through one transit upstream provider over another. So again, using BGP policies you can
match routes from one AS path and then apply the needed policy changes.
In some situations, moving a whole AS from one interface to another may not be the optimum
solution. For example, the traffic for that AS may be too great for the new interface which will
just move the problem to another interface.
In this situation another good solution is to have visibility of the traffic per external IP prefix, so
moving traffic for some prefixes would provide more granular traffic control between interfaces.
The same report can help make the decision on which prefix can be shifted and to monitor the
traffic shift after applying the changes from the network side.
• To affect inbound traffic, you need to manipulate your own BGP prefix
announcement
• Sightline reports can help you identify traffic distribution for you own resources,
you can then use this to identify which prefix or service need reallocation
• Use below sightline reports to identify internal traffic distribution
➢ Interface Customer (Reports > Interfaces > Customers)
➢ Interface Profile (Reports > Interfaces > Profiles)
➢ Interface Communities (Reports > Interfaces > BGP > Communities)
Now let’s go through the options to manage inbound traffic of a utilized interface, using the
same logic. In order to manipulate inbound traffic, you will need to manipulate the outgoing
route advertisement.
So here the challenge would; be what are the correct routes that can be moved from one
interface to another?
Again, Sightline reports provide the required visibility on which internal resources are utilizing
the inbound traffic of an interface so that you can redistribute those resources among other
free network interfaces.
The Interface Customers report provides you with the interface traffic breakdown per internal
customer managed object, so in order to offload inbound traffic from that interface you can
choose a customer or group of customers and apply the required BGP policy changes to shift
them to another interface.
Below are some options that can be used:
• Stop announcing customer route on old interface and announce through new interface
• Or use AS Path prepending if redundancy is required
• Or use longest prefix match rule
Profile managed objects are usually used to define internal services within the network, so by
using the Interface Profiles report you can gain some visibility on the services utilizing inbound
interface bandwidth.
Similar to customer managed objects you can also apply routing policy changes to move
the traffic of those services to other free interfaces.
Service providers commonly use BGP communities to group prefixes by area or service or
even QoS profiles. If BGP communities are being used within your network then you can use
Sightline to view a traffic breakdown by BGP community.
BGP communities can later be used within BGP routing policies to change the advertisement
of all prefixes using a specific community. This provides more flexibility, so you don’t need to
track per prefix advertisement. You can use communities instead.
If traffic re-allocation is not an option due to lack of sufficient resources, then it is time to plan
your future resource requirements. Usually service providers will not wait until they reach full
network congestion on all interfaces, rather when defining the high utilization thresholds, they
consider some room for worst case scenarios. For example they use 75% instead of 95%.
Identifying the exact value of resources required within a future time period is a challenging
process. With the use of historical traffic utilization trends, data analysts can extract
approximate values for the required resources.
For this purpose, data analysts require this historical data in a raw data format. Sightline
provides assistance in this situation because you can easily extract the raw data of any report
in CSV format.
Navigate to the desired report. For example, if you want the historical data for a peering
interface go to the report of that interface and download as CSV file.
Solution: Q1 = d ; Q2 = b ; Q3 = a ; Q4 = c
BGP
– Suboptimal routing
– Diminished network performance
Sightline uses BGP for enriching the dataset delivered by Netflow and SNMP
The BGP information is exchanged to Sightline from the monitored routers in the form of the
internal and external routing table
Using the information gleaned from the BGP routing tables, Sightline provides external visibility
reports for end-to-end traffic reporting
BGP derived reports on Sightline includes transit and non-transit reports
19 Fields
Enhanced Flow Record Information
Source Destination Src Dst Proto In Out ToS Flags Bytes pkts
IP Prefix Nexthop AS Path Com IP Prefix Nexthop AS Path Com Port Port Intf Intf
From BGP
Sightline combines BGP and Netflow datasets to produce enriched annotated flow
The annotated flow contains additional fields comprising of all the BGP attributes
The enhanced flow data thus produced internally on Sightline, offers a detailed drill down into
the traffic seen in the network
Reports > Routers > BGP > ASNs Peer Reports > Routers > BGP/BGP Transit > ASNs Peer
to list a few…
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 7
The entire gamut of BGP based traffic reports rely on BGP dataset on Sightline, for its
functionality
Peer reports, network-level transit and non-transit reports, customer/profile ASN reports are all
produced from the annotated enhanced flow data set
19 Fields
Enhanced Flow Record Information
Source Destination Src Dst Proto In Out ToS Flags Bytes pkts
IP Prefix Nexthop AS Path Com IP Prefix Nexthop AS Path Com Port Port Intf Intf
BGP
No BGP or erroneous / incomplete BGP information can impact flow annotation. Lack of BGP
data leads to absence of BGP based enhancement to flow data
Reports > Network > BGP > ASN Origin Reports > Customers > BGP > ASNs Origin
Reports > Customers > BGP > ASNs Origin Reports > Routers > BGP > ASNs Peer
Reports > Routers > BGP > ASNs Peer Reports > Peers > Peering Traffic Exchange
Issues with the BGP dataset – could be complete or partial failure of BGP, can affect the
reliability of external visibility reporting on Sightline
No BGP or erroneous / incomplete BGP information thereby subsequently impacts traffic
reporting and visibility especially for external traffic
✓ Available
▪ BGP peering sessions should be up/stable
✓ Accurate
▪ Full BGP routing table
✓ Reliable
▪ BGP data must be consistent and
trustworthy
On the Sightline platform, the following aspects/characteristics of BGP data are extremely
critical:
• Availability/stability – Monitored routers must have well established and stable BGP peering
with Sightline
• Completeness – The BGP routing table must be received in full on Sightline
• Accuracy - Correct information must be received from router BGP tables
• Reliability – Reporting will be impacted if BGP received is unreliable
If the BGP info is inaccurate, we lose information on external visibility and cannot rely on the
reporting in the system.
In this section, we will explore the available BGP fault detection and analysis tools on the
Sightline GUI
On completion of this section, the users should be more familiar with the approaches to
analyzing the deployment for BGP issues and how to resolve them
The Sightline platform has both alerting and reporting options to validate the availability and
accuracy of BGP data received from routers.
Since a good part of external visibility is obtained from BGP data, it is important the operators
regularly monitor the GUI for BGP events and alerts.
One of the most important considerations is to validate that the BGP sessions are up and
running from all the peering routers that are provisioned as BGP peers to the TRA collectors.
Multiple sections of the GUI provides this information. A high-level view of the BGP session
status can be obtained from the per-appliance status page. From here, one can move on to
more specific analysis to identify the specific router that has BGP issues.
The per-router status provides specific information on which router BGP session is currently
down along with the associated collector.
In addition to the BGP status, this page also provides information on the routes received from
each router. This would be a good starting point for investigating BGP down alerts on a per-
router basis.
BGP alerting provides notifications whenever a BGP peering session goes down. Use the
‘wizard’ button to search for alerts related to ‘BGP down’ type. Alerts provide the exact start
time and duration of the event in addition to the other details.
Use Wizard to search for other BGP Alerts such as:
• ac: Alert Class
• at: Alert Type
Some examples include:
• ac: Data
• at: BGP Down
Results: Displays the BGP peering sessions that are down.
• ac: BGP
• at: BGP Instability
Results: Displays the router instances that exceed the preconfigured BGP instability
threshold.
• ac: BGP
• at: BGP Route Hijack
Results: Displays instances of Sightline detecting a BGP route announcement from an
external ASN for a prefix within the defined local address space. This alert indicates either a
potential hijacking of local address space or a misconfiguration of the local address space.
• The number should match with the router’s BGP table, to confirm data accuracy
Each router that peers with a TRA must be configured to send its full routing table to the TRA.
Full routing table helps the TRA to derive and report completely and accurately on external
traffic.
It is equally important therefore to validate whether all the routers configured as BGP peers
advertise their complete routing table to the Sightline platform. If the number of routes received
on the Sightline TRA is less than the actual number of routes in the router’s route table, it
becomes important to understand why it is so, and address the issue by changing the
configurations as required to have the router advertise its full routing table.
BGP Instability refers to any event that causes the BGP peering sessions to repeatedly or
frequently go down. This leads to abnormal route updates from the affected router. An unstable
BGP session is not desirable and can negatively impact the quality and accuracy of external
BGP based traffic reporting on Sightline.
Look for and analyze BGP instability events on the environment using the wizard menu of the
system error alerts menu. The BGP instability alerts are detected based on pre-configured
thresholds set by the administrator. The thresholds are evaluated on a 5-minute interval and
the administrator can set these thresholds based on the expected normal number of route
updates typically seen in the network.
Spike/Max
values suggest
anomaly in BGP
updates –
Instability event
BGP Instability reports give detailed information and graphs related to instability events both
network-wide and per-router. Network-wide reports are a good place to start the drill down.
The report contains the following types of data (if observed):
• ANN announcement updates
• AADIFF routes implicitly withdrawn and replaced by an alternate route to the same prefix
(indicates forwarding instability)
• AADUP routes implicitly withdrawn and replaced by a duplicate of the original route
• TUP new, previously unseen prefixes being announced
• TDOWN routes being withdrawn
• UPDATES total number of BGP updates
• WWDUP duplicate withdrawn updates
• WITH total number of withdrawals
Spike/Max
values suggest
anomaly in BGP
updates –
Instability event
From the network wide perspective of instability, the investigation can move to the router-level
view. This report clearly identifies periods of spikes of BGP updates that can be a clear
indicator that there might be issues related to the overall stability of BGP peering from that
router. The analysis can now be focused on the router(s) exhibiting the instability issue.
The ’Analyze Instability’ report provides a clear view of the factors contributing to BGP
instability on the selected router – information such as per ASN and per prefix updates can be
used to quickly understand what are the top contributors to the BGP instability behaviour on
the router. Once identified, the triggering elements can be reviewed with the router team to
troubleshoot and resolve.
The Explore BGP Routing Instability page (Explore > Routing > IPv4 Analyze Instability) has
several menus, that are explained below:
• Withdraw - The number of BGP withdrawals
• Announce - The number of BGP announcements
• Number of Unique Prefixes - The number of unique prefixes
Top Origin ASN section:
• ASN - The origin ASN
• Top Origin ASNs Number of Updates - The number of BGP updates for this ASN over the
timeframe
• Top Origin ASNs Percentage - The percentage of BGP updates in the timeframe that the
system applied to an ASN
Top BGP Prefixes section:
• Prefix - The BGP prefix
• Top BGP Prefixes Number of Updates - The number of BGP updates for a prefix over the
given time period
• Top BGP Prefixes Percentage - The percentage of BGP updates in the specified time
period that the system applied to a prefix
Frequent changes in
BGP attributes for
the same prefix,
confirms issue
Once the prefix(es) or ASN(s) contributing to BGP instability are identified, it now becomes
easy to query the updates corresponding to the identified prefix/ASN and display the entries
related to the instability events.
The Explore BGP Updates page (Explore > Routing > IPv4 Updates) allows you to view
announcements and withdrawals in a router’s BGP table. BGP event descriptions:
Event:
• A - Announced
• W - Withdrawn
PEER DOWN - The peering session with the specified router went down, causing all routes to
be withdrawn.
PEER UP - The peering session with the specified router came up.
Solution: Q1 = a + c ; Q2 = b ; Q3 = y ; Q4 = d
• BGP relevance for optimal Sightline external visibility reporting was described
OTT is the delivery of 3rd-party applications over standard IP networks independent of any
last-mile provider ISP. OTT is opposed to dedicated legacy infrastructure (phone, radio,
television, …)
“Over The Top” means over the IP networks. The service is delivered “over the top” of another
platform, hence the moniker. It was initially named in reference to devices going “over the top”
of the cable box to give users access to content.
OTT services circumvent traditional media distribution channels such as telecommunications
networks or cable television providers. As long as you have access to an internet connection
— either locally or through a mobile network — you can access the complete service at your
leisure. It is typically monetized.
An OTT platform provider is an online solution that hosts live and on-demand content that is
broadcast over the internet.
OTT services are typically monetized via paid subscriptions.
The type of OTT service most users probably interact with most regularly is video OTT.
Services like Netflix, Hulu or Disney+HotStar are video OTT services, which provide users with
a number of programming options, both in terms of a licensed library of TV shows and films, as
well as original programming.
Another major OTT market is audio, with services such as Spotify now almost synonymous
with music streaming. Users can access a massive library of recording artists and podcasts via
an internet connection.
Similarly, voice OTT services, like Skype or WhatsApp, are increasingly common instead of
phone calls.
Remember text messages? Most users now use OTT messaging services like WhatsApp,
Telegram or Signal, which allow them to use their internet connection to share information.
• Rich content
– Unique, cheap
• On-demand access
– Watch whenever you
want
• Easy access
– Internet capable device
OTT Services have become mainstream, making identification
– Multiple capability
and classification simultaneously more important and even
• Growth potential more difficult.
– Trending
OTT provides rich content. If you’ve been on Netflix or Amazon Prime TV recently, you’ll notice
a lot of original content exclusively for those OTT channels. This exclusivity makes it easier for
consumers to access your content, and you also retain customer loyalty.
OTT has on-demand access. Not only do you have access to thousands of movies, TV series,
and documentaries at a reasonable price, but you also can watch them wherever and
whenever you like. More than ever, consumers are able to find exactly what they want to watch
and only pay for that content.
It is easy to access OTT services. Unlike traditional broadcasting, you don’t always need your
TV to get access to your favorite shows. All you need is a reliable internet connection and a
Wi-Fi capable device. Then download the necessary apps and register with the particular
service.
OTT is supported by multiple platforms such as your phone, PC, Smart TV, Video Console
etc.
There is a huge amount of growth potential. Lots of companies are entering the OTT space,
leading to a wide variety of options for consumers, and increasing quantities of ad inventory for
marketers. And with the trend pointing upwards, most media companies are compelled to join
the wave.
Management Product
Identify long term trends
Marketing
Understand user behaviors
New or enhanced services
Network Network
Planning Operations
Optimize network build-outs Reduce network down times
and investments
Empower ISP decision makers and operations - for the executive team, visibility of OTT
traversing the network is important, as Telcos have been fighting hard for over a decade to not
be seen as packet pushers or commodity traffic pipes. Knowing what services are used is
paramount to understand trends and identify new business models.
Similarly, visibility is key to product management. The business intelligence derived from user
behaviour analytics can help to better position and market broadband and enterprise
connectivity services.
Lastly, for network engineers, accurate metrics of the network utilization can vastly help in
network upgrades, impact analysis and future planning, but also for reducing downtime and
troubleshooting performance issues
In this section we aim to explain how OTT services and networking has evolved in recent
years, starting with some background information about OTT and the way the Internet is
architected. Let's start with a rudimentary introduction of the Internet.
Tier 1 networks provide the backbone of the Internet and, unsurprisingly, are often called
backbone Internet providers. These providers have infrastructure such as the Atlantic Internet
sea cables, sometimes owning, leasing or just operating submarine connections. They provide
traffic to all other Internet providers, but not to end users. Without Tier 1 providers, Internet
traffic could not be exchanged between continents and countries.
Tier 2 networks are an ISP that peers, usually for free, with some other Tier 2 networks, but
rely on its transit connection to reach the whole of the Internet. Tier 2 usually have built and
deploy their own regional infrastructure.
Tier-3 networks can be considered access ISPs.
The picture shows us:
Tier-1 Sells Internet connectivity to Tier 2 Regional ISPs (peering with each other no money)
Tier-2 Sells Internet connectivity to Tier 3 Access ISPs (peering with each other money)
Tier-3 Owns the last mile (cable/fiber/dsl) and provides consumers/enterprises with Internet
connectivity
Internet Exchange Points are an important milestone for Internet evolution that causes OTT
infrastructure to spread faster.
An Internet exchange point (IX or IXP) is the physical infrastructure through which Internet
service providers (ISPs) and content delivery networks (CDNs) exchange Internet traffic
among their networks (autonomous systems) and peer together.
To facilitate peering of the Access ISP, in need of higher throughput for their customers and
reduction of transit cost, Internet Exchange started to emerge. They are local facilities which
allow Access ISP to physically peer with each other, at no cost. Content Providers benefit from
free IXPs. Content Providers negotiate direct peering with Access ISPs.
A content delivery network (CDN) is a distributed servers that speed up the delivery of content.
Internet is for subscribers, and we are demanding more and more traffic. The development of
content delivery networks sought to deal with extreme bandwidth pressures.
CDN is a geographically distributed network of servers and their data centers that help in
content distribution to users with minimal delay.
As you see from the picture at right, CDNs speed up the delivery of web content by bringing it
closer to where users are. If you’re in London, and you watch a YouTube video, that video is
served to you from a London datacenter. A person in San Francisco, watching the same video,
gets it from a datacenter in San Francisco. Both users get fast local performance, and it’s a
CDN that makes it happen.
• OTT content is mostly delivered via CDN ✓ Large capacity public CDN offerings
✓ Attractive Pricing
• Multiple techniques and types
– Public CDNs
– Private CDNs
– Multi-CDNs
• OTT Visibility
– Which CDN delivers OTT content ?
In the past, traffic analysis and engineering based on criteria such as IP address allocation or
ASN was sufficient. For example: traffic from home users to Google, or governments, or banks
was clearly seen and reliably understood since those services were discrete and provided in-
house by those organizations. Network engineers and operators were able to easily
understand their traffic loads and routing and engineer their environments accordingly.
Today a single company might serve millions of customers, spreading its offerings across
multiple cloud providers regionally or even globally, and not even have a registered BGP ASN
or IP block allocation. The old methods may no longer suffice.
The challenge has become determining what services are being utilized when IP addresses or
ASNs are not providing enough insight. Therefore we must begin to correlate additional
resources with the IP connectivity to ascertain what is happening. Utilizing DNS provides a
significant insight into the nature of a connection. By correlating IP connectivity with DNS
requests, we can begin to more accurately and finely categorize this traffic as well as establish
user intention.
Additionally, Over-the-Top (OTT) services (predominantly streaming video such as Netflix and
HBO, which historically were delivered over cable and satellite mediums) have become
mainstream, making identification and classification simultaneously more important and even
more difficult.
and added
• Two types matching Arbor Sightline
– IP based
– DNS based
AIF managed objects are profile managed objects that match traffic flows for over-the-top
(OTT) services such as video streaming, gaming, and VoIP.
The ATLAS team configures the match settings for AIF managed objects based on ATLAS
traffic data for OTT services. When the ATLAS team changes those match settings or makes
new AIF managed objects available, your deployment automatically receives the latest AIF
managed object configurations at the next update.
There are two types of matching in AIF managed objects:
• CIDR blocks are used by specific high-volume OTT service providers.
• The match settings for an AIF managed object can also include domains for dynamic DNS
matching.
• Automatically populated by
AIF Feed
– Non editable, no deletion
– Licensed
• No detection, alerting,
mitigation FEEDS
• Read-Only Tags
– All have ATLAS and profile
tags
– Various tags per their type,
service, function etc.
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 17
AIF managed objects are automatically populated by the AIF Feed and they cannot be edited.
If your deployment has the AIF for Sightline licensed capability, Sightline automatically
downloads AIF managed objects from the AIF server through the AIF managed object feed.
AIF managed objects are different from standard managed objects. They have no detection,
alerting or mitigation functionality.
You can go to Administration > Monitoring > Managed Objects to list all managed objects.
Then use tag:ATLAS filtering to list only AMOs.
All AIF managed objects have the profile and ATLAS tag applied and are read-only. They can
also have other tags depending on the OTT traffic that they match, such as CDN and public
cloud. You can use tags to help you search for AIF managed objects.
Let's start with AMO types. IP Based ATLAS managed objects usually represent large
infrastructures identified by ASN, IP Block or Whois/RIR database.
AMOs are more granular, you can have infrastructure granularity such as Amazon ASN Origin
versus Amazon infra (EC2, CloudFront, S3 AMO which are all behind the same Amazon ASN
Origin). You can have services granularity such as “EPIC Games” (IP AMO) servers which are
behind the Amazon ASN.
AMOs capture more traffic than classic Sightline reports in some cases. For example,
AKAMAI-ASN1 (AS20940) doesn’t include Edge Servers hosted by the operator’s ASN with IP
Block registered under Akamai Org.
Another AMO type is DNS based. As we mentioned before it can be downloaded via AIF Feed
and they are immutable. But they can also be configured manually to track OTT content which
is not provided via AIF.
DNS based AMOs require the Dynamic DNS Matching feature. This matches traffic for
frequently changing service IP addresses for the domains.
The screenshot is from an ATLAS managed object with dynamic DNS matching configured.
This is automatically downloaded by Sightline.
The dynamic matching feature can be used to manually configure managed objects to match
OTT traffic.
To receive real-time DNS data for dynamic DNS matching, your Sightline leader must be set
up to communicate with one or more NETSCOUT InfiniStream® (ISNG) appliances in your
network. ISNG sends DNS to IP mapping information to Sightline. Sightline has the ability to
match using domain names.
– Explore Page
There are two ways to access to the reports for managed objects:
• Explore > Traffic
• Reports > Profiles
We recommend using the Explore Traffic page in most cases, this has the Profile Tag search
function as well.
You should select Filter 1:Profile to access to the ATLAS managed object. Remember that all
AMOs are profile managed objects.
You can use tag filtering to find the intended AMO. Multiple tags with spaces between each is
a good way to filter as demonstrated in the screenshot.
AMOs that have the tag filter combination are listed. Then you can see traffic for this AMO, or
select Filter2 for a further breakdown.
It is useful to see cumulative traffic for the requested tag. If you select the Gaming tag, the
report will show you cumulative traffic on all managed objects with that tag. If there are two
AMOs with the Gaming tag, where AMO-1 has total traffic of 100Mbps and AMO-2 has total
traffic of 200Mbps, Profile Tag:Gaming will report 300Mbps.
Your network traffic utilization by all OTT services is important for planning purposes in the
long-term. You can see how much OTT traffic has increased for the last year etc.
Instead of listing each AMO, you can see a cumulative number for all AMOs by using a Profile
Tag with the ATLAS filter.
Alternatively, you could select each AMO (no more than 100) and calculate the cumulative
traffic value, but this is much more time consuming than simply using tags.
Let's say you need a list and traffic level for each OTT service in your network.
Reports > Profiles is not a useful option here because it will only list a single AMO rather than
the required list of AMOs and their traffic levels.
Also, if we select use Explore Traffic and Filter1:Profile without any value, it will provide us the
list of all profile managed objects, so this is again is not a useful option since the list may
include non-OTT traffic as well.
Explore > Traffic with Filter 1:Profile Tag and Filter 2:None will give us traffic breakdown for all
tags.
See that a Profile Tag check box is also included in the list. You can deselect this option
because we are not interested in the entire list of profile managed objects, only AMOs.
You can also remove the ATLAS row too because it will show all AMO (OTT) traffic in your
network.
Other rows show us traffic levels for each tag:
• How much Apple OTT services traffic is being used?
• How much CDN traffic is in my network?
– Cost-effective resource
management
– Apply different network
and security policies etc.
• Use “Profile Tag” with
“ATLAS” value to match
all OTT traffic
– Use second filter
“application” without a
value
You need to know which routers are forwarding traffic for Apple services. Apple services
(Apple Store or IOS Update) can cause big changes in the network traffic profile. The aim is to
improve user experience by applying specific policies to routers.
Use the “Profile Tag” with “Apple” value and select second filter “Routers”. Profile Tag with
Apple matches all Apple AMOs.
You can also use the Profile filter with more specific Apple AMOs like Apple Store, IOS
Update, Apple TV, iTunes etc.
Your company is deploying a solution to optimize video streaming. Streaming video on popular
sites such as YouTube, Netflix, and Twitch, and the growing demand for ultra-high-quality
video is increasing bandwidth requirements and network congestion.
Which of my customers has been using Streaming content?
OTT reports are useful and brings content visibility in your reports. You can use this visibility to
monetize your resources and offer new services to your customers.
Use the “Profile Tag” with “Streaming” value and filter “Customer” to match all your streaming
traffic with that configuration. You can also see a break down by customer by using an
additional “Customer” filter:
• IN: Destined to the Streaming service, sourced from the customer
• OUT: Sourced from the Streaming service, destined to the customer
AMOs can also provide CDN visibility so you can control how content is cached. You have
visibility over where your content is cached so you can plan and invest accordingly. It also
helps to accelerate applications and prevent heavy pages, and long distances from the origin
can slow down webpages.
The reports help you to ensure availability - overloaded or unavailable infrastructure prevents
users from accessing applications.
• Streaming demand
is growing and SSI
streaming traffic by
peers is required
INTERNET2-IP2X
– Private peering
opportunities
– Reduce peer traffic
Netflix demand is growing in the network and companies need to have visibility of Netflix traffic
by peers.
Close to 95% of Netflix traffic globally is delivered via direct connections between Open
Connect (Netflix cache servers) and the residential ISPs our members use to access the
Internet.
Netflix openly peer with any network at IXP locations where they are mutually present, and
private interconnection is considered appropriate.
This helps to improve their customers' Netflix user experience by localizing Netflix traffic and
minimizing the delivery of traffic that is served over a transit provider.
Netflix itself also provides a report for ranking Netflix performance per ISP
(https://ispspeedindex.netflix.net/).
• Which of my customers
are accessing the CDN
networks mostly? VISIBILITY + THREAT
Solution: Q1 = d ; Q2 = c ; Q3 = a + c ; Q4 = b
• OTT data delivery that is correlated with Internet evolution was reviewed
• OTT and Sightline relationship was clarified by explaining Sightline ATLAS and
AIF
Managed objects must be defined to get the most from the reporting in the system. Here are
some examples of possible managed objects that can be created.
Managed object traffic is only counted at the defined boundary to eliminate double counting
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 5
Managed object traffic is only counted where the boundary is defined (to avoid double
counting).
• The Network/Global boundary is the default boundary.
• This cannot count traffic flowing inside the network.
Sightline uses global boundaries to define all entry and exit points to the network that it
monitors. It uses algorithms to determine which monitored interfaces connect to external BGP
ASNs, and it labels these interfaces as “external.” Sightline considers in and out traffic on
these external interfaces for managed objects that use the global boundary.
Sightline uses boundary-based counting to ensure accuracy while eliminating the double
counting of flows. It aggregates information across multiple boundary interfaces and routers to
track traffic in and out of the network, each router, or user configured managed objects. Every
object the system tracks has a boundary on which the system counts data.
Managed Object traffic is counted only at the defined boundary to eliminate double counting
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 6
Managed object traffic is only counted where the boundary is defined (to avoid double
counting).
The Local boundary needs to be defined if you do not want to use the default Global/Network
boundary.
Traffic that flows only on the network will be counted.
Local Boundary
Used when monitoring routers directly connected to the managed object – i.e. customer
aggregation router.
Used when you want to configure a more detailed boundary than the network boundary for the
customer so you can capture backbone traffic from the customer. This could be done on the
actual customer interfaces if they're monitored, on an aggregation router, or even on a
POP/regional gateway router that connects a region to the main network backbone.
Counts all data for the managed object and not just the traffic that goes across the network
provider boundary.
Traffic is counted along specifically configured boundary interfaces.
Managed object traffic reporting can give you great insight into the traffic for a particular
resource.
Deep Dive
?
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 9
• Top talkers for traffic to/from managed What does the traffic profile
look like? Source/Destination of my
customer traffic?
object Where does the traffic go to?
Routers mostly
used
Get insight into which routers are carrying most of this managed objects traffic.
Interfaces
mostly used
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 13
Top Talker Internal = IPs carrying most traffic inside the managed object
In case of congestion, is there someone responsible from the customer site? Check for the
most active IP addresses within the customer managed object space
Top Talker External = IPs carrying most of the traffic outside the managed object
In case of congestion, where is the traffic going to or coming from? Check for the most active
remote end points that are used to exchange data
Which Nexthops
are mostly used
For the managed object, which nexthops (peer external interface IPs) are used for traffic
leaving the network?
Peers mostly
used
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 17
Which ASNs are mostly receiving/sending traffic for this managed object – Google, Facebook,
Netflix etc?
Protocols/Ports
mostly used
If other managed objects (profiled) have been defined/created, then they can be used to see
how much of the customer managed object traffic is going to these profiled managed objects.
Get an idea of the traffic growth over time for the managed object.
You can use the fixed reports which always have one reference which is broken down by a
subclass.
When using Explore > Traffic you can select multiple references at a time to be used with the
selected subclass.
d) Top talkers for traffic to/from managed object d) What deny/allow lists are in use by the customer
Q2: Why is the managed object boundary Q4: Where can I view a Sankey Diagram of my
important to understand? customer’s traffic?
a) Comparing Applications Dashboard
a) Validates which peers are connected by BGP
b) More Reports tab on the Customer Dashboard
b) Displays how far customer’s traffic is transported
c) Relationships selection on Explore Traffic page
c) Identifies the volume of current traffic d) Customer Network Summary
d) Used to eliminate double counting of traffic
Solution: Q1 = a + b + c + d ; Q2 = d ; Q3 = b ; Q4 = c
Big data is a term that describes large, hard-to-manage volumes of data that inundate
businesses on a day-to-day basis. But it’s not the amount of data that’s important. It’s what
organizations do with the data that matters.
Organizations collect data from a variety of sources, including business transactions, smart
(IoT) devices, industrial equipment, videos, social media and more.
Data is growing exponentially with the growth of the Internet of Things. Data streams into
businesses at an unprecedented speed and must be handled in a timely manner.
Data comes in all formats, and it is variable – from structured, numeric data in traditional
databases to unstructured text documents, emails, videos, audios, stock ticker data and
financial transactions.
In the energy industry, big data helps oil and gas companies identify potential drilling locations
and monitor pipeline operations. Financial services firms use big data systems for risk
management and real-time analysis of market data.
Other government uses include emergency response, crime prevention and smart city
initiatives.
Big data solutions provide additional capabilities to network visibility, you can:
• Access more granular data flexibly and gain clearer business insight.
• Make better design decisions with access to complete, hi-fidelity historical data.
• Conduct deep forensics into past events to determine root causes.
• Pre-configured, pre-
populated reports
• Many built-in reports
• Economical storage
• Optimized for
managed objects
• Data granularity
diminishes over time
Sightline has more than 400 pre-configured reports such as customer traffic by routers, peer
traffic by BGP ASN origin etc. Network reports are an example of pre-populated reports. They
are being populated as soon as the system starts receiving flows. The report data is stored in
Sightline databases.
It is economical because no extra storage is required for these reports, and you can see
historical data for years.
These reports are generated for each managed object. If you don’t create managed objects for
your resources, you can not see reports belonging to that resource. When you create managed
objects, it triggers Sightline to start additional processing in terms of reporting.
Sightline is not only responsible for reporting, it also has DDoS detection and mitigation
functions. Data kept for reporting does not store real flow records because this would require
huge storage. Instead, Sightline stores traffic values for each report in a timely-manner and
granularity is decreased over the time.
• Insight is an add-on to
Sightline powered by Big-
Data technology
• Insight provides
– Advanced traffic analytics
– Security investigation
• It transforms a Sightline
deployment into a rich
network traffic explorer
Insight transforms a Sightline deployment into a rich network traffic explorer that extends
Sightline with new reporting and forensics capabilities. Intuitive workflow enables “speed of
thought” traffic analytics and forensics.
It enables network and security engineers to flexibly slice and visualize the data to answer
network, security and business questions.
Insight supports all major use cases - root cause analysis and debugging, DDoS forensics,
transit/peering analysis and network planning.
Network visibility and advanced analytics play an ever more critical role in maintaining
optimum network operations and making intelligent business decisions.
Insight is built to maximize the analytical power of bigdata and puts it within reach of network
operators and network professionals.
See the diagram to understand how Insight is installed on top of Sightline to bring additional
features.
Flow data from routers goes to the TRA appliances and not directly to Insight.
TRAs still continue to bin data and perform their legacy reporting duties. Insight receives is the
raw, annotated flow data directly from TRAs.
• Additional hardware is
required called an Insight
cluster
• Scalable
– Grows with demand
• Flexible and depends on:
– Number of flows in your
Sightline deployment
– Duration of historical data
– Redundancy
The collective hardware that is added to your Sightline deployment to use Insight is called
an Insight cluster. An Insight cluster is made up of nodes. Each node performs specific
tasks.
Insight is a scalable solution that can grow as your needs change. You can add appliances
to your cluster to increase processing capacity, data retention and
redundancy.
Insight is a flexible system with many options for tuning performance. Some things to consider
are:
• The number of flows per second (fps) that Sightline sends to the Insight cluster
• The duration of historical data that you want to explore using Insight (for example, two
weeks, two months, or six months)
• The redundancy of the data stored
• Enhanced forensic
investigation
The new big-data technologies that have come about in recent years take advantage of the
increased ubiquity of hardware resources while also enabling multi-dimensional access over
all stored data, meaning that any combination of any number of correlations can be retrieved
and analyzed with a simple yet powerful data exploration interface.
Insight also enables high fidelity historical and forensics analysis by retaining raw traffic flows
for as long as storage allows, providing a photographic memory of high-detail historical traffic
patterns as well as facilitating DDoS and other security investigations.
Insight provides multi-dimensional analysis for many existing network visibility challenges. For
instance, when conducting peering analysis, multi-dimensional analysis brings ASN origin,
peers, routers, interfaces, customers and more aspects together all at one time and makes it
easy to see traffic move from the source to the destination and how it touches the intermediate
steps.
Insight means you can move freely and intuitively through historical data in less time to learn
more about past attacks, targets and indicators without being overwhelmed by multi-
screen/multi-report process.
annotated annotated
flow records flow records
Insight
Sightline database
report database
Forensics over SAMPLED raw flows Forensics over ALL raw flows
COPYRIGHT © 2021 NETSCOUT SYSTEMS, INC. 13
Sightline has a simple, high-capacity raw flow records database, while Insight is built on top of
a scalable, distributed analytics data store that significantly extends flow storage to archive
flow data for even the largest networks with sub-second accuracy.
annotated annotated
flow records flow records
managed managed
object object
matched matched
Sightline Insight
report database database
We see 4 annotated flows in the diagram, 3 of them match with managed objects. This means
that for these 3 annotated flows, data is being processed by the Sightline reporting daemon
and reflected to managed object reports for later usage. This does not apply to network and
router reports, but managed object reports such as customer, ASN origin etc.
If you don’t have a managed object for the flow being processed, the data is not reflected to
most of the Sightline reports (not all), and you can not access this data later. This is why
creating managed objects for important resources is recommended in Sightline. This does not
affect DDoS detection and all annotated flows (matching or non-matching) are considered
here.
Insight is different. It stores every single annotated flow data in its database to access later.
You can access all details of your resources even you didn’t previously create a managed
object for it. As can be seen in the diagram, all 4 annotated flows are stored in the Insight
database, even those that do not match any managed object.
2 weeks
00:00 00:55 1 year
00:00 00:55
With Sightline, the granularity of the data decreases with the passage of time, while the
granularity of the data is retained with Insight.
Sightline is not designed to store every annotated flow. Daily and frequent usage of the reports
besides fast access is Sightline’s key functionality. In Sightline, you lose details and binning
cycles with the passage of time. For example, if you have 12 binning points for a 1-hour report,
this reduces to 2 binning points after 1 week, and 1 binning point after 2 weeks as was seen in
the previous unit.
Insight brings big data functionality to Sightline and is designed to store every single annotated
flow. That stored data is never aggregated. Even after 1 year, you will see the same granular
data.
Sightline allows you to filter traffic data using two facets of the traffic.
Insight allows you to create a filter using as many available facets as you want to display data
of interest. Each facet that is added to a filter narrows the scope of the traffic data that is
displayed.
Remember that Sightline filters provide you the binned data while Insight brings you the actual
annotated flow data.
time series
Sightline allows you to display traffic as a time series, while Insight also allows you to visualize
the relationships between multiple facets. This is particularly useful when you want to identify
the elements of your network that are carrying the largest amounts of traffic within the duration
of a given time period, rather than individual high-traffic and low-traffic incidents that occur in a
time period.
Sightline allows you to investigate traffic data from cities internal to your network and regions
and countries external to your network, while Insight allows you to investigate traffic data for
cities, countries, and/or regions either internal or external to your network.
Time: Select the time period for the traffic displayed on the graph and table in each tab.
Units: Select the units that are used to express traffic data displayed on the Insight page. After
you change the selected units, click Update to apply and display the changes. The options are
bps, pps and fps (flow per second).
• Calculation: Select the desired method for calculating the data to display on the Insight
page. The following methods are available:
• Last: displays the values of the last traffic logged during the selected time period.
• Average: displays the average of all traffic during the expanded time period.
• Max: displays the maximum of all traffic during the expanded time period.
• PCT95: displays the 95th percentile of all traffic during the expanded time period.
• Total: displays the total amount of all traffic during the expanded time period.
Fidelity:
• High (Standard): Insight runs a query on 100% of all traffic data for the time period and
returns all traffic data that matches the settings in the control bar. High-fidelity queries take
more time to return than lower-fidelity queries. This is the default setting.
• Moderate (Faster): Insight runs a query on 10% of all traffic data for the time period and
scales the traffic that matches the settings in the control bar to account for sampling.
Moderate-fidelity queries take less time to return than high-fidelity queries.
• Low (Fastest): Insight runs a query on 1% of all traffic data for the time period and scales
the traffic that matches the settings in the control bar to account for sampling. Low fidelity
queries take significantly less time to return than higher-fidelity queries.
(continued)
(continued)
Filter: Allows you to select the view and the facets for which traffic is displayed on
the Insight page. Facet means any dimension or aspect and we will cover it in more detail at
upcoming slides.
Tabs:
• Summary: Displays a summary of the traffic specified in the control bar, both in graph and
in table form.
• Relationships: Displays a Sankey diagram that allows you to visualize the volume of traffic
moving between facets specified in the Filter box.
• Top Contributors: Displays tables and graphs that allow you to see traffic for the top
contributors within the facets you select on this tab.
• Raw Flows: Displays predefined information for about 50 raw flow records for the traffic
specified in the control bar. This information includes various aspects of the traffic, such as
IP protocol, source and destination port numbers, and source and destination IP addresses.
VIEW FILTER
• From which perspective you • Define the traffic you want to
want to view traffic? investigate with filters
– as it entered or exited the – traffic that passed through a
network certain interface, router etc.
– wherever it was seen – traffic that uses certain ports,
– when it crossed a certain protocols etc.
customer boundary – traffic that was sent to a
– when it crossed a customer specific host or prefix, ASN etc.
boundary with certain tag
The Insight Filter box is where you define the traffic you want to investigate in Insight.
When you use the Filter box, ask yourself the following two basic questions:
1. What kind of traffic do I want to see?
Example answers:
• I want to see traffic that started or ended at a certain customer
• I want to see traffic that passed through a certain interface
• I want to see traffic that was sent to a specific host or prefix
• I want to see the traffic of the ports that received the most traffic
This is determined by the facets you select in the Filter box
2. What perspective do I want to see the traffic from?
Example answers:
• I want to see the traffic as it entered or exited the network
• I want to see the traffic wherever it was seen
• I want to see the traffic when it crossed a certain customer's boundary
This is determined by the View selector in the Filter box.
The View selector is a component of the Filter box. It allows you to select the perspective you want to
view traffic that is displayed on the Insight page.
Network: Displays traffic that crosses the network boundary.
Customer: Displays traffic that crosses a customer managed object's boundary and matches the
managed object's match value.
Peer: Displays traffic that crosses a peer managed object's boundary and matches the managed object's
match value.
Profile: Displays traffic that crosses a profile managed object's boundary and matches the managed
object's match value.
Customer Tag: Displays traffic that crosses a customer managed object's boundary, matches the
managed object's match value, and matches a tag assigned to the managed object.
Peer Tag: Displays traffic that crosses a peer managed object's boundary, matches the managed object's
match value, and matches a tag assigned to the managed object.
Profile Tag: Displays traffic that crosses a profile managed object's boundary, matches the managed
object's match value, and matches a tag assigned to the managed object..
All Flows: All flows are displayed. Boundaries are not considered.
Note: When using All Flows Insight may count traffic multiple times if the traffic matches the filter criteria.
For example, if you are investigating a destination host, and traffic passes through three routers to arrive
at that host, Insight counts and displays the traffic three times when using the All Flows view.
If you set the View to Customer and select the managed objects for customer A and customer B, Insight
displays traffic that:
• crossed the boundary of customer A and matched the match value of customer A, or,
• crossed the boundary of customer B and matched the match value of customer B
You can add as many facets and facet values to the Filter box as you want, but more facets,
more values, and a longer time period increase the time it takes to collect and display the
traffic information on the Insight page.
• Click Is (=) to select an “equals” operation. For example, to display the traffic of Customer
A, filter by “Customer = A”.
• Click Is Not (!=) to select a “does not equal” operation. For example, to display traffic from
customers other than Customer A, filter by “Customer != A”.
Insight processes multiple values within the same facet with an OR operator. If you specify
multiple values for a facet, Insight displays traffic that matches any of those values. For
example, if you selected TCP Port facet and values 443, 80 it brings you data if it is either TCP
port 443 or TCP port 80.
Q : Show the top combinations of router and destination origin ASN traffic
used by customer A.
The Summary tab displays a graph of the traffic specified in the control bar.
Each column represents a facet selected; each row represents the dataset in graph.
The Relationships tab makes it easy to visualize the volume of traffic moving between facets
specified in the Filter box. It is particularly useful when you want to identify the elements of
your network that are carrying the largest amounts of traffic within the duration of a given time
period, rather than individual high-traffic and low-traffic incidents that occur in a time period.
Insight indicates the volume of traffic that moves between facets. Traffic is visualized using
gray connections of varying thicknesses; thicker connections indicate higher traffic volume,
and thinner connections indicate lower traffic volume. To display additional information:
Hover your mouse pointer over a connection to display details about the traffic moving
between the facet on the left and the facet on the right.
Hover your mouse pointer over a specific facet to display details about the combination of all
traffic moving between the facets on the left and the facets on the right.
The Top Contributors tab allows you to display the top traffic contributors of certain facets
within the traffic specified by the Filter box.
For example, you can see which customers and routers have the most traffic on destination
port 80 by setting Destination Port = 80 in the Filter box, and then setting Customer and Router
on the Top Contributors tab.
The Raw Flows tab lists actual flows for forensic purposes. It is not possible to list all individual
flow records, they are randomly selected by considering filter and timeframe criteria. The more
filters you do, the more precise list you will get.
– Faster query
performance Low: 1% of all traffic data. Faster than others.
Fidelity selectors allow the user to query the sampled data. It allows longer queries (weeks to
months) to finish earlier while providing enough fidelity to answer long term trend analysis
questions. Insight uses sampled data sources for this purpose.
Tip: If you run query after query in an effort to find certain traffic patterns or anomalies, you can
set the Fidelity selector to Moderate (Fast) or Low (Fastest) to process your queries quickly.
After you find the traffic that you are interested in, you can set the Fidelity selector to High
(Standard) and view the results at full fidelity.
Filter with interested items and then use High Fidelity to access all flows
Let's say you are checking for source IP addresses sending the highest traffic for the last 3
hours. This could result in a lot of IP addresses!
Use Low Fidelity for a faster query in the smaller dataset to gather the source IP list.
In the Low Fidelity results, select the most interesting source IPs and use High Fidelity to
retrieve all flows belong to these IPs. This will give increased granularity and a higher
response time, but all the data for the selected IPs will be in the results.
Useful to re-query
Save the current Use “Saved
complex or daily
query settings Queries” to access
requirements
Insight allows you to save the current control bar settings and then reload them later from
the Saved Queries tab. Each saved query contains all of the information in the control bar that
is necessary to reload the currently displayed Insight page.
Does not
Last 25 Current UI The results are
query Insight
queries session in the browser
cluster
Session History tab allows you to display the results of the last 25 queries from the current UI
session. The query results are stored temporarily in your web browser, which means that you
can display them again quickly without re-querying the Insight cluster. The session history is
lost when you do any of the following:
• Close the browser window or tab
• Refresh the Insight page
You can use the traffic filtering features of Insight to detect very specific types of traffic, and
then trigger an alert when that traffic exceeds a certain threshold. This is called Smart alerting.
You use the Insight page (Explore > Insight) to create Smart alert configurations. Use the
Insight page to display the traffic that you want to detect and then click the Set Smart
Alert button to start creating a configuration. Set the detection details, including the traffic
threshold for triggering alerts, and save the configuration.
Sightline uses the details of each Smart alert configuration to query the Insight cluster every
five minutes. When Insight detects that the traffic specified in a Smart alert configuration
exceeded the specified threshold, Sightline creates a Smart alert. When Insight detects that
the traffic dropped below the specified threshold, Sightline stops the alert.
Just as with other alert types, Sightline displays Smart alerts on the alert listing pages. When
you click the ID of a Smart alert, you can display the traffic that triggered the alert on the
Insight page and investigate the incident.
Let's assume there is a customer suffering from DNS Amplification attacks. We can create a
Smart Alert specifically for this customer and attack type and get notifications once traffic
exceeds the configured threshold.
Currently, Smart alerts cannot be mitigated.
The result is displayed by using a relationship graph. This filter does not anchor the
directionality (source IP, input interface etc.), therefore the graph is bidirectional.
In other words, the graph shows both the traffic sourced from Customer A and destined to
Customer A.
When you hover with the mouse, you can see IN and OUT values for the pairs.
Here the data is anchored by selecting a filter that limits it to traffic in one direction. The filter is
for Source Customer being A or B and now the traffic behaves in a more predictable fashion.
This example displays the entire network’s traffic breakdown from source origin ASN to
destination origin ASN. It shows which peers the traffic is coming from and going to, and which
router(s) are forwarding that traffic. It's a good summary to understand network wide routing
decisions.
The report also gives a breakdown of router input and output interfaces to understand routing
decisions at each router level.
This example report is to understand the traffic utilization for each peer. Peers are being used
to access which Origin ASNs, and for what application types.
If you need a high-level view of the traffic matching certain signatures sourced from your
customer, you could use these facets.
It is common that new malware communication quickly becomes popular worldwide. You can
use Insight flexible filtering to match this traffic in your network and see a source country and
destination IP breakdown. By using this you can understand where the malware traffic comes
from and is destined to which of IPs in your network.
Raw flows gives you the actual flow records that match the malware traffic to get more insight
about the traffic pattern.
Sightline has alerts that give you detailed information about the alert traffic. Insight has also
flexible filters to report this traffic. If you know your certain IPs or prefixes that are under attack,
you can use facets to understand the sources and types of traffic.
b) Aggregated flow data to minimize long-term Q4: What displays a Sankey diagram to visualize
storage needs the traffic between facets specified in the Filter?
c) Multi-dimensional visual analysis for complex a) Summary tab
queries
b) Raw Flow Tab
d) Alternative reporting on deep packet inspection
c) Relationships tab
d) Top Contributors tab
Solution: Q1 = b ; Q2 = c ; Q3 = b ; Q4 = c
• Big Data definition, services and usage in network visibility were discussed