F5 Analytics - Lab Guide - Final
F5 Analytics - Lab Guide - Final
F5 Analytics
Presented by: Ashish Zalani
Agility 2017 F5 Analytics Lab Guide
What’s inside
Introduction 1
Lab Environment Setup 1
Accessing the Lab Environment 2
f5.com i
Agility 2017 F5 Analytics Lab Guide
Introduction
In this lab session, you will be introduced to a few different ways to gather,
visualize and analyze traffic information available on a BIG-IP platform. It is
assumed that you are familiar with the basics of setting up a BIG-IP device for
various ADC functions. Hence, in order to focus maximum time on the Analytics
portion of the lab, the lab environment has been setup with an HA pair of BIG-IP
Virtual Editions (VEs) that have been pre-configured for a few web applications.
Your task will be to configure the BIG-IPs to generate Analytics data so that you
may visualize and analyze this data.
In the interest of time, the following components have been setup with basic
configurations for you:
The Windows client is used as a jump host to access the BIG-IPs as well as
the Linux server. It also has a traffic generator, pre-configured to generate a
significant volume of traffic for the web applications.
The BIG-IP Virtual Editions are running BIG-IP version 12.1.2 (HF2) and
have been pre-licensed and provisioned for Local Traffic Manager (LTM),
as well as Analytics Visibility and Reporting (AVR). The BIG-IPs have also
been paired together in an Active/Standby HA device cluster.
f5.com 1
Agility 2017 F5 Analytics Lab Guide
To access the lab environment, you will require a web browser and Remote
Desktop Protocol (RDP) client software. The web browser will be used to access
the Lab Training Portal to retrieve the IP address for your Windows jump host that
you will RDP into to access the entire lab environment.
3. Establish an RDP connection to your jump box with the IP address retrieved
from Step 2 and login with the following credentials:
User: external_user
Password: password
User: admin
Password: AgilityLab17
f5.com 2
Agility 2017 F5 Analytics Lab Guide
The Application Visibility and Reporting (AVR) module provides detailed charts
and graphs to give you more insight into the performance of web applications,
TCP traffic, DNS traffic, as well as system performance (CPU, memory, etc.). You
can use this module to visualize the traffic being processed by your BIG-IP device,
and gain a better understanding of where the traffic is originating from (client IP
addresses / subnets as well as geographical regions), the nature and volume of
request and response traffic (Total Transactions as well as Average and Max
Transactions/sec), the most commonly requested URLs, Server Latency and
Page Load times, Virtual Server and Pool member performance, and many more
metrics. This lab will give you a brief introduction on how to setup the AVR module
to generate these charts / reports and how to visualize them on your BIG-IP.
In this lab, you will first configure an Analytics profile to attach to your existing
applications (Virtual Servers), and then generate some traffic for these
applications. You will then view the analytics graphs and charts on the BIG-IP to
gain more insight into the traffic patterns for incoming traffic for your applications.
You will perform all configuration tasks from the Windows jump box
On the Windows Jump box, open the Chrome browser, and then use the
bookmark in the bookmark bar to access BIGIP_A.
Username: admin
Password: AgilityLab17
f5.com 3
Agility 2017 F5 Analytics Lab Guide
Task 1 – Create a new Analytics profile and attach it to your Virtual Servers
2. Click Create
f5.com 4
Agility 2017 F5 Analytics Lab Guide
7. Click Finished
9. Click on F5_Demo_HTTPS_VS
10. Use the pull-down menu to change the Configuration from Basic to
Advanced
f5.com 5
Agility 2017 F5 Analytics Lab Guide
11. Scroll down and change the HTTP Analytics Profile from None to
Custom_HTTP_Analytics
12. Click Update at the bottom to save the Virtual Server configuration
1. Minimize the Chrome browser window, and launch JMeter from the
Desktop shortcut
2. Once the Apache JMeter window opens, go to File > Open, and open the
F5_Analytics_Demo.jmx file
f5.com 6
Agility 2017 F5 Analytics Lab Guide
While you wait for the traffic to be generated and sent to the BIG-IPs, and for AVR
to gather and analyze the data, let us use this time to explore the setup we have.
Apache JMeter is a traffic generator that we are using in this lab in order to
simulate user traffic, since we do not have any actual traffic hitting our applications
or BIG-IPs. In your own environment, you probably would not need to use JMeter,
since you presumably have actual users accessing your applications through your
BIG-IP devices.
The Application Visibility and Reporting (AVR) module is a built-in module that is
available on all BIG-IP platforms starting software version 11.x onwards. This is a
special module that does not need a separate license (it is included by default).
However, it does need to be provisioned as a module on your BIG-IP in order to
use it. You can verify that we have the module provisioned on our BIG-IPs by
going to the BIG-IP GUI (in the Chrome browser), and going to System >
Resource Provisioning, and verifying that AVR is provisioned (check marked
under the Provisioning column). In your own environment, you will need to
provision the AVR module on your BIG-IPs before you can use it. Note that
provisioning (or de-provisioning) any module will re-start some services on the
BIG-IP, which could disrupt some traffic, hence you may want to only do so during
a maintenance window.
BIG-IP Virtual Servers – randomizing the client source IPs and User Agents
Since we do not have actual user traffic in the lab environment, we have setup the
lab to simulate traffic originating from several different client source IP addresses
and HTTP User Agent strings. This allows us to get some more interesting
f5.com 7
Agility 2017 F5 Analytics Lab Guide
reports for Analytics. The way this is accomplished is by using “proxy” Virtual
Servers (Src_IP_Randomizer_HTTP / Src_IP_Randomizer_HTTPS) which
intercept the incoming traffic from the Win7_Client, change the source IP to a
randomly chosen IP address, and then forward the request to the actual
application Virtual Server that will process the traffic. The application Virtual
Servers (F5_Demo_HTTP_VS / F5_Demo_HTTPS_VS) then further change the
HTTP User-Agent header to a randomly chosen value before forwarding the
request to the pool member (application web server). All of this is accomplished
via iRules attached to these Virtual Servers. We encourage you to take a look at
how the whole setup works by going to Local Traffic > Network Map. Look at the
configuration for the virtual servers named Src_IP_Randomizer_HTTP/HTTPS.
There is just a simple iRule (Src_IP_Randomizer / _HTTPS) attached to these.
Click on the iRule name to view its details. Similarly, view the
Src_UA_Randomizer iRule attached the application Virtual Servers. Ask the
instructor if you would like help understanding how these iRules work.
In order to see all this in action, open a new tab in the Chrome browser window,
and click on either the Demo1 (HTTP) or Demo2 (HTTPS) bookmarks in the
browser bookmark bar. When the page loads, scroll down to the HTTP Request
and Response Information section and click the link for Request and
Response Headers. Now refresh this page several times (Ctrl-Shift-R), and
observe the Client IP address/port field and the User-Agent field displayed on
the page, and notice how they change every time you refresh the page.
NOTE: in your own environment, you will not need to use these tricks to change
the client Source IPs and User Agent strings, since you would presumably have
traffic from actual users originating from different IP addresses and using different
HTTP User-Agent strings.
f5.com 8
Agility 2017 F5 Analytics Lab Guide
Task 3: Generate detailed charts and reports to visualize the analytics data
Once we have had some traffic received by the application Virtual Servers and
processed by the Analytics profile, we can now go in and view and analyze this
data.
1. In the BIG-IP GUI, go to Statistics > Analytics > HTTP > Overview
2. In the Override time range to pull-down menu at the top, change the
value to Last Hour
3. This page now shows you details about the traffic received by all the
Virtual Servers that had the HTTP Analytics profile attached.
f5.com 9
Agility 2017 F5 Analytics Lab Guide
5. You can move the widgets around and re-arrange the page by simply
dragging-and-dropping the widgets from the top-left corner of each widget.
add more widgets on the page by clicking the Add Widget button at the
bottom of the page.
f5.com 10
Agility 2017 F5 Analytics Lab Guide
6. Similarly, you can modify the tables on the right side of the page, and add
another table by clicking the Add Widget button below the last table on
the right.
7. Once you have updated the page to show you the data you want, you can
create a report by clicking the Export button at the top-right of the page.
Bonus Lab:
Similar to the HTTP Analytics profile that we used in this lab, create a custom TCP
Analytics profile, attach it to your virtual servers, generate some more traffic, and
then view the results in the Analytics pages on your BIG-IP.
f5.com 11
Agility 2017 F5 Analytics Lab Guide
In this lab, we will integrate our BIG-IPs to send data into Splunk, and use Splunk
to visualize and analyze the data from a single centralized location rather than
viewing/analyzing it on an individual BIG-IP.
In order to get Splunk to process and display Analytics data from your BIG-IPs,
you need to configure it to accept this data, parse and process it, and display it in
a meaningful way for you to get the most out of it. In order to help with this, F5
has written a Splunk app that is available as a free add-on to your Splunk
deployment. This F5 Analytics Splunk app can be downloaded from the
Splunkbase web-site here:
https://splunkbase.splunk.com/app/3161/
For your convenience, we have already downloaded this Splunk app onto the
Windows jump box, so we can just go ahead and install it within our Splunk
instance.
f5.com 12
Agility 2017 F5 Analytics Lab Guide
1. In the Chrome browser on your Windows jump box, click the bookmark for
Splunk to launch the Splunk web UI
3. In the Splunk Web GUI, click on the settings button next to Apps (on the
left) to Manage Apps
6. In the file browser window, navigate to Desktop > Analytics Lab Files,
and choose the f5-networks-analytics-new_100.tgz file and click Open
f5.com 13
Agility 2017 F5 Analytics Lab Guide
7. Click Upload
8. Once the upload is complete, you should see the F5 Networks app listed
in the Apps table, with the Status set to Enabled
9. Click the Splunk logo in the top-left to go to the start page. You should
now see the F5 Networks app listed on the left
f5.com 14
Agility 2017 F5 Analytics Lab Guide
10. Now click the Settings menu in the top-right, and choose Data inputs
f5.com 15
Agility 2017 F5 Analytics Lab Guide
15. For the Name, enter F5-Analytics, and then Click Next > at the top
16. On the Input Settings page, scroll down till you see Default Index, and
then click the Create a new index link
f5.com 16
Agility 2017 F5 Analytics Lab Guide
17. In the New Index window, enter f5-default for the Index Name, and click
Save
20. Ensure your settings match those shown in the screenshot below, then
click Submit
f5.com 17
Agility 2017 F5 Analytics Lab Guide
21. Once your token has been created, highlight the Token Value for the
newly created Token, and copy it to your clipboard (Ctrl-C or Right-click
> Copy). We will use this later.
NOTE: Your token value will be different from the one shown above
f5.com 18
Agility 2017 F5 Analytics Lab Guide
22. Click on the Splunk logo in the top-left to go back to the Splunk start
page.
https://support.f5.com/csp/article/K07859431
Note that the F5 Analytics iApp template itself does not ship with the product, but
can be downloaded from the F5 downloads site (https://downloads.f5.com).
For your convenience, we have already downloaded the iApp template on the
Windows jump box, so we can just import it into our BIG-IP.
Task 2: Import and configure the F5 Analytics iApp template on the BIG-IP
1. Open a new tab in your Chrome browser, and click on the bookmark for
BIGIP_A to connect to the BIG-IP GUI
Username: admin
Password: AgilityLab17
f5.com 19
Agility 2017 F5 Analytics Lab Guide
7. Click Upload
8. Once the file is finished uploading, you should see it listed in the iApp
Templates table.
f5.com 20
Agility 2017 F5 Analytics Lab Guide
f5.com 21
Agility 2017 F5 Analytics Lab Guide
NOTE: If you are not familiar with what all the different settings refer to, you
may want to keep the inline help enabled. For now, we have disabled it just to
reduce the amount of additional text on the configuration screen.
f5.com 22
Agility 2017 F5 Analytics Lab Guide
16. Leave all settings under Module Log Stream Capture and Local
Logging Capture sections at their default values
17. Under Application Mapping, leave all settings at their default values,
except in the Mapping Table, enter the following:
a. Order: 10
b. Type: App Name
c. From: Virtual Name
d. Regex: (.*)_HTTP[S]*_VS
e. Action: Map
f. AppendPrefix: <leave blank>
g. DirectMapping: <leave blank>
f5.com 23
Agility 2017 F5 Analytics Lab Guide
1. Minimize the Chrome browser window, and launch JMeter from the
Desktop shortcut
2. Once the Apache JMeter window opens, go to File > Open, and open the
F5_Analytics_Demo.jmx file
f5.com 24
Agility 2017 F5 Analytics Lab Guide
While you wait for the traffic to be generated and sent to the BIG-IPs, and for
Splunk to gather and analyze the data, let us explore the setup for this lab.
This iApp template is designed to gather a large number of statistics and event
information from a variety of different sources, and export the data to different
kinds of data collectors / SIEM systems. The sources of information that the iApp
gathers include system performance metrics (CPU, memory usage, throughput,
connection rates, etc.), tmstats (statistics collected by the Traffic Management
Microkernel / TMM regarding the traffic that is being handled/processed by TMM),
event logs (from the /var/log directory), SNMP trap-related information, and AVR
data. The options in the iApp allow the user fine-grained control on what data will
be collected and bundled up to be sent to external receivers. Further, the iApp
also provides the ability to customize the output format for different receivers,
including F5 BIG-IQ, Splunk, as well as other 3rd-party systems. Lastly, the iApp
provides for options to group together and/or map different pieces of information
(Virtual Servers and their associated objects, etc.) into Facilities (e.g. data
centers), tenants (for multi-tenant environments), and applications, where a single
application could consist of multiple virtual servers (for example, a web application
could consist of both, an HTTP and an HTTPS virtual server that serve the host
the same application). This application mapping can also be applied across
multiple BIG-IPs so that the same application hosted in different locations can be
grouped together under a single application name. For more details on the iApp,
please see the iApp Deployment Guide, which can be found here:
https://www.f5.com/pdf/deployment-guides/f5-analytics-dg.pdf
f5.com 25
Agility 2017 F5 Analytics Lab Guide
The data presented in the F5 Networks Splunk app includes a lot of data that
cannot be easily visualized on a BIG-IP, such as tmstats information, virtual server
and pool member health stats, system performance information, and even syslog
event information. Additionally, this app provides the ability to collate and present
data across multiple BIG-IP devices, even BIG-IPs in different locations, allowing
a user to view all their devices and their data in one single central location, rather
than having to view it separately on each individual BIG-IP device.
Note that this lab guide walks you through some simple setup options for both, the
Splunk app as well as the iApp, in order to help you get up-and-running quickly.
However, these configuration options are by no means the only way to configure
these. To get a better understanding of all the configuration options we ask that
you refer to the F5 analytics iApp deployment guide referenced earlier which also
has a section on configuring the Splunk app.
f5.com 26
Agility 2017 F5 Analytics Lab Guide
1. In your Chrome browser window, open a new tab, and click on the Splunk
bookmark to launch the Splunk Web UI
3. On the Home tab of the F5 Splunk app, change the Time pull-down to
Last 60 minutes
f5.com 27
Agility 2017 F5 Analytics Lab Guide
5. Scroll down to view other widgets. You may find that your BIG-IP devices
are shown under the Unhealthy Devices and/or the F5_Demo application
is shown in the Unhealthy Applications. Let’s investigate:
7. Just under the Overview table showing the scores on different metrics,
you can also see a table showing the Device Status, with details on the
devices included in the group
8. If your devices had any error conditions that generated some Diagnostic
information, you could see that in the Diagnostics section.
f5.com 28
Agility 2017 F5 Analytics Lab Guide
9. Now click on the System Performance tab. This will show you details on
the CPU and Memory usage of your BIG-IP devices, including a
breakdown of processes consuming the most amount of CPU or memory
10. Next, click on Interface Status & Statistics. This will show you detailed
Interface and VLAN stats. You can change the options in the pull-down
menus to view different information.
11. Next, click on the Events tab. This tab shows you Syslog events, with a
time-chart of when different kinds of events occurred. If you see any
f5.com 29
Agility 2017 F5 Analytics Lab Guide
12. Feel free to explore the other tabs as well to view additional information
13. Now, let us look at our applications to view more details about them. In the
red menu bar at the top, click on Application > Application Dashboard.
14. This will show you a listing of all the applications across all your BIG-IPs,
based on the application grouping and mappings that you defined in the
f5.analytics iApp on the BIG-IP. In our case, we just have a single
application. Click on the F5_Demo application name to go into the
Application Drilldown dashboard
f5.com 30
Agility 2017 F5 Analytics Lab Guide
15. The Application Drilldown dashboard shows you a lot of detailed statistics
about your application(s). You can view the various metrics for your
application(s) on the Overview tab. To get more details, you can click the
link for the View in Application Health Dashboard, which will give you even
more detailed metrics and charts that are used in calculating the health
scores for the various metrics:
16. In the Application Health Dashboard that opens up in a new browser tab,
you can view the various metrics as well as charts for those metrics that
make up the overall Application Health score
17. Now go back to the browser tab for the Application Drilldown
dashboard, and then click on the Application Resources tab. This tab
shows you various components that make up your applications, including
the facility, virtual servers, pools, pool members, and even iRules. In our
case, our F5_Demo application is hosted in a single Facility (F5 Lab), and
is made up of 2 Virtual Servers: F5_Demo_HTTP_VS and
F5_Demo_HTTPS_VS. Each virtual server has its own pool with their
f5.com 31
Agility 2017 F5 Analytics Lab Guide
corresponding pool members. You can view details for all these
components in the tables below.
18. Next, click on the Traffic Overview, Latency, & Analytics tab. This tab
shows you detailed traffic-related stats, similar to the data available via the
AVR charts and reports you saw in Lab 1.
19. Next, click on the Client Visibility tab. This tab provides a lot of visibility
into the traffic between the end-clients and the BIG-IP, including
connection stats, throughput information, TCP stats, HTTP information
(HTTP requests, HTTP version, HTTP compression info, etc.), SSL
information (SSL throughput, SSL protocol info, ciphers, SSL
renegotiations, etc.). Similarly, the Server Visibility tab provides similar
information for the traffic between the BIG-IPs and the back-end
application servers.
20. The Pool Statistics tab provides details on the various pools and pool
members across all the BIG-IPs and each application / virtual server on
each BIG-IP.
21. Feel free to explore other tabs including the System Performance and
the Alerts and Logs tabs.
This concludes all the lab steps for the Splunk Integration lab. Feel free to
explore other portions of the F5 Splunk app, or try out other settings in the
f5.analytics iApp. Note that this lab environment does not include other F5
modules (DNS/GTM, AFM, ASM, or APM). However, if you have these other
modules enabled on your BIG-IP devices in your own environment, you can
view data for these modules as well in the F5 Splunk app.
The End
f5.com 32
F5 Networks, Inc. | f5.com
US Headquarters: 401 Elliott Ave W, Seattle, WA 98119 | 888-882-4447 // Americas: [email protected] // Asia-Pacific: [email protected] // Europe/Middle East/Africa: [email protected] // Japan: f5j [email protected]
©2017 F5 Networks, Inc. All rights reserved. F5, F5 Networks, and the F5 logo are trademarks of F5 Networks, Inc. in the U.S. and in certain other countries. Other F5 trademarks are identified at f5.com. Any other products,
services, or company names referenced herein may be trademarks of their respective owners with no endorsement or affiliation, express or implied, claimed by F5. These training m aterials and documentation are
F5 Confidential Information and are subject to the F5 Networks Reseller Agreement. You may not share these training materials and documentation with any third party without the express written permission of F5.