VMware Validated Design™

Monitoring and Alerting Guide

VMware Validated Design for Software-

Defined Data Center 2.0

This document supports the version of each product listed

and supports all subsequent versions until the document is
replaced by a new edition. To check for more recent
editions of this document, see
http://www.vmware.com/support/pubs.

EN-002204-00
 VMware Validated Design Monitoring and Alerting Guide

You can find the most up-to-date technical documentation on the VMware Web site at:
http://www.vmware.com/support/
The VMware Web site also provides the latest product updates.
If you have comments about this documentation, submit your feedback to:
[email protected]

© 2016 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright
and intellectual property laws. This product is covered by one or more patents listed at
http://www.vmware.com/download/patents.html.
VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other
jurisdictions. All other marks and names mentioned herein may be trademarks of their respective
companies.

VMware, Inc.
3401 Hillview Avenue
Palo Alto, CA 94304
www.vmware.com

© 2016 VMware, Inc. All rights reserved.

Page 2 of 80
 VMware Validated Design Monitoring Guide

Contents

1. Purpose and Intended Audience ...................................................... 5

2. Enabling Alerts in vRealize Log Insight ............................................ 6
2.1 View the Full List of Alerts for a Management Product .................................................. 6
2.2 Enable the Alerts for vSphere Resources ...................................................................... 7
2.3 Enable Alerts for Storage Resources........................................................................... 12
2.4 Enable the Alerts for Virtual SAN ................................................................................. 19
2.5 Enable the Alerts for vSphere Networking ................................................................... 24
2.6 Enable the Alerts for NSX for vSphere ........................................................................ 28
2.7 Enable Alerts for vRealize Operations Manager .......................................................... 33
2.8 Enable the Alerts for vRealize Automation .................................................................. 37
2.9 Enable the Alerts for vRealize Orchestrator ................................................................ 42

3. Creating Custom SDDC vRealize Operations Dashboards ............ 49

3.1 Configure a Dashboard that Shows Capacity Trends.................................................. 49
3.2 Configure a Dashboard that the Provides an Overview of SDDC Operation .............. 56
3.2.1 Create an Application for vRealize Log Insight ...................................................................... 56
3.2.2 Create an Application for vRealize Orchestrator ................................................................... 58
3.2.3 Collect the SDDC Objects in a Group ................................................................................... 59
3.2.4 Configure a Dashboard that Provides an Overview of the SDDC State ................................ 60

4. Configure vRealize Operations Manager to Notify of SDDC Issues66

4.1 Create Notifications in vRealize Operations Manager ................................................. 66
4.2 List of Notifications for vRealize Operations Manager ................................................. 68
4.2.1 Notification Delivery Properties ............................................................................................. 69
4.2.2 Virtual Machine and Host Notifications .................................................................................. 69
4.2.3 Networking Notifications ........................................................................................................ 71
4.2.4 Storage Notifications ............................................................................................................. 72
4.2.5 Notifications about vRealize Operations Manager ................................................................. 73

5. Monitor vSphere Data Protection Jobs by Email ............................ 75

6. Configure vRealize Automation System Notification Events ........... 78

© 2016 VMware, Inc. All rights reserved.

Page 3 of 80
 VMware Validated Design Monitoring Guide

List of Tables
Table 1. Delivery Properties of vRealize Operations Manager Notifications ........................................ 69
Table 2. VM and Host Notifications in SDDC ........................................................................................ 69
Table 3. Networking Notifications in SDDC .......................................................................................... 71
Table 4. Storage Notifications in SDDC ................................................................................................ 72
Table 5. Notifications of vRealize Operations Manager Issues ............................................................ 73

© 2016 VMware, Inc. All rights reserved.

Page 4 of 80
 VMware Validated Design Monitoring Guide

1. Purpose and Intended Audience

VMware Validated Design Monitoring and Alerting Guide provides step-by-step instructions for
configuring vRealize Operations Manager and vRealize Log Insight for monitoring of the operations in
the SDDC, and for enabling notifications about issues in your environment, configuring, and operating
a software-defined data center (SDDC) based on the VMware Validated Design for Software-Defined
Data Center.
After you deploy the Software-Defined Data Center from this VMware Validated Design, you can
monitor the parameters that are most important for environment management by using a set of
dashboards for alerts and log events.

Note The VMware Validated Design Monitoring and Alerting Guide is compliant and validated with
certain product versions. See Introducing VMware Validated Design for more information
about supported product versions.

VMware Validated Design Monitoring and Alerting Guide is intended for cloud architects,
infrastructure administrators, cloud administrators and cloud operators who are familiar with and want
to use VMware software to deploy in a short time and manage an SDDC that meets the requirements
for capacity, scalability, backup and restore, and extensibility for disaster recovery support.

© 2016 VMware, Inc. All rights reserved.

Page 5 of 80
 VMware Validated Design Monitoring Guide

2. Enabling Alerts in vRealize Log Insight

Use the vRealize Log Insight known event signature engine to monitor key events. You can use a set
of alerts to send to vRealize Operations Manager and via SMTP for operations team notification.
The integration between vRealize Log Insight and vRealize Operations Manager allows for
implementing the following cross-product event tracking:
 Sending alerts from vRealize Log Insight to vRealize Operations Manager which automatically
maps them to the target objects
 Launching in context from a vRealize Operations Manager object to the objects logs in vRealize
Log Insight
 Launching in context from a vRealize Log Insight event to the objects in vRealize Operations
Manager
For applications that are failed over between regions, such as vRealize Automation and vRealize
Operations Manager, configure alerting in both regions to avoid missing any alerts when applications
move between regions.

2.1 View the Full List of Alerts for a Management Product

Explore the alerts and queries that are available in vRealize Log Insight for the management products
in the SDDC such as vSphere, NSX for vSphere, vRealize Automation, and so on. The alerts and
queries are handled by the content packs for these products.
Procedure
1. Log in to the vRealize Log Insight user interface
a. Open a Web browser and go to the following URL.

Region vRealize Log Insight URL

Region A https://vrli-cluster-01.sfo01.rainpole.local

b. Log in using the following credentials.

Setting Value

User name Admin

Password vrli_admin_password

2. Locate the content pack for the management product.

a. In the vRealize Log Insight UI, click the configuration drop-down menu icon and select
Content Packs.
b. Under Installed Content Packs, select the pack.
c. Click Alerts or Queries to view the full list of alerts for the product.

© 2016 VMware, Inc. All rights reserved.

Page 6 of 80
 VMware Validated Design Monitoring Guide

2.2 Enable the Alerts for vSphere Resources

Use the built-in problem and alert signatures in vRealize Log Insight for ESXi host and vCenter Server
to enable alerts about issues.
For basic monitoring the vSphere components, use the following alerts:

Alert Name Purpose Severity

*** CRITICAL *** Hardware: The purpose of this widget is to notify Critical
Physical event detected when the following physical hardware
events have been detected, which
indicates a hardware problem. Under
most normal conditions, this widget
should return no results. The
following types of hardware events
are returned:
• Advanced Programmable
Interrupt Controller (APIC)
• Machine Check Exception
(MCE)
• Non-Maskable Interrupt (NMI)

Hardware: Faulty memory During the previous boot of an ESXi Critical

detected host faulty memory was detected.
Unless a corresponding corrected
message is seen, the memory should
be replaced.

*** CRITICAL *** ESX/ESXi: A core dump has been detected, Critical
Core dump detected which indicates the failure of a
component in ESX/ESXi. This issue
may lead to VM crashes and/or host
PSODs.

© 2016 VMware, Inc. All rights reserved.

Page 7 of 80
 VMware Validated Design Monitoring Guide

Alert Name Purpose Severity

*** CRITICAL *** ESX/ESXi: The purpose of this alert is to notify Critical
Stopped logging when an ESXi host has stopped
sending syslog to a remote server.

*** CRITICAL *** ESX/ESXi: A root file system has reached its Critical
RAM disk / inode table is full resource pool limit. Various
administrative actions depend on the
ability to write files to various parts of
the root file system and might fail if
the RAM disk and/or inode table is
full.

ESX/ESXi: HA isolated During a health check, HA Critical

events by hostname determined that a host was isolated.
Depending on how HA is configured
this may mean that VMs have been
failed over from the isolated host.

vCenter Server: HA A HA cluster has detected one or Critical

connection failure detected more unresponsive ESXi hosts. If the
host(s) are marked as dead then
VMs running on those hosts will be
migrated to other systems.

Procedure
1. Log in to the vRealize Log Insight user interface
a. Open a Web browser and go to the following URL.

Region vRealize Log Insight URL

Region A https://vrli-cluster-01.sfo01.rainpole.local

b. Log in using the following credentials.

Setting Value

User name Admin

Password vrli_admin_password

2. In the vRealize Log Insight UI, click Interactive Analytics.

3. Click the icon and select Manage Alerts. You see all available alerts.
4. Select the alerts that are related to vSphere.

© 2016 VMware, Inc. All rights reserved.

Page 8 of 80
 VMware Validated Design Monitoring Guide

a. In the search box of the Alerts dialog box, enter Hardware, ESX/ESXi or vCenter
Server as a search phrase, and select the alerts from the results.
b. Repeat the step until you select all the alerts related to vSphere.

Alert Name

*** CRITICAL *** Hardware: Physical event detected

Hardware: Faulty memory detected

*** CRITICAL *** ESX/ESXi: Core dump detected

*** CRITICAL *** vCenter Server: ESX/ESXi stopped logging

*** CRITICAL *** ESX/ESXi: RAM disk / inode table is full

vCenter Server: HA connection failure detected

ESX/ESXi: HA isolated events by hostname

5. Enable the alerts.

a. In the Alerts dialog box, click Enable.
b. In the Enable Alerts dialog box, configure the following alert settings and click Enable.

© 2016 VMware, Inc. All rights reserved.

Page 9 of 80
 VMware Validated Design Monitoring Guide

Setting Value

Email Email address to send alerts to

Send to vRealize Operations Manager Selected

Default Object SFO01-Mgmt01

Criticality critical

6. In the Alerts dialog box, set the Raise an alert option for each enabled alert.
a. Click the Edit button on the first enabled vSphere alert

© 2016 VMware, Inc. All rights reserved.

Page 10 of 80
 VMware Validated Design Monitoring Guide

b. In the Edit Alert dialog box, under Raise an alert, select On any match, and click Save.

© 2016 VMware, Inc. All rights reserved.

Page 11 of 80
 VMware Validated Design Monitoring Guide

c. Repeat the steps for the other enabled alerts.

d. Close the Alerts dialog box.

2.3 Enable Alerts for Storage Resources

Use the inbuilt problem and alert signatures in vRealize Log Insight for storage monitoring.
For monitoring storage in the Software-Defined Data Center, you can use the following alerts in
vRealize Log Insight.

© 2016 VMware, Inc. All rights reserved.

Page 12 of 80
 VMware Validated Design Monitoring Guide

Alert Name Purpose Severity

** CRITICAL *** Storage: All One or more datastores has Critical

Paths Down (APD) experienced an All Paths Down
(APD) outage situation. This
indicates that one or more
datastores is or was unavailable.
As a result of this issue, VMs are
or were unavailable and
ESX/ESXi hosts may have been
disconnected from vCenter
Server. This issue requires
immediate attention.

*** CRITICAL *** Storage: VSAN A Virtual SAN storage device Critical
device offline that backs up the datastores
might fail.
This occurs due to a faulty
device firmware, physical media,
or storage controller or when
certain storage devices are not
readable or writeable.
Typically, such failures are
irreversible. In some instances,
permanent data loss might also
occur, especially when data is
not replicated on other nodes
before failure. Virtual SAN
automatically recovers data
when new devices are added to
the storage cluster, unless data
lost is permanent.

Storage: NFS connectivity issue The purpose of this alert is to Critical

notify when an NFS connectivity
issue was detected. This means
an NFS datastore is or was
unavailable. Do to this issue,
one or more VMs may be
unavailable.

Storage: NFS lock file issue The purpose of this alert is to Critical
notify when an NFS lock file
issue has been detected. Stale
NFS lock files can prevent VMs
from powering on.

© 2016 VMware, Inc. All rights reserved.

Page 13 of 80
 VMware Validated Design Monitoring Guide

Alert Name Purpose Severity

Storage SCSI Path dead The purpose of this alert is to Critical

notify when a SCSI path has
become unavailable. Assuming
multiple paths are in use and the
other paths are online this
means reduced redundancy and
performance. If all paths to a
storage device become
unavailable then VMs running on
the storage device will become
unavailable.

Storage: Snapshot consolidation The purpose of this alert is to Critical

required notify when a snapshot
consolidation is required. A
failed snapshot consolidation
operation that is not manually
addressed can lead to a full
datastore.

© 2016 VMware, Inc. All rights reserved.

Page 14 of 80
 VMware Validated Design Monitoring Guide

Procedure
Open the vRealize Log Insight user interface.
a. Open a Web browser and go to the following URL.

Region vRealize Log Insight URL

Region A https://vrli-cluster-01.sfo01.rainpole.local

b. Log in using the following credentials.

Setting Value

User name admin

Password vrli_admin_password

In the vRealize Log Insight UI, click Interactive Analytics.

Click the icon and select Manage Alerts.

Select the alerts that are storage related.
a. In the search box of the Alerts dialog box, enter storage as a search phrase and select the
following alerts.
b. Repeat the step until you select all the storage related alerts

Alert Name

*** CRITICAL *** Storage: All Paths Down (APD)

*** CRITICAL *** Storage: VSAN device offline

Storage: NFS connectivity issue

Storage: NFS lock file issue

Storage SCSI Path dead

Storage: Snapshot consolidation required

© 2016 VMware, Inc. All rights reserved.

Page 15 of 80
 VMware Validated Design Monitoring Guide

Enable the alerts.

a. In the Alerts dialog box, click Enable.
b. In the Enable Alerts dialog box, configure the following alert settings and click Enable.

Setting Value

Email Email address to send alerts to

Send to vRealize Operations Manager Selected

Default Object SFO01-Mgmt01

Criticality critical

© 2016 VMware, Inc. All rights reserved.

Page 16 of 80
 VMware Validated Design Monitoring Guide

In the Alerts dialog box, set the Raise an alert option for each enabled alert.
a. Click Edit button on the first enabled Storage Resources alert.

© 2016 VMware, Inc. All rights reserved.

Page 17 of 80
 VMware Validated Design Monitoring Guide

b. In the Edit Alert dialog box, under Raise an alert, select On any match, and click Save.

© 2016 VMware, Inc. All rights reserved.

Page 18 of 80
 VMware Validated Design Monitoring Guide

c. Repeat the step for the other enabled alerts.

d. Close the Alerts dialog box.

2.4 Enable the Alerts for Virtual SAN

Use the inbuilt problem and alert signatures in vRealize Log Insight for Virtual SAN monitoring.
For monitoring Virtual SAN in the Software-Defined Data Center, you can use the following alerts in
vRealize Log Insight.

Alert Name Purpose Severity

VSAN - SSD health change to This alert will fire when the state of Critical
unhealthy state any SSD changes to unhealthy. The
reason could be either because of
permanent disk failure, disk
decommissioning, node shutdown,
etc.

© 2016 VMware, Inc. All rights reserved.

Page 19 of 80
 VMware Validated Design Monitoring Guide

Alert Name Purpose Severity

VSAN - Configuration failure - This alert indicates that we cannot Critical

Insufficient space create a configuration for a new
object(VM) in the VSAN cluster
because sufficient space is not
available in the cluster. If we see
this error, please check the error
logs and try the provisioning
operation after adding new
hosts/disks.

VSAN - Device offline This alarm will trigger if a particular Critical

device goes offline. In this case,
please check the device
configuration and other cluster
state.

VSAN - Object component This alert will be triggered when Critical

state changed to degraded VSAN object state changes to
degraded state.Check the state of
the adapters, disks and network
settings associated with the VSAN
cluster.

Procedure
1. Open the vRealize Log Insight user interface.
a. In a Web browser, go to the following URL.

Region vRealize Log Insight URL

Region A https://vrli-cluster-01.sfo01.rainpole.local

b. Log in using the following credentials.

Setting Value

User name Admin

Password vrli_admin_password

2. In the vRealize Log Insight UI, click Interactive Analytics.

3. Click the icon and select Manage Alerts.

Select the alerts that are Virtual SAN related.
a. In the search box of the Alerts dialog box, enter vsan as a search phrase and select the
following alerts.
b. Repeat the step until you select all the alerts related to Virtual SAN.

© 2016 VMware, Inc. All rights reserved.

Page 20 of 80
 VMware Validated Design Monitoring Guide

Alert Name

VSAN - SSD health change to unhealthy state

VSAN - Configuration failure - Insufficient space

VSAN - Device offline

VSAN - Object component state changed to degraded

Enable the alerts.

a. In the Alerts dialog box, click Enable.
b. In the Enable Alerts dialog box, configure the following alert settings and click Enable.

Setting Value

Email Email address to send alerts to

Send to vRealize Operations Manager Selected

Default Object SFO01-Mgmt01

Criticality critical

© 2016 VMware, Inc. All rights reserved.

Page 21 of 80
 VMware Validated Design Monitoring Guide

6. In the Alerts dialog box, set the Raise an alert option for each enabled alert.
a. Click Edit button on the first enabled Storage Resources alert.

© 2016 VMware, Inc. All rights reserved.

Page 22 of 80
 VMware Validated Design Monitoring Guide

b. In the Edit Alert dialog box, under Raise an alert, select On any match, and click Save.

© 2016 VMware, Inc. All rights reserved.

Page 23 of 80
 VMware Validated Design Monitoring Guide

c. Repeat the step for the other enabled alerts.

d. Close the Alerts dialog box.

2.5 Enable the Alerts for vSphere Networking

Use the inbuilt problem and alert signatures in vRealize Log Insight for networking to enable alerts
about issues.
For basic monitoring the vSphere networking components, use the following alerts.

© 2016 VMware, Inc. All rights reserved.

Page 24 of 80
 VMware Validated Design Monitoring Guide

Alert Name Purpose Severity

Network: ESXi physical NIC ESXi has reported that a physical Critical
down NIC has become unavailable.
Assuming other NICs are still online
this indicates a lack of redundancy
and a potential performance impact.
If all physical NICs for a
vSwitch/dvSwitch are unavailable
then communication problems to
VMs and/or the ESXi host may be
possible.

Network: ESX/ESXi uplink Only one physical NIC is currently Critical

redundancy lost connected, one more failure will
result in a loss of connectivity.

Network: Out of Memory Under certain circumstances hosts Critical

with NetQ enabled may run out of
memory when using jumbo frames.
Out of memory conditions could
lead to lost connectivity between
vCenter Server, NFS datastores,
and the virtual machine level.

Procedure
1. Open the vRealize Log Insight user interface.
a. In a Web browser, go to the following URL.

Region vRealize Log Insight URL

Region A https://vrli-cluster-01.sfo01.rainpole.local

b. Log in using the following credentials.

Setting Value

User name admin

Password vrli_admin_password

2. In the vRealize Log Insight user interface, click Interactive Analytics.

3. Click the icon and select Manage Alerts.

4. Select the alerts that are related to vSphere networking.
a. In the search box of the Alerts dialog box, enter network as a search phrase and select the
following alerts.
b. Repeat the step until you select all the alerts related to vSphere networking.

© 2016 VMware, Inc. All rights reserved.

Page 25 of 80
 VMware Validated Design Monitoring Guide

Alert Name

Network: Out of Memory

Network: ESX/ESXi uplink redundancy lost

Network: ESXi physical NIC down

5. Enable the alerts.

In the Alerts dialog box, click Enable.
a. In the Enable Alerts dialog box, configure the following alert settings and click Enable.

Setting Value

Email Email address to send alerts to

Send to vRealize Operations Manager Selected

Default Object SFO01-Mgmt01

Criticality critical

© 2016 VMware, Inc. All rights reserved.

Page 26 of 80
 VMware Validated Design Monitoring Guide

6. In the Alerts dialog box, set the Raise an alert option for each enabled alert.
a. Click the Edit button on the first enabled Network alert.

© 2016 VMware, Inc. All rights reserved.

Page 27 of 80
 VMware Validated Design Monitoring Guide

b. In the Edit Alert dialog box, under Raise an alert, select On any match, and click Save.

c. Repeat the step for the other enabled alerts.

d. Close the Alerts dialog box.

2.6 Enable the Alerts for NSX for vSphere

Use the inbuilt problem and alert signatures in vRealize Log Insight for NSX for vSphere.
For monitoring the NSX for vSphere configuration in the Software-Defined Data Center, you can use
the following alerts in vRealize Log Insight.

Alert Name Purpose Severity

VMW_NSX_Manager - Host This event will be generated when NSX Manager fails to Critical
Communication Errors receive heartbeat from UserWorld Agent on the host within
the threshold period. The output is grouped by host-id. The
host-id can be found from vCenter.

VMW_NSX_VXLAN This alert indicates VXLAN dataplane lost connection to Critical

dataplane lost connection to controller.
controller

© 2016 VMware, Inc. All rights reserved.

Page 28 of 80
 VMware Validated Design Monitoring Guide

Alert Name Purpose Severity

VMW_NSX_VXLAN This alert is generated when VXLAN configuration pushed Critical

configuration issue to host before host was prep'ed - host must be rebooted to
initialize configuration in correct order.

VMW_NSX_Firewall critical Firewall critical events: Critical

errors
 301501 - This is vsm side event if host failed to respond
with in time out window
 301503 - This is vsm side event if vsm failed while
provisioning firewall rule
 301506 - This is vsm side event if vsm failed to send
exclude list update
301031 - Failed to receive/parse/Update firewall
config. Key value will have context info like generation
number and also other debugging info

Procedure
Open the vRealize Log Insight user interface.
a. In a Web browser, go to the following URL.

Region vRealize Log Insight URL

Region A https://vrli-cluster-01.sfo01.rainpole.local

a. Log in using the following credentials.

Setting Value

User name admin

Password vrli_admin_password

In the vRealize Log Insight UI, click Interactive Analytics.

Click the icon and select Manage Alerts.

Select the alerts that are related to vSphere.
b. In the search box of the Alerts dialog box, enter nsx as a search phrase, and select the
alerts from the results.
c. Repeat the step until you select all the alerts related to NSX for vSphere.

Alert Name

VMW_NSX_Manager - Host Communication Errors

VMW_NSX_VXLAN dataplane lost connection to controller

© 2016 VMware, Inc. All rights reserved.

Page 29 of 80
 VMware Validated Design Monitoring Guide

Alert Name

VMW_NSX_VXLAN configuration issue

VMW_NSX_Firewall critical errors

Enable the alerts.

a. In the Alerts dialog box, click Enable.
b. In the Enable Alerts dialog box, configure the following alert settings and click Enable.

Setting Value

Email Email address to send alerts to

Send to vRealize Operations Manager Selected

Default Object Mgmt01nsxm01.sfo01

Criticality critical

© 2016 VMware, Inc. All rights reserved.

Page 30 of 80
 VMware Validated Design Monitoring Guide

In the Alerts dialog box, set the Raise an alert option for each enabled alert.
a. Click the Edit button on the first enabled vSphere alert.

© 2016 VMware, Inc. All rights reserved.

Page 31 of 80
 VMware Validated Design Monitoring Guide

b. In the Edit Alert dialog box, under Raise an alert, select On any match, and click Save.

© 2016 VMware, Inc. All rights reserved.

Page 32 of 80
 VMware Validated Design Monitoring Guide

c. Repeat the steps for the other enabled alerts.

d. Close the Alerts dialog box.

2.7 Enable Alerts for vRealize Operations Manager

Use the inbuilt problem and alert signatures in vRealize Log Insight for vRealize Operations Manager.
For monitoring the vRealize Operations Manager deployment in the Software-Defined Data Center,
you can use the following alerts in vRealize Log Insight.

© 2016 VMware, Inc. All rights reserved.

Page 33 of 80
 VMware Validated Design Monitoring Guide

Alert Name Purpose Severity

vRops: VC The purpose of this alert is to notify when vROPs instance is not able to Critical
stats query get the data back from vCenter instance
timed out within the 5 minute interval and the metrics back up and get dropped with
occurred the error: Communication Error:
com.integrien.adapter.vmware.VcCollector.collectMetrics - Vc stats query
timed out (ms): 300377. This is usually
due to intermittent connection issues with the vCenter and hosts or down
to the network not able to handle the request and timing out.

vRops: Out This alert gets generated when OutOfMemoryError: Java heap space Critical
of Memory occurs.
errors This could indicate memory issues and could lead to degradation in
occurred performance.

Procedure
Open the vRealize Log Insight user interface.
a. Open a Web browser and go to the following URL.

Region vRealize Log Insight URL

Region A https://vrli-cluster-01.sfo01.rainpole.local

a. Log in using the following credentials.

Setting Value

User name admin

Password vrli_admin_password

In the vRealize Log Insight UI, click Interactive Analytics.

Click the icon and select Manage Alerts.

Select the alerts that are related to vRealize Operations Manager.
a. In the search box of the Alerts dialog box, enter vrops as a search phrase, and select the
alerts from the results.
Repeat the step to select the other alert related to vRealize Operations Manager.

Alert Name

vRops: VC stats query timed out occurred

vRops: Out of Memory errors occurred

© 2016 VMware, Inc. All rights reserved.

Page 34 of 80
 VMware Validated Design Monitoring Guide

Enable the alerts.

a. In the Alerts dialog box, click Enable.
b. In the Enable Alerts dialog box, configure the following alert settings and click Enable.

Setting Value

Email Email address to send alerts to

Send to vRealize Operations Manager Selected

Default Object vrops-mstrn-01

Criticality critical

© 2016 VMware, Inc. All rights reserved.

Page 35 of 80
 VMware Validated Design Monitoring Guide

In the Alerts dialog box, set the Raise an alert option for each enabled alert.
a. Click the Edit button on the first enabled Network alert.

© 2016 VMware, Inc. All rights reserved.

Page 36 of 80
 VMware Validated Design Monitoring Guide

b. In the Edit Alert dialog box, under Raise an alert, select On any match, and click Save.

Repeat the steps for the other enabled alerts.

Close the Alerts dialog box.

2.8 Enable the Alerts for vRealize Automation

Use the inbuilt problem and alert signatures in vRealize Log Insight for vRealize Automation.
For monitoring the vRealize Automation deployment in the Software-Defined Data Center, you can
use the following alerts in vRealize Log Insight.

© 2016 VMware, Inc. All rights reserved.

Page 37 of 80
 VMware Validated Design Monitoring Guide

Alert Name Purpose Severity

*** CRITICAL *** vRA A vRA CAFE service has become unavailable. This may happen Critical
CAFE service because:
unavailable!
 A service has failed - if the service does not automatically
restart this may impact vRA's ability to function
 A service is blocked and cannot response at the moment -
this may indicate increased load within the environment
vRA is starting and certain dependencies of the component are
not available yet - this issue should clear automatically as all
services come online.

*** CRITICAL *** vRA A vRA service has become unavailable. This may happen Critical
IaaS Services Stopped because:
 A service has failed - if the service does not automatically
restart this may impact vRA's ability to function
 A service is blocked and cannot response at the moment -
this may indicate increased load within the environment
 vRA is starting and certain dependencies of the component
are not available yet - this issue should clear automatically
as all services come online.

*** CRITICAL *** vRA  A vRA IaaS service has become unavailable. This may Critical
disk is full happen because:
 A service has failed - if the service does not automatically
restart this may impact vRA's ability to function
 A service is blocked and cannot response at the moment -
this may indicate increased load within the environment
Management Agent- in an HA deployment, only ONE
Management Agent instance should be running.
If more than one is running, this will cause issues with normal
functioning of the system.

Procedure
1. Open the vRealize Log Insight user interface.
a. Open a Web browser and go to the following URL.

Region vRealize Log Insight URL

Region A https://vrli-cluster-01.sfo01.rainpole.local

b. Log in using the following credentials.

Setting Value

User name admin

© 2016 VMware, Inc. All rights reserved.

Page 38 of 80
 VMware Validated Design Monitoring Guide

Password vrli_admin_password

2. In the vRealize Log Insight UI, click Interactive Analytics.

3. Click the icon and select Manage Alerts.

4. Select the alerts that are related to vRealize Automation.
a. In the search box of the Alerts dialog box, enter vra as a search phrase, and select the
alerts from the results.
b. Repeat the step until you select all the alerts related to vRealize Automation.

Alert Name

*** CRITICAL *** vRA CAFE service unavailable!

*** CRITICAL *** vRA IaaS Services Stopped

*** CRITICAL *** vRA disk is full

© 2016 VMware, Inc. All rights reserved.

Page 39 of 80
 VMware Validated Design Monitoring Guide

5. Enable the alerts.

a. In the Alerts dialog box, click Enable.
b. In the Enable Alerts dialog box, configure the following alert settings and click Enable.

Setting Value

Email Email address to send alerts to

Send to vRealize Operations Manager Selected

Default Object vra01srv01a

Criticality critical

© 2016 VMware, Inc. All rights reserved.

Page 40 of 80
 VMware Validated Design Monitoring Guide

6. In the Alerts dialog box, set the Raise an alert option for each enabled alert.
a. Click the Edit button on the first enabled Network alert.

b. In the Edit Alert dialog box, under Raise an alert, select On any match, and click Save.

© 2016 VMware, Inc. All rights reserved.

Page 41 of 80
 VMware Validated Design Monitoring Guide

c. Repeat the steps for the other enabled alerts.

d. Close the Alerts dialog box.

2.9 Enable the Alerts for vRealize Orchestrator

Use the inbuilt problem and alert signatures in vRealize Log Insight for vRealize Orchestrator.
For monitoring the vRealize Orchestrator deployment in the Software-Defined Data Center, you can
use the following alerts in vRealize Log Insight

© 2016 VMware, Inc. All rights reserved.

Page 42 of 80
 VMware Validated Design Monitoring Guide

Alert Name Purpose Severity

vRO: Orchestrator The Orchestrator server state switched to STANDBY mode. In Critical
STANDBY Alert general there could be two reasons:
 There are enough RUNNING nodes in the cluster and
current node will stay on standby
playing a role of back-up node waiting to switch to
RUNNING state if needed.
 Problems with critical components as database or
authentication provider has been detected
which prevents the normal functioning of the server node.
The current node is considered unhealthy.
The server will monitor that critical components and try to
recover as soon as the problems are solved.
The current work won't accept any new requests and all its
workflows will be resumed on other healthy nodes.

vRO: Invalid Login Alert Failed login attempt has been detected. The reason could be Critical
wrong credentials
used to login to the server or there could be malicious attempt
to access the server.

vRO: Orchestrator Reboot Orchestrator server has been started or rebooted. The cause Critical
Alert could be planned reboot or result of unwanted action.

vRO: Workflow The content of some workflow has been modified. This could Critical
Modification Alert be a planned
workflow content update or result of unwanted malicious
actions.

vRO: Orchestrator Orchestrator worklow run failures have been detected. Critical
Workflow Failure Alert This could be due to infrastructure problems with external
systems.

vRO: Configuration Some modifications in the configuration elements of the Critical

Modification Alert Orchestrator have been detected.

Procedure
1. Open the vRealize Log Insight user interface.
a. Open a Web browser and go to the following URL.

Region vRealize Log Insight URL

Region A https://vrli-cluster-01.sfo01.rainpole.local

b. Log in using the following credentials.

Setting Value

© 2016 VMware, Inc. All rights reserved.

Page 43 of 80
 VMware Validated Design Monitoring Guide

User name admin

Password vrli_admin_password

2. In the vRealize Log Insight UI, click Interactive Analytics.

3. Click the icon and select Manage Alerts.

4. Select the alerts that are related to vRealize Orchestrator.

a. In the search box of the Alerts dialog box, enter vro as a search phrase, and select the
alerts from the results.
b. Repeat the step until you select all the alerts related to vRealize Orchestrator.

Alert Name

vRO: Orchestrator STANDBY Alert

vRO: Invalid Login Alert

vRO: Orchestrator Reboot Alert

vRO: Workflow Modification Alert

vRO: Orchestrator Workflow Failure Alert

vRO: Configuration Modification Alert

© 2016 VMware, Inc. All rights reserved.

Page 44 of 80
 VMware Validated Design Monitoring Guide

Enable the alerts.

a. In the Alerts dialog box, click Enable.
b. In the Enable Alerts dialog box, configure the following alert settings and click Enable.

Setting Value

Email Email address to send alerts to

Send to vRealize Operations Manager Selected

Default Object vra01vro01.rainpole.local

Criticality critical

© 2016 VMware, Inc. All rights reserved.

Page 45 of 80
 VMware Validated Design Monitoring Guide

In the Alerts dialog box, set the Raise an alert option for each enabled alert.

© 2016 VMware, Inc. All rights reserved.

Page 46 of 80
 VMware Validated Design Monitoring Guide

a. Click the Edit button on the first enabled Network alert.

© 2016 VMware, Inc. All rights reserved.

Page 47 of 80
 VMware Validated Design Monitoring Guide

In the Edit Alert dialog box, under Raise an alert, select On any match, and click Save.

a. Repeat the steps for the other enabled alerts.

b. Close the Alerts dialog box.

© 2016 VMware, Inc. All rights reserved.

Page 48 of 80
 VMware Validated Design Monitoring Guide

3. Creating Custom SDDC vRealize Operations

Dashboards
Monitoring the Software-Defined Data Center (SDDC) is critical to the health of the environment. You
create custom vRealize Operations Manager dashboards to provide centralized SDDC dashboards.
Using such dashboards simplifies monitoring the health of the SDDC as opposed to having to switch
between multiple product-specific dashboards.
To create custom dashboards, verify that you have deployed vRealize Operations Manager according
to implementation guides. You must have the following management packs installed and configured:
 vRealize Operations Manager Management Pack for VMware vSphere
 vRealize Operations Manager Management Pack for NSX-vSphere
 vRealize Operations Manager Management Pack for vRealize Automation
 vRealize Operations Manager Management Pack for Storage Devices
 vRealize Operations Manager Management Pack for Log Insight

3.1 Configure a Dashboard that Shows Capacity Trends

In the SDDC, monitor the trends in the CPU and memory consumption on the ESXi hosts and the
storage capacity of the datastores from a common dashboard.
Procedure
Log in to vRealize Operations Manager.
a. Open a Web browser and go to https://vrops-cluster-01.rainpole.local
b. Log in using the following credentials.

Setting Value

User name admin

Password vrops_admin_password

On the Home page, from the Actions menu select Create Dashboard.
In the Dashboard Configuration section of the New Dashboard dialog box, configure the
following settings.

Dashboard Option Value

Name SDDC Capacity Trends

Is default No

© 2016 VMware, Inc. All rights reserved.

Page 49 of 80
 VMware Validated Design Monitoring Guide

Add widgets to the dashboard.

a. Expand the Widget List section.
b. Drag 3 Scoreboard widgets to the layout pane on the right.
Configure the first Scoreboard widget to show metrics for CPU usage.
a. In the upper-right corner of the first Scoreboard widget, click the Edit icon.

b. Configure the following settings at the top of the Edit Scoreboard dialog box.

Scoreboard Widget Option Value

Title SDDC CPU Usage

© 2016 VMware, Inc. All rights reserved.

Page 50 of 80
 VMware Validated Design Monitoring Guide

Refresh Content On

Refresh Interval 300s

Self Provider On

Show Sparkline On

c. Click the Object Types tab, and add the metrics that this widget displays.

Object Type Attribute Value

Adapter Type vCenter Adapter

Object Type Host System

d. In the metrics pane on the right, enter Capacity Usage in the search box, and press Enter.
e. In the search result, expand the CPU metric category and double-click the Capacity Usage
(%) metric to add it to the list of metrics that the widget displays.
f. In the metrics list, double-click each of the following collection options, customize it and click
Update.

Collection Option Value

Color Method Custom

Yellow Bound 60

Orange Bound 75

Red Bound 90

g. In the Edit Scoreboard dialog box, click Save.

© 2016 VMware, Inc. All rights reserved.

Page 51 of 80
 VMware Validated Design Monitoring Guide

Configure the second Scoreboard widget to show metrics for memory usage.
a. In the upper-right corner of the second Scoreboard widget, click the Edit icon.
b. Configure the following settings at the top of the Edit Scoreboard dialog box.

Scoreboard Widget Option Value

Title SDDC Memory Overview

Refresh Content On

Refresh Interval 300s

Self Provider On

Show Sparkline On

c. Click the Object Types tab, and add the metrics that this widget displays.

Object Type Attribute Value

Adapter Type vCenter Adapter

Object Type Host System

d. In the metrics pane on the right, enter Us ag e / Us able in the search box, and press
Enter.

© 2016 VMware, Inc. All rights reserved.

Page 52 of 80
 VMware Validated Design Monitoring Guide

e. In the search result, expand the Memory metric category and double-click the Usage /
Usable (%) metric to add it to the list of metrics that the widget displays.
f. In the metrics list, double-click each of the following collection options, customize it and click
Update.

Collection Option Value

Color Method Custom

Yellow Bound 70

Orange Bound 80

Red Bound 90

© 2016 VMware, Inc. All rights reserved.

Page 53 of 80
 VMware Validated Design Monitoring Guide

g. In the Edit Scoreboard dialog box, click Save.

Configure the third Scoreboard widget to show metrics for storage usage.
a. In the upper-right corner of the third Scoreboard widget, click the Edit icon.
b. Configure the following settings at the top of the Edit Scoreboard dialog box.

Scoreboard Widget Option Value

Title SDDC Storage Consumption

Refresh Content On

Refresh Interval 300s

Self Provider On

Show Sparkline On

c. Click the Object Types tab, and add the metrics that this widget displays.

Object Type Attribute Value

Adapter Type vCenter Adapter

Object Type Datastore

d. In the metrics pane on the right, enter Used Space in the search box, and press Enter.

© 2016 VMware, Inc. All rights reserved.

Page 54 of 80
 VMware Validated Design Monitoring Guide

e. In the search result, expand the Capacity metric category and double-click the Used Space
(%) metric to add it to the list of metrics that the widget displays.
f. Enter Number of VMs in the search box and press Enter.
g. In the search result, expand the Summary metric category and double-click the Total
Number of VMs metric to add the number of VMs in the datastore to the list of metrics in the
widget.
h. In the metrics list at the bottom, double-click each metric row, customize the following
collection attribute for the metric and click Update.

Collection Option Value

Color Method Custom

Yellow Bound 60

Orange Bound 70

Red Bound 80

i. In the Edit Scoreboard dialog box, click Save.

In the New Dashboard dialog box, click Save.

The SDDC Capacity Trends dashboard becomes available on the Home page of the vRealize
Operations Manager user interface.

© 2016 VMware, Inc. All rights reserved.

Page 55 of 80
 VMware Validated Design Monitoring Guide

3.2 Configure a Dashboard that the Provides an Overview of

SDDC Operation
Create a dashboard in vRealize Operations Manager where you can monitor the objects of the SDDC
management stack.
To model the SDDC overview dashboard, perform the following tasks:
 Define applications for the virtual machines of the SDDC management applications, such
vRealize Log Insight and vRealize Orchestrator
 Create a custom group to track the state of the management applications that are connected to
vRealize Operations Manager.
 Create and configure a dashboard that shows the state of the hosts and virtual machines in the
management cluster.
 Create an Application for vRealize Log Insight
 Create an Application for vRealize Orchestrator
 Collect the SDDC Objects in a Group
 Configure a Dashboard that Provides an Overview of the SDDC State

3.2.1 Create an Application for vRealize Log Insight

Create an application in vRealize Operations Manager to group the monitoring data about the virtual
machines of vRealize Log Insight.

vRealize Operations Manager builds an application to determine how your environment is affected
when one or more components in an application experiences problems. You can also monitor the
overall health and performance of the application.

vRealize Operations Manager collects data from the components in the application and displays the
results in a summary dashboard for each application with a real-time analysis for any or all of the
components.

© 2016 VMware, Inc. All rights reserved.

Page 56 of 80
 VMware Validated Design Monitoring Guide

Because the Management Pack for vRealize Log Insight does not collect monitoring data about the
virtual machines of the vRealize Log Insight deployment, you create an application to watch their
state.
Procedure
In a Web browser, open the main page of vRealize Operations Manager.
 If you use the public interface to the SDDC, go to https://vrops-cluster-01.rainpole.local
Use the adm in user name and the vro ps_ adm in _pas swo rd password to log in.
In the left pane of vRealize Operations Manager, click Environment menu and click
Applications.
On the Applications tab page, click Add icon to add an application.
In the Add Application dialog box, select Custom and click OK.

The Application Management dialog box appears where you select the objects for the
application.
In the Application Management dialog box, in the Name text box enter v R ea liz e L og
Ins ight .
In the Tiers pane, click Add Tier, enter Log In si gh t VM s as Tier Name and click Update.
In the objects list underneath, enter vrli in the search box, and press Enter.
Select the virtual machine objects of vRealize Log Insight and drag them to the Tier Objects
pane.

VMs

vrli-mstr-01.sfo01

vrli-wrkr-01.sfo01

vrli-wrkr-02.sfo01

© 2016 VMware, Inc. All rights reserved.

Page 57 of 80
 VMware Validated Design Monitoring Guide

Click Save.

3.2.2 Create an Application for vRealize Orchestrator

Create an application in vRealize Operations Manager to group the monitoring data for the virtual
machines of vRealize Orchestrator.
vRealize Operations Manager builds an application to determine how your environment is affected
when one or more components in an application experiences problems. You can also monitor the
overall health and performance of the application.
vRealize Operations Manager collects data from the components in the application and displays the
results in a summary dashboard for each application with a real-time analysis for any or all of the
components.
Because the Management Pack for vRealize Automation does not collect monitoring data about the
virtual machines of the vRealize Orchestrator cluster, you create an application to watch their state.
Procedure
1. In a Web browser, open the main page of vRealize Operations Manager.
a. If you use the public interface to the SDDC, go to https://vrops-cluster-01.rainpole.local
b. Use the adm in user name and the vro ps_ adm in _pas swo rd password to log in.
2. In the left pane of vRealize Operations Manager, click Environment menu and click
Applications tab.
3. On the Applications tab, click the Add icon to add an application.
4. In the Add Application dialog box, select Custom and click OK.

© 2016 VMware, Inc. All rights reserved.

Page 58 of 80
 VMware Validated Design Monitoring Guide

5. In the Application Management dialog box, in the Application text box enter v R e al iz e
O r ch est r at o r .
6. In the Tiers pane, click Add Tier, enter O r ch e str at o r VM s as Tier Name and click Update.
7. In the objects list underneath, enter vravro in the search box and press Enter.
8. Select the virtual machine objects of vRealize Orchestrator and drag them to the Tier
Objects pane

VMs

vra01vro01a

vra01vro01b

9. Click Save.

3.2.3 Collect the SDDC Objects in a Group

Create a custom group for each management application to monitor the health of the entire
application stack as opposed to individual virtual machine health.
Procedure
In a Web browser, open the main page of vRealize Operations Manager.
a. If you use the public interface to the SDDC, go to https://vrops-cluster-01.rainpole.local
b. Use the adm in user name and the v rop s _a dm in_ pa ssw or d password to log in.
In the left pane of vRealize Operations Manager, click the Environment menu.
Click the Groups tab, and click Add icon to add a custom group.
In the New group dialog box, enter SDDC Management in the Name text box.
From the Group Type drop-down menu, select Function.
Expand the Define membership criteria section.
To add the vRealize Automation objects, from the Select the Object Type that matches all of
the following criteria drop-down menu, select vRealize Automation MP > vRealize
Automation MP Instance.
Click Add another criteria set and repeat Step 7 to add each of the following object types
representing the other management applications in the SDDC.
a. vCenter Adapter > vSphere World
b. NSX-vSphere Adapter > NSX-vSphere Environment
c. vRealize Operations Adapter > vRealize Operations Cluster

© 2016 VMware, Inc. All rights reserved.

Page 59 of 80
 VMware Validated Design Monitoring Guide

Expand the Objects to always include section.

Add the following vRealize Log Insight and vRealize Orchestrator application objects.
a. Under Filtered objects, expand Custom Groups and select Applications.
b. Select vRealize Log Insight and vRealize Orchestrator, and click the Add button to add the
application objects to Objects to always include list on the right.
Click OK.

3.2.4 Configure a Dashboard that Provides an Overview of the SDDC State

Create a central dashboard that you can use to track the overall state of the SDDC.

The SDDC overview dashboard has the following two aspects:

 Display the main indicators for the state of CPU, memory, connectivity and storage in the
management cluster that hosts the management applications.
 Show the overall state of the management applications in the SDDC.
You use the SDDC Management custom group that represents a common object for all
management applications.
Procedure
In a Web browser, open the main page of vRealize Operations Manager.
a. If you use the public interface to the SDDC, go to https://vrops-cluster-01.rainpole.local
b. Use the admin user name and the vrops_admin_password password to log in.
On the Home page, from the Actions menu select Create Dashboard.
In the Dashboard Configuration section of the New Dashboard dialog box, configure the
following settings.

© 2016 VMware, Inc. All rights reserved.

Page 60 of 80
 VMware Validated Design Monitoring Guide

Dashboard Option Value

Name SDDC Overview

Is default Yes

Add widgets to the dashboard.

a. Expand the Widget List section.
b. Drag 12 Heatmap widgets to the layout pane on the right and align them so that they are all
approximately equal in size.
c. Drag a Health widget to the layout pane on the right.
Configure the heatmap widgets.
a. In the upper-right corner of each widget, click the Edit icon and configure the widget.
b. In the Edit Heatmap dialog box, configure the following settings of the heatmap widget and
click Save.
Widgets are labeled Heatmap x.y where x is the column number and y is the row number in
the dashboard.

Widget Heatmap 1.1 Heatmap 1.2 Heatmap 1.3

Option

Title Physical CPU Physical Network I/O Storage Read Latency

Remaining

Refresh On On On
Content

Refresh 300s 300s 300s

Interval

Description Management Hosts Management Hosts Datastores

Group by vCenter Adapter > vCenter Adapter > Datacenter vCenter Adapter >
Datacenter Datacenter

Then by - - -

Mode Instance Instance Instance

Object type vCenter Adapter > Host vCenter Adapter > Host System vCenter Adapter >
System Datastore

Attribute type CPU > Capacity Network I/O > Usage Rate > Datastore I/O -> Read
Remaining (%) Capacity Remaining (%) Latency (ms)

Min Value 0 0 0

Max Value 25 20 30

© 2016 VMware, Inc. All rights reserved.

Page 61 of 80
 VMware Validated Design Monitoring Guide

Widget Heatmap 1.1 Heatmap 1.2 Heatmap 1.3

Option

Widget Heatmap 2.1 Heatmap 2.2 Heatmap 2.3

Option

Title Physical Memory Physical Dropped Packets Storage Write Latency

Remaining

Refresh On On On
Content

Refresh 300s 300s 300s

Interval

Description Management Hosts Management Hosts Datastores

Group by vCenter Adapter > vCenter Adapter > Datacenter vCenter Adapter >
Datacenter Datacenter

Then by - - -

Mode Instance Instance Instance

Object type vCenter Adapter > Host vCenter Adapter > Host System vCenter Adapter >
System Datastore

Attribute type Memory > Capacity Network I/O > Packets Dropped Datastore I/O > Write
Remaining (%) Latency (ms)

Min Value 0 0 0

Max Value 25 1 30

Widget Option Heatmap 3.1 Heatmap 3.2 Heatmap 3.3

Title Management VM CPU Management VM Memory Management VM Storage

Used Used Latency

Refresh On On On
Content

Refresh 300s 300s 300s

Interval

Description Management VMs Management VMs Management VMs

Group by vCenter Adapter > vCenter Adapter > vCenter Adapter >
Datacenter Datacenter Datacenter

Then by - - -

Mode Instance Instance

© 2016 VMware, Inc. All rights reserved.

Page 62 of 80
 VMware Validated Design Monitoring Guide

Widget Option Heatmap 3.1 Heatmap 3.2 Heatmap 3.3

Object type vCenter Adapter > Virtual vCenter Adapter > Virtual vCenter Adapter > Virtual
Machine Machine Machine

Attribute type CPU > Usage (%) Memory > Usage (%) Virtual Disk > Total
Latency

Min Value 80 50 0

Max Value 100 90 30

© 2016 VMware, Inc. All rights reserved.

Page 63 of 80
 VMware Validated Design Monitoring Guide

Widget Option Heatmap 4.1 Heatmap 4.2 Heatmap 4.3

Title Management VM CPU Management VM Swap Management VM Disk Free

Contention Rate

Refresh On On On
Content

Refresh 300s 300s 300s

Interval

Description Management VMs Management VMs Management VMs

Group by vCenter Adapter > vCenter Adapter > vCenter Adapter >
Datacenter Datacenter Datacenter

Then by - - -

Mode Instance Instance Instance

Object type vCenter Adapter > Virtual vCenter Adapter > Virtual vCenter Adapter > Virtual
Machine Machine Machine

Attribute type CPU > CPU Contention Memory > Swapped (KB) Disk Space > Capacity
(%) Remaining (%)

Min Value 0 0 5

Max Value 2 1 20

Configure the Health Chart widget.

a. In the upper-right corner of the widget, click the Edit icon and configure the widget.
b. In the Edit Heatmap dialog box, configure the following settings of the Health Chart widget.

Health Widget Option Value

Title Management Applications

Refresh Content On

Refresh Interval 300s

Self Provider On

Mode Children

Order By Value Asc

Pagination number 15

Period Length Last 6 hours

© 2016 VMware, Inc. All rights reserved.

Page 64 of 80
 VMware Validated Design Monitoring Guide

Health Widget Option Value

Metric Health

c. From the objects list at the bottom, expand Function and select the SDDC Management
custom group.
In the New Dashboard dialog box, click Save.
The SDDC Overview dashboard becomes available on the Home page of the vRealize Operations
Manager user interface.

© 2016 VMware, Inc. All rights reserved.

Page 65 of 80
 VMware Validated Design Monitoring Guide

4. Configure vRealize Operations Manager to Notify of

SDDC Issues
Create a set of notifications in vRealize Operations Manager so that data center operators receive
alerts about issues in the SDDC main functions.
 Create Notifications in vRealize Operations Manager
 List of Notifications for vRealize Operations Manager

4.1 Create Notifications in vRealize Operations Manager

Create email notifications in vRealize Operations Manager so that the SDDC operators know of
issues in the main monitoring parameters of the environment.
Procedure
In a Web browser, open the main page of vRealize Operations Manager.
a. If you use the public interface to the SDDC, go to https://vrops-cluster-01.rainpole.local
b. Use the admin user name and the vrops_admin_password password to log in.
In the left pane of vRealize Operations Manager, click Content and click the Notifications tab.
On Notifications tab, click the Add icon and configure the following notification settings in the
Add Rule dialog box.

Notification Option Value

Name Virtual machine is projected to run out of disk space

Method Standard Email Plugin

Instance Rainpole Alert Mail Relay

Recipients [email protected]

Filtering Criteria

Scope Object Type

vCenter Adapter > Virtual Machine

© 2016 VMware, Inc. All rights reserved.

Page 66 of 80
 VMware Validated Design Monitoring Guide

Configure the trigger for the notification.

a. In the Filtering Criteria section of Add Rule dialog box, select Alert Definition from the
Notification Trigger drop-down menu.
b. Click the Select an Alert Definition button.
c. In the search box of the Alert Definitions dialog box, enter d i sk s pa c e and press Enter.
d. Select the Virtual machine is projected to run out of disk space alert definition and click
Select.

© 2016 VMware, Inc. All rights reserved.

Page 67 of 80
 VMware Validated Design Monitoring Guide

In the Add Rule dialog box, click Save.

Repeat the steps to create the notifications that are defined in List of Notifications for vRealize
Operations Manager.

4.2 List of Notifications for vRealize Operations Manager

Configure vRealize Operations Manager to send email notifications about important alerts in the
SDDC.

© 2016 VMware, Inc. All rights reserved.

Page 68 of 80
 VMware Validated Design Monitoring Guide

You define notifications from the Content > Notifications page in vRealize Operations user interface.
See Create Notifications in vRealize Operations Manager.

4.2.1 Notification Delivery Properties

When you define notifications from vRealize Operations Manager, use the following properties to
direct them by email to the operations team in your organization.
Table 1. Delivery Properties of vRealize Operations Manager Notifications

Notification Delivery Property Value

Method Standard Email Plugin

Instance Rainpole Alert Mail Relay

Recipients [email protected]

4.2.2 Virtual Machine and Host Notifications

Create notifications for most important virtual machines and ESXi host issues.
Table 2. VM and Host Notifications in SDDC

Name Scope Notification Alert Definition

Trigger

Virtual machine is projected Object Type Alert Virtual machine is projected to

to run out of disk space* Definition run out of disk space
vCenter
* Step-by-step instructions
Adapter >
are provided in Create
Virtual Machine
Notifications in vRealize
Operations Manager.
Virtual machine has CPU
Object Type Alert Virtual machine has CPU
contention caused by co-stop
Definition contention due to multi-vCPU
vCenter
scheduling issues (co-stop)
Adapter >
caused by too many vCPUs
Virtual Machine

Virtual machine has Object Type Alert Virtual machine has

unexpected high CPU Definition unexpected high CPU workload
vCenter
workload
Adapter >
Virtual Machine

Virtual machine has Object Type Alert Virtual machine has

unexpected high memory Definition unexpected high memory
vCenter
workload workload
Adapter >
Virtual Machine

Virtual machine has disk I/O Object Type Alert Virtual machine has disk I/O
latency problem caused by Definition latency problem caused by
vCenter
snapshots snapshots
Adapter >
Virtual Machine

© 2016 VMware, Inc. All rights reserved.

Page 69 of 80
 VMware Validated Design Monitoring Guide

Name Scope Notification Alert Definition

Trigger

Virtual Machine is running out Object Type Alert Virtual Machine is running out
of disk space Definition of disk space
vCenter
Adapter >
Virtual Machine

Virtual machine has large Object Type Alert Virtual machine has large disk
disk snapshots Definition snapshots
vCenter
Adapter >
Virtual Machine

Not enough resources for Object Type Alert Not enough resources for
vSphere HA to start the Definition vSphere HA to start the virtual
vCenter
virtual machine machine
Adapter >
Virtual Machine

vSphere HA cannot perform Object Type Alert vSphere HA cannot perform a

a failover operation for the Definition failover operation for the virtual
vCenter
virtual machine machine
Adapter >
Virtual Machine

Host has CPU contention Object Type Alert Host has CPU contention
caused by overpopulation of Definition caused by overpopulation of
vCenter
virtual machines virtual machines
Adapter > Host
System

Host has memory contention Object Type Alert Host has memory contention
caused by overpopulation of Definition caused by overpopulation of
vCenter
virtual machines virtual machines
Adapter > Host
System

vSphere DRS enabled cluster Object Type Alert DRS-enabled cluster has CPU
has CPU contention caused Definition contention caused by
vCenter
by overpopulation of virtual overpopulation of virtual
Adapter >
machines machines
Cluster
Compute
Resource

vSphere DRS enabled cluster Object Type Alert DRS-enabled cluster has
has unexpected high CPU Definition unexpected high CPU workload
vCenter
workload
Adapter >
Cluster
Compute
Resource

© 2016 VMware, Inc. All rights reserved.

Page 70 of 80
 VMware Validated Design Monitoring Guide

Name Scope Notification Alert Definition

Trigger

vSphere DRS enabled cluster Object Type Alert DRS-enabled cluster has
has memory contention Definition memory contention caused by
vCenter
caused by overpopulation of overpopulation of virtual
Adapter >
virtual machines machines
Cluster
Compute
Resource

vSphere DRS enabled cluster Object Type Alert DRS-enabled cluster has
has unexpected high memory Definition unexpected high memory
vCenter
workload and contention workload and contention
Adapter >
Cluster
Compute
Resource

vSphere HA failover Object Type Alert vSphere High Availability (HA)

resources are insufficient Definition failover resources are
vCenter
insufficient
Adapter >
Cluster
Compute
Resource

4.2.3 Networking Notifications

Create notifications for most important networking issues in distributed switches and NSX
components.
Table 3. Networking Notifications in SDDC

Name Scope Notification Alert Definition

Trigger

Object Type
Distributed switch Distributed Switch
configuration is out of vCenter Adapter > Alert Definition configuration is out of
sync vSphere Distributed sync
Switch

Host NSX messaging Object Type Host's NSX messaging

infrastructure is reporting vCenter Adapter > Host Alert Definition infrastructure is reporting
an issue System an issue

Object Type
NSX Manager resource Manager resource usage
NSX-vSphere Adapter > Alert Definition
usage is high is high
NSX-vSphere Manager

Object Type
NSX Manager API calls Manager API calls are
NSX-vSphere Adapter > Alert Definition
are failing failing
NSX-vSphere Manager

© 2016 VMware, Inc. All rights reserved.

Page 71 of 80
 VMware Validated Design Monitoring Guide

Name Scope Notification Alert Definition

Trigger

Object Type
VXLAN segment range VXLAN segment range
NSX-vSphere Adapter > Alert Definition
has been exhausted has been exhausted
NSX-vSphere Manager

Object Type
Less than three NSX NSX-vSphere Adapter - Less than three
Alert Definition
Controllers are active > NSX-vSphere controllers are active
Controller Cluster

Object Type
Edge resource usage is Edge resource usage is
NSX-vSphere Adapter > Alert Definition
high high
NSX-vSphere Edge

Object Type
The Edge is not highly The Edge is not highly
NSX-vSphere Adapter > Alert Definition
available available
NSX-vSphere Edge

Edge VM is not Object Type Edge VM is not

responding to health NSX-vSphere Adapter > Alert Definition responding to health
check NSX-vSphere Edge check

One or more Load Object Type One or more Load

Balancer pool members NSX-vSphere Adapter > Alert Definition Balancer pool members
are down NSX-vSphere Edge are down

4.2.4 Storage Notifications

Create notifications for most important storage issues.
Table 4. Storage Notifications in SDDC

Name Scope Notification Alert Definition

Trigger

Datastore is running out Object Type Alert Definition Datastore is running out
of disk space of disk space
vCenter Adapter >
Datastore

Datastore is projected to Object Type Alert Definition Datastore is projected to

run out of disk space run out of disk space
vCenter Adapter >
Datastore

Virtual SAN cluster Object Type Alert Definition VirtualSAN cluster

partitioned partitioned
Storage Devices ->
VirtualSAN Cluster

© 2016 VMware, Inc. All rights reserved.

Page 72 of 80
 VMware Validated Design Monitoring Guide

4.2.5 Notifications about vRealize Operations Manager

Create notifications for most important issues in the operation of vRealize Operations Manager.
Table 5. Notifications of vRealize Operations Manager Issues

Name Scope Notification Alert Definition

Trigger

Object Type
One or more vRealize One or more vRealize
vRealize Operations Alert
Operations services are Operations services are
Adapter > vRealize Definition
down down
Operations Node

Object Type
Disk space on a vRealize
vRealize Operations Alert
Operations Manager node is Disk space on node is low
Adapter > vRealize Definition
low
Operations Node

Object Type
Node processing queue is vRealize Operations Alert Node processing queue is
backing up Adapter -> vRealize Definition backing up
Operations Node

Object Type
vRealize Operations Alert
FSDB corrupted files Fsdb corrupted files
Adapter -> vRealize Definition
Operations Fsdb

Object Type
FSDB failed to repair vRealize Operations Alert Fsdb failed to repair
corrupted files Adapter -> vRealize Definition corrupted files
Operations Fsdb

Object Type
vRealize Operations Alert
FSDB overload Fsdb high load
Adapter -> vRealize Definition
Operations Fsdb

Number of objects Object Type Number of Objects being

monitored by this vRealize vRealize Operations monitored by this vRealize
Alert
Operations Manager node Adapter > vRealize Operations Node exceeds
Definition
exceeds the configured limit. Operations the configured limit.
Possible loss of data Analytics Possible loss of data

Object Type
Remote Collector one or vRealize Operations One or more vRealize
Alert
more vRealize Operations Adapter > vRealize Operations services are
Definition
services are down Operations Remote down
Collector

© 2016 VMware, Inc. All rights reserved.

Page 73 of 80
 VMware Validated Design Monitoring Guide

Name Scope Notification Alert Definition

Trigger

Object Type
Remote Collector not vRealize Operations Remote Collector not
Alert
reporting correct number of Adapter -> vRealize reporting correct number of
Definition
services Operations Remote services
Collector

Object Type
vRealize Operations Cluster vRealize Operations Cluster
vRealize Operations Alert
processes might be out of processes may not have
Adapter -> vRealize Definition
memory enough memory
Operations Cluster

© 2016 VMware, Inc. All rights reserved.

Page 74 of 80
 VMware Validated Design Monitoring Guide

5. Monitor vSphere Data Protection Jobs by Email

vSphere Data Protection supports email notifications about the status of backup jobs. Configure
vSphere Data Protection to send daily emails. You can regularly check them as a part of your daily
monitoring activities.
Open the vSphere Web Client
a. In a Web browser, go to the following address.

Region vCenter Server URL

Region A https://mgmt01vc01.sfo01.rainpole.local/vsphere-client

b. Use the [email protected] user name and the

vsphere_admin_password password to log in.
On the vSphere Web Client Home page, click VDP.
On the Welcome to vSphere Data Protection page, select the region-specific vSphere Data
Protection instance from the VDP Appliance drop-down menu and click Connect.

Region vSphere Data Protection Instance

Region A mgmt01vdp01

On the Configuration tab, click the Email button and click Edit.

Configure the following settings for email notification and click Save.

Email Notification Settings Value

Enable vSphere Data Protection Selected

alarm email notification

Enable email reports Selected

© 2016 VMware, Inc. All rights reserved.

Page 75 of 80
 VMware Validated Design Monitoring Guide

Email Notification Settings Value

Send time 6:00 AM

Send day(s) All days

Enable the CSV attachment in Selected

email report

Outgoing mail server smtp.rainpole.local

From address [email protected]

To address(es) [email protected]

Click the Send test email hyperlink and verify that you receive the test email.

© 2016 VMware, Inc. All rights reserved.

Page 76 of 80
 VMware Validated Design Monitoring Guide

© 2016 VMware, Inc. All rights reserved.

Page 77 of 80
 VMware Validated Design Monitoring Guide

6. Configure vRealize Automation System Notification

Events
You can receive automatic notifications for several types of events, such as the successful completion
of a catalog request or a required approval.
System administrators can configure global email servers that handle email notifications.
Tenant administrators can override the system default servers, or add their own servers if no global
servers are specified. Tenant administrators select which events, also known as scenarios,
trigger notifications. Each component, such as the service catalog or IaaS, can define events that can
trigger notifications.
Each user can choose whether to receive notifications. Users either receive all notifications
configured by the tenant administrator or no notifications, they do not have the fine-grained control
over which notifications to receive.
Prerequisites
 Verify that vRealize Automation has the inbound and outbound email servers configured. See
Configure the Default Email Servers (Region A).
Procedure
Open the vRealize Automation management console for the Rainpole tenant.
a. In a Web browser, go to https://vra01svr01.rainpole.local/vcac/org/rainpole.
b. Use the ITAC-TenantAdmin user name, the rainpole_tenant_admin_password
password and ra inp ol e. l oc al domain to log in.

On the Home page of the vRealize Automation management console, click the Administration
tab and click Notifications.
Configure the scenarios to receive notifications about. By default all scenarios are active.
a. On the Notifications page, select Scenarios in the navigator.

© 2016 VMware, Inc. All rights reserved.

Page 78 of 80
 VMware Validated Design Monitoring Guide

b. If you do not want to be alerted on a scenario, select it and click the Suspend button.
c. Verify that each of the scenarios you want receive notifications about is Active.

Subscribe to notifications from vRealize Automation.

a. Click Preferences next to the ITAC-TenantAdmin username.

b. Under Notifications, select English (United States) from the Language drop-down menu.
c. Select Enabled next to the Email protocol, click Apply and click Close.

© 2016 VMware, Inc. All rights reserved.

Page 79 of 80
 VMware Validated Design Monitoring Guide

Notifications are now enabled for the ITAC-TenantAdmin account.

© 2016 VMware, Inc. All rights reserved.

Page 80 of 80