Cisco Expressway Cluster Creation and Maintenance Deployment

Guide (X14.3)
First Published: 2023-05-17
Last Modified: 2023-06-27

Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of
the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional
and coincidental.

All printed copies and duplicate soft copies of this document are considered uncontrolled. See the current online version for the latest version.

Cisco has more than 200 offices worldwide. Addresses and phone numbers are listed on the Cisco website at www.cisco.com/go/offices.

The documentation set for this product strives to use bias-free language. For purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on
age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that
is hardcoded in the user interfaces of the product software, language used based on standards documentation, or language that is used by a referenced third-party product.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL:
https://www.cisco.com/c/en/us/about/legal/trademarks.html. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a
partnership relationship between Cisco and any other company. (1721R)
© 2023 Cisco Systems, Inc. All rights reserved.
 CONTENTS

CHAPTER 1 About This Guide 1

Information Covered 1
Change History 2

CHAPTER 2 Clustering Basics 5

Overview 5
Benefits of Clustering 5
About Capacity Gain 5
About Licensing 6

CHAPTER 3 Clustering Requirements 7

Do Not Mix Expressway-C and Expressway-E 7
Platform and Software Versions Match 7
Network Conditions Are Met 8
Basic Configuration Is Done 8
DNS Configuration Is Done 9
TMS Is Configured (When Essential) 9
Clusters With Mixed CE1200 and CE1100 Physical Appliances 9
Mix Cluster Deployment With Expressway and Expressway Select 10

CHAPTER 4 How to Form a Cluster 11

Overview 11
Preparing Expressway to Join a Cluster 12
Create a New Cluster of Expressway Peers 13
Next Steps 16
Add a Peer to a Cluster 17

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

iii
 Contents

Checks 18
Next Steps 18

CHAPTER 5 (Optional) Use Fully Qualified Domain Names to Form a Cluster 19

Cluster Address Mapping for Expressway-E Clusters 19

Do You Need Cluster Address Mapping? 19
How Cluster Address Mapping Works 20
Where Does the Suggested Mapping Come From? 20
Configure Cluster Address Mapping (Expressway-E Clusters) 20
Change Cluster to Use FQDNs 21
Enforce TLS Verification 24
Before You Begin 24
Process to Enforce TLS Verification 24
Usage Note for Expressway-E Traversal Zones 24

CHAPTER 6 How to Change a Cluster 25

Before You Change a Cluster 26

Remove a Live Peer From a Cluster (Permanently) 26
On the Expressway That You Are Removing From the Cluster 27
On the Primary Expressway 28
On All the Remaining Subordinate Expressway Peers 28
Remove a Dead Peer From a Cluster (Permanently) 28
Clear Configuration From This Peer 29
Recovery Of an Expressway Cluster Peer 30
Disband a Cluster 30
Change the Primary Peer 31
Change the Peer Identity 32
Replace a Peer 32
Replace a Peer and Migrate Its Configuration 33

CHAPTER 7 How to Connect the Expressway Cluster to Other Systems 35

Neighboring Between Expressway Clusters 35

Configure Endpoints to Work With a Cluster 35
SIP Endpoints 36

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

iv
 Contents

Option 1 – SIP Outbound (Preferred) 36

Option 2 – DNS SRV (2nd Choice) 36
Option 3 – DNS Round-Robin (3rd Choice) 37
Option 4 – Static IP (Least Preferred) 37
H.323 Endpoints 38
Option 1 – DNS SRV (Preferred) 38
Option 2 – DNS Round-Robin (2nd Choice) 38
Option 3 – Static IP (Least Preferred) 39
Add the Expressway to Cisco TMS 39
On the Expressway 39
On the Cisco TMS 40

CHAPTER 8 Troubleshooting 41

How to Rebuild a Cluster from Backup 41

Restarting Sequence 41
Check Replication Status 42
Force Refresh in Cisco TMS 42
Expressway Alarms and Warnings 42
Cluster name not configured: if FindMe or clustering are in use a cluster name must be defined 42
Cluster replication error: (details) manual synchronization of configuration is required 42
Cluster replication error: (details) restart node 43
Cluster replication error persists after ForceConfigUpdate 43
Cluster replication error: the NTP server is unreachable 43
Cluster replication error: the local Expressway does not appear in the list of peers 44
Cluster replication error: automatic replication of configuration has been temporarily disabled because
an upgrade is in progress 44
Invalid clustering configuration: H.323 mode must be turned On - clustering uses H.323
communications between peers 44
Expressway database failure: Please contact your Cisco support representative 44
Cisco TMS Warnings 44
Cisco TMS Cluster Diagnostics 44
Conference Factory Template Does Not Replicate 44
Expressway’s External Manager Protocol Keeps Getting Set to HTTPS 45

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

v
 Contents

CHAPTER 9 Reference 47

Peer-specific Items 47
Sample Firewall Rules for Protecting Intracluster TLS Port 49
Cluster Name and DNS SRV Records 50
DNS SRV Configuration for Mobile and Remote Access 51
DNS SRV Configuration for Video Conferencing 52
Checking DNS SRV Settings 53
Clusters in Isolated Networks 54
NAPTR Records 55
NAPTR Record Format 56
Impact of Clustering in Other Expressway Applications 57
Conference Factory (Multiway™) 57
Microsoft Interoperability 58

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

vi
 CHAPTER 1
About This Guide
This chapter explains the following:
• Information Covered, on page 1
• Change History, on page 2

Information Covered
From version X12.5 onwards, this guide applies only to the Cisco Expressway Series product (Expressway)
and no longer applies to the Cisco VCS product (VCS). Older VCS guides on Cisco.com are still valid for
the VCS versions they apply to—as specified on the title page of each guide.
The guide covers the following topics:
• Clustering Requirements
Describes the required network environment and minimum configuration of the peer Expressways before
you can cluster them.
• How to Form a Cluster
How to form a cluster of one, add peers to a cluster, and configure cluster address mapping if necessary.
• How to Change a Cluster
Processes like upgrading, taking peers offline, changing the primary peer, and disbanding the cluster.
• How to Connect the Expressway Cluster to Other Systems
How to connect the cluster with external systems like Cisco TMS, other Expressways, and endpoints.
• Troubleshooting
Guidance that may assist if the cluster is not working as expected.
• Reference
Additional material that may be relevant to your environment.

For information about license usage and capacity with clustered systems, refer to the Expressway Administrator
Guide on the Cisco Expressway Series Maintain and Operate Guides page.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

1
 About This Guide
Change History

Change History
Table 1: Change History

Date Change Reason

June 2023 First published for X14.3 release X14.3 release

Included a section "Usage Note for
Expressway-E Traversal Zones" in
chapter "(Optional) Use Fully
Qualified Domain Names to Form
a Cluster"
Included a Note to the section "Mix
Cluster Deployment With
Expressway and Expressway
Select" in chapter "Clustering
Requirements"

July 2021 Updates for X14.0.2 release X14.0.2 release

Addressed a few CDETs

April 2021 First published for X14.0 release X14.0 release

Included a few "Expressway
Alarms and Warning" in the
chapter "Troubleshooting"

June 2020 • Updated for X12.6. Also X12.6 release and document
remove cluster license usage correction
and capacity guidelines, which
are now in the Expressway
Administrator Guide.
• Update Clustering
Requirements to clarify DNS
SRV record with A or AAAA
records per peer for B2B
deployments is recommended
but not required.

March 2019 Clarify that removal of a cluster Clarification

peer deletes all configuration for
the LAN2 interface in dual NIC
deployments.

February 2019 Updated for X12.5. X12.5 release

From this version, the guide applies
only to Cisco Expressway Series
and not to Cisco VCS.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

2
About This Guide
Change History

Date Change Reason

February 2019 Cluster Address Mapping section Documentation defect, X8.11.4

edited. Software version updated release
to X8.11.4 maintenance release.
Other superficial enhancements to
text.

September 2018 Updated for Webex and Spark X8.11.1 release

platform rebranding, CE1200
appliance, and X8.11.1
maintenance release.

August 2018 Corrected text and example in Correction

“Cluster Name and DNS SRV
Records” section.

July 2018 Updated for X8.11. X8.11 release

November 2017 Updated round trip delay and Update

maximum hop distances in
“Prerequisites” section.

October 2017 Strengthened advice on cluster Clarification

upgrade order.

August 2017 Added note that all cluster peers Omission

should be configured in the same
domain.

July 2017 Updated for X8.10. X8.10 release

April 2017 Added section and related edits for X8.9.2 release
cluster address mapping.

December 2016 Added section on clusters in X8.9 release

isolated networks in relation to
TLS.

June 2016 Cluster communications now use X8.8 release

TLS. Registrations, FindMe,
TMSPE support introduced on
Expressway.

November 2015 Updated for X8.7.

July 2015 Updated for X8.6. New procedure

for replacing a peer.

April 2015 Menu path changes for X8.5

onwards. Republished with X8.5.2.

December 2014 Updated for X8.5.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

3
 About This Guide
Change History

Date Change Reason

June 2014 Republished for X8.2.

April 2014 Updated for Expressway X8.1.1:

• New 'Upgrading a cluster'
section for Expressway
• New 'Replacing an
Expressway peer' section
• Updates to 'IP ports and
protocols' appendix

December 2013 First release of Expressway version

of this document. For older VCS
versions see VCS Configuration
Guides page.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

4
 CHAPTER 2
Clustering Basics
This chapter explains the following:
• Overview, on page 5

Overview
An Expressway can be part of a cluster of up to six Expressways. Each Expressway in the cluster is a peer of
every other one in that cluster. When you create a cluster you nominate one peer as the primary, from which
its configuration is replicated to the other peers.
Every Expressway peer in the cluster must have the same routing capabilities — if any Expressway can route
a call to a destination it is assumed that all Expressway peers in that cluster can route a call to that destination.
If routing is different on different Expressway peers, then you need to use separate Expressways / Expressway
clusters.

Benefits of Clustering
Clustered Expressways can provide benefits for both capacity and resilience:
• Capacity. Clustering can increase the capacity of an Expressway deployment by a maximum factor of
four, compared with a single Expressway.
• Resilience. Clustering can provide redundancy while an Expressway is in maintenance mode, or in case
it becomes inaccessible due to a network or power outage, or other reason. The Expressway peers in a
cluster share bandwidth usage as well as routing, zone, and other configuration. Endpoints can register
to any of the peers in the cluster, so if an endpoint loses connection to its initial peer, it can re-register
to another one in the cluster.

About Capacity Gain

There is no capacity gain after four peers. So in a six-peer cluster for example, the fifth and sixth Expressways
do not add extra call capacity to the cluster. Resilience is improved with the extra peers, but not capacity.
The Small Expressway VMs are intended for Cisco Business Edition 6000 customers, so clustering of Small
VMs only provides redundancy and does not provide any additional scale benefit.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

5
 Clustering Basics
About Licensing

About Licensing
Capacity licensing is done on a per-cluster basis, and all capacity licenses installed on a cluster peer are
available to any peer in the cluster. This includes Rich Media Session licenses and room system & desktop
system registration licenses. More details are provided in License Usage Within a Cluster

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

6
 CHAPTER 3
Clustering Requirements
Before setting up a cluster of Expressway peers or adding an Expressway to a cluster, ensure that the following
requirements are met:
This chapter explains the following:
• Do Not Mix Expressway-C and Expressway-E, on page 7
• Platform and Software Versions Match, on page 7
• Network Conditions Are Met, on page 8
• Basic Configuration Is Done, on page 8
• DNS Configuration Is Done, on page 9
• TMS Is Configured (When Essential), on page 9
• Clusters With Mixed CE1200 and CE1100 Physical Appliances, on page 9
• Mix Cluster Deployment With Expressway and Expressway Select, on page 10

Do Not Mix Expressway-C and Expressway-E

A cluster must contain only Expressway-C nodes or only Expressway-E nodes. They cannot be mixed in the
same cluster.

Platform and Software Versions Match

• All clusters peers are running the same Expressway software version. The only case when different peers
are allowed to run different versions of code is for the short period of time while a cluster is being
upgraded from one version of code to another, during which time the cluster operates in a partitioned
fashion.
• Each peer is using a hardware platform (appliance or virtual machine) with equivalent capabilities. For
example, you can cluster peers that are running on standard appliances with peers running on 2 core
Medium VMs, but you can't cluster a peer running on a standard appliance with peers running on 8 core
Large VMs.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

7
 Clustering Requirements
Network Conditions Are Met

Network Conditions Are Met

• Each peer has a different LAN configuration (a different IPv4 address and a different IPv6 address, where
enabled).
• Expressway supports a round trip delay of up to 80ms. This means that each Expressway in the cluster
must be within a 40ms hop of all other peers in the cluster.
• Each peer in a cluster is directly routable to each and every other Expressway in or to be added to the
cluster. (There must be no NAT between cluster peers – if there is a firewall ensure that the required
ports are opened.)
• External firewalls are configured to block access to the clustering TLS ports.
• The network connections between the peers must be reliable during cluster forming or changing procedures.
Clustering procedures must be carried out in the correct sequence, and the primary peer must start first.
If other peers start first they can try to assume control of the cluster, resulting in inconsistent configuration
state that is hard to recover from.

Basic Configuration Is Done

• Each peer has a different system name to all other peers.
• Configure all cluster peers in the same domain.
• Each peer has a certificate that identifies it to other peers (minimum required for default of TLS
Verification mode set to Permissive).

Note Although we support using one certificate for multiple Expressways in one cluster,
this isn't recommended due to the security risk. That is, if one private key is
compromised on one device, all devices in the cluster are compromised.

• If you have systems that still use option keys, all peers have the same set of option keys installed, with
the following exceptions:
• RMS licenses
• Room system registration licenses
• Desktop system registration licenses

• Enable H.323 mode on each peer (Configuration > Protocols > H.323, and for H.323 mode select On).
The cluster uses H.323 signaling between peers to determine the best route for calls, even if all endpoints
are SIP endpoints.
• Configure the firewall rules on each peer to block connections to clustering TLS ports, from all IP
addresses except its peers.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

8
 Clustering Requirements
DNS Configuration Is Done

DNS Configuration Is Done

DNS server configuration does not replicate so you must enter the DNS server address(es) on each peer.
• The DNS servers used by the Expressway peers must support both forward and reverse DNS lookups of
Cisco TMS and all Expressway peer addresses. The DNS servers must also provide address lookup for
any other DNS functionality required, such as:
• NTP servers or the external manager if they configured using DNS names
• Microsoft FE Server FQDN lookup
• LDAP server forward and reverse lookup (reverse lookups are frequently provided through PTR
records)

Note Expressway-E typically uses a public DNS, but it's undesirable to use the public
DNS to resolve private IP addresses. It's also undesirable to cluster on the public
addresses of the Expressway-E peers. For these reasons, we recommend you use
cluster address mapping to resolve the peers' FQDNs to private IP addresses.

For details, see the Cisco Expressway Cluster Creation and Maintenance Deployment Guide for your
version, on the Cisco Expressway Series Configuration Guides page.
• A DNS SRV record is recommended for the cluster, which contains A or AAAA records for each peer.
This configuration is advised for video interoperability and business to business (B2B) video calling,
but not for Mobile and Remote Access.
• (For MRA) Create a collab-edge SRV record for each peer in the Expressway-E cluster.
• (For B2B only) The Expressway-E cluster has a DNS SRV record that defines all cluster peers.

TMS Is Configured (When Essential)

• Cisco TMS, if used, is running version 13.2 or later (12.6 or later is permitted if you are not using Cisco
TMS for provisioning or FindMe).
• If Cisco TMS is to be used for replicating FindMe and/or Provisioning data, ensure that Provisioning
Extension mode functionality is enabled on Cisco TMS (see Cisco TMS Provisioning Extension
Deployment Guide for details).

Clusters With Mixed CE1200 and CE1100 Physical Appliances

To add a CE1200 appliance to an existing cluster that has CE1100 models in it, configure the Type option to
match the other peers (Expressway-E or Expressway-C) through the service setup wizard on the Status >
Overviewpage, before you add the CE1200 to the cluster.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

9
 Clustering Requirements
Mix Cluster Deployment With Expressway and Expressway Select

If you have clusters with mixed appliance types in them, be aware that every peer must run the same software
version. Not all appliance types support all software versions - please check first in the appliance installation
guides that the units you want to mix can all support the same software version.

Mix Cluster Deployment With Expressway and Expressway

Select

Note Expressway clusters consisting of Expressway and Expressway Select peers are not supported. Instead, all
peers within a cluster must either run the Expressway software image or the Expressway Select software
image.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

10
 CHAPTER 4
How to Form a Cluster
This chapter explains the following:
• Overview, on page 11

Overview
• You can have up to six Expressways in a cluster, including the primary.
• Add the peers to the cluster one by one.
• You should only make configuration changes on the primary Expressway.

Caution Do not adjust any cluster-wide configuration until the cluster is stable with all
peers running. Cluster database replication will be negatively impacted if any
peers are upgrading, restarting, or out of service when you change the cluster's
configuration.

Any changes made on other peers are not reflected across the cluster, and will be overwritten the next
time the primary’s configuration is replicated across the peers. The only exceptions to this are some
Peer-specific Items.
You may need to wait up to one minute before changes are updated across all peers in the cluster.
• Cluster communication failure alarms are raised while the cluster is forming. Alarms should clear when
you are finished.
• Configuration replication is suspended to new Expressways before they have properly joined the cluster.
• If the new Expressway peer has two network interfaces, the Peer N address must not specify the external
interface. If you need to enforce TLS between peers (that is, TLS verify is ON), you have to use the Peer's
FQDN as it appears on the peer's certificate for the Peer N address. You must also use Cluster Address
Mapping because the DNS resolution of the FQDN will likely resolve to the public IP address. See
Cluster Address Mapping for Expressway-E Clusters, to map FQDNs to the private IP addresses.
• If Expressway server has single NIC and static NAT enabled on the server, Peer N address must not be
the static NAT address. If you need to enforce TLS between peers (that is, TLS verify is ON), you have
to use the Peer's FQDN as it appears on the peer's certificate for the Peer N address. You must also use
Cluster Address Mapping because the DNS resolution of the FQDN will likely resolve to the public IP

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

11
 How to Form a Cluster
Preparing Expressway to Join a Cluster

address. See Cluster Address Mapping for Expressway-E Clusters, to map FQDNs to the private IP
addresses.

Preparing Expressway to Join a Cluster

• If necessary, take the new peer out of service:
1. Enable maintenance mode (Maintenance > Maintenance mode) and wait for all calls to clear and
registrations to timeout on this peer.
2. If the Expressway is in a cluster, remove it from its existing cluster then restart it.
3. Factory reset the Expressway (unless you already did this, because of the restart in previous step).

• Check that the address of your Expressway is not a peer of any other Expressway in your organization.
• Check that the Expressway is not a neighbor, traversal client, or traversal server of any other Expressway.
• Review and modify the configuration to ensure that the Expressway has:
• A valid Ethernet speed (System > Network interfaces > Ethernet).
• Valid IP address and IP gateway (System > Network interfaces > IP).
• A valid and working NTP server configured (System > Time; in the Status section, the State should
be “Synchronized”).
• At least one valid DNS server configured, and that if unqualified DNS names are used elsewhere
(e.g. for the NTP server), that the correct Domain name is also configured (Domain name is added
as a suffix to an unqualified DNS name to make it into an FQDN) (System > DNS).
• Go to System > DNS and ensure that System host name is the DNS hostname for this Expressway
(typically the same as the System name in System > Administration, but excluding spaces, and
unique for each Expressway in the cluster). If it is not configured correctly, set it up appropriately
and click Save.

Note <System host name>.<DNS domain name> = FQDN of this Expressway

• No peers configured (on System > Clustering – all Peer N address fields on this page should be
blank).

Caution If you clear all the peer address fields from the clustering page and save the
configuration, then the Expressway will factory reset itself the next time you do
a restart. This means you will lose all existing configuration except basic
networking for the LAN1 interface, including all configuration that you do
between when you clear the fields and the next restart.

If this Expressway is already a member of a cluster, you should remove it from that cluster and
restart it before you use it in another cluster.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

12
 How to Form a Cluster
Create a New Cluster of Expressway Peers

• If you have systems that use option keys, ensure that the same set of option keys is installed as those
that will be installed on all other peers of the cluster (Maintenance > Option keys). The number
of call/RMS/device/room licenses may differ between peers; all other license keys must be identical
on each peer.
• H.323 Mode set to On (Configuration > Protocols > H.323)

• If this Expressway is joining a cluster that is integrated with Cisco TMSPE, Add the Expressway to Cisco
TMS, then:
1. Check that the new Expressway can see Cisco TMS.
To do this, go to System > External manager and in the Status section, ensure that the State is
'Active'.
2. Check that Cisco TMS knows the Host Name of the Expressway:
a. Go to Systems > Navigator (and any required sub folders).
b. Select this Expressway.
c. Select the Connection tab.
d. Set Host Name to be the FQDN of this subordinate peer, for example vcs3.uk.company.com.
e. Click Save/Try.
You can ignore any error messages such as “DNS
config failure resolving <DNS name>:
Did not find system IP address () in DNS: <Server IP>”

f. Ensure that Cisco TMS updates its DNS.

1. Select the Settings tab.
2. Click Force Refresh

g. Check that Cisco TMS can communicate with the new Expressway.
To do this, on Cisco TMS go to System > Navigator (and any required sub folders) then click
on the name of the Expressway and ensure that it says:
“✓System has no open or acknowledged tickets”

• Go to System > Alarms. If there is an alarm that the Expressway must be restarted, go to Maintenance >
Restart options and then click Restart.

Create a New Cluster of Expressway Peers

This process initiates a cluster of a single Expressway. Do not use this process if the cluster already exists.

Important You must create a cluster of one (primary) peer first, and restart the primary, before you add other peers. You
can add more peers after you have established a “cluster of one”.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

13
 How to Form a Cluster
Create a New Cluster of Expressway Peers

1. Decide which Expressway will be the primary peer. The primary Expressway peer will be the source
of the configuration information for all Expressway peers in the cluster. Subordinate Expressway peers
will have most of their configuration deleted and replaced by that from the primary.
2. Check that the Expressway is running X12.6 software.
3. Backup the Expressway (Maintenance > Back and restore).
4. Review and modify the configuration to ensure that the Expressway has:
• A valid Ethernet speed (System > Network interfaces > Ethernet).
• Valid IP address and IP gateway (System > Network interfaces > IP).
• A valid and working NTP server configured (System > Time; in the Status section, the State
should be “Synchronized”).
• At least one valid DNS server configured, and that if unqualified DNS names are used elsewhere
(e.g. for the NTP server), that the correct Domain name is also configured (Domain name is added
as a suffix to an unqualified DNS name to make it into an FQDN) (System > DNS).
• Go to System > DNS and ensure that System host name is the DNS hostname for this Expressway
(typically the same as the System name in System > Administration, but excluding spaces, and
unique for each Expressway in the cluster). If it is not configured correctly, set it up appropriately
and click Save.

Note <System host name>.<DNS domain name> = FQDN of this Expressway

• No peers configured (on System > Clustering – all Peer N address fields on this page should be
blank).

Caution If you clear all the peer address fields from the clustering page and save the
configuration, then the Expressway will factory reset itself the next time you do
a restart. This means you will lose all existing configuration except basic
networking for the LAN1 interface, including all configuration that you do
between when you clear the fields and the next restart.

If this Expressway is already a member of a cluster, you should remove it from that cluster and
restart it before you use it in another cluster.
• If you have systems that use option keys, ensure that the same set of option keys is installed as
those that will be installed on all other peers of the cluster (Maintenance > Option keys). The
number of call/RMS/device/room licenses may differ between peers; all other license keys must
be identical on each peer.
• H.323 Mode set to On (Configuration > Protocols > H.323)

5. Ensure that this Expressway does not list any of the Expressways that are to be peers in this new cluster
in any of its neighbor zones or traversal zones (Configuration > Zones > Zones, then check each
neighbor and traversal zone).

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

14
How to Form a Cluster
Create a New Cluster of Expressway Peers

6. Set the H.323 Time to live to an appropriate value for the size of your deployment. A smaller number,
like 60 (seconds), means that if one Expressway becomes inaccessible, the endpoint will quickly register
with another peer (Configuration > Protocols > H.323).

Note By reducing the registration time to live too much, you risk flooding the Expressway with registration requests,
which will severely impact performance. This impact is proportional to the number of endpoints, so you should
balance the need for occasional quick failover against the need for continuous good performance.

7. Go to System > DNS and ensure that System host name is the DNS hostname for this Expressway
(typically the same as the System name in System > Administration, but excluding spaces, and unique
for each Expressway in the cluster). If it is not configured correctly, set it up appropriately and click
Save.

Note <System host name>.<DNS domain name> = FQDN of this Expressway

8. Go to Configuration > Call routingand set Call signaling optimization to On.

9. Click Save.
10. Enable maintenance mode (Maintenance > Maintenance mode) and wait for all calls to clear and
registrations to timeout on this peer.
11. (Not applicable to MRA deployments) Go to System > Clustering and ensure that Cluster name is
the routable Fully Qualified Domain Name used in SRV records that address this Expressway cluster,
for example cluster1.example.com (See Cluster Name and DNS SRV Records.
Change the Cluster name if necessary.
12. Click Save.
13. On the Clustering page configure the fields as follows:

Configuration Primary 1

Cluster IP version Choose IPv4 or IPv6 to match the underlying network addressing
scheme.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

15
 How to Form a Cluster
Next Steps

TLS Verification mode Options: Permissive (default) or Enforce.

Permissive means that the peers do not validate each others' certificates
when establishing intracluster TLS connections.
Enforce is more secure, but requires that each peer has a valid certificate
and that the signing CA is trusted by all other peers.
We recommend you form a cluster using FQDN and TLS verification
as follows: form your cluster using IP addresses in Permissive mode
and then change the peer addresses to FQDNs. You can then switch
TLS verification mode to Enforce.
If you are clustering Expressway-E peers in an isolated network, you
also need to configure cluster address mappings. For detailed steps,
see Cluster Address Mapping for Expressway-E Clusters.

Peer 1 address Enter the address of this Expressway (the primary peer).
If TLS verification mode is set to Enforce, then you must enter an
FQDN that matches the subject CN or a SAN on this peer's certificate.

14. Click Save.

To the right of the Peer 1 address field the words “This system” should appear (though this may require
the page to be refreshed before they appear).
15. Restart the Expressway (go to Maintenance > Restart options, then click Restart and confirm OK).
16. Check that configuration data exists as expected:
• If FindMe is in use, check that the expected FindMe entries still exist (Status > Applications >
TMS Provisioning Extension Services > FindMe > Accounts).
• Check configuration for items from the System, Configuration and Application menus.

17. Check that maintenance mode is disabled.

a. Go to Maintenance > Maintenance mode.
b. Set Maintenance mode to Off.
c. Click Save.

18. Backup the Expressway (Maintenance > Backup and restore).

You've now finished forming a cluster (of one Expressway)

Next Steps
• Go to Status > Alarms and ensure that all alarms are acted upon and cleared.
• Add other Expressways to the cluster using Add a Peer to a Cluster.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

16
 How to Form a Cluster
Add a Peer to a Cluster

Add a Peer to a Cluster

This procedure adds a new peer to an existing X12.6 cluster (of one or more peers) and replicates the primary
peer’s configuration onto the Expressway.
If you do not have an existing cluster, see Create a New Cluster of Expressway Peers.
1. Go to System > Clustering on the primary Expressway.
One or more of the Peer N address fields should be empty.
2. In the first empty field, enter the address of the new Expressway peer.
3. Click Save.
Peer 1 should indicate 'This system'. The new peer may indicate 'Unknown' and then with a refresh should
indicate 'Failed' because it has not fully joined the cluster yet.
4. Go to System > Clustering on one of the subordinate peers already in the cluster, and edit the following
fields:

Cluster name Identical to the Cluster name configured on the

primary Expressway

Configuration primary Same number as chosen on the primary Expressway

Cluster IP version Same version as chosen on the primary Expressway

TLS verification mode Same setting as chosen on the primary Expressway*

Peer 1 address …Peer 6 address The addresses should be the same, and in the same
order, as those entered on the primary Expressway

*If you intend to use cluster address mapping, all devices in the cluster should be in Permissive mode
initially. For more information, see Cluster Address Mapping for Expressway-E Clusters.
Save the new clustering configuration.
5. Repeat the previous step for each of the subordinate peers already in the cluster.
6. Go to System > Clustering on the new peer:

Cluster name Identical to the Cluster name configured on the

primary Expressway

Configuration primary Same number as chosen on the primary Expressway

Cluster IP version Same version as chosen on the primary Expressway

TLS verification mode Same setting as chosen on the primary Expressway*

Peer 1 address …Peer 6 address The addresses should be the same, and in the same
order, as those entered on the primary Expressway

*If you intend to use cluster address mapping, all devices in the cluster should be in Permissive mode
initially. For more information, see Cluster Address Mapping for Expressway-E Clusters.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

17
 How to Form a Cluster
Checks

Save the new clustering configuration.

7. The Expressway raises a cluster communication failure alarm. The alarm clears after the required restart.
8. Restart the Expressway (go to Maintenance > Restart options, click Restart and confirm OK).

Checks
1. After the restart, wait approximately 2 minutes – this is the frequency with which configuration is copied
from the primary.
2. Check the Cluster database status.
3. Check that configuration data exists as expected:
• If FindMe is in use, check that the expected FindMe entries still exist (Status > Applications > TMS
Provisioning Extension Services > FindMe > Accounts).
• Check configuration for items from the System, Configuration, and Application menus.

Next Steps
• Add more peers if necessary.
• If you are using Conference Factory (Multiway™) in your cluster, see Impact of Clustering in Other
Expressway Applications.
• If you want peers to resolve their FQDNs to their private IP addresses, see Cluster Address Mapping for
Expressway-E Clusters.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

18
 CHAPTER 5
(Optional) Use Fully Qualified Domain Names to
Form a Cluster
This chapter is about changing a cluster, that was formed using IP addresses, so that the peers use FQDNs to
form the cluster. This is necessary if you want to enforce TLS verification between peers. If you have not yet
formed your cluster,see How to Form a Cluster.
If you are creating a cluster of Expressway-Es, they might be in an isolated network such as a DMZ, and you'll
need to use a local mapping if you want to enforce TLS verification. If you're forming a cluster of
Expressway-Cs, you should not need to use cluster address mapping.
This chapter explains the following:
• Cluster Address Mapping for Expressway-E Clusters, on page 19
• Configure Cluster Address Mapping (Expressway-E Clusters), on page 20
• Change Cluster to Use FQDNs, on page 21
• Enforce TLS Verification, on page 24

Cluster Address Mapping for Expressway-E Clusters

For secure deployments like MRA, each Expressway-E peer must have a certificate with a SAN containing
its public FQDN. The FQDN is mapped in the public DNS to the Expressway-E's public IP address. This
configuration enables external entities, like MRA endpoints, to discover the Expressway-E's public interface
and establish a secure connection.

Do You Need Cluster Address Mapping?

• If you simply want to cluster Expressway-E peers and you don't need TLS verification between them,
then you can form the cluster using the nodes' private IP addresses. You don't need cluster address
mapping.
• If you want the Expressway-E peers in a cluster to verify each other's identities using certificates, you
could allow them to use DNS to resolve cluster peer FQDNs to their public IP addresses. This is a perfectly
acceptable way to form a cluster if the Expressway-E nodes have only one NIC, are not using static NAT,
and have routable IP addresses. You don't need cluster address mapping.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

19
 (Optional) Use Fully Qualified Domain Names to Form a Cluster
How Cluster Address Mapping Works

• If your security policy dictates that you enforce TLS verification between the peers, and if the
Expressway-Es are using static NAT, or dual NIC, or both, then we do not recommend using the external
interfaces or the static NAT addresses to form the cluster.
Also, do not try to use the public DNS to map the peers' public FQDNs to their private IP addresses,
because you will break external connectivity.
You should use cluster address mapping in these situations.

How Cluster Address Mapping Works

When you use Fully Qualified Domain Names to form the cluster, peers must be able to translate those names
into IP addresses. This translation is the main reason for DNS but, if the peers have no access to DNS, or if
you need to translate the FQDN into a private IP address, then you can populate the cluster address mapping
table to provide a local alternative to the DNS.
Cluster address mappings are FQDN:IP pairs which are shared around the cluster, one pair for each peer. The
peers consult the mapping table before they query DNS and, if they find a match, they do not query DNS.
If you choose to enforce TLS, the peers must also read the names from the SAN field of each other's certificates,
and check each name against the FQDN side of the mapping. If the SAN matches the FQDN side of the
mapping, and if the IP address that presented the certificate matches the IP side of the mapping, then the peer
trusts the other peer and they can establish the TLS connection.
Without using DNS, cluster address mapping is the only way to achieve this verification.

Where Does the Suggested Mapping Come From?

If the cluster is already formed, using IP addresses, and the peers already have a System host name and a
DNS name configured on the System > DNS page, then you have the option to automatically populate the
cluster address mapping table with assumed mappings as follows:
Peer1Hostname.Peer1DNSName maps to <Peer1 Private IP address>

….

Peer6Hostname.Peer6DNSName maps to <Peer6 Private IP address>

Note This automatic mapping may be incorrect! If the peers' certificates do not contain these assumed FQDNs
in their SAN fields, then the cluster will not form when TLS verification mode is changed to Enforce. You
must check that the SANs contain the entries that you place in the peer FQDN fields.

Configure Cluster Address Mapping (Expressway-E Clusters)

We strongly recommend that you enter the mappings on the primary peer. Address mappings replicate
dynamically through the cluster.
The mapping order is unimportant, but if you are using address mappings you must create mappings for every
cluster peer, using only private IP addresses.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

20
 (Optional) Use Fully Qualified Domain Names to Form a Cluster
Change Cluster to Use FQDNs

Step 1 Form your cluster using IP addresses (as described in Create a New Cluster of Expressway Peers, on page 13 and Add
a Peer to a Cluster, on page 17) with TLS verification mode set to Permissive.
Step 2 Verify that the cluster is correctly formed, by checking for green Clustering status messages against the peer address
fields.
You will also see blue Certificate: Invalid ...status messages. This is because your certificates should not correspond with
internal/private IP addresses, assuming they are correctly formed to identify peers by FQDN. This is expected behavior
and does not prevent you from proceeding.

Step 3 Go to System > Clustering on the primary peer, and change the Cluster address mapping enabled drop-down to On
(default is Off).
The Cluster address mapping fields display.

Step 4 [Optional, see note above] Click Suggest mappings based on system information to autofill the mapping fields for
each cluster peer. This uses the System host name and DNS name configured on the System > DNS pages of each peer,
and maps them to the IP addresses of the inward facing NICs.
Step 5 [If you used the autofill option] Check that the suggested mappings correspond to the names in the peers' certificates,
and the IP addresses of the NICs that you want to cluster. (The data is built up from information which may not match
the certificate or DNS.)
Step 6 Edit the mappings so that the public FQDNs of the Expressway-E peers correspond to the IP addresses of their internal
facing NICs.
(You can check the public FQDNs in the certificate SAN fields, or by querying DNS).

Step 7 Click Save.

The mappings are saved and copied to the other cluster peers.
Note The cluster is still formed using IP addresses and is still using the Permissive mode of TLS verification. The
cluster will start using these mappings when you change the Peer N address fields to the public FQDNs and
change the TLS verification mode to Enforce.

Change Cluster to Use FQDNs

This topic describes how to systematically change the peer address, replacing the IP addresses with FQDN.
You can change one peer address at a time, across the whole cluster, before moving on to the next address.
To change an Expressway-E cluster to use FQDNs, you use the addresses that are entered in the mapping
table (see Cluster Address Mapping for Expressway-E Clusters, on page 19).

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

21
 (Optional) Use Fully Qualified Domain Names to Form a Cluster
Change Cluster to Use FQDNs

Note While you are changing a peer address, communications between peers are temporarily impacted and you
will see alarms that persist until the changes are complete and the cluster agrees on the new addresses.

Step 1 Sign in to all the cluster peers and navigate to System > Clustering on each.
Step 2 Choose which peer address you are going to change first. We recommend starting at Peer 1 address, because you need
to repeat the following process, one by one, for all peer addresses in the list.
Step 3 On every peer in the cluster:
• Change the chosen peer address field from the IP address to the corresponding FQDN (if you did mappings, they
should be replicated in all peers at this stage).
• Click Save.

Caution Make sure you only change one peer address on each box.

Step 4 Switch to the peer that is identified by the peer address you are currently changing and restart this peer (go to
Maintenance > Restart options, then click Restart and confirm OK.
Note A single restart is needed when changing a peer address across all peers.

Step 5 Wait for any transient clustering alarms to resolve.

You've successfully changed this peer's clustering address, from an IP address to an FQDN, across the whole cluster.

Step 6 Choose which peer address you are going to change next, and then repeat steps 3-5. Repeat this loop until you have
changed all peer addresses and restarted all of the peers.
The whole cluster should now be operating on FQDNs, and the cluster is still in Permissive mode.
If the cluster is an Expressway-E cluster, and you are aiming to enforce TLS verification between the peers, then the peer
address fields should match the identities presented in the certificates. Check that both Clustering and Certificate status
messages are green.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

22
(Optional) Use Fully Qualified Domain Names to Form a Cluster
Change Cluster to Use FQDNs

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

23
 (Optional) Use Fully Qualified Domain Names to Form a Cluster
Enforce TLS Verification

Enforce TLS Verification

Before You Begin

Caution Verify that your certificate SANs contain the FQDNs that are in the Peer N address fields. You should see
green status messages for clustering and certificate next to each address field before you proceed.

Process to Enforce TLS Verification

Step 1 On the primary peer, set TLS verification mode to Enforce.

Caution A warning will display if any certificates are invalid and will prevent the cluster working properly in enforced
TLS verification mode.
The new TLS verification mode replicates throughout the cluster.

Step 2 Verify that TLS verification mode is now Enforce on each other peer.
Step 3 Click Save and restart the primary peer.
Step 4 Sign in to each other peer and then restart the peer.
Step 5 Wait for the cluster to stabilize, and check that Clustering and Certificate status is green for all peers.

Usage Note for Expressway-E Traversal Zones

This point is about operational usage rather than initial setup, but is provided here for convenience. Be aware
that the FQDN of the Expressway-C cluster will need to be configured in the TLS verify subject name field
of the Expressway-E traversal zone. Expressway uses the SAN attribute (Subject Alternate Name) to validate
the received certificate, not the CN (Common Name).

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

24
 CHAPTER 6
How to Change a Cluster
When your cluster is connected to other systems, any changes to the cluster could impact the integrated
systems. When you change a cluster, remember to:
• Check other Expressways that are neighbors, clients, or servers of this cluster and update their zone
configurations. For example, you need to update the peer address lists on neighbor zones towards this
cluster when you add or remove peers from it.
• Check connections to other systems that integrate with the cluster. For example, Cisco Unified
Communications Manager may have trunks to the cluster, or there may be auto-generated MRA zones
that need to be refreshed on new cluster peers
• Check that endpoints which register to the Expressway cluster are aware of new or removed peers, so
that they register equally to the changed cluster's peers.
• Change the DNS entries for this cluster if you add or remove peers, or change IP addresses or FQDNs.
• If you use Expressway physical appliances:
• To add a CE1200 appliance to an existing cluster that has CE1100 models in it, configure the Type
option to match the other peers (Expressway-E or Expressway-C) through the service setup wizard
on the Status > Overview page, before you add the CE1200 to the cluster.
If you are adding a more recent model than existing appliances in the cluster, upgrade the Expressway
software on the existing peers to the same version as the new appliance, before you create the backup
to be later restored onto the new appliance. (A backup can only be restored onto the same software
version that it was created on.) Not all appliance types support all software versions - please
check first in the appliance installation guides that the units you want to mix can all support the
same software version.
• Re-export SAML metadata and copy it to the IDP. Whenever you add, remove, or replace a peer
in a cluster of Expressway-Cs, you will change the SAML metadata of the cluster. If the cluster is
configured for SSO of MRA-connected clients, then SSO will fail some of the time until you update
the IDP with the cluster's new SAML metadata. This is because the (unique) serial numbers of the
peers are used to generate the cluster's metadata. For details, see the Mobile and Remote Access
Through Cisco Expressway Deployment Guide on the Expressway configuration guides page.

Note With the cluster wide SAML metadata, it is not sufficient to export metadata,
you need to regenerate SAML certificate which contains FQDN information for
all expressway cluster peers.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

25
 How to Change a Cluster
Before You Change a Cluster

Note For instructions about upgrading a cluster to a new software version, please refer to the release notes for the
relevant version.

This chapter explains the following:

• Before You Change a Cluster, on page 26
• Remove a Live Peer From a Cluster (Permanently), on page 26
• Remove a Dead Peer From a Cluster (Permanently), on page 28
• Recovery Of an Expressway Cluster Peer, on page 30
• Disband a Cluster, on page 30
• Change the Primary Peer, on page 31
• Change the Peer Identity, on page 32
• Replace a Peer, on page 32

Before You Change a Cluster

• Systems that are configured as peers must not also be configured as neighbors to each other, and vice
versa.
• If peers are deployed on different LANs, there must be sufficient connectivity between the networks to
ensure a low degree of latency between the peers.
• Cluster peers can be in separate subnets. Peers communicate with each other using H.323 messaging,
which can be transmitted across subnet boundaries.
• Deploying all peers in a cluster on the same LAN means they can be configured with the same routing
information such as local domain names and local domain subnet masks.
• To remove a peer from the cluster you must clear all peer address fields on that peer, save, then restart.

Caution If you clear all the peer address fields from the clustering page and save the configuration, then the Expressway
will factory reset itself the next time you do a restart. This means you will lose all existing configuration
except basic networking for the LAN1 interface, including all configuration that you do between when you
clear the fields and the next restart.

The Expressway displays a banner to remind you that it is pending a factory reset.
If you need to prevent the factory reset, restore the clustering peer address fields exactly as they were. Replace
the original peer addresses in the same order, and then save the configuration to clear the banner and prevent
the reset.

Remove a Live Peer From a Cluster (Permanently)

This process removes one Expressway peer from an existing cluster.
• If you are disbanding the whole cluster, see Disband a Cluster, on page 30 instead.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

26
 How to Change a Cluster
On the Expressway That You Are Removing From the Cluster

• If you want to remove the primary peer, make a different peer the primary before you remove this one.
See Change the Primary Peer, on page 31
• If you cannot access the peer you want to remove, see Remove a Dead Peer From a Cluster (Permanently),
on page 28.
• If you want to recover an expressway cluster peer, see Recovery Of an Expressway Cluster Peer, on page
30.

On the Expressway That You Are Removing From the Cluster

Step 1 Go to System > Clustering.

Step 2 Delete all entries in the Peer N address fields.
Step 3 Save.
Caution If you clear all the peer address fields from the clustering page and save the configuration, then the Expressway
will factory reset itself the next time you do a restart. This means you will lose all existing configuration
except basic networking for the LAN1 interface, including all configuration that you do between when you
clear the fields and the next restart.

If you need to avoid the factory reset, restore the clustering peer address fields as they were. Replace the original peer
addresses in the same order, and then save the configuration to clear the banner.

Step 4 Restart the Expressway (go to Maintenance > Restart options , then click Restart and confirm OK).
The factory reset is automatically triggered when the peer restarts, to remove sensitive data and clustering configuration.
The reset clears all configuration except the basic networking information listed below, which is preserved for the LAN1
interface so that you can still access the Expressway. If you use the dual-NIC option, be aware that any LAN2 configuration
is removed completely by the reset.
Configuration that is preserved (for LAN1) after the reset:
• IP addresses
• Admin and root accounts and passwords
• SSH keys
• Option keys
• HTTPS access enabled
• SSH access enabled

Note From version X12-6 the factory reset removes the server certificate, associated private key, and CA trust
store settings from the peer. In earlier Expressway software versions these settings were preserved.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

27
 How to Change a Cluster
On the Primary Expressway

On the Primary Expressway

Step 1 Go to System > Clustering.

Step 2 Delete the address of the Expressway that has been removed.
Step 3 If the Expressway being removed is not the last field in the list, move any other addresses up the list so that there are no
empty fields between entries.
Step 4 If the primary Expressway peer’s address has been moved up the list in the previous step, alter the Configuration primary
value to match its new location.
Step 5 Click Save.

On All the Remaining Subordinate Expressway Peers

Step 1 Go to System > Clustering.

Step 2 Edit the Peer N address and Configuration primary fields so that they are identical to those configured on the primary
Expressway
Step 3 Click Save.
Step 4 Repeat for all remaining subordinate Expressway peers until they all have identical clustering configuration.
You have finished removing the live Expressway from the cluster.

Remove a Dead Peer From a Cluster (Permanently)

This procedure removes an out-of-service peer from a cluster if it needs to be RMAd, or cannot be accessed
for some other reason.
• If you are disbanding the whole cluster, see Disband a Cluster, on page 30.
• If you can access the peer you want to remove, see Remove a Live Peer From a Cluster (Permanently),
on page 26
• If you want to remove the primary peer, make a different peer the primary before you remove this one.
See Change the Primary Peer, on page 31.
• If you want to recover an expressway cluster peer, see Recovery Of an Expressway Cluster Peer, on page
30.

Note This procedure does not clear configuration from the Expressway. If you manage to revive the system, you
must not start using it until you have reset its default configuration (factory reset).

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

28
 How to Change a Cluster
Clear Configuration From This Peer

On the primary Expressway:

1. Go to System > Clustering.
2. Delete the address of the Expressway that has been removed.
3. If the Expressway being removed is not the last field in the list, move any other addresses up the list so
that there are no empty fields between entries.
4. If the primary Expressway peer’s address has been moved up the list in the previous step, alter the
Configuration primary value to match its new location.
5. Click Save.

On all the remaining subordinate Expressway peers:

1. Go to System > Clustering.
2. Edit the Peer N address and Configuration primary fields so that they are identical to those configured
on the primary Expressway.
3. Click Save.
4. Repeat for all remaining subordinate Expressway peers until they all have identical clustering configuration.

You have removed the inaccessible peer from the Expressway cluster.

Clear Configuration From This Peer

If you ever recover the peer that you removed, you must clear its configuration before you reconnect it to the
network.

Step 1 Go to System > Clustering.

Step 2 Delete all entries in the Peer N address fields.
Step 3 Click Save.
Step 4 Restart the Expressway (go to Maintenance > Restart options, then click Restart and confirm OK). The Expressway
initiates a factory reset when you do the restart. It comes back up with all configuration removed, except:
The factory reset is automatically triggered when the peer restarts, to remove sensitive data and clustering configuration.
The reset clears all configuration except the basic networking information listed below, which is preserved for the LAN1
interface so that you can still access the Expressway. If you use the dual-NIC option, be aware that any LAN2
configuration is removed completely by the reset.
Configuration that is preserved (for LAN1) after the reset:
• IP addresses
• Admin and root accounts and passwords
• SSH keys
• Option keys
• HTTPS access enabled
• SSH access enabled

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

29
 How to Change a Cluster
Recovery Of an Expressway Cluster Peer

Note From version X12-6 the factory reset removes the server certificate, associated private key, and CA trust
store settings from the peer. In earlier Expressway software versions these settings were preserved.
Now you can bring it back into the cluster, see Add a Peer to a Cluster, on page 17

Recovery Of an Expressway Cluster Peer

Expressways are in a cluster. You cannot re-insert a peer involuntarily taken out of a cluster to its original
position. In such situations,
• Remove the peer from the cluster before re-inserting
• Make it the last peer in the cluster list
• Any back-up of such a peer becomes useless since it bears a different Fully Qualified Domain Name
(FQDN) in the cluster list after re-insertion.

Disband a Cluster
This process removes all Expressway peers from an existing cluster. FindMe and configuration replication
will be stopped, as will provisioning, and the cluster will be deleted from Cisco TMS.
Each Expressway will retain enough configuration to enable you to access the web interface, but all other
configuration is cleared.
The procedure involves removing peers one by one, and finally clearing the clustering configuration from the
primary peer. In X8.11 and later, clearing the clustering configuration prepares the Expressway for a factory
reset. You must be certain you want to factory reset the primary, because there are some situations where you
need to have Expressway configured as a 'cluster of one'.
To disband the cluster:

Step 1 Remove any peers that you cannot access. See Remove a Dead Peer From a Cluster (Permanently), on page 28.
Step 2 If you're using Cisco TMSPE, sign in to Cisco TMS and stop provisioning to the cluster:
a. Select Systems > Navigator (and any required sub folders), then click on any Expressway in the cluster.
b. Select the Provisioning tab.
c. Disable all 4 services (clear checkboxes).
d. Click Save.

Step 3 Remove each of the subordinate peers turn. See Remove a Live Peer From a Cluster (Permanently), on page 26.
When you remove the last subordinate peer, you should only have the primary peer remaining in the cluster.
The cluster is now a 'cluster of one' and you can stop here if you want to retain this Expressway with its configuration.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

30
 How to Change a Cluster
Change the Primary Peer

Step 4 If you want to factory reset the primary peer, sign in to it and follow the process to Remove a Live Peer From a Cluster
(Permanently), on page 26.
You've finished disbanding the cluster.

Change the Primary Peer

You can do this process even if the current primary peer is not accessible. Make sure you follow the steps in
the order listed here, to avoid putting the cluster in a state where multiple peers are contending to be the
primary.
Typically you only need to change the Configuration primary to take the primary Expressway unit out of
service or if the original primary peer fails.

Note No changes are required for Cisco TMS, which will see the primary change on the Expressway cluster and
report this appropriately.

Step 1 On the "new" primary Expressway, go to System > Clustering.

Step 2 From the Configuration primary drop-down menu select the ID number of the peer entry that says ‘This system’.
Step 3 Click Save.
While changing the primary peer, ignore any alarms on Expressway that report 'Cluster primary mismatch' or 'Cluster
replication error' – they will be rectified as part of this procedure.

Step 4 On all other Expressway peers, starting with the “old” primary peer (if it is still accessible), go to System > Clustering.
Step 5 From the Configuration primary drop-down menu select the ID number of the “new” primary Expressway.
Step 6 Click Save.
Any alarms raised on the Expressway peers that relate to 'Cluster primary mismatch' and 'Cluster replication error'
should clear automatically after approximately 2 minutes

Step 7 Confirm that the change to the Configuration primary has been accepted by going to System > Clustering and
refreshing the page.
Step 8 If any Expressways have not accepted the change, repeat the steps above.
Step 9 Check that the cluster database status reports as Active.
Step 10 If you are changing the primary peer because the “old” primary is not accessible, see Remove a Dead Peer From a
Cluster (Permanently), on page 28 procedure.
Step 11 If there is any chance of reviving the “old” primary, you must isolate it from the other peers and factory reset it if
possible.
No further steps are required if you are using FQDNs, with valid cluster address mapping configured.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

31
 How to Change a Cluster
Change the Peer Identity

Change the Peer Identity

To change the IP address, hostname, or Fully Qualified Domain Names (FQDN) of an Expressway peer, you
must remove the Expressway from the cluster, change its IP address, hostname, or FQDN and then add the
Expressway back into the cluster.
The process is as follows:

Step 1 Ensure that the Expressway whose IP address, hostname, or FQDN you want to change is not the primary Expressway.
If it is the primary Expressway, follow the steps in Change the Primary Peer, on page 31 to make a different peer the
primary.

Step 2 Carry out the process documented in Remove a Live Peer From a Cluster (Permanently), on page 26
Step 3 Change the IP address, hostname, or FQDN of the Expressway.
Step 4 Carry out the process documented in Add a Peer to a Cluster, on page 17.

If you are using an Expressway-E with dual NIC and want to change the IP address, hostname, or FQDN of
the External NIC, there is no need to disband the cluster, as this IP address, hostname, or FQDN is not used
for clustering.

Replace a Peer
This section summarizes the procedure for replacing a cluster peer Expressway with a different unit.

Step 1 Ensure that the Expressway to be replaced is not the primary Expressway.
If it is the primary Expressway, follow the steps in Change the Primary Peer, on page 31 to make a different peer the
primary.

Step 2 Remove the existing peer from the cluster:

a. If the cluster peer to be replaced is not accessible, use the procedure defined in Remove a Dead Peer From a Cluster
(Permanently), on page 28.
b. If the cluster peer to be replaced is accessible, use the procedure defined in Remove a Live Peer From a Cluster
(Permanently), on page 26.

Step 3 Add the replacement peer to the cluster using the procedure defined in Add a Peer to a Cluster, on page 17.
Important additional information if you have clusters with physical appliances

To add a CE1200 appliance to an existing cluster that has CE1100 models in it, configure the Type option to match the
other peers (Expressway-E or Expressway-C) through the service setup wizard on the Status > Overview page, before
you add the CE1200 to the cluster.
If you are adding a more recent model than existing appliances in the cluster, upgrade the Expressway software on the
existing peers to the same version as the new appliance, before you create the backup to be later restored onto the new
appliance. (A backup can only be restored onto the same software version that it was created on.) Not all appliance types

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

32
 How to Change a Cluster
Replace a Peer and Migrate Its Configuration

support all software versions - please check first in the appliance installation guides that the units you want to mix can
all support the same software version.

Replace a Peer and Migrate Its Configuration

This procedure assumes that you are replacing an accessible Expressway peer with a different Expressway.

Step 1 Ensure that the Expressway to be replaced is not the primary Expressway.
If it is the primary Expressway, follow the steps in Change the Primary Peer, on page 31 to make a different peer the
primary.

Step 2 Remove the peer by deleting its clustering configuration, but do not restart it yet. See Remove a Live Peer From
a Cluster (Permanently), on page 26.
Step 3 Backup the configuration of the removed peer before you restart it.
Step 4 If required, generate and apply any option keys needed for the new Expressway. Apply the same set of keys that are
applied to the other peers.
Step 5 Restore the backup from the removed peer onto the new Expressway.
Step 6 Check the DNS configuration of the new Expressway is the same as the other peers, and synchronize it with the same
NTP servers.
Step 7 Add the replacement peer to the cluster using the procedure defined in Add a Peer to a Cluster, on page 17.
You should use the new peer's address in place of the removed peer's address when following that procedure.
The most important steps are summarized here:
a. Add the new peer's address to the clustering configuration on the primary, in place of the old peer's address.
b. Add the new peer's address to the clustering configuration on other existing peers, in place of the old peer's address.
c. Enter the new clustering configuration on the new peer (cluster name, shared secret, ordered peer list).

Step 8 Restart the new peer.

Step 9 Wait for approximately five minutes, then check the cluster status and resolve any alarms.
Step 10 Restart the removed peer to initiate a factory reset and clear the old configuration.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

33
 How to Change a Cluster
Replace a Peer and Migrate Its Configuration

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

34
 CHAPTER 7
How to Connect the Expressway Cluster to Other
Systems
This chapter explains the following:
• Neighboring Between Expressway Clusters, on page 35
• Configure Endpoints to Work With a Cluster, on page 35
• Add the Expressway to Cisco TMS, on page 39

Neighboring Between Expressway Clusters

You can neighbor your local Expressway cluster to a remote cluster. The remote cluster might be a neighbor,
traversal client, or traversal server to the local system. When a call is received on the local Expressway and
is passed via the relevant zone to the remote cluster, it gets routed to whichever peer in that neighbor cluster
has the lowest resource usage (peers in maintenance mode are not considered). That peer then forwards the
call to one of the following:
• A locally registered endpoint, if the endpoint is registered to that peer
• A peer, if the endpoint is registered to another peer in the cluster
• An external zone, if the endpoint is located elsewhere

Configuration instructions are provided in the Expressway Administrator Guide.

Configure Endpoints to Work With a Cluster

When configuring endpoints it's desirable for them to know about all the Expressway peers in a cluster. So
that at initial registration or later, if endpoints lose connection to their Expressway peer, they can register with
another peer in the cluster. This section lists the available configuration methods (in preferred order) for SIP
endpoints and H.323 endpoints respectively.
For more details about DNS SRV and round-robin DNS, see the URI dialing section in the Expressway
Administrator Guide and Cluster Name and DNS SRV Records, on page 50.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

35
 How to Connect the Expressway Cluster to Other Systems
SIP Endpoints

Note SIP and H.323 endpoints behave differently.

SIP Endpoints
The options are listed in preference order for providing resilience of connectivity of endpoints to a cluster of
Expressways where one or more Expressway cluster peers become inaccessible. The choice of option will
depend on what functionality the endpoint you are using supports.

Option 1 – SIP Outbound (Preferred)

Important For endpoints running Cisco Collaboration Endpoint software, this option is not supported from version CE8.0.

SIP outbound allows an endpoint to be configured to register to two or more Expressway peers simultaneously.
The benefit of this is that if the connection between one peer and the endpoint breaks, a connection from the
endpoint to the other peer remains. With the endpoint registering to both peers simultaneously, there is no
break in service while the endpoint realizes that its registration has failed, before it registers to a different
peer. Thus, at no time is the endpoint unreachable.
Configuration of SIP outbound is endpoint specific, but typically will be:
• Proxy 1
• Server discovery = Manual
• Server Address =DNS name of cluster peer or IP address of cluster peer

• Proxy 2
• Server discovery = Manual
• Server Address =DNS name of a different cluster peer or IP address of a different cluster peer

• Outbound = On

Option 2 – DNS SRV (2nd Choice)

To use this option, there must be a DNS SRV record available for the DNS name of the Expressway cluster
that defines an equal weighting and priority for each cluster peer.
On each SIP endpoint configure the SIP Settings as:
• Server discovery = Manual
• Server Address = DNS name of the Expressway cluster

If the endpoint supports DNS SRV, on startup the endpoint issues a DNS SRV request and receives a DNS
SRV record back defining an equal weighting and priority for each cluster peer.
The endpoint then tries to register with a relevant cluster peer (having taken into account the priority /
weightings). If that peer is not available, the endpoint will try and register to another listed peer at the same

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

36
 How to Connect the Expressway Cluster to Other Systems
Option 3 – DNS Round-Robin (3rd Choice)

priority, or if all peers at that priority have been tried, a peer at the next lower priority. This is repeated until
the endpoint can register with a Expressway.
The endpoint will continue to use the first Expressway that it registered to for re-registrations and for calls.
If it ever loses connection to its Expressway, it will use the DNS SRV entry to find a new Expressway to
register to, starting at the highest priority.
To minimize DNS traffic, the DNS SRV cache timeout should be set to a fairly long time, such as 24 hours.

Option 3 – DNS Round-Robin (3rd Choice)

To use this option, there must be a DNS A-record available for the DNS name of the Expressway cluster that
supplies a round-robin list of IP addresses.
On each SIP endpoint configure the SIP Settings as:
• Server discovery = Manual
• Server Address = DNS name of the Expressway cluster

If the endpoint does not support DNS SRV, on startup the endpoint will perform a DNS A-record lookup.
The DNS server will have been configured to support round-robin DNS, with each of the cluster peer members
defined in the round-robin list.
The endpoint will take the address given by the DNS lookup and will then try and register with the relevant
cluster peer. If that is not available, then the endpoint will perform another DNS lookup and will try to connect
to the new Expressway peer that it is given. (The DNS server will have supplied the next cluster peer’s IP
address.) This is repeated until the endpoint can register with a Expressway.
The endpoint will continue to use the first Expressway that it registered to for re-registrations and for calls.
If it ever loses connection to its Expressway it will perform another DNS lookup to find a new Expressway
to register to (the DNS server providing a Expressway in the round-robin sequence).
DNS cache timeout should be set to a fairly short time (for example, 1 minute or less) so that if a Expressway
is not accessible the endpoint is quickly pointed at a different Expressway.

Option 4 – Static IP (Least Preferred)

Use this option if the Expressway cluster does not have a DNS name.
On each SIP endpoint configure the SIP Settings as:
• Server discovery = Manual
• Server Address = IP address of a Expressway peer

On startup the endpoint will try and register with the Expressway at the specified IP address. If that is not
available, then the endpoint will continue trying at regular intervals. This is repeated until the endpoint can
register with the Expressway.
The endpoint will continue to use the first Expressway that it registered to for re-registrations and for calls.
If it ever loses connection then it will keep on trying to register to that Expressway until it is accessible again.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

37
 How to Connect the Expressway Cluster to Other Systems
H.323 Endpoints

H.323 Endpoints
The options are listed in preference order for providing resilience of connectivity of endpoints to a cluster of
Expressways where 1 or more Expressway cluster peers become inaccessible. The choice of option will depend
on what functionality the endpoint you are using supports.

Option 1 – DNS SRV (Preferred)

To use this option, there must be a DNS SRV record available for the DNS name of the Expressway cluster
that defines an equal weighting and priority for each cluster peer.
On each H.323 endpoint, configure the Gatekeeper Settings as:
• Discovery = Manual
• IP Address = DNS name of the Expressway cluster

If the endpoint supports DNS SRV, on startup the endpoint issues a DNS SRV request and receives a DNS
SRV record back defining an equal weighting and priority for each cluster peer.
The endpoint then tries to register with a relevant cluster peer (having taken into account the priority /
weightings). If that peer is not available, the endpoint will try and register to another listed peer at the same
priority, or if all peers at that priority have been tried, a peer at the next lower (higher numbered) priority.
This will be repeated until the endpoint can register with a Expressway. On registering with the Expressway,
the Expressway will respond with the H.323 “Alternate Gatekeepers” list containing the list of Expressway
cluster peer members.
The endpoint will continue to use the first Expressway that it registered to for re-registrations and for calls.
If it ever loses connection to its Expressway then it will select an “Alternate Gatekeeper” from the list it was
supplied with.
DNS SRV cache timeout should be set to a fairly long time (e.g. 24 hours) to minimize DNS traffic.

Option 2 – DNS Round-Robin (2nd Choice)

To use this option, there must be a DNS A-record available for the DNS name of the Expressway cluster that
supplies a round-robin list of IP addresses.
On each H.323 endpoint configure the Gatekeeper Settings as:
• Discovery = Manual
• IP Address = DNS name of the Expressway cluster

If the endpoint does not support DNS SRV, on startup the endpoint will perform a DNS A-record lookup.
The DNS server will have been configured to support round-robin DNS, with each of the cluster peer members
defined in the round-robin list.
The endpoint will take the address given by the DNS lookup and will then try and register with the relevant
cluster peer. If that peer is not available, then the endpoint will perform another DNS lookup and will try to
connect to the new Expressway peer that it is given. (The DNS server will have supplied the next cluster
peer’s IP address.)
This will be repeated until the endpoint can register with a Expressway. On registering with the Expressway,
the Expressway will respond with the H.323 ‘Alternate Gatekeepers’ list containing the list of Expressway
cluster peer members.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

38
 How to Connect the Expressway Cluster to Other Systems
Option 3 – Static IP (Least Preferred)

The endpoint will continue to use the first Expressway that it registered to for re-registrations and for calls.
If it ever loses connection then it will select an “Alternate Gatekeeper” from the list it was supplied with.
DNS cache timeout should be set to a fairly short time (e.g. 1 minute or less) so that on failure to reach a
Expressway at startup, the endpoint is quickly pointed at a different Expressway.

Option 3 – Static IP (Least Preferred)

Use this option if the Expressway cluster does not have a DNS name.
On each H.323 endpoint configure the Gatekeeper Settings as:
• Discovery = Manual
• IP Address = IP address of a Expressway peer

On startup the endpoint will try and register with the Expressway at the specified IP address. If that is not
available, then the endpoint will continue trying at regular intervals.
This will be repeated until the endpoint can register with the Expressway. On registering with the Expressway,
the Expressway will respond with the H.323 “Alternate Gatekeepers” list containing the list of Expressway
cluster peer members.
The endpoint will continue to use the first Expressway that it registered to for re-registrations and for calls.
If it ever loses connection then it will select an “Alternate Gatekeeper” from the list it was supplied with.

Add the Expressway to Cisco TMS

For more detail about Cisco TMS administration, see Cisco TelePresence Management Suite Administrator
Guide , for your version, at Cisco Telepresence Management Suite (TMS Maintain and Operate Guide page)

On the Expressway

Step 1 Go to System > SNMP.

a) SNMP mode is set to v3 plus Cisco TMS support or v2c.
b) Community name is set to public.
(If SNMP was previously disabled, an alarm may appear indicating the need for a restart. If so, restart the system via
Maintenance > Restart options..)

Step 2 Go to System > External manager and ensure that:

a) Address is set to the IP address or FQDN of Cisco TMS.
b) Path is set to tms/public/external/management/SystemManagementService.asmx.
c) If the Protocol is HTTPS and Certificate verification mode is On then you must load the relevant certificates before
the connection can become ‘Active'.
(If the Protocol is HTTP or Certificate verification mode is Off, no certificates need to be loaded.)

Step 3 Click Save.

The Status section of the External manager page should show a State of ‘Active’ or ‘Initialising’1.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

39
 How to Connect the Expressway Cluster to Other Systems
On the Cisco TMS

On the Cisco TMS

Step 1 Select Systems > Navigator.

Step 2 Select (or create) an appropriate folder in which to put the Expressway (in the example below the folder has been called
“Cluster”):

Step 3 Click Add Systems.

Step 4 In section 1. Specify Systems by IP addresses or DNS names , enter the IP address or DNS name of the Expressway.
Step 5 Click Next.
Step 6 Look for "green tick" sign for System added.
Note When you add an Expressway to TMS, the TMS UI shows it as a VCS . This is a known issue.

Step 7 Click Finish Adding Systems, Add System despite warnings or Add More Systems as appropriate.

1
Cisco TMS may force the protocol to be HTTPS. The configuration for this is found in Administrative Tools > Configuration > Network settings. The
protocol will be forced to HTTPS if, in the TMS Services section Enforce Management Settings on Systems is set to On and in the Secure-Only Device
Communication section Secure-Only Device Communication is set to On.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

40
 CHAPTER 8
Troubleshooting
This chapter explains the following:
• How to Rebuild a Cluster from Backup, on page 41
• Restarting Sequence, on page 41
• Check Replication Status, on page 42
• Force Refresh in Cisco TMS, on page 42
• Expressway Alarms and Warnings, on page 42
• Cisco TMS Warnings, on page 44

How to Rebuild a Cluster from Backup

1. Bring down all the nodes in the existing cluster.
2. Install new OVA in the Primary and bring the device up.
3. Restore the backup on to the primary. Restart the primary.
4. Install OVA on first peer and restore from backup.
5. Repeat step 5 for all the peer nodes one by one.
6. Reboot the primary node.

Restarting Sequence
Whenever you have formed, connected, upgraded, or changed a cluster, you should check whether any peers
need restarting. Sometimes you will need to restart only one peer, typically if you've made a peer-specific
configuration change.
When you're working with the cluster configuration, you sometimes need to restart more than one peer. In
this case, you should always restart in the following sequence:
1. Restart the primary peer, and wait for it to be accessible via web interface.
2. Check cluster replication status on the primary and status of all peers. Wait a few minutes, refreshing the
peer's web interfaces occasionally.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

41
 Troubleshooting
Check Replication Status

3. Restart other peers, if required, one at a time. Each time, wait a few minutes after it is accessible and
check its replication status.

Check Replication Status

You may need to wait about 5 minutes after making clustering changes before the Expressway peers report
successful status.
1. Go to System > Clustering on each peer and check that the cluster database status reports as Active.
If there is a failure status, refresh the browser first. If the status is still not Active, check the alarms.

Force Refresh in Cisco TMS

If you're using Cisco TMS, check that Cisco TMS has all the correct settings for the cluster by forcing a refresh
as follows:

Step 1 On Cisco TMS, go to Systems > Navigator.

Step 2 Find and click on the name of the Expressway.
Step 3 Select the Settings tab.
Step 4 Click Force Refresh.
Step 5 Repeat for all Expressway peers in the cluster (including the primary Expressway).

Expressway Alarms and Warnings

Cluster name not configured: if FindMe or clustering are in use a cluster name
must be defined
Ensure that the same cluster name is configured on each Expressway in the cluster.
The Cluster name should be the routable Fully Qualified Domain Name used in SRV records that address
this Expressway cluster, for example “cluster1.example.com”. (See Cluster Name and DNS SRV Records,
on page 50).

Cluster replication error: (details) manual synchronization of configuration is

required
This may be:
• “Cluster replication error: manual synchronization of configuration is required”

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

42
 Troubleshooting
Cluster replication error: (details) restart node

• “Cluster replication error: configuration primary ID is inconsistent, manual synchronization of

configuration is required”
• “Cluster replication error: this peer's configuration conflicts with the primary's configuration, manual
synchronization of configuration is required”

If a subordinate Expressway reports an alarm: “Cluster replication error – <details> synchronization of

configuration”
On that subordinate Expressway:
1. Log in as admin on an SSH or other CLI interface.
2. At the command prompt type: xcommand ForceConfigUpdate

This will delete the subordinate Expressway configuration and then force it to update its configuration
from the primary Expressway.

Caution Use this command only if the configuration on the primary Expressway is known to be in a good state. We
recommend that you take a backup before running this command.

Cluster replication error: (details) restart node

This may be:
“Cluster replication error: cannot find primary or this subordinate’s peer configuration file, restart the node”

Cluster replication error persists after ForceConfigUpdate

In X8.11 we introduced a unique encryption key per cluster peer. Also, in some upgrade cases, for example,
if peers are upgraded in the wrong order, subordinate peers may not synchronize with the primary. These two
issues compound each other, allowing peers to be in a state where they cannot decrypt the configuration from
the primary.
The symptom of this is that the Cluster replication alarm persists after you tried xcommand forceconfigupdate
on a subordinate peer. This is probably after a recent upgrade to X8.11 on the primary peer.
You can avoid the problem by always upgrading the primary first, but if you have got this persistent error,
you can resolve it as follows:
1. Sign in to the primary peer and check that it is in a good state.
2. Ensure that the clustering configuration shows this peer to be the primary.
3. Upgrade the primary again, using the same package that you originally used to upgrade.

The replication alarm clears after the primary peer has upgraded and rebooted. This normally happens within
ten minutes after reboot, but could be up to twenty minutes after reboot.

Cluster replication error: the NTP server is unreachable

Configure an accessible NTP server on the Expressway System > Time page.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

43
 Troubleshooting
Cluster replication error: the local Expressway does not appear in the list of peers

Cluster replication error: the local Expressway does not appear in the list of
peers
Check and correct the list of peers for this Expressway on the primary Expressway, and copy to all other
Expressway peers (System > Clustering).

Cluster replication error: automatic replication of configuration has been

temporarily disabled because an upgrade is in progress
Wait until the upgrade has completed.

Invalid clustering configuration: H.323 mode must be turned On - clustering

uses H.323 communications between peers
Ensure that H.323 mode is on (see Configuration > Protocols > H.323).

Expresswaydatabasefailure:PleasecontactyourCiscosupportrepresentative
The support representative will help you work through the following steps:
1. Take a system snapshot and provide it to your support representative.
2. Remove the Expressway from the cluster using: Remove a Live Peer From a Cluster (Permanently), on
page 26.
3. Restore that Expressway’s database by restoring a backup taken on that Expressway previously.
4. Add the Expressway back to the cluster using Add a Peer to a Cluster, on page 17.

Cisco TMS Warnings

Cisco TMS Cluster Diagnostics
If Cisco TMS cluster diagnostics reports a difference in configuration on Expressway peers, it is comparing
the output of https://<ip address>/alternatesconfiguration.xml for each Expressway.
To manually check the differences, on a Unix / Linux system, run:
wget --user=admin --password=<password> --no-check-certificate https://<IP or FQDN of
Expressway>/alternatesconfiguration.xml

for each of the Expressway peers, then use diff to check for differences.

Conference Factory Template Does Not Replicate

This is by design; the Conference Factory %% value is NOT shared between cluster peers and the Conference
Factory application configuration is NOT replicated across a cluster.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

44
 Troubleshooting
Expressway’s External Manager Protocol Keeps Getting Set to HTTPS

See Impact of Clustering in Other Expressway Applications, on page 57.

Expressway’s External Manager Protocol Keeps Getting Set to HTTPS

Cisco TMS can be configured to force specific management settings on connected systems. This includes
ensuring that a Expressway uses HTTPS for feedback. If enabled, Cisco TMS will (on a time period defined
by Cisco TMS) re-configure the Expressway’s System > External Manager Protocol to HTTPS.
If HTTPS must be used for Expressway to supply feedback to Cisco TMS, see Add the Expressway to Cisco
TMS, on page 39 for information about how to set up certificates.
Cisco TMS will force HTTPS on Expressway if:
• TMS Services > Enforce Management Settings on Systems = On (Administrative Tools >
Configuration > Network Settings)
and
• Secure-Only Device Communication > Secure-Only Device Communication = On (Administrative
Tools > Configuration > Network Settings)

Set Enforce Management Settings on Systems to Off if Cisco TMS does not need to force the management
settings.
Set Secure-Only Device Communication to Off if it is unnecessary for Expressway to provide feedback to
Cisco TMS using HTTPS (if HTTP is sufficient).

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

45
 Troubleshooting
Expressway’s External Manager Protocol Keeps Getting Set to HTTPS

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

46
 CHAPTER 9
Reference
This chapter explains the following:
• Peer-specific Items, on page 47
• Sample Firewall Rules for Protecting Intracluster TLS Port, on page 49
• Cluster Name and DNS SRV Records, on page 50
• Clusters in Isolated Networks, on page 54
• NAPTR Records, on page 55
• Impact of Clustering in Other Expressway Applications, on page 57

Peer-specific Items
Most items of configuration are applied via the primary peer to all peers in a cluster. However, the following
items (marked with a † on the web interface) must be specified separately on each cluster peer.

Note You should not modify configuration data that applies to all peers on any peer other than the primary peer.
At best it will result in the changes being overwritten from the primary; at worst it will cause cluster replication
to fail.

Cluster configuration (System > Clustering)

The list of Peer N addresses (including the peer's own address) that make up the cluster has to be specified
on each peer and they must be identical on each peer.
The Cluster name, Configuration primary, and Cluster IP version must be specified on each peer and
must be identical for all peers.

Note If you need to enable cluster address mapping, we recommend forming the cluster on IP addresses first. Then
you will only need to add the mappings on one peer.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

47
 Reference
Peer-specific Items

Ethernet speed (System > Network interfaces > Ethernet)

The Ethernet speed is specific to each peer. Each peer may have slightly different requirements for the
connection to their Ethernet switch.

IP configuration (System > Network interfaces > IP)

LAN configuration is specific to each peer.
• Each peer must have a unique IP address, whether that is an IPv4 address, an IPv6 address, or both.
• IP gateway configuration is peer-specific. Each peer can use a different gateway.

Note that the IP protocol is applied to all peers, because each peer must support the same protocols.

IP static routes (System > Network interfaces > Static routes)

Any static routes you add are peer-specific and you may create different routes on different peers if required.
If you want all peers in the cluster to be able to use the same static route, you must create the route on each
peer.

System name (System > Administration)

The System name must be different for each peer in the cluster.

DNS servers and DNS host name (System > DNS)

DNS servers are specific to each peer. Each peer can use a different set of DNS servers.
The System host name and Domain name are specific to each peer.

NTP servers and time zone (System > Time)

The NTP servers are specific to each peer. Each peer may use one or more different NTP servers.
The Time zone is specific to each peer. Each peer may have a different local time.

SNMP (System > SNMP)

SNMP settings are specific to each peer. They can be different for each peer.

Logging (Maintenance > Logging)

The Event Log and Configuration Log on each peer only report activity for that particular Expressway. The
Log level and the list of Remote syslog servers are specific to each peer. We recommend that you set up a
remote syslog server to which the logs of all peers can be sent. This allows you to have a global view of
activity across all peers in the cluster.

Security certificates (Maintenance > Security)

The trusted CA certificate, server certificate and certificate revocation lists (CRLs) used by the Expressway
must be uploaded individually per peer.

Administration access (System > Administration)

The following system administration access settings are specific to each peer:

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

48
 Reference
Sample Firewall Rules for Protecting Intracluster TLS Port

• Serial port / console

• SSH service
• Web interface (over HTTPS)
• Redirect HTTP requests to HTTPS
• Automated protection service

Option keys (Maintenance > Option keys)

Option keys that control features are specific to the peer where they are applied. Option keys that control
licenses are pooled for use by the whole cluster.
Each peer must have an identical set of feature option keys installed, which means you must purchase a key
for each peer in the cluster.
License option keys can be applied to one or more peers in the cluster, and the sum of the installed licenses
is available across the cluster. This license pooling behavior includes the following option keys:
• Expressway: Rich media sessions
• Expressway: Telepresence room systems
• Expressway: Desktop systems
• VCS: Traversal calls
• VCS: Non-traversal calls

Note In some cases a peer will raise an alarm that it has no key to enable licenses the peer needs, even though there
are licenses available in the cluster. You can acknowledge and ignore this category of alarm, unless the only
peer that has the required licenses is out of service.

Active Directory Service (Configuration > Authentication > Devices > Active Directory Service)
When configuring the connection to an Active Directory Service for device authentication, the NetBIOS
machine name (override), and domain administrator Username and Password are specific to each peer.

Conference Factory template (Applications > Conference Factory)

The template used by the Conference Factory application to route calls to a conferencing server must be unique
for each peer in the cluster.

Sample Firewall Rules for Protecting Intracluster TLS Port

To protect your cluster peers against denial-of-service attacks, we encourage you to use the Expressway's
in-built firewall rules to filter all TCP access to the clustering ports.
On each peer:
1. Go to System > Protection > Firewall rules > Configuration.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

49
 Reference
Cluster Name and DNS SRV Records

2. Add a rule to drop TCP connections to ports 4371 and 4372, for all IP addresses in the appropriate (IPv4
or IPv6) range.
3. Add lower priority rules, one for each of the other peers' IP addresses, that allow TCP connections to
those ports.
(Lower numbered rules are implemented before higher numbered rules.)
4. Activate the firewall rules.

Figure 1: Creating a custom rule to allow a specific peer to connect to this peer's clustering ports

Figure 2: Example list of rules, showing recommended priority order

Cluster Name and DNS SRV Records

Using DNS SRV to convert a domain to an IP address has a number of benefits:

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

50
 Reference
DNS SRV Configuration for Mobile and Remote Access

• The structure of the lookup includes service type and protocol as well as the domain, so that a common
domain can be used to reference multiple different services which are hosted on different machines (e.g.
HTTP, SIP, H.323).
• The DNS SRV response includes priority and weighting values which allow the specification of primary,
secondary, tertiary etc groups of servers, and within each priority group, the weighting defines the
proportion of accesses that should use each server.
• As the DNS SRV response contains details about priorities and weights of multiple servers, the receiving
device can use a single lookup to search for an in-service server (where some servers are inaccessible)
without the need to repeatedly query the DNS server. (This is in contrast to using round robin DNS which
does require repeated lookups into the DNS server if initial servers are found to be inaccessible.)

The generic format of a DNS SRV query is:

• _service._protocol.<fully.qualified.domain>

The DNS SRV response is a set of records in the format:

• _service._ protocol.<fully.qualified.domain>. TTL Class SRV Priority Weight Port Target
where Target is an A-record defining the destination.

Further details on DNS SRV can be found in Expressway Administrator Guide and RFC 2782.

DNS SRV Configuration for Mobile and Remote Access

This section summarizes the public (external) and local (internal) DNS requirements for MRA. For more
information, see the Cisco Jabber Planning Guide for your version on the Jabber Install and Upgrade Guides
page.

Important From version X8.8 onward, you must create forward and reverse DNS entries for all Expressway-E
systems, so that systems making TLS connections to them can resolve their FQDNs and validate their
certificates.

Public DNS (external domains)

The public, external DNS must be configured with _collab-edge._tls.<domain> SRV records so that endpoints
can discover the Expressway-Es to use for Mobile and Remote Access. You also need SIP service records
for general deployment (not specifically for MRA). For example, for a cluster of 2 Expressway-E systems:

Table 2:

Domain Service Protocol Priority Weight Port Target Host

example.com collab-edge tls 10 10 8443 expe1.example.com

example.com collab-edge tls 10 10 8443 expe2.example.com

example.com sips tcp 10 10 5061 vcse1.example.com

example.com sips tcp 10 10 5061 vcse2.example.com

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

51
 Reference
DNS SRV Configuration for Video Conferencing

Local DNS (internal domains)

Although we recommend that the local, internal DNS is configured with _cisco-uds._tcp.<domain> SRV
records, from X12.5 this is no longer a requirement. Example records:

Table 3:

Domain Service Protocol Priority Weight Port Target Host

example.com cisco-uds tcp 10 10 8443 cucmserver1.example.com

example.com cisco-uds tcp 10 10 8443 cucmserver2.example.com

Create internal DNS records, for both forward and reverse lookups, for all Unified Communications nodes
used with MRA. This allows Expressway-C to find the nodes when IP addresses or hostnames are used instead
of FQDNs.
Ensure that the cisco-uds SRV records are NOT resolvable outside of the internal network, otherwise the
Jabber client will not start MRA negotiation via the Expressway-E.

DNS SRV Configuration for Video Conferencing

The format of DNS SRV queries for sip (RFC 3263) and H.323 used by Expressway are:
• _sips._tcp.<fully.qualified.domain>
• _sip._tcp.<fully.qualified.domain>
• _sip._udp.<fully.qualified.domain> - not recommended for video calls, only use for audio-only calls
• _h323ls._udp.<fully.qualified.domain> - for UDP location (RAS) signaling, such as LRQ
• _h323cs._tcp.<fully.qualified.domain> - for H.323 call signaling

The format of DNS SRV queries for sip (RFC 3263) and H.323 typically used by an endpoint are:
• _sips._tcp.<fully.qualified.domain>
• _sip._tcp.<fully.qualified.domain>
• _sip._udp.<fully.qualified.domain> - not recommended for video calls, only use for audio-only calls
• _h323ls._udp.<fully.qualified.domain> - for UDP location (RAS) signaling, such as LRQ
• _h323cs._tcp.<fully.qualified.domain> - for H.323 call signaling
• _h323rs._udp.<fully.qualified.domain> - for H.323 registrations

UDP is not a recommended transport medium for video signaling; SIP messaging for video system is too
large to be reliably carried on datagram-based (rather than stream-based) transports.
The Expressway Cluster name (configured on the System > Clustering page) should be an FQDN, where
the domain part is the domain used for the SRV records which point to that Expressway cluster.
Example
DNS SRV records for 2 peers of an Expressway-E cluster for example.com
where:

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

52
 Reference
Checking DNS SRV Settings

• FQDN of Expressway-E peer 1: expe1.example.com

• FQDN of Expressway-E peer 2: expe2.example.com
• FQDN of Expressway-E cluster: cluster.example.com

_sips._tcp.example.com. 86400 IN SRV 1 1 5061 expe1.example.com.

_sips._tcp.example.com. 86400 IN SRV 1 1 5061 expe2.example.com.
_sip._tcp.example.com. 86400 IN SRV 1 1 5060 expe1.example.com.
_sip._tcp.example.com. 86400 IN SRV 1 1 5060 expe2.example.com.
_h323ls._udp.example.com. 86400 IN SRV 1 1 1719 expe1.example.com.
_h323ls._udp.example.com. 86400 IN SRV 1 1 1719 expe2.example.com.
_h323cs._tcp.example.com. 86400 IN SRV 1 1 1720 expe1.example.com.
_h323cs._tcp.example.com. 86400 IN SRV 1 1 1720 expe2.example.com.
_h323rs._udp.example.com. 86400 IN SRV 1 1 1719 expe1.example.com.
_h323rs._udp.example.com. 86400 IN SRV 1 1 1719 expe2.example.com.

Note • Priorities are all the same. Only use different priorities if you have different clusters allowing failover
from one primary cluster to another (secondary) cluster. In that case the primary cluster peers should
have one value and the other (secondary) cluster peers should have a larger value.
• Weights should be the same – so that there is equal use of each peer.

Checking DNS SRV Settings

Check DNS SRV Connectivity from Expressway
1. Go to Maintenance > Tools > Network utilities > Connectivity Test.
2. Enter a Service Record Domain you want to query, for example, call.ciscospark.com.
3. Enter the Service Record Protocols you want to test, for example, _sips._tcp.
Use commas to delimit multiple protocols, for example, _sip._tcp, _sips._tcp.
4. Click Run.

The Expressway queries DNS for SRV records comprised of the service, protocol and domain combinations,
for example: _sip._tcp.call.ciscospark.com and _sips._tcp.call.ciscospark.com.
By default the system will submit the query to all of the system's default DNS servers (System > DNS).

Use DNS Lookup Tool on Expressway

1. Go to Maintenance > Tools > Network utilities > DNS lookup.
2. Enter the SRV path in the Host field.
3. Click Lookup.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

53
 Reference
Clusters in Isolated Networks

nslookup
nslookup -query=SRV _sip._tcp.example.com

dig
dig _sip._tcp.example.com SRV
; <<>> DiG 9.4.1 <<>> _sip._tcp.example.com SRV
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44952
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 4
;; QUESTION SECTION:
;_sip._tcp.example.com. IN SRV
;; ANSWER SECTION:
_sip._tcp.example.com. 1183 IN SRV 1 0 5060 expe1.example.com.
_sip._tcp.example.com. 1183 IN SRV 1 0 5060 expe2.example.com.
;; AUTHORITY SECTION:
example.com. 87450 IN NS ns1.mydyndns.org.
example.com. 87450 IN NS ns2.mydyndns.org.
;; ADDITIONAL SECTION:
expe1.example.com. 1536 IN A 194.73.59.53
expe2.example.com. 1376 IN A 194.73.59.54
ns1.mydyndns.org. 75 IN A 204.13.248.76
ns2.mydyndns.org. 10037 IN A 204.13.249.76
;; Query time: 0 msec
~ #

Clusters in Isolated Networks

Note The background information in this appendix is valid, but the issue and workaround described have been
invalidated by a fix in X8.9.2. That fix allows privately mapping peer FQDNs to peer IP addresses, instead
of using the IP addresses returned by DNS lookup.

As of X8.8, Expressway peers use TLS to communicate with each other. You have the option of permissive
TLS - the certificates are not verified - or enforced TLS where the certificates are verified.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

54
Reference
NAPTR Records

In the latter case, each peer will need to DNS look up the common name (CN), and perhaps also subject
alternate names (SANs), that they read from their peers' certificates. They compare the returned IP addresses
against the IP addresses that gave them the certificates and if they match, the connection is authenticated.
In isolated networks, the peers will not typically be able to reach the internal DNS servers, because that would
require unsolicited inbound requests. In a dual-NIC setup, you probably also don't want to put the peers' private
IP addresses into the public DNS.
The issue is compounded by not being able to use IP addresses as common names or subject alternate names
on server certificates: certificate authorities do not advocate this and probably will not issue such certificates.

Expressway-E peers have dual NICs, with no static NAT

You can enforce TLS between cluster peers:
1. Enter public DNS servers on the DNS configuration of each peer.
2. Choose which of the LAN interfaces will take the public facing address.
3. Configure the public DNS to resolve each peer's FQDN to its public IP address.
4. Populate the CN of all peer certificates with the same cluster FQDN, and populating each peer certificate's
SAN with that peer's FQDN.
5. Enter the cluster FQDN and peer FQDNs on the clustering configuration page and set the TLS verification
mode to Enforce.

The peers will now use the public DNS to verify each others' identities, as presented on their certificates.

Expressway-E peers have dual NICs, with static NAT enabled

In addition to its private IP address in the isolated network, you can give one of the NICs a public IP address
that translates to its private address. In this case, you cannot use FQDNs to form the cluster.
This is because the public DNS record for each peer's FQDN would match its translated (public) IP address,
but the peers would see each other's private addresses when swapping certificates. The mismatch between IP
addresses would prevent the TLS connection being established, and the cluster would not form.
To form the cluster:
1. Enter public DNS servers on the DNS configuration of each peer.
2. Choose which of the LAN interfaces on each peer will have static NAT enabled.
3. Enter the private IP addresses of the other LAN interfaces on the clustering configuration pages, and set
the TLS mode to Permissive.

The peers will now use the private IP addresses to form the cluster, but will not check the contents of the
certificates against the DNS records.

NAPTR Records
NAPTR records are typically used to specify various methods to connect to a destination URI, for example
by email, by SIP, by H.323. They can also be used to specify the priority to use for those connection types,
for example to use SIP TLS in preference over using SIP TCP or SIP UDP.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

55
 Reference
NAPTR Record Format

NAPTR records are also used in ENUM, when converting a telephone number into a dialable URI. (For further
details on ENUM see ENUM Dialing on Expressway Deployment Guide).

NAPTR Record Format

Example: SIP access to example.com, and for enum lookups for 557120, 557121, and 557122.
$ORIGIN example.com.
IN NAPTR 10 100 "s" "SIPS+D2T" "" _sips._tcp.example.com.
IN NAPTR 12 100 "s" "SIP+D2T" "" _sip._tcp.example.com.
IN NAPTR 14 100 "s" "SIP+D2U" "" _sip._udp.example.com.

$ORIGIN www.example.com.
IN NAPTR 10 100 "s" "http+I2R" "" _http._tcp.example.com.
IN NAPTR 10 100 "s" "ftp+I2R" "" _ftp._tcp.example.com.

$ORIGIN 0.2.1.7.5.5.enum.lookup.com.
IN NAPTR 10 100 "u" "E2U+sip" "!^.*[email protected]!"
.
IN NAPTR 12 100 "u" "E2U+h323" "!^.*[email protected]!"
.
IN NAPTR 10 100 "u" "mailto+E2U" "!^.*$!mailto:[email protected]!"
.

$ORIGIN 1.2.1.7.5.5.enum.lookup.com.
IN NAPTR 10 100 "u" "E2U+sip" "!^.*[email protected]!" .

$ORIGIN 2.2.1.7.5.5.enum.lookup.com.
IN NAPTR 10 100 "u" "E2U+h323" "!^.*[email protected]!" .
IN = Internet routing NAPTR = record type
10 = order value (use lowest order value first)
100 = preference value if multiple entries have the same order value
"u" = the result is a routable URI
"s" = the result is a DNS SRV record
"a" = the result is an ‘A’or ‘AAAA’ record
"E2U+sip" to make SIP call
"E2U+h323" to make h.323 call
Regular expression:
! = delimiter
"" = no expression used
… usual Regex expressions can be used
Replace field; . = not used

Looking Up an ENUM NAPTR Record

dig 4.3.7.8.enum4.example.com. NAPTR
; <<>> ;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38428
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 2
;; QUESTION SECTION:
;4.3.7.8.enum4.example.com. IN NAPTR
;; ANSWER SECTION:
4.3.7.8.enum4.example.com. 60 IN NAPTR 10 100 "u" "E2U+sip" "!^.*[email protected]!" .
4.3.7.8.enum4.example.com. 60 IN NAPTR 10 100 "u" "E2U+h323" "!^.*[email protected]!" .

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

56
 Reference
Impact of Clustering in Other Expressway Applications

;; AUTHORITY SECTION:
enum4.example.com. 60 IN NS int-server1.example.com.
;; ADDITIONAL SECTION:
int-server1.example.com. 3600 IN A 10.44.9.144
int-server1.example.com. 3600 IN AAAA 3ffe:80ee:3706::9:144
;; Query time: 0 msec

Looking Up a Domain NAPTR Record

Example: NAPTR record allowing endpoints to detect that they are in the public (external) network. The flag
“s” is extended to “se” to indicate that it is “external”.
~ # dig -t NAPTR example.com
; <<>> DiG 9.4.1 <<>> -t NAPTR example.com
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1895
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 6, ADDITIONAL: 4
;; QUESTION SECTION:
;example.com. IN NAPTR
;; ANSWER SECTION:
example.com. 2 IN NAPTR 50 50 "se" "SIPS+D2T" "" _sips._tcp.example.com.
example.com. 2 IN NAPTR 90 50 "se" "SIP+D2T" "" _sip._tcp.example.com.
example.com. 2 IN NAPTR 100 50 "se" "SIP+D2U" "" _sip._udp.example.com.
;; AUTHORITY SECTION:
example.com. 320069 IN NS nserver2.example.com.
example.com. 320069 IN NS nserver.euro.example.com.
example.com. 320069 IN NS nserver.example.com.
example.com. 320069 IN NS nserver3.example.com.
example.com. 320069 IN NS nserver4.example.com.
example.com. 320069 IN NS nserver.asia.example.com.
;; ADDITIONAL SECTION:
nserver.example.com. 56190 IN A 17.111.10.50
nserver2.example.com. 57247 IN A 17.111.10.59
nserver3.example.com. 57581 IN A 17.22.14.50
nserver4.example.com. 57452 IN A 17.22.14.59
;; Query time: 11 msec

Impact of Clustering in Other Expressway Applications

Conference Factory (Multiway™)
When using Conference Factory (Multiway) in a cluster, note that:
• The Conference Factory application configuration is NOT replicated across a cluster.
• The Conference Factory template MUST be DIFFERENT on each of the Expressway peers.

When configuring a cluster to support Multiway:

1. Set up the same Conference Factory alias (the alias called by the endpoint to initiate a Multiway conference)
on each peer.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

57
 Reference
Microsoft Interoperability

2. Set up a different Conference Factory template on each peer (so that each peer generates unique Multiway
conference IDs).

For example, if the MCU service prefix for ad hoc conferences is 775 then the primary Expressway may have
a template of 775001%%@domain, peer 2 a template of 775002%%@domain, and peer 3 a template of
775003%%@domain. In this way, whichever Expressway serves the conference ID, it cannot serve a conference
ID that any other Expressway could have served.
The same applies across a network. If there is more than one Expressway or Expressway cluster that provides
Conference Factory functionality in a network, each and every Expressway must provide values in a unique
range, so that no two Expressways can serve the same conference ID.
See Cisco TelePresence Multiway Deployment Guide for more information.

Microsoft Interoperability
If Microsoft infrastructure is deployed with an Expressway cluster, see Expressway and Microsoft Infrastructure
Deployment Guide.

Cisco Expressway Cluster Creation and Maintenance Deployment Guide (X14.3)

58