Webex For Cisco Broadworks Troubleshooting Guide: Americas Headquarters
Webex For Cisco Broadworks Troubleshooting Guide: Americas Headquarters
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
© 2021 Cisco Systems, Inc. All rights reserved.
CONTENTS
Contacts 3
Useful Log Files 3
Reading List 4
Known Issues and Limitations 5
Serviceability Connector 5
Escalating an Issue 7
What Client Information to Collect 7
Check User Details in Help Desk 8
View Customer Organization in Help Desk 8
Retrieve User Logs from Partner Hub 8
How to Find Client Version 9
Client Check for Calling Service 9
Get Client Logs or Feedback 10
Get Calling Environment Data 10
Reset Webex Database 11
Verify that Webex Should Register to BroadWorks 11
Analyze PSLog for User Provisioning Issues 14
Analyze XSP Logs to Troubleshoot Subscriber Log in 15
Change History
Date Version Change
July 15, 2021 1.11 Added error message 100006 to End User Error Codes. Also
updated Users Sign In Issues.
July 14, 2021 1.10 Added topic with link to Known Issues and Limitations article
July 02, 2021 1.9 Updated product name for Webex rebranding.
June 08, 2021 1.7 Added Suggested Action column to End User Error Codes table
June 04, 2021 1.6 Correction to End User Error Codes table
April 22, 2021 1.4 Updated End User Error Codes with two additional codes: 200016
and 200054
December 08, 2020 1.2 Updated document. Rebranding Webex Teams to Webex (app).
Added End User Error Codes
Contacts
Note Starting in October 2020, we are migrating BroadSoft customer support to Cisco CX support processes and
tools. This means that Webex for Cisco BroadWorks partners need to move from using Xchange for case
management to using Support Case Manager (SCM).
We expect the migration run for approximately 3 months and through the end of calendar year 2020. The
BroadWorks/UCaaS TAC team will start supporting cases in CSOne / Lightning instead of BroadSoft Jira
when you are migrated over. You may need to refer to cases in both systems during the migration period.
See Legacy BroadSoft Support Transition for details.
Reading List
• Webex for Cisco BroadWorks Partner Knowledge Portal: https://www.cisco.com/go/wbx4bwks.
• XSP Platform Configuration Guide (R23): https://xchange.broadsoft.com/node/1033484
• BroadWorks Software Management Guide (R23): https://xchange.broadsoft.com/node/1034035
• Cisco BroadWorks Device Management Configuration Guide (R23):
https://xchange.broadsoft.com/node/1031995
• Broadworks Xsp Command Line Interface Administration Guide:
https://xchange.broadsoft.com/node/1034192
• Long-Lived Authentication Token Feature Description Release 23.0 https://xchange.broadsoft.com/node/
1027115
• SAML Authentication integration guide, R22: https://xchange.broadsoft.com/node/1031642
• Cisco BroadWorks SSL Support Options Guide: https://xchange.broadsoft.com/node/1032051
• Cisco CI Support Feature Description: https://xchange.broadsoft.com/node/1052408
• Notification Push Server (Feature Description): https://xchange.broadsoft.com/node/485737
Serviceability Connector
The Webex Serviceability service increases the speed with which Cisco technical assistance staff can diagnose
issues with your infrastructure. It automates the tasks of finding, retrieving and storing diagnostic logs and
information into an SR case. The service also triggers analysis against diagnostic signatures so that TAC can
more efficiently identify and resolve issues with your on-premises equipment.
For details on how to deploy the Serviceability Connector, go to Deployment Guide for Cisco Webex
Serviceability Connector at https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cloudCollaboration/spark/
hybridservices/serviceability/cmgt_b_deployment-guide-spark-hybrid-service-connector.html.
Escalating an Issue
After you have followed some of the troubleshooting guidance, you should have a reasonable idea of where
the issue is rooted.
Procedure
Step 1 Collect as much information as you can from the systems related to the issue
Step 2 Contact the appropriate team at Cisco to open a case (see Contacts section)
• User identifier: CI email address or User UUID (this is the Webex identifier, but if you also get the user's
BroadWorks identifier, that will help)
• Organization identifier
• Approximate time frame during which the issue was experienced
• Client platform and version
• Send or collect logs from the client
• Record the tracking ID if shown on client
Step 4 Click Copy if you need to use this information in another tool, or attach it to a Cisco case.
Procedure
Step 3 Open the Settings/Preferences menu and go to the Phone Services section. You should see the status SSO
Session You're signed in.
(If a different phone service, such as Webex Calling, is shown, the user is not using Webex for Cisco
BroadWorks.)
This verification means:
• The client has successfully traversed the required Webex microservices.
• The user has successfully authenticated.
• The client has been issued a long-lived JSON web token by your BroadWorks system.
• The client has retrieved its device profile and has registered to BroadWorks.
Note Send logs is silent. However, if a user sends feedback, it goes to the Webex App devops team. Be sure to
record the user’s feedback number if you want to follow up with Cisco. For example:
Procedure
Step 1 In the client, click the profile picture, then click Help > Export Calling Environment Data.
Step 2 Save the resulting file ced.dat for troubleshooting calling issues for this user.
Important: Logout from or restart the client clears the internal cache. If you export ced.dat after that, the
exported data will not correspond with any logs that were sent before the cache.
Note A Webex for Cisco BroadWorks user should NOT have the bc-sp-standard entitlement if they are intending
to use Webex for Cisco BroadWorks. This is the entitlement for “Webex Calling (Broadcloud)” which is
Webex app calling through a Cisco-managed cloud calling service.
Procedure
Step 2 You can also check the original HTTP POST for any suspect values that could cause provisioning to fail.
The POST contains a subscriberProfile XML structure. Inside this, useful nodes to check are:
• bwuserid: Use this to find the subscriber profile if you need to edit it in BroadWorks.
• group: If the template is in "Service Provider mode", this is lowercased and becomes the name of the
Customer org you see in Partner Hub.
• serviceProvider: If the template is in "Enterprise mode", this is lowercased and becomes the name of
the Customer org you see in Partner Hub.
• primaryPhoneNumber: Must exist. Provisioning fails without it.
• email: Becomes the user ID in Webex. Must be valid and unique to Webex, otherwise provisioning fails.
Note Ignore the services stanza: it is created by AS, and accepted but not used by Webex.
GET /com.broadsoft.xsi-actions/v2.0/user/[email protected]/profile
In the XsiActionsLog:
• Look for the profile GET request from Webex (2.1 in diagram). It has the Webex user ID. E.g.
GET /com.broadsoft.xsi-actions/v2.0/user/[email protected]/profile
• The XSP then does OCI-P Basic authentication against BroadWorks (AuthenticationVerifyRequest and
AuthenticationVerifyResponse, like any other application doing basic authentication via Xsi) and also
a UserGetRequest and ServiceProviderGetRequest to gather the subscriber information.
• The Xsi response to Webex contains an XML Profile block containing the (BroadWorks) userId and
other details (2.2 in diagram).
and StatusCode=200 which you can associate with the original request using the trackingid: CLIENT…
header.
In the XsiActionsLog:
• The client is now able to present the long-lived token at Xsi-Actions interface to get its device profile (6
in diagram). E.g.:
GET /com.broadsoft.xsi-actions/v2.0/user/bwksUserId%40example.com/profile/device
With the headers authorization: Bearer token and user-agent: WebexTeams (variant/version)
• The Xsi-Actions interface POSTs the token to the authservice (configured to be on the loopback interface)
e.g.: 127.0.0.1:80 POST http://127.0.0.1:80/authService/token
which you can correlate with the trackingid: CLIENT… header in the GET and the
X-BROADSOFT-CORRELATION-ID : CLIENT… header in the POST.
In the XsiActionsLog:
• Having received 200 OK from authservice, which validated the client’s token, the Xsi-Actions application
now sends OCI-P request for UserPrimaryAndSCADeviceGetListRequest
• Receives OCI-P UserPrimaryAndSCADeviceGetListResponse containing the accessDeviceTable XML
structure.
• The OCI-P response is encoded as Xsi response to client, including AccessDevices XML structure,
which has the deviceTypes e.g. Business Communicator – PC and the urls where the client can retrieve
the device configuration files.
BroadWorks Enterprise Integrated IM&P setting Partner Hub Customer Template setting
This failed provisioning request should be creating a new Customer Organization in Webex, but
your template is configured to prevent new Customer Organizations to be created. If you want to
allow new organizations, for email domains that do not match existing customers in Webex, then
you can reconfigure your template in Partner Hub and retest the provisioning request. However, if
you are not expecting a new organization to be created for this user, perhaps the email address is
mistyped (specifically the domain part). Check the user’s email id in BroadWorks.
403 Forbidden
The provisioning request has no chance of succeeding. You will need to investigate the request and
response in this case. For example, if you see an IP address as the target of the provisioning request –
instead of the appropriate provisioning bridge URL for your organization (see the firewall configuration
topics in Solution Guide) – it could indicate that your Application Server is missing a required patch
(ap373197).
Check that all required patches are applied to the Application Server, and that you completed the related
configuration for successful flowthrough provisioning.
409 Conflict
The provisioning request cannot proceed because there is an existing user in Webex that matches the
email address in the request.
User Already in CI
Get the subscriber email out of the HTTP POST request and search for it in Help Desk.
You may not see the user if you are not permitted, but you may also see that the user is in a ‘free’ organization
e.g. “Consumer”.
You can ask this user to delete their free account, or you can use a different email address to provision them.
See https://help.webex.com/ndta402.
Make sure the user has entered upper case and lower case characters correctly.
This is an unsuccessful registration. There are multiple reasons why a Webex app client would fail registration
with BroadWorks:
Multiple Calling Services Being Tested with Same Clients
This known issue can be caused by the client changing between different calling back ends. It is most likely
to occur during trials of different calling services offered via (the same) Webex app clients. You can reset the
client database (link) to remedy this issue.
Misconfiguration of Authentication Service
Check the XSP(s) hosting the authentication service against the Solution Guide (see Configure Services on
your Webex for Cisco BroadWorks XSPs). Specifically:
• The RSA keys (that you generate on one XSP) are copied onto all the XSPs
• The authentication service URL has been provided to the web container on all XSPs, and entered correctly
in the cluster in Partner Hub
• External authentication by certificates is configured:
XSP_CLI/System/CommunicationUtility/DefaultSettings/ExternalAuthentication/CertificateAuthentication>get
allowUserApp = false
allowClientApp = true
• When using MTLS, you must upload the Webex client certificate to the XSPs (you can getthe certificate
from Partner Hub, on the BroadWorks Settings page)
• Verify the required client configuration template is deployed and CSW tags are properly set. (See the
Call Settings Webview section in the Webex for Cisco BroadWorks Solution Guide).
• Verify the Webex app is registered for calling in BroadWorks.
• Check that the Webex app is a recent version that supports CSWV.
If you claim any domains in error and need to remove a claim, refer to the Manage Your Domains Webex
article.
Note This is not an exhaustive list of error codes. The table lists only existing error codes for which the Webex app
does not currently provide clear direction to the user.
100006 Login failed: User ID/Password is incorrect Check that the user is using the correct
password. If the template that you
used to provision the user's Customer
Organization (in Partner Hub) is
configured for BroadWorks
authentication, the user should be
entering their BroadWorks "Web
Access" password. The user may also
need to enter their BroadWorks User
ID if their email address is not
configured as an Alternate User ID.
Make sure the user has entered upper
case and lower case characters
correctly.
200010 Failed validating credentials as BroadWorks user User should try a different username
unauthorized and password combination.
Otherwise, admin must reset password
in BroadWorks.
200016 Failed validating credentials as session not found User should refresh browser and retry
the username/password.
200018 Failed validating credentials as user is locked out User should wait 10 minutes and then
try again.
200019 Failed validating credentials as add user failed for self Admin should check self-activation
activation settings in Control Hub
200022 Failed to send email as user is unauthenticated User should retry onboarding and
entering credentials.
200026 Failed validating email due to pre-check failure or Admin should inform the user that
pending user incorrect state for PartnerOrgUUID : they entered the wrong email address
{partnerOrgUUID} , BroadoworksUUID : as the email address is associated to a
{broadworksUUID} , ConfigSetUUID : different organization.
{configSetUUID}
200039 Failed validating email as emailId already in use in a User should try onboarding again to
different Org the same verification link, but using
a different BroadWorks User ID.
Otherwise, the customer organization
administrator from the different org
should delete the existing user
account.
200040 Failed validating email as configSet is not matching Admin should compare the
with configSet in customerConfig verification link that the user used
against the link that's configured in
Control Hub. The two links and
configSets must match.
200041 Failed validating email as user is already entitled for User should try onboarding again to
another conflicting service, conflicting entitlements the same verification link using a
different BroadWorks User ID.
Otherwise, the customer org
administrator who manages the
conflicting service should delete the
conflicting service or entitlements.
200042 Failed validating email as email is already associated User should try again with different
with another BroadWorks UserId email address.
Otherwise, admin must delete the
other user that uses this email address.
200043 Failed validating email as user customer config User should try again with different
mapping is incorrect email address.
Otherwise, admin must delete the
other user that uses this email address.
200044 Failed validating email as userId is already in use on User should try again with different
this BroadWorks cluster email address.
Otherwise, the customer organization
administrator who manages the
existing user account that uses this
email address must delete that user
account.
200045 Failed adding user through self activation as user is User should retry onboarding, but with
already part of a different org a different email address.
Otherwise, the customer organization
administrator who administers the
different org should delete the existing
account.
200046 Failed adding user through self activation as multiple Admin should delete the pending users
pending users exists with same broadworksUserId from Control Hub
under same BroadWorks cluster
200047 Failed adding user through self activation as userId User should try again with different
is already in use on this BroadWorks cluster email address.
Otherwise, the customer organization
administrator who manages the
existing user account should delete
that existing user account or remove
other entitlements.
200048 Failed adding user through self activation as email User should try again with different
address was already provisioned with a different email address.
BroadWorks userId
200049 Failed adding user through self activation as userId User should try again with different
is already in use on this BroadWorks cluster email address.
Otherwise, the customer organization
administrator who manages the
existing user account should delete
that existing user account or remove
other entitlements..
200050 Failed adding user through self activation as The admin should compare the
provisioningID doesn't match expected provisioningID verification link that the user used
of subscriber's enterprise against the link that's configured in
Control Hub.
The two links and configSets must
match.
200051 Failed adding user through self activation as The admin should check existing orgs
spEnterpriseId specified in this request conflicts with in Control Hub and make sure that
a Service Provider or Enterprise already provisioned they are not creating an org with a
from this BroadWorks Cluster name that already exists.
200054 Failed validating email as the region of the customer The admin should check the partner
org and partner org mismatch org and customer org settings in
Control Hub and make sure that the
regions match.