Inside Cisco IT: Saving Costs and Reducing

Complexity by Consolidating Voice Circuits using SIP

Greg Moon - Sr. IT Manager | Kees Gerritsen – Sr. IT Engineer

BRKCOC-2008
Agenda

• SIP Technology Primer

• Introduction: Plan and Scope
• Selling the Business Case
• Engineering and Implementing the Solution
• CUBE: Cisco Unified Border Element

• Lessons Learned
• Conclusion
Who are We?
Greg Moon Kees Gerritsen

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
SIP Technology Primer
 Session Initiation Protocol
What is SIP? (SIP) is an application-layer
control (signaling) protocol
for creating, modifying, and
terminating sessions with
one or more participants.

These sessions include

Internet telephone calls,
multimedia distribution, and
multimedia conferences.

- RFC 3261
BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
What is SIP in This Context?

SIP is the protocol used to provide delivery of large

capacity data pipes to carry in and outbound carrier
traffic to and from Cisco

SIP circuits replace legacy Telecom circuits such as PRI,

BRI, ISDN, etc.

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Why SIP, Why Now?
Why Should You Care?

 Service Provider models are changing

 Dispersed and unique to centralized

and standardized
How?

 Shrink footprint and costs

 Scalability in real time

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Plan and Scope
Introduction: Where We Began

Replace Legacy Voice at Cisco: 120 US Locations

• PRIs terminated at each location from ~17 different providers
• DIDs provided independently
• Invoicing inconsistent

• WAN used for internal call routing and Tail-end Hop Off (TEHO)
• SRST used for local site recovery
• Two existing US-based SIP trunks used for outbound LD and some
WebEx traffic

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Before
Branch
Office

Pipe PR
Data I

Cisco WAN PSTN Inbound and

Outbound
D at I
a Pi PR
pe
Branch
Office

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
After
Branch
Office

Pipe
Data

Cisco WAN

D at
a Pi
pe
SIP Trunk Branch
Office

Provider SIP
Cloud Inbound and Outbound PSTN

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
Introduction: Defining the Scope

• 3 US-based Termination points to aggregate traffic

• Strong desire to use a Single Carrier for cost effectiveness
and technology consistency
• ~120 offices on the target list

* Top Tip – we started “local” to help us learn.

Other Global regions followed

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
Selling the Business Case
Selling the Business Case: Overview and Benefits
What are the Operational Gains?
• Simplified Architecture and resiliency for inbound calls
• Centralized Administration – Reduced Ops complexity
• Adaptive Capacity – Scale bandwidth up or down in real time

What is the Business Value?

• Single Service Provider = better rates
• Centralized Hardware
• Centralized circuits

Why is the Technology Important?

• Disaster Recovery – in our hands!
• Consistency – all sites functionally the same
• Future-proofing

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Selling the Business Case: Caveats and Risks

• The WAN is the Lynchpin!

• No SRST - mobile phone becomes default backup
during WAN outages
• Some sites may need more (or better) bandwidth
• Emergency (911,112, 999) requirements
• Service Provider Compatibility
• Support team’s learning curve
• Have a back-out plan!

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Selling the Business Case: Dollars and Cents

Cost Element Today Tomorrow Delta

PRIs + Usage $100,000 $10,000 - $90,000

SIP Circuits $0 $10,000 + $10,000

SIP Usage $0 $30,000 + $30,000

1-Time Costs $0 $25,000 + $25,000

Total Year 1 - $25,000

20-30%
Over Total Year 2 - $50,000
Time

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Selling the Business Case: Things to Remember

• Cost savings potential is there, but

will take multiple quarters or
years to realize, depending on
scope and complexity
• WAN reliability is crucial
• Emergency Services must be
addressed

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Engineering and Implementing the
Solution
Engineering the Solution: US Overview

• Cisco User base in America, 40% of company

• Very big campus in San Jose CA
• 5 Smaller campus locations
• 100+ Field sales offices
• 75% of outbound peak traffic is WebEx Conference
• Peak 6400 Concurrent calls

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Engineering the Solution: Parameters

Some of the Solution Design parameters:

• Redundant – survive ‘the worst scenario’
• Multiple Locations
• Multiple Regions
• Ability to serve 100+ offices and high volume traffic
• Align with our existing UCM Cluster Infrastructure
• Ability to grow – shrink, multiple SP
• QOS, Voice, 911 and Fax Support
• ROI
BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Engineering the Solution: Parameters

• Cisco has 3 large Datacenters in Texas – California – NC

• 4 UCM Clusters are in these locations
• Carrier Neutral Facilities (CNFs) are Touchpoint for N&T providers
• CNF using Cisco ONS, providing L1,2 dark fiber rings to IT DC, IDC and NOC’s
• CAPnet, world wide production network, ring 10 GB network in USA
• QOS Voice, Video and Signaling

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Engineering the Solution: Direction for Field Offices

Cisco Branch Offices

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
Engineering the Solution: The Idea
BEFORE Outbound – (Int-)National AFTER In & Outbound – (Int-)National
Local Telco – DID Range All office – DID Ranges

IP PSTN
IP PSTN

Cisco Campus
Cisco Campus

WAN
WAN

Local Telco – DID Range

SRST
SRST Message
PSTN No local
PSTN

100> Cisco Branch Offices

100> Cisco Branch Offices
BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
CUBE: Cisco Unified Border
Element
CUBE: Interoperability
• Validated with Service
Providers World-Wide
• Independently Tested
with 3-Party PBXs in
tekVizion Labs
• Standards based
Verified by

Proven Interoperability and Interworking with

Service Providers Worldwide
Cisco Interoperability Portal:
[Link]/go/interoperability

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
CUBE: Enterprise Product Portfolio
50-150 ASR 1004/6 RP2

ASR 1002-X
50-100
ASR 1001-X

ISR 4451-X
ISR 4431
20-35 3900E Series ISR-G2
(3925E, 3945E)
CPS

ISR 4351 ASR1004 RP2 Selected

17

3900 Series ISR-G2 (3925, 3945)

• Networking Standard HW
ISR-4K (4321, 4331)
• Spares – Growth - Mgt
8-12

2900 Series
ISR-G2 (2901, 2911, 2921, 2951)

<5

800 ISR

4 <50 500-600 900-1000 2000-2500 4000 4500-6000 7000-10,000 12K-14K 14-16K

Active Concurrent Voice Calls Capacity

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
CUBE: Understanding the Call Flow
Incoming VoIP Call Leg Outgoing VoIP Call Leg
Matches an Incoming Dial-peer Matches an Outbound Dial-peer

[Link] CUBE [Link]

1000 2000
INVITE /w SDP
[Link] INVITE /w SDP [Link]
c= [Link]
c= [Link]
m=audio abc RTP/AVP 0
m=audio xxx RTP/AVP 0
100 TRYING 100 TRYING

180 RINGING 180 RINGING

200 OK
200 OK c= [Link]
c= [Link] m=audio uvw RTP/AVP 0
m=audio xyz RTP/AVP 0 ACK
ACK

RTP (Audio)
[Link] [Link] [Link] [Link]
BYE BYE
200 OK
200 OK
BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
CUBE: High Availability Options
• Inbox redundancy
• ASR 1006,
• Stateful failover
• Local redundancy
• Preserves signaling & media

• L2 Box-to-Box redundancy
Active
• ISR G2/4K (Stateful failover)
SIP SP
• ASR 1001-X/2-X/4/6 (Stateful failover)
• Local redundancy (Both routers must be CUBE
Virtual
IP
physically located on the same Ethernet LAN)
• Not supported across data centers CUBE
• Preserves both signaling and media Standby

• Clustering with load balancing

• All platforms
• Load balancing by
• SP call agent SIP SP
• Cisco Unified SIP Proxy CUSP CUSP
• Local and geographical redundancy

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
CUBE: Call Access Control (CAC) Enhancements
CUBE provides various CAC mechanisms to safeguard your network from SIP based attacks and to enforce policies based on:
• Total calls • Maximum connections per destination
• CPU & Memory • Dial-peer or interface bandwidth
• Call spike detection

Total Calls, Call Spike call spike call-number [steps

High Water Mark Detection number-of-steps size milliseconds]
CPU, Memory call spike 10 steps 5 size 200
Low Water Mark

CUBE CUBE

call threshold global [total/mem/cpu] calls low xx high yy If a call spike is detected,
call treatment on reject calls

Max Calls per Max Bandwidth Call #3 Rejected

Call #3
Destination Rejected by based by CUBE
Call #1 CUBE Call #1 – 80Kbps
Call #2 Call #2 – 80 Kbps
Call #3 CUBE Call #3 – 80 Kbps
CUBE

dial-peer voice 1 voip dial-peer voice 1 voip

max-conn 2 max-bandwidth 160

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
CUBE: Enhancements
Multiple Incoming Patterns Under Same Incoming/Outgoing Dial-peer
voice class e164-pattern-map 100
e164 4071002...
e164 4081001... Provides the ability to combine multiple
Site A (407)100-2xxx e164 4091001...
destination-patterns targeted to the
(408)100-1xxx dial-peer voice 1 voip same destination to be grouped into a
Site B
destination e164-pattern-map 100 single dial-peer
codec g711uLaw Up to 5000 entries in a text file
Site C (409)100-1xxx
session target ipv4:[Link]

A SIP Trunk SP SIP Trunk IP PSTN

CUBE

(
! This is an example of the contents

)
Site A voice class e164-pattern-map 200
(919)200-2010 url flash:[Link] of E164 patterns text file
stored in flash:e164-pattern-
Site B (510)100-1010 dial-peer voice 1 voip [Link]
destination e164-pattern-map 200
codec g711ulaw 9192002010
Site C (408)100-1010
session target ipv4:[Link] 5101001010
4081001010
G711 Sites
BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
CUBE: Enhancements
Destination Server Group
• Supports multiple destinations (session targets) be defined in a group and applied to a single outbound
dial-peer
• Once an outbound dial-peer is selected to route an outgoing call, multiple destinations within a server
group will be sorted in either round robin or preference [default] order
• This reduces the need to configure multiple dial-peers with the same capabilities but different
destinations. E.g. Multiple subscribers in a cluster

voice class server-group 1 dial-peer voice 100 voip

hunt-scheme {preference | round-robin} description Outbound DP
ipv4 [Link] preference 5 destination-pattern 1234
ipv4 [Link] session protocol sipv2
ipv4 [Link] port 3333 preference 3 codec g711ulaw
ipv6 [Link] port 2323 preference 3 dtmf-relay rtp-nte
ipv6 [Link] port 2222 session server-group 1

* DNS target not supported in server group

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
CUBE: Enhancements
OOD OPTIONS Ping Keepalive
voice class sip-options-keepalive 1 Sample Show command output
description UDP Options consolidation CUBE#sh voice class sip-options-keepalive 1
down-interval 49
Voice class sip-options-keepalive: 1 AdminStat: Up
up-interval 180
retry 7 Description: UDP Options consolidation
Single OOD Option
transport udp Ping Group applied Transport: udp Sip Profiles: 0
to multiple dial-peers Interval(seconds) Up: 180 Down: 49
dial-peer voice 1 voip with same session
Retry: 7
destination-pattern 6666 targets
session protocol sipv2
session target ipv4:[Link] Peer Tag Server Group OOD SessID OOD Stat IfIndex
voice-class sip options-keepalive profile 1 -------- ------------ ---------- -------- -------
1 4 Active 9
dial-peer voice 2 voip
destination-pattern 5555 2 4 Active 10
session protocol sipv2 OOD SessID: 4 OOD Stat: Active
session target ipv4:[Link] Target: ipv4:[Link]
voice-class sip options-keepalive profile 1
Transport: udp Sip Profiles: 0

• With OOD Options Ping Keepalive group, an options ping keepalive connection is established on per remote target base as opposed
an options ping keepalive connection established per dial-peer basis. Up to 10,000 “voice class sip-options-keepalive <tag>” can be
defined per system

• Either legacy “sip options-keepalive” or the new “sip options-keepalive profile <tag>” can be configured on a dial-peer. Dial-peers with
Destination Server Group instead of Session Target IP must use Options Keepalive Profile and not the legacy CLI.

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
CUBE: Enhancements
HA ASR Design Box-to-Box Redundancy
Single Attached Dual Attached
Active Active
GE0/0/2 GE0/0/0
SIP SP GE0/0/1 GE0/0/0

CUBE CUBE
Virtual GE0/0/2 Virtual
IP Virtual
IP IP
SIP SP
GE0/0/2

GE0/0/2 CUBE GE0/0/0

Standby GE0/0/1 GE0/0/0
CUBE
Standby

• ISR4K/ASR Uses Redundancy Group (RG) Infrastructure Protocol – ISR G2 uses HSRP
• Only active calls are checkpointed (Calls that are connected - 200OK / ACK transaction completed)
• GE0/0/0 (and GE0/0/1) are referred to as traffic (SIP/RTP) interfaces and GE0/0/2 is RG
(Redundancy Group) Control/data interface
• RG Control/Data Interface (GE0/0/2) can be a Port channel interface starting IOS-XE 16.3.1
• All signaling/media is sourced from/to the Virtual IP Address
• When configuration is applied and saved, the box must go through a reload cycle
BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Engineering the Solution: Region Overview

• CPE: ISR 4451

• SBC: ASR 1004
• NAM: Prime 2320 Appliance
• L2&3 Switch: Catalyst 4500-X

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Engineering the Solution: Call flows
Inbound (call towards Cisco)
IP SIP Vendor
• Vendor SBC will use the 3 HA
Telco DC
Telco DC Telco DC
SBC nodes
• Grouping of DN ranges

Outbound
MPLS – IP SIP
• Load balance – round robin
• Traffic steering capability per
CNF – Sunnyvale
CNF CNF CNF
vendor SBC
CAL 3
Level NC Texas
Design around issue:
ITaaC

ITaaC

ITaaC
• Vendor doesn’t detect fail in
California North Carolina Texas
Cisco Network
Campus• Vendor doesn’t mark the
Cisco WAN Backbone destination down
Branch Campus DC Branch Campus DC Branch Campus DC

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
Implementation: Network Resiliency & Traffic Steering

Telco DC Telco DC Telco DC

GRE / BGP Overlay

• Virtual network/routing zone

• GRE Tunnel to interconnect SBC’s
MPLS – SP • BGP to deliver routing intelligence

Packet loss
Benefits
Disaster Disaster
Recovery CNF – Sunnyvale CNF
Recovery CNF
Level 3 • Network resiliency / Disaster Recovery
• Traffic steering capability
ITaaC

ITaaC

ITaaC
California North Carolina Texas GRE / BGP Overlay

Cisco WAN Backbone

Branch Campus DC Branch Campus DC Branch Campus DC

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Implementation: Failover and Redundancy

UCM - Enable Options Ping Trunk, Route List circular, enable Route on all nodes
CUBE HA – one active IP per location, HA B2B failover, Server-groups, SIP-UA tuning

voice class sip-profiles 999

request INVITE sip-header Diversion add
voice class server-group 10 "Diversion: \"Cisco Systems\" = sip-ua
ipv4 [Link] <sip:747412345@[Link]>" retry invite 1
ipv4 [Link] ! retry response 2
ipv4 [Link] voice class sip-options-keepalive 999 retry bye 2
description Server Group for IP PSTN description Options PING List retry cancel 2
hunt-scheme round-robin up-interval 30 connection-reuse
! retry 3
transport udp
sip-profiles 999
!
voice class e164-pattern-map 1091
description Patterns for West Cluster
e164 6055539..
e164 60533333[01].
e164 6116601[23]..
!

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
Implementation: CUBE Security

ACL on interfaces: ip access-list extended sip-white-list

Voice Service Voip: ip address trusted list
DOS protection: Call spike
Route - only existing - owned - e164 numbers to UCM layer
Use Network Appliance Monitor – TranslatorX SIP Messages
Future plan: CUBE network based recording (forking stream)

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Implementation: FAX Services @ Cisco
voice service voip
fax protocol t38 version 0 ls-redundancy 0 hs-redundancy 0 fallback
none
!
voice class e164-pattern-map 78
description Contains all patterns to T38 Fax Server
e164 4103094889
e164 4109538091
e164 7326353057
MFD IP Fax server CUBE IP PSTN !
voice class server-group 10
ipv4 [Link]
4103094889 (Xmedius) description Server Group for IP PSTN
hunt-scheme round-robin
Ipv4: [Link] !
voice class server-group 78
ipv4 [Link] preference 1
ipv4 [Link] preference 2
4109538091 description Preferred Server Group for T38 Fax Server
!
voice translation-profile outgoing-cld
translate called 888
[Link] Ipv4: [Link] !
7326353057 voice translation-rule 888
rule 1 /^8889\(.*\)/ /\1/
!
dial-peer voice 7381 voip
description Inbound FAX from IP PSTN to T38 Fax Server
session protocol sipv2

T.38 T.30 session server-group 78

destination e164-pattern-map 78
codec g711ulaw
fax rate 14400
T.38 T.30 !
ip qos dscp af32 signaling

!
dial-peer voice 8882 voip
description Outgoing FAX calls to IP PSTN
translation-profile outgoing outgoing-cld
destination-pattern 888T
session protocol sipv2
session server-group 10
voice-class sip profiles 999
codec g711ulaw
fax rate 14400
ip qos dscp ©af32
BRKCOC-2008 2016 signaling
Cisco and/or its affiliates. All rights reserved. Cisco Public 41
 Implementation: (E)911 Services

1. TP9911 converts to 911

2. RP 911 routes to CER
3. CER returns ELIN as Calling Party 1. 911 call matches on Calling party ANI,
4. UCM routes call to SIP TRUNK with updated CP 2. gets routed to Local PSAP based on DID address map
User dials:
911 or 9911

Site A Region A
IP PSTN PSAP
UCM
Region B
PSAP

Site B CER
Emergency Location Identification Number (ELIN) is the phone
number (Caller ID), which is associated with an Location in
CER. (Subnet based – SNMP sw layer)
SIP Vendor blocks 911call if
CP ANI is not known! That is presented to the PSAP so they can match the caller ID
number to the Location Information (Caller's Address) and
provide a call back number to the PSAP in case of a call
disconnect. BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
Lessons Learned
Plan and Be Patient!
• Test Plans: Service Provider interop, CUBE, UCM, # ranges
• Use a phased approach to manage cost control
• Understand your contracts with local Bells
• Scaling up or down is much simpler
• Learn SIP and forget older protocols
• IT(s) fun!

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Conclusion
Conclusion: Get Ahead of the Curve

 Service Provider models are changing

 Dispersed and unique to centralized and standardized

 Shrink footprint and costs

 Enable scalability in real time

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
References:
Cisco Cube resource: [Link]/go/cube

Complete feature Presentations, Lab Guide, Free Hands-on Lab access &
Application Notes : [Link]

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
Complete Your Online Session Evaluation
• Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner will
receive a $750 Amazon gift card.
• Complete your session surveys
through the Cisco Live mobile
app or from the Session Catalog
on [Link]/us.

Don’t forget: Cisco Live sessions will be available

for viewing on-demand after the event at
[Link]/Online

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Lunch & Learn
• Meet the Engineer 1:1 meetings
• Related sessions

BRKCOC-2008 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Thank you