SAPT DC Network Migration

High Level Design

28 Dec 2023

Enabling Technology
Objectives

The objectives of this presentation are to discuss;

Project Scope
The existing Datacenter Network design/topology of SAPT Network architecture
High Level Design for SAPT DCN Revamp
Edge Firewall design
DC FW Design
TOR Switching Design
High Availability / Load Balancing
Management and Security Features
Proposed Design Decisions
Migration Strategy
Proposed Project Plan
Project Scope

Supply of DC/Edge/Access Network and NGFW equipment as per the required BoQ at Karachi SAPT Port
Design of DCN Architecture
Physical – HA within Primary DC
Logical – L2/L3 Topology
TOR Switching Design – Loop free topology
Optimal Edge/DC NGFW Design with no or minimal down time
High Availability
Use Cases
Traffic Flows
Migration Strategy
Phased Approach
Transit to End State
Documentation of proposed design and changes
Deployment and Migration of Edge/DC NGFW and TOR switches at SAPT Network environment
Deployment of Huawei SecoManager and AAA
Integration with existing Core, Access LAN and NMS
Industry best practices for implementing recommended security features
Existing Network Design

Currently 2 DC sites, Primary and Failover, both located at Karachi Port

Terminal
Primary DC core switching layer consists a pair of Cisco Nexus 9504
switches with VPC configuration
DC servers connected directly to the TOR switches i.e. Cisco Nexus 9372
TOR switches connects to core layer using VPC multiple 10G uplinks
A pair of DC firewalls i.e. Cisco ASA 5585 in Active/Standby mode is directly
connected with the core layer
DC Firewalls currently act as gateway for Server Farm
Existing Network Design

OSPF routing protocol is widely used, while a limited static routing is also
configured
Edge segment consist a pair of Cisco ASA 5516 firewalls in Active/Standby
mode
A partial implementation of BGP exists at Internet Routers with 3 x ISPs
(PTCL, Cybernet & TWA)
Existing Network Topology

TOWARDS HONG KONG

TOWARDS KICTL
ISP 1 ISP 2 ISP 3
PTCL CYBERNET TWA

DMZ SEGMENT
PRIMARY DATACENTER
EDGE FIREWALL
CISCO ASA 5516

B2B SEGMENT
ACTIVE/STANDBY

DC FIREWALL
CISCO ASA 5585

ACTIVE/STANDBY

CISCO NEXUS CORE SWITCH 9K IT/OT SEGMENT

F5 LB

Peer Link

Keep Alive Link

F5 LB
VPC

CISCO NEXUS TOR SWITCH 9K CISCO NEXUS TOR SWITCH 9K CISCO NEXUS TOR SWITCH 9K

Peer Link Peer Link Peer Link

Keep Alive Lin k

DC FAILOVER/
Keep Alive Lin k
SECONDARY
Keep Alive Lin k

VPC VPC VPC

DATACENTER
SERVER FARM

DISTRIBUTION SWITCHES

ACCESS SWITCHES
Proposed High Level Network Topology Routers
replaced by

TOWARDS HONG KONG

ISP 1
PTCL
ISP 2
CYBERNET
ISP 3
TWA Huawei
i-Stack Switches Edge Firewalls
HUAWEI S5731
PRIMARY DATACENTER
EDGE SWITCHES
Migration
DMZ SEGMENT HUAWEI USG6635
EDGE FIREWALL

ACTIVE/STANDBY
eBis Firewall
HUAWEI USG6685F
DC FIREWALL
eBis-FW
CISCO ASA 5516
Migration

ACTIVE/STANDBY
DC Firewalls
CISCO NEXUS CORE SWITCH 9K IT/OT SEGMENT
Peer Link
F5 LB
Migration
Keep Alive Link
F5 LB
VPC

i-Stack i-Stack
TOR Switches
Migration
i-Stack

DC FAILOVER/
HUAWEI TOR SWITCH HUAWEI TOR SWITCH
CE6881 CE6881
HUAWEI TOR SWITCH
CE6863E
SECONDARY
DATACENTER
SERVER FARM

Huawei
SecoManager
SECOMANAGER - VM AAA iMaster NCE

DISTRIBUTION SWITCHES & AAA

ACCESS SWITCHES
Existing Edge Design
ISP 1 ISP 2 ISP 3
PTCL CYBERNET TWA

CISCO ASA5516
EDGE FIREWALL

DMZ SEGMENT
ACTIVE/STANDBY

CISCO NEXUS CORE SWITCH 9K

Peer Link

Keep Alive Link

VPC
Proposed Edge Design
ISP 1 ISP 2 ISP 3 INTERNET
PTCL CYBERNET TWA

i-Stack
CISCO
DUO MFA SSL REMOTE VPN
SSL REMOTE VPN
BGP AS#
HUAWEI S5731
• BGP Configuration
EDGE SWITCHES • Routing Policies
• Traffic Filtering
Eth-Trunk Eth-Trunk

HUAWEI USG6635
EDGE FIREWALL
DMZ SEGMENT • Policies/Services Migration
• SSL VPN with MFA

TOWARDS CORE SWITCH

ACTIVE/STANDBY
ACTIVE STANBY
 DC FIREWALL
Existing DC FW Design CISCO ASA 5585

ACTIVE/STANDBY

CISCO NEXUS CORE SWITCH 9K

Peer Link

Keep Alive Link

VPC

CISCO NEXUS TOR SWITCH 9K CISCO NEXUS TOR SWITCH 9K CISCO NEXUS TOR SWITCH 9K

Peer Link Peer Link Peer Link

Keep Alive Link Keep Alive Link Keep Alive Link

VPC VPC VPC

SERVER FARM
PROPOSED DC FW Design
HUAWEI USG6685F DC FIREWALL

FAILOVER LINK/HEARTBEAT

VRRP(Virtual Router Redundancy Protocol) STANDBY

ACTIVE
Eth-Trunk LACP

• OSPF Routing CISCO NEXUS CORE SWITCH 9K

• Policies Optimization

North South Traffic

Peer Link

Keep Alive Link

VPC
East West Traffic Default Gateway DC FW

SERVER FARM

SECOMANAGER - VM AAA iMaster NCE

PROPOSED TOR Switching Design
CISCO NEXUS CORE SWITCH 9K

Peer Link

• Virtualized Stack (i-stack)

• Ether-Trunking VPC
Keep Alive Link
• Loop-free Topology

10G 10G
Eth-Trunk LACP 10G
i-Stack i-Stack

Eth-Trunk LACP
Eth-Trunk LACP

10G

HUAWEI CE6863E HUAWEI CE6881

FIBER TOR SWITCH COPPER TOR SWITCH

SERVER FARM

SECOMANAGER - VM AAA iMaster NCE

High Availability

Failover Scenarios
Primary DC
Active Edge Firewall Fails – Traffic will continue to flow from Standby
FW
Primary ISP Link Fails - Traffic will continue to flow from secondary ISP
Internet Switch Fails - Traffic will continue to flow from secondary
switch in virtualized stack
Active DC FW Fails - Traffic will continue to flow from Standby DC FW
Primary TOR Switch Fails - Traffic will continue to flow from Secondary
switch in virtualized stack
In case of dual failure of a critical component e.g. both DC FW fail –
manual intervention may be required to run the services from
secondary/failover DC
Load Balancing

Load Balancing of traffic will be achieved at DC switching layer by utilizing

multiple 10G uplinks in LACP/Ether-Trunk mode while avoiding congestion
and giving better user experience
Load balancing of internet traffic (incoming & outgoing) will be configured
at Edge switching layer using BGP routing policies
In case of failure of any link, the available link will take the full load
No load balancing at FW layer (either DC or Edge) as HA mode is
Active/Standby
Security and Management Features

Users Authentication (AAA) via Radius / TACACS+ (iMaster NCE)

Firewall Policies Optimization (SecoManager)
BPDU / Root Guard
UDLD (Unidirectional Link Detection)
BCP38 and Bogon filtering
SNMPv3
SSHv2
Syslog
Best security practices for device hardening e.g. Management, Control and
Forwarding plane
Design Decisions

Design Decision 1: NAT will be performed at Huawei Edge Firewall. Currently it is being done at Internet Routers

Design Decision 2: BGP configuration and routing policies will be done at Huawei Edge Switches

Design Decision 3: Edge switches will form Ether-Trunk (Port Channel) with ISPs if possible, to provide full
redundancy of switch failure

Design Decision 4: Separate Ether-Trunks (Port Channels) for each DC firewall at Core layer

Design Decision 5: Virtualized stacking i.e. Huawei i-Stack will be used at TOR switching layer, forming Ether-
Trunk (Port channel) with the Cisco VPC at core switches
Migration Strategy

Phased Approach – The Strategy is to have the Parallel setup as much as possible while
maintaining the zero or minimal downtime for services migration –
Description of Multiple milestones

Phase 1 – Deployment of Internet Edge design (FW & Internet Switches) – Service
Migration

Phase 2 – Deployment of TOR switches – Service Migration

Phase 3 – Deployment of DC FW – Service Migration

Proposed Design Changes to achieve the smooth transition
End State Design
Proposed Project Plan
Proposed Project Plan
Thank you!