Introduction

For more than three decades, network management has been entirely
based on the command-line interface (CLI) and legacy protocols such as
SNMP. These protocols and methods are severely limited. The CLI, for ex-
ample, is vendor specific, lacks a unified data hierarchy (sometimes even
for platforms from the same vendor), and was designed primarily as a
human interface. SNMP suffers major scaling problems, is not fit for writ-
ing configuration to devices, and overall, is very complex to implement
and customize.

In essence, automation aims at offloading as much work from humans as

possible and delegating that work to machines. But with the aforemen-
tioned legacy interfaces and protocols, machine-to-machine communica-
tion is neither effective nor efficient; and at times, close to impossible.

Moreover, device configuration and operational data have traditionally

lacked a proper hierarchy and failed to follow a data model. In addition,
network management workflows have always been far from mature,
compared to software development workflows in terms of versioning,
collaboration, testing, and automated deployments.

Enter network programmability. Programmability revolves around pro-

grammable interfaces, commonly referred to as application program-
ming interfaces (APIs). APIs are interfaces that are designed primarily to
be used for machine-to-machine communication. A Python program ac-
cessing a network router to retrieve or push configuration, without hu-
man intervention, is an example of a machine-to-machine interaction.
Contrast this with the CLI, where a human needs to manually enter com-
mands on a device and then visually inspect the output.

Network equipment vendors (for both physical and virtual equipment)

are placing ever-increasing emphasis on the importance of managing
their equipment using programmable interfaces, and Cisco is at the fore-
front of this new world. This new approach to managing a network pro-
vides several benefits over legacy methods, including the following:

Normalizing the interface for interaction with network platforms by

abstracting communication with these platforms and breaking the de-
pendency of this communication on specific network OS scripting lan-
guages (for example, NX-OS, IOS XR, and Junos OS)
Providing new methods of interacting with network platforms and, in
the process, enabling and aligning with new technologies and archi-
tectures, such as SDN, NFV, and cloud
Harnessing the power of programming to automate manual tasks and
perform repetitive tasks efficiently
Enabling rapid infrastructure and service deployment by using work-
flows for service provisioning
Increasing the reliability of the network configuration process by
leveraging error checking, validation, and rollback and minimizing
human involvement in the configuration process
Using common software development tools and techniques for net-
work configuration management, such as software development
methodologies, versioning, staging, collaboration, testing, and continu-
ous integration/continuous delivery

This book covers all the major programmable interfaces used in the mar-
ket today for network management. The book discusses the protocols,
tools, techniques, and technologies on which network programmability is
based. Programming, operating systems, and APIs are not new technolo-
gies. However, programmable interfaces on network platforms, and using
these programmable interfaces to fully operate and maintain a network,
along with the culture accompanying these new methods and protocols,
may be (relatively) new. This book explains, in detail, all the major com-
ponents of this new ecosystem.

Goals and Methods of This Book

This is a “fundamentals” book aimed at transitioning network engineers

from a legacy network-based mindset to a software-based (and associated
technologies) mindset. A book covering fundamentals generally struggles
to cover as many subjects as possible with just enough detail. The fine
balance between breadth and depth is challenging, but this book handles
this challenge very well.

This book introduces the emerging network programmability and auto-

mation ecosystem based on programmable interfaces. It covers each pro-
tocol individually, in some significant detail, using the relevant RFCs as
guiding documents. Protocol workflows, messages, and other protocol nu-
ances tend to be dry, and at times boring, so to keep things interesting,
practical examples are given wherever possible and relevant. You, the
reader, can follow and implement these examples on your machine,
which can be as simple as a Linux virtual machine with Python Version
3.x installed, and free tools to work with APIs, such as Postman and cURL.
This book makes heavy use of the Cisco DevNet sandboxes, so in the ma-
jority of cases, you do not need a home lab to test and experiment with
physical equipment.

A whole section of the book is dedicated to putting the knowledge and

skills learned throughout the book to good use. One chapter covers pro-
gramming Cisco platforms and another covers programming non-Cisco
platforms. A third chapter in that same section is dedicated exclusively to
Ansible. This book provides an abundance of hands-on practice.

The last chapter provides a way forward, discussing tools and technolo-
gies that you might want to explore after you are done with this book.

Who This Book Is For

This book is meant for the following individuals and roles, among others:

Network architects and engineers who want to integrate programma-

bility into their network designs
NOC engineers monitoring and operating programmable networks or
those who rely on network management systems that utilize pro-
grammability protocols
Network engineers designing, implementing, and deploying new net-
work services
Software engineers or programmers developing applications for net-
work management systems
 Network and software engineers working with networks or systems
involving SDN, NFV, or cloud technologies
Network engineers pursuing their Cisco DevNet certifications

Whether you are an expert network engineer with no prior programming

experience or knowledge, or a software engineer looking to utilize your
expertise in the network automation domain, after reading this book, you
will fully understand the most commonly used protocols, tools, technolo-
gies, and techniques related to the subject, and you will be capable of ef-
fectively using the newly learned material to design, implement, and op-
erate full-fledged programmable networks and the associated network
automation systems.

How This Book Is Organized

This book covers the information you need to transition from having a fo-
cus on networking technology to focusing on software and network pro-
grammability. This book covers six main focus areas:

Operating systems: Linux

Software development: Python
Transport: HTTP, REST, and SSH
Encoding: XML, JSON, and YAML
Modeling: YANG
Protocols: NETCONF, RESTCONF, gRPC, and service provider
programmability
Practical programmability: Cisco platforms, non-Cisco platforms, and
Ansible

Each chapter in this book either explicitly covers one of these focus areas
or prepares you for one of them. Special consideration has been given to
the ordering of topics to minimize forward referencing. Following an in-
troduction to the programmability landscape, Linux is covered first be-
cause to get anything done in network programmability, you will almost
always find yourself working with Linux. The book next covers Python
because the vast majority of the rest of the book includes coverage of
Python in the context of working with various protocols. The following
chapters present an organic flow of topics: transport, encoding, modeling,
and the protocols that build on all the previous sections. For example, un-
derstanding NETCONF requires you to understand SSH, XML, and YANG,
and understanding RESTCONF requires that you understand HTTP,
XML/JSON, and YANG. Both NETCONF and RESTCONF require knowledge
of Python, most likely running on a Linux machine.

How This Book Is Structured

The book is organized into nine parts, described in the following sections.

PART I, “Introduction”

Chapter 1, “The Network Programmability and Automation

Ecosystem”: This chapter introduces the concepts and defines the terms
that are necessary to understand the protocols and technologies covered
in the following chapters. It also introduces the network programmability
stack and explores the different components of the stack that constitute a
typical network programmability and automation toolbox.

PART II, “Linux”

Chapter 2, “Linux Fundamentals”: Linux is the predominant operating

system used for running software for network programmability and auto-
mation. Linux is also the underlying operating system for the vast major-
ity of network device software, such as IOS XR, NX-OS, and Cumulus
Linux. Therefore, to be able to effectively work with programmable de-
vices, it is of paramount importance to master the fundamentals of Linux.
This chapter introduces Linux, including its architecture and boot
process, and covers the basics of working with Linux through the Bash
shell, such as working with files and directories, redirecting input and
output, performing system maintenance, and installing software.

Chapter 3, “Linux Storage, Security, and Networks”: This chapter

builds on Chapter 2 and covers more advanced Linux topics. It starts
with storage on Linux systems and the Linux Logical Volume Manager. It
then covers Linux user, group, file, and system security. Finally, it ex-
plains three different methods to manage networking in Linux; the ip
utility, the NetworkManager service, and network configuration files.
Chapter 4, “Linux Scripting”: This chapter builds on Chapters 2 and 3
and covers Linux scripting using the Bash shell. The chapter introduces
the grep, awk, and sed utilities and covers the syntax and semantics of
Bash scripting. The chapter covers comments, input and output, variables
and arrays, expansion, operations and comparisons, how to execute sys-
tem commands from a Bash script, conditional statements, loops, and
functions. It also touches on the Expect programming language.

PART III, “Python”

Chapter 5, “Python Fundamentals”: This chapter assumes no prior

knowledge of programming and starts with an introduction to program-
ming, covering some very important software and computer science con-
cepts, including algorithms and object-oriented programming. It also dis-
cusses why programming is a foundational skill for learning network pro-
grammability and covers the fundamentals of the Python programming
language, including installing Python Version 3.x, executing Python pro-
grams, input and output, data types, data structures, operators, condi-
tional statements, loops, and functions.

Chapter 6, “Python Applications”: This chapter builds on Chapter 5 and

covers the application of Python to different domains. The chapter illus-
trates the use of Python for creating web applications using Django and
Flask, for network programmability using NAPALM and Nornir, and for
orchestration and machine learning. The chapter also covers some very
important tools and protocols used in software development in general,
such as Git, containers, Docker and virtual environments.

PART IV, “Transport”

Chapter 7, “HTTP and REST”: This is one of the most important chapters
in this book. It introduces the HTTP protocol and the REST architectural
framework, as well as the relationship between them. This chapter covers
HTTP connections based on TCP. It also covers the anatomy of HTTP mes-
sages and dives into the details of HTTP request methods and response
status codes. It also provides a comprehensive explanation of the most
common header fields. The chapter discusses the syntax rules that govern
the use of URIs and then walks through working with HTTP, using tools
such as Postman, cURL, and Python libraries, such as the requests
library.

Chapter 8, “Advanced HTTP”: Building on Chapter 7, this chapter moves

to more advanced HTTP topics, including HTTP authentication and how
state can be maintained over HTTP connections by using cookies. This
chapter provides a primer on cryptography for engineers who know
nothing on the subject and builds on that to cover TLS, and HTTP over
TLS (aka HTTPS). It also provides a glimpse into HTTP/2 and HTTP/3, and
the enhancements introduced by these newer versions of HTTP.

Chapter 9, “SSH”: Despite being a rather traditional protocol, SSH is still

an integral component of the programmability stack. SSH is still one of
the most widely used protocols, and having a firm understanding of the
protocol is crucial. This chapter discusses the three sub-protocols that
constitute SSH and cover the lifecycle of an SSH connection: the SSH
Transport Layer Protocol, User Authentication Protocol, and Connection
Protocol. It also discusses how to set up SSH on Linux systems as well as
how to work with SSH on the three major network operating system: IOS
XR, IOS XE, and NX-OS. Finally, it covers SFTP, which is a version of FTP
based on SSH.

PART V, “Encoding”

Chapter 10, “XML”: This chapter covers XML, the first of three encoding
protocols covered in this book. XML is the oldest of the three protocols
and is probably the most sophisticated. This chapter describes the general
structure of an XML document as well as XML elements, attributes, com-
ments, and namespaces. It also covers advanced XML topics such as creat-
ing document templates using DTD and XML-based schemas using XSD,
and it compares the two. This chapter also covers XPath, XSLT, and work-
ing with XML using Python.

Chapter 11, “JSON”: JSON is less sophisticated, newer, and more human-
readable than XML, and it is therefore a little more popular that XML.
This chapter covers JSON data formats and data types, as well as the gen-
eral format of a JSON-encoded document. The chapter also covers JSON
Schema Definition (JSD) for data validation and how JSD coexists with
YANG.
Chapter 12, “YAML”: YAML is frequently described as a superset of JSON.
YAML is slightly more human-readable than JSON, but data encoded in
YAML tends to be significantly lengthier than its JSON-encoded counter-
part. YAML is a very popular encoding format and is required for effec-
tive use of tools such as Ansible. This chapter covers the differences be-
tween XML, JSON, and YAML and discusses the structure of a YAML docu-
ment. It also explains collections, scalers, tags, and anchors. Finally, the
chapter discusses working with YAML in Python.

PART VI, “Modeling”

Chapter 13, “YANG”: At the heart of the new paradigm of network pro-
grammability is data modeling. This is a very important chapter that cov-
ers both generic modeling and the YANG modeling language. This chapter
starts with a data modeling primer, explaining what a data model is and
why it is important to have data models. Then it explains the structure of
a data model. This chapter describes the different node types in YANG
and their place in a data model hierarchy. It also delves into more ad-
vanced topics, such as augmentations and deviations in YANG. It de-
scribes the difference between open-standard and vendor-specific YANG
models and where to get each type. Finally, the chapter covers a number
of tools for working with YANG modules, including pyang and pyang-
bind.

PART VII, “Protocols”

Chapter 14, “NETCONF and RESTCONF”: NETCONF was the first proto-
col developed to replace SNMP. RESTCONF was developed later and is
commonly referred to as the RESTful version of NETCONF. Building on
earlier chapters, this chapter takes a deep dive into both NETCONF and
RESTCONF. The chapter covers the protocol architecture as well as the
transport, message, operations, and content layers of each of the two pro-
tocols. It also covers working with these protocols using Python.

Chapter 15, “gRPC, Protobuf, and gNMI”: The gRPC protocol was ini-
tially developed by Google for network programmability that borrows its
operational concepts from the communications models of distributed ap-
plications. This chapter provides an overview of the motivation that
drove the development of gRPC. It covers the communication flow of
gRPC and protocol buffers (Protobuf) used to serialize data for gRPC com-
munications. The chapter also shows how to work with gRPC using
Python. The chapter then takes a deep dive into gNMI, a gRPC-based spec-
ification. Finally, the chapter shows how gRPC and gNMI are used to man-
age a Cisco IOS XE device.

Chapter 16, “Service Provider Programmability”: Service providers

face unique challenges due to the typical scale of their operations and the
stringent KPIs that must be imposed on their networks, especially given
the heated race to adopt 5G and associated technologies. This chapter dis-
cusses how such challenges influence the programmability and automa-
tion in service provider networks and provides in-depth coverage of
Segment Routing, BGP-LS, and PCEP.

PART VIII, “Programmability Applications”

Chapter 17, “Programming Cisco Platforms”: This chapter explores the

programmability capabilities of several Cisco platforms, covering a wide
range of technology domains. In addition, this chapter provides several
practical examples and makes heavy use of Cisco’s DevNet sandboxes.
This chapter covers the programmability of IOS XE, IOS XR, NX-OS,
Meraki, DNA Center, and Cisco’s collaboration platforms, with a use case
covering Webex Teams.

Chapter 18, “Programming Non-Cisco Platforms”: This chapter covers

the programmability of a number of non-Cisco platforms, such as the
Cumulus Linux and Arista EOS platforms. This chapter shows that the
knowledge and skills gained in the previous chapters are truly vendor
neutral and global. In addition, this chapter shows that programmability
using APIs does in fact abstract network configuration and management
and breaks the dependency on vendor-specific CLIs.

Chapter 19, “Ansible”: This chapter covers a very popular tool that has
become synonymous with network automation: Ansible. As a matter of
fact, Ansible is used in the application and compute automation domains
as well. Ansible is a very simple, yet extremely powerful, automation tool
that provides a not-so-steep learning curve, and hence a quick and effec-
tive entry point into network automation. This is quite a lengthy chapter
that takes you from zero to hero in Ansible.
PART IX, “Looking Ahead”

Chapter 20, “Looking Ahead”: This chapter builds on the foundation

covered in the preceding chapters and discusses more advanced tech-
nologies and tools that you might want to explore to further your knowl-
edge and skills related to network programmability and automation.