DevOps

Topics

• Introduction - devops
• Tools in devops
• Git
• Maven
• Docker
 3

Introduction to DevOps
Devops is the new SDLC method to reduce the gap between Development team
and Operations team, encouraging the improved collaboration between the both
teams, so that everyone can be involved with the code at any point in the
lifecycle.
 4

Design-Code-Test-Deploy- Waterfall
Design-Code-Test-Code-Test-Code-Test-Deploy -Agile
Design-Code-Test-Deploy-Design-Code-Test-Deploy-Design-Code-Test-
Deploy -DevOps
 5

Agile Way of working

 In agile - teams are empowered through values, principles, and practices to
deliver valuable products & services to respective customers.
 4 values of agile:
 Individuals and Interactions over Processes and Tools
 Working Software over Comprehensive Documentation
 Customer Collaboration over Contract Negotiation
 Responding to Change over Following a Plan
 6

Principles of Agile:
•Customer Satisfaction: satisfy the customer through early and
continuous delivery of valuable software.
•Welcome Change: Agile welcomes changing requirements, even late in
development,
to provide a competitive advantage.
•Frequent Delivery: Deliver working software frequently, with a
preference for shorter timescales.
•Collaboration: Business people and developers must work together
daily throughout the project.
•Motivated Individuals: Build projects around motivated individuals,
provide them with the environment
and support they need, and trust them to get the job done.
•Face-to-Face Communication: The most efficient and effective
method of conveying
information is face-to-face conversation.
•Working Software: Working software is the primary measure of
progress.
 7

Continuation
•Sustainable Pace: Agile processes promote sustainable development.
Teams should be able to maintain a constant pace indefinitely.
•Technical Excellence: Continuous attention to technical excellence
and good design enhances agility.
•Simplicity: The art of maximizing the amount of work not done
essentially focusing on simplicity is essential.
•Self-Organizing Teams: The best architectures, requirements, and
designs emerge from self-organizing teams.
•Reflection and Adjustment: At regular intervals, the team reflects on how
• to become more effective and adjusts its behavior accordingly.
 8

•Iterative Development:
Key concepts of Agile:
Agile projects are broken down into small, manageable units called iterations
or sprints, usually lasting 1-4 weeks. Each iteration results in a potentially
shippable product increment.
•Incremental Delivery:
Instead of delivering the entire product at once, Agile teams deliver small,
working parts of the product at regular intervals. This allows for early and
continuous delivery of value.
•Collaboration and Communication:
Agile encourages close collaboration among all stakeholders, including
developers, testers, product owners, and customers. Daily stand-up
meetings, often called "scrums," are a common practice to keep everyone
aligned and informed.
•Customer Feedback:
Agile relies on constant customer feedback. By involving customers
throughout the development process, Agile ensures that the final product
meets their needs and expectations.
•Flexibility and Adaptability:
Agile methodologies welcome changes in requirements, even late in the
project. This flexibility allows teams to adapt to new information and
 9

Scrum & Kanban

 Kanban is about the visualization of one’s your work, limiting the work in
progress, and maximizing efficiency (or flow). Kanban teams focus on reducing
the time a project takes (or a user story) from start to finish. They do this by using
a kanban board and continuously improving their flow of work.
 The kanban card quickly evolved into a visual artifact that tracks a work item as
it flows through a workflow. At its core, the kanban card encourages teamwork by
helping team members communicate and share information visually.
 Scrum teams commit to completing an increment of work, which is potentially
shippable, through set intervals called sprints. Their goal is to create learning
loops to quickly gather and integrate customer feedback. Scrum teams adopt
specific roles, create special artifacts, and hold regular ceremonies to keep things
moving forward.
 10

ITIL:
 ITIL is Information Technology Infrastructure Library, where both
ITIL and Devops both frameworks focus on improving IT service
management and software development practices.
 The Key difference between ITIL and Devops is, ITIL provides a
comprehensive framework for IT service management, while
DevOps focuses on improving collaboration and efficiency in
software development and deployment.
 Also the scope of ITIL is more towards broader IT service
management within large organizations and where as DevOps
more towards agile and dynamic environment.
 11

Why DevOps?

 It enhances simultaneous deployment across different platforms

 Automation is a core principle of DevOps. By automating repetitive
tasks such as testing, deployment, and monitoring, organizations
can increase efficiency, reduce manual errors, and free up time for
teams to focus on higher-value activities.
 By improving efficiency, reducing downtime, slowness or waiting
time, and minimizing manual effort, organizations can lower
operational costs and increase return on investment.
 12

 What is CI?
 The enablement of multiple people working together without affecting the
original deployed application is called CI
 And CD?
 Continuous Delivery is a practice where code changes are automatically
built, tested, and prepared for release to production.
 Continuous Deployment takes Continuous Delivery one step further by
automating the deployment of every code change that passes through
the pipeline directly to production.
 13

Main crux in how DevOps is different

from Agile:
 14

Continuation:
 DevOps: DevOps emphasis strongly on automation across the entire delivery
pipeline. This includes automated testing, continuous integration, continuous
deployment, and infrastructure automation. Tools like Jenkins, Docker, Kubernetes,
and Ansible are central to the DevOps approach.

End-to-End Delivery:
• Agile: Focuses on the development phase, ensuring that small increments of
software are delivered frequently and that the team can adapt to changes in
requirements.
• DevOps: Takes an end-to-end view of the software delivery process, ensuring that
the software can be developed, tested, deployed, and monitored continuously. It
addresses the entire lifecycle from code commit to production deployment and
beyond, with a focus on minimizing downtime and increasing the reliability of
releases.
 15

Continuation:
 16

Delivery Pipeline
 17

Addressing Bottle necks in Delivery

Pipeline:
 18

Continuation
 19

Tools for Identifying Bottlenecks

 20

Tools in DevOps
 21

Why tools ?
 Git: To store different versions of an application
 Maven, Gradle: Build tools to take code from different repos & builds a
complete application
 Automation(scripted) testing tools: Se, Junit for software quality
 Jenkins: This is an open-source solution comprising an automation server to
enable continuous integration and continuous delivery (CI/CD), automating the
various stages of software development such as test, build, and deployment.
 Tools that control and manage the deployment of code in prod env: xebialabs,
bms
 Once software is deployed ops team configure servers, provision them with
required resources : docker, k8s
 Monitoring - Spying on the issues: Nagios
 22

Monilithic Vs Microservices architecture:

Monolithic Architecture is a traditional approach where an entire application is built as a
single, unified unit. All the components of the application are tightly coupled and run as a
single process.
Advantages:
 Easy deployment – One executable file or directory makes deployment easier.
 Development – When an application is built with one code base, it is easier to develop.
 Performance – In a centralized code base and repository, one API can often perform the
same function that numerous APIs perform with microservices.
 Simplified testing – Since a monolithic application is a single, centralized unit, end-to-end
testing can be performed faster.
 Easy debugging : it’s easier to follow a request and find an issue, since all in same place
 23

Continuation:
Disadvantages: Monolithic applications can be quite effective until they grow too
large and scaling becomes a challenge.
 Slower development speed : If application is too large
 Scalability – You can’t scale individual components.
 Reliability – If there’s an error in any module, it could affect the entire application’s
availability.
 Barrier to technology adoption – Any changes in the framework or language affects
the entire application, making changes often expensive and time-consuming.
 Lack of flexibility – A monolith is constrained by the technologies already used in the
monolith.
 Deployment – A small change to a monolithic application requires the redeployment of
the entire monolith.
 24

Microservices Architecture:
 A microservices architecture, also simply known as microservices, is an
architectural method that relies on a series of independently deployable
services.
 These services have their own business logic and database with a specific
goal. Updating, testing, deployment, and scaling occur within each service.
 Microservices decouple major business, domain-specific concerns into
separate, independent code bases.
 Microservices don’t reduce complexity, but they make any complexity visible
and more manageable by separating tasks into smaller processes that
function independently of each other and contribute to the overall whole.
 25

Continuation:
Advantages:
 Agility – Promote agile ways of working with small teams that deploy
frequently.
 Flexible scaling – If a microservice reaches its load capacity, new instances
of that service can rapidly be deployed to the accompanying cluster.
 Continuous deployment – We now have frequent and faster release cycles.
 Highly maintainable and testable – Teams can experiment with new
features and roll back if something doesn’t work.
 High reliability – You can deploy changes for a specific service, without the
threat of bringing down the entire application.
 26

Continuation:
Disadvantages:
 Debugging challenges – Each microservice has its own set of logs, which
makes debugging more complicated. Plus, a single business process can run
across multiple machines, further complicating debugging.
 Exponential infrastructure costs – Each new microservice can have its own
cost for test suite, deployment playbooks, hosting infrastructure, monitoring
tools, and more.
 Lack of clear ownership – As more services are introduced, so are the
number of teams running those services. Over time it becomes difficult to
know the available services a team can leverage and who to contact for
support.
 27

Architecture rules of thumb in DevOps:

 Design for Automation
Automate the deployment pipeline, including code integration, testing, deployment,
and monitoring. Automation reduces manual errors and speeds up the release cycle.
 Microservices over Monoliths
Decouple Services: Break down the application into independent, self-contained
services (microservices) that can be developed, deployed, and scaled
independently.
 Design for Failure and Resilience:
Redundancy and High Availability: Implement redundancy in critical system
components to ensure high availability, using load balancers, failover strategies,
and backup systems.
 28

Continuation:
 Continuous Delivery and Integration
CI/CD Pipelines: Implement Continuous Integration and Continuous Delivery
(CI/CD) pipelines to automate testing and deployment. This ensures that
changes are deployed quickly and reliably.
• Scalability and Elasticity
Horizontal Scaling: Design the system to scale horizontally by adding more
instances rather than vertically increasing the size of a single instance.
 29

Continuation
• Security:
Zero Trust Architecture: Adopt a zero-trust approach where all access is
authenticated and authorized, regardless of the origin of the request.
 Monitoring and Observability:
Full-Stack Monitoring: Implement end-to-end monitoring across all services
and infrastructure components to detect issues early and respond quickly.
 Decentralized Governance:
Empower Teams: Decentralize governance by allowing teams to choose the
best tools and practices for their specific needs, within the bounds of
organizational standards.
 30

Continuation:
 Use of Containers and Orchestration:
a. Containerization: Use containers to package applications and their
dependencies consistently across different environments.
b. Orchestration: Implement container orchestration platforms like
Kubernetes to manage containerized applications at scale.
 Feedback Loops and Continuous Improvement
Rapid Feedback: Ensure that feedback loops are short and continuous,
enabling quick identification and resolution of issues.
 31

Introduction to Git
 What is version control system?
A Version Control System (VCS) is a tool or software that helps track changes
to files and directories over time. It helps developers to manage different
versions of files, and revert to previous states if needed.
Two main types of VCS: centralized and distributed
Centralized example: SVN(SubVersion) CVS(Concurrent Version System)
Distributed example: Git, Bitbucket, gitlab
 Source Code Management (SCM)?
Source Code Management (SCM) is a broader term that encompasses the
management of source code files, including version control. It involves
practices and tools for managing, organizing, and tracking changes to source
code throughout its lifecycle.
Github: An online platform to host our folders/repos online
 32

Centralized version control system: Central

repos
It is two tiered, with centralized repos Push Pull

Drawbacks: A B C
If we lost central repo the project code is gone
There can be conflict errors where for example: both developers A, B makes
Remoteto keep.
changes to same code System doesn’t know which updates
repos
Distributed VCS Push
Pull
Index or
This is a three tier architecture. Add, Staging
comm area
Several drawbacks of centralized vcs are solved here. it
merge
Developers can work together A
Automatic backups
Helps to track changes in the code
 33

The need for source code

control:
 Branching: Version control allows the use of branches to develop from
the main codebase. Once a feature branch or any branch is complete, it
can be merged back into the main branch, making it easy to manage
parallel development and avoid conflicts.
 Auditability: This also ensures the detailed audit trail which is crucial for
maintaining a software project long term. Which makes easier for
troubleshooting issues or to involve new developers.
 Backup And Recovery: VCS serves as a backup for the entire codebase.
If code is lost, corrupted, or a bug is introduced, developers can quickly
revert to an earlier, stable version of the software. This ensures code
stability and minimizes the risk of losing critical work.
 34

Continuation:
 Tracking the changes: It allows to track all changes, details like who committed,
and what are the specific lines added or deleted to the codebase which helps
developers to easily review previous versions, also helps in tracing back origin of
bugs, in which they can easily revert back if changes are not required.
 Working Together: The Source code control helps in multiple developers to work
along simultaneously without overwriting each other changes, that allows different
team members to add new features, fixing bugs also ensuring the collaboration to
run without hectics.
 CI/CD: Version control systems often integrate with tools that facilitate code
reviews and pull requests. This allows developers to collaborate more effectively
by reviewing each other's code, ensuring higher quality and more maintainable
software.
 35

The history of source code

management:
 Manual: At early days multiple copies of same file with different names are stored
where collaboration was limited and poor version tracking which led to error prone.
 Revision Control System: This is an early version control system designed to
manage multiple revisions of files in a project. But it has limitations like,
 only one person at a time can change the file since it applies lock on it.
 Also it requires manual merging.
 RCS works on individual files only and does not manage entire project workspace or
repositories.
 Centralized VCS: In this type of VCS, the version history of files are stored on
central server, where the developers can checkout files make changes and commit
back then to central repository. But there are some disadvantages for C-VCS like:
 36

Continuation:
 If the central server goes down, entire codebase cannot be accessed, neither
comitted nor retrieved.
 Also C-VCS is slower because every operation is based on central server and has
limited offline capabilities.
 Also conflict resolution is manual
 But the centralized VCS is helpful for managing backups and policies easily
 Distributed Version Control System:
 D-VCS as a famous example git is created by Linus Torvalds for the development
of the Linux kernel, Git became revolutionary for its distributed model, allowing
every developer to have a complete copy of the repository. This enabled
branching, merging, and collaboration on a massive scale.
 37

Continuation:
 Pros and Cons of D-SCM: D-SCM is fault tolerant, since any copy of repo can be
used if main server crashes and local operations like review history of commits
are faster which does not require internet access. But cons can be issue of
more disk space for projects.
 Modern SCM and Cloud Integration: This can be considered with examples of
the existing platforms like GitHub, GitLab, and Bitbucket which are integrated
version control systems with cloud-based services, enabling collaboration,
code reviews, continuous integration (CI), and project management.
 The main difference between Distributed and Modern SCM is that, there are
limited tools for automation in D-SCM where are there are built in CI/CD
pipelines, automated tests and deployment tools in modern SCM
 38

Source Code Management System

& Migration:
 A Source Code Management (SCM) system, also known as Version Control
System (VCS), is a tool that helps developers track and manage changes to
source code over time. SCM systems are essential for maintaining a history of
changes, collaborating with others, and managing different versions of code.
 The Key aspects of SCM are Version Control, Backup and recovery,
Collaboration, Change Tracking, Branching and Merging
 Migration refers to the process of moving data, applications, or other business
elements from one environment to another. There can be different types of
migration like :
 Application Migration: Moving an application from one environment to another,
such as from on-premises servers to the cloud
 39

Continuation:
 Code Migration: Updating or porting code to run on a different
platform or framework. This can involve refactoring code to be
compatible with a new version of a programming language or
library.
 Database Migration: Moving a database from one system to
another that can involve data transformation, schema changes
also ensuring data integrity.
 SCM systems here, keep track of different versions of the code,
which helps in ensuring that the code being migrated is the
correct version and includes all necessary changes.
 40

Shared Authentication:
 Shared Authentication refers to use the single authentication
mechanism or service to provide access to multiple applications
or systems.
 This approach simplifies the user experience by allowing users to
log in once and gain access to various services without needing
to authenticate separately for each one
 Single Sign On: SSO is a common implementation of shared
authentication which allows users to authenticate once and gain
access to multiple applications or services without the need for log
in separately for each one. This happens via authentication token or
a session creation when user logs in This token is then used to
access other applications or services that are part of the SSO
 41

Continuation:
 Another way is Centralized Authentication Providers:
 Centralized authentication providers manage user identities and
authentication across multiple systems or services from a single point of
control. This centralization helps streamline user access, improve security, and
simplify administration. Examples are Identity Providers like: LDAP, Oauth etc.,
 LDAP: LDAP is lightweight Directory Access Protocol, where LDAP directories are
organized hierarchically with entries structured in a tree format. Each entry
represents an object (example: user, group) and has attributes (example:
username, email).
 LDAP supports various operations, such as searching for entries, reading
attributes, adding new entries, modifying existing entries, and deleting entries.
 Users authenticate by binding to the LDAP server with their credentials. The
server verifies these credentials and returns an authentication response.
 42

Continuation:
 Oauth:
 The Oauth is Open Authorization is generally used to grant the third
party applications with limited permissions without exposing the
user’s credentials
 Generally for example there will be a third party app requesting
access, and there is authentication server that issue access tokens
after authenticating the user and granting consent. And also a
resource server, that hosts the protected resources and accepts
access tokens to grant access.
 Authentication request->User Consent->Authorization code-
>Access code->Resource access.
 43

Pros and Cons of Shared

Authentication:
 Pros:
 It is easy for users to remember only a set of credentials
 Centralized authentication allows for consistent security policies and
easier management of user accounts.
 Cons:
 If the centralized authentication service goes down, users may be
unable to access any dependent applications.
 Again there is always a risk in compromise of shared authentication
which could allow multiple users systems get attacked
 44

Hosted Git Servers

 Hosted Git Servers are services that provide a centralized platform for hosting
Git repositories. These platforms facilitate version control, collaboration, and
code management for development teams.
 There are few most widely used git servers like: GitHub, GitLab, Bitbucket etc,
 GitHub: GitHub is one of the most widely used platforms for hosting Git
repositories. It provides a collaborative environment for developers to share,
review, and manage code. And is founded in the year 2008, acquired by
Microsoft in 2018.
 GitHub has its main features like:
 Host repos, create and fork
 Review and discuss on modifications before merging
 45

Continuation:
 Allows users to host static websites
 Provides a marketplace for third-party integrations and applications to extend
GitHub’s functionality.
 GitHub Actions enables CI/CD workflows, allowing automated testing, building,
and deployment of code.
 BitBucket:
 This is known for its integration with Atlassian tools.
 Bitbucket Pipelines is a CI/CD service integrated into Bitbucket, providing
automated build, test, and deployment capabilities.
 Provides issue tracking with support for issue management and integration with
Jira.
 Facilitates code review with pull requests and inline comments.
 46

Continuation:
 GitLab:
 This is similar to GitHub’s pull requests, GitLab’s merge requests facilitate code
review and discussion. And also hosts Git repositories with features for version control
and collaboration.
 GitLab positions itself as a complete DevOps lifecycle tool. It integrates various
stages of software development, from planning and coding to testing, deployment,
and monitoring, into a single application. GitLab provides a suite of tools for
monitoring, security, and infrastructure management, including features for container
management and Kubernetes integration.
 Whereas, GitHub’s primary focus is on source code management and collaboration. It
provides robust features for version control, code review, and collaborative
development. GitHub Actions to provide CI/CD capabilities in github, While GitHub has
strong CI/CD capabilities, it relies on integrations with third-party tools for certain
DevOps functionalities.
 47
Git Architecture:
Working Local staging Remote
directory area Local Repository
Repository

git add
git
commit Push to
remote repo

git pull
git checkout

git
merge

Git merge- after you are done with all changes you can merge the new
branches to master branch
 48
 The git init command initializes a new Git repository. Here’s what happens
when you run git init:
Details:
 Create a New Git Repository: It sets up a new Git repository in the current
directory,
 Allowing to start tracking changes to files in that directory.
 Create a .git Directory: A hidden directory named .git is created in the root of
your project.
 This directory contains all the metadata and objects needed for version
control, including:
 Configuration Files: Settings for the repository, such as user information and
remote repositories.
 Commit History: Logs of all commits and changes made to the repository.
 Branches and Tags: Information about different branches and tags within the
repository.
 Objects and Index: Data files and the index (staging area) used to track changes.
 This initializes basic git structures such as default branch main or master to
start accepting commits
 The adding files to staged area can be done using git add and commit
 49

Let’s practice:
 Gitbash download: https://git-scm.com/downloads
 Creating the organization - https://github.com/settings/organizationsGit
Commands:
* git init – initializes the empty git repo(the .git file is created that stores history of
changes made)

this makes the initial configuration settings.

* git status – status of the files or changes
* git add . – adding to staging area
* git commit –m “message here” – commit to local repo
 50

* git log – to see all the commits we made

* git reset <commit_hashid> - resets to one commit, other commits are in unstaged area
* git stash – instead of commits putting them to back(kind of storing)
* git stash pop – adding the most recent set of stashed changes to the working directory
* git stash clear – clearing the stash
* git remote add origin <url to add> - attaching remote url to your folder(origin is name of url)
* git remote –v :shows all urls attaching to the folder
* git branch : list of existing branches
* git branch feature – creating a new branch
* git checkout feature – head pointer to feature (next commit happens to this branch)
* git checkout main
* git pull origin master
* git merge feature – commits to main branch
* git push origin main
* git remote remove origin
 51

* git clone <url> - fork any project and clone to local

* git remote add upstream
* git push origin <branchname>
* git rebase –i <commitid> - merge some commits to single commit
* git reset --soft 9f57ad6e759eb5368fded9420f2fa454f64acbca
* git reset --hard 9f57ad6e759eb5368fded9420f2fa454f64acbca
* git remote -v add origin https://<token>@github.com/<name >/
<codebase>
Example : git remote -v add origin
https://[email protected]/Kypa-
Jeevitha/GDCPortal_MasterCodeBasePreProd
* git remote set-url jorigin https://<token>@github.com/<name >/
<codebase>
*git remote –v add jorigin
https://[email protected]/Jeevitha7
71/MyGitPractice
52
53
54
55
56
57
58
59
60
61
 62

Little Theory
 Fast Forward merge occurs when branch that is
merging(newbranch) is directly ahead of branch we are merging
(master), then master can simply move forward to include the
new commits without creating a new commit. This happens when
there are no diverging changes between the branches.
 A merge commit has two parent commits: one from the the
master branch and one from the newbranch This commit
represents the point where the branches are combined Git
creates a new commit (called a merge commit) to combine the
histories of both branches. This happens when When a fast-
forward merge is not possible because both branches have
diverged
63
64
65
66
67
68
69
70
71
72
73
74
75
76
 Docker 77

What is Docker and why we use it?

 Docker is a virtualization software that makes developing and deploying
applications much easier
 It packages application with all necessary dependencies, configuration,
system tools and runtime
 It has an ability to create a standardized unit that has everything the
application needs to run
 It helps in creating a portable artifact that can be easily shared and
distributed.
 78

Installation steps:
Installing wsl in windows:
 Open PowerShell or Windows Command Prompt in administrator mode by right-clicking and
selecting "Run as administrator",enter the below command, then restart your machine.
 wsl –install: This command will enable the features necessary to run WSL and install the Ubuntu
distribution of Linux
 curl -fsSL https://get.docker.com -o get-docker.sh
 DRY_RUN=1 sudo sh ./get-docker.sh
 sudo service docker start
 sudo service docker status
 sudo curl -SL https://github.com/docker/compose/releases/download/v2.11.2/docker-compose-linux-
x86_64 -o /usr/local/bin/docker-compose
 sudo chmod +x /usr/local/bin/docker-compose
 sudo docker-compose version
 79

 Docker Desktop Installation link:

https://docs.docker.com/desktop/install/windows-install/
 80

Before Docker:
 Each developer needs to install and configure all the services directly on their
OS of their local machine.
 Drawbacks: Installations versions differs in each OS
 Many steps, may go wrong during installation
 Application dependent services are all needed to be installed by the developer

After Docker:
 It has own isolated environment
 Packages any service with all the dependencies
 Starts the service with 1 docker run command, and this command is same for
all the OS and all the services.
 Ex: docker run redis
 81

How is OS ?
 OS has mainly 2 main layers OS Application layer

 The kernel is the core of the operating system

and it interacts between hardware and software
OS Kernel
components

Hardware layer
 82

Differences

 Docker couple of MB  VM images needs GB

 Containers takes few seconds  These takes minutes to start
to start
 83

Docker Containers:
84
85
86
 87

 A dedicated team is responsible for reviewing and publishing all the content
in docker official image repositories. Which has security experts and software
maintainers.
 This collaboration happens openly in github which enables anyone to
participate.
Examples:
 Dockerhub is a largest public registry, anyone can search and download
required docker images. It also enables to host both public and private repos
of the applications
 Private registries need prior authentication, example: ECR(Amazon elastic
container registry), google cloud container registry, azure container registry,
Registry Vs Repository?
Registry is a service that provides storage for storing different repositories.
While the repository is the collection of similar images of different
versions/tags.
 88

Dockerfile creation:
 FROM: Docker files start from a parent image or base image and we choose this base
image depending on which tools we need
 WORKDIR: specify a working directory inside the container where commands will be
executed.
 COPY: Copy files from host to image by COPY instruction
 RUN: Run will execute any command in a shell inside the container environment
 CMD: the instruction that has to be executed when docker container starts, there can be
only one CMD
 Each instruction in the docker file creates one layer in the docker image.

FROM <base_image>:<tag>
WORKDIR /path/to/directory
COPY <src> <dest>
RUN <command>
CMD [“executable”, “server.js”] (executable can be: node, python etc...,)
> docker build –t <image-name>:<version/tag> .
 89

Commands:
>docker pull <image>
>docker images
>docker run <imagename>:<tag>
>docker ps (shows running containers)
>docker ps –a (shows all including stopped containers)
Generally the containers ports are not exposed to the host and are only exposed
inside the docker network. So to access the application or make service available to
outside hosted by specific container, port binding of that docker container’s port and
host machine’s port needs to be done.
>docker stop <container-id>
-d (detach mode), -p{host port: container’s port} (publish container’s port to host)
>docker run –d –p {host port: container’s port} <imageid>:<tag> --name
<container-name>
>docker logs <containerId>
 90

 Dockerfile creation example:

FROM node:18-bullseye
COPY package.json /app/
COPY src /app/
WORKDIR /app/
RUN npm install
CMD ["node", "index.js"]
>sudo docker build -t <image:tag>
>sudo docker run -d -p 3000:3000 node-app:10
>sudo docker rm <container-id>
> sudo docker rmi <image-id>
> docker network ls
> docker network create mongo-network
>docker exec –it <container-id> /bin/bash
91
 92

The Docker architecture:

 Docker Client: The Docker client is the primary interface that users
interact with to manage containers. We can perform commands like
docker run, docker build, docker pull through client
 The Docker client communicates with the Docker daemon (Docker
Engine) through a REST API, either locally or remotely.
 Docker Daemon (Docker Engine): This is responsible for building, running,
and managing Docker containers. It listens for API requests and performs
the necessary tasks.
 The daemon communicates with both the Docker client and containerd
(container runtime). It also interacts with registries (like Docker Hub) to
pull and push container images.
 93

Continuation:
 Therefore, dockerd is responsible for,
 Build and Management of container images
 Handling container lifecycle
 Network configurations
 Data Volumes
 Container RunTime(Containerd):
 This is a container runtime that directly interfaces with the Linux
kernel features (cgroups, and namespaces) to create and manage
containers. It handles all the low-level details of managing the
container lifecycle. (create, execute, delete), networking etc
 94

Differences:
Dockerd(Docker Daemon) Containerd(Container Runtime)
 Higher-level management of Docker  Low-level management of container
containers, images, volumes, networks, lifecycle (running, stopping, deleting
and registries. containers).
 Used as Full Docker platform.  Core container runtime, can be used
 Communicates with containerd to run standalone or with Kubernetes.
and manage containers and with  Communicates with the Linux kernel
Docker clients for container and interacts with dockerd or other
management. orchestrators
 Full lifecycle management including  Container runtime only, focused on
images, build, volumes, networks.
low-level container operations.
 95

Kubernetes:
 Kubernetes is a container orchestration
technology used to orchestrate the
deployment and management of
hundreds and thousands of containers
in a clustered environment.
 The process of automatically deploying
and managing containers is known as
Container Orchestration.
 Under circumstances where an
application relies on other containers
such as database or messaging services
or other backend services or when the
number of users increase/decrease, the
application needs to be scaled up/down
and for this an underlying platform with
a set of resources will be required.
 96

 The platform needs to orchestrate the connectivity between the containers

and automatically scale up or down, based on the load.
 There are multiple such technologies available today – Docker has its own
tool called Docker Swarm, Kubernetes from Google and Mesos from
Apache.
 Docker Swarm is really easy to set up and get started, it lacks some of the
advanced autoscaling features required for complex applications.
 And others are difficult to setup,
 Kubernetes – arguably the most popular of it all, is a bit difficult to set up
and get started, but provides a lot of options to customize deployments
and supports deployment of complex architectures.
 Kubernetes is now supported on all public cloud service providers like GCP,
Azure and AWS, and the Kubernetes project is one of the top ranked
projects in Github.
 97

Advantages of K8S:
• Due to the multiple instances running on different nodes, an application will
now be highly available as hardware failures do not bring the application down.
• User traffic is load balanced across the various containers.
• More instances of the application can be seamlessly deployed when demand
increases, and the number of nodes can be scaled up/down without having to
take down the application when hardware resources are depleted.

K8S Architecture:
• A Node is a worker machine in Kubernetes, interchangeably called as Minion. It
is the smallest unit of computing hardware in Kubernetes and will likely be
either a physical machine in a datacenter or virtual machine hosted on a cloud
provider.
• A Cluster is a set of nodes that run containerized applications. Cluster
intelligently handles distributing work to the individual nodes, and will shift
around work as necessary if any nodes are added or removed from the cluster.
98
 99

 The Master is another node with Kubernetes installed in it, and is configured
as a Master. The master watches over the nodes in the cluster and is
responsible for the actual orchestration of containers on the worker nodes.
 A POD is a single instance of an application and the smallest object that can
be created in Kubernetes. The ultimate aim of using Kubernetes is to deploy
an application in the form of containers on a set of machines that are
configured as worker nodes in a cluster. Kubernetes does not deploy these
containers directly on the worker nodes, but encapsulates the containers into
PODs.
 100

 PODs usually have a one-to-one relationship with containers running an

application. To scale up, new PODs are created and to scale down, PODs
are deleted. Additional containers cannot be added to an existing POD
to scale up the application.
Kubernetes command line tool - kubectl
 The Kubernetes command-line tool, kubectl, is used to run commands against
Kubernetes clusters. Kubectl, also called Kube Control, can be used to deploy
and manage applications on a kubernetes cluster, to get cluster information, to
get the status of nodes in the cluster, to inspect, to manage cluster resources
and to view logs.
 The kubectl cluster-info command is used to view information about the cluster,
and the kubectl get pod command is used to list all the nodes part of the cluster.

https://github.com/mmumshad/kubernetes-training-answers
 101

Commands:
102
103
104
 105

4.Integrating the System:

 Build Systems:
 The Build Systems are used to convert a code into deployable
format.
 A build system has a set of software tools which are used to
perform several steps like compilation of code, linking it with
necessary libraries, packaging the software (like jar file in java
applications.) While all these tasks are automated in build
systems, which reduces probability of human error.
 The efficiency to use these build systems comes into place
because most of the build systems will detect the changes in the
components or specific files and rebuild them. Also enabling the
 106

Continuation:
 It also helps in automatically managing of dependencies between the
components, ensuring that files are compiled in the correct order and
that all necessary libraries are included.
Examples of Build Systems are :
 Apache Ant: A Java-based build system that uses XML files for
configuration, making it highly flexible and extensible.
 Maven: Also Java-based, Maven goes beyond compilation to manage
project dependencies, documentation, and site deployment.
 Gradle: A modern build system for Java and other languages that
combines the best features of Ant and Maven with a Groovy-based DSL
for configuration.
 Maven 107

 Before Maven:
 Need of adding jars
 Minor mistake the servlet structure does not give right output
 Build and deploy takes lot of time

 After:
 Maven is a automation tool that helps in build and manage projects in short time.
 It is a product from apache that uses a process called pom.xml(project
object model)
 Pom.xml has information about the project details, configuration, plugins
and dependencies, this can be created by maven architype
 we get from maven central repository
 Users has to only declare dependency requirements, maven will download
and deploy the dependencies with applications.(needs jdk latest)
 Maven practice: 108

Linux Env:
 wget
https://dlcdn.apache.org/maven/maven-3/3.8.8/binaries/apache-maven-3.8.8-bin.tar.gz
~> download maven repo
 tar –zxvf <file-name> ~> unzip
 ll ~>check contents
 cd <foldername>, ll
 sudo amazon-linux-extras install java-openjdk11 ~>download jdk
 yum install maven –y ~> maven package install from repo
 java –version , maven –version ~> version check
 mvn archetype:generate ~> mvn recources download
 GroupId: project name
 Artifactid: package name(this has final code ex: devops)
 This has pom.xml, src (main, test)
 ls src/main/java/batch/

 mvn compile(at package(devops) folder) ~>target file gets generated

 109

 >ll
 >ls /target/classes/batch ~> has app.class file(class file generated)
 >mvn test
 >mvn package ~> jar file creation
 >mvn install ~> to place the the jar file in local repo
 >mvn deploy ~>deploying into artifactory
 Java: sudo amazon-linux-extras install java-openjdk11
 110

Windows:
 https://dlcdn.apache.org/maven/maven-3/3.9.6/binaries/apache-maven-3.9.6-bin.zip
~installation
 Add path to user variables, MAVEN_HOME, M2_HOME, also needs jdk
 Add bin path to system variable path
 >mvn –version or mvn -v
 >mvn archetype:generate ~>creates project from existing template, downloads plugins if
not found, there are many archetypes we can chose a template by number version
choose, groupId : unique (ex: com.hello), artifactId: name of the project(this has src,
pom.xml), version(dev)
 If jar/war file is created it will be with name: artifactId-version.extension (ex: my-maven-
prjct-1.0-SNAPSHOT.jar) package name: here class, test dirs are created.
 <follow the same steps>
 Maven lifecycle: 111

Commands:
yum install java-1.8.0-openjdk –y
yum install maven –y
yum install git –y
git clone <maven-repo>
Cd <dirname>
mvn validate : validates the syntaxes
mvn compile : .class file, target file is generated,
mvn test : testing before converting to jar file
mvn package : creating a jar/war file
mvn install : stores in local repo(.m2)
mvn deploy: deploys to artifactory
(If you are installing any webapplication jar file it can be copied into tomcat
webapps folder to run the webapplication on your system)
 112

Jenkins build server:

 Jenkins is defined as an open-source solution comprising an automation server to enable
continuous integration and continuous delivery (CI/CD), automating the various stages of
software development such as test, build, and deployment.
 Jenkins is a Java-based open-source automation platform with plugins designed for
continuous integration.
 It is used to continually create and test software projects, making it easier for developers
and DevOps engineers to integrate changes to the project and for consumers to get a new
build. It also enables you to release your software continuously by interacting with various
testing and deployment methods.
 Jenkins incorporates a variety of development life-cycle operations, such as build, document, test, package,
stage, deploy, static analysis, and more.
 Jenkins is a well-known continuous integration tool developed initially by Hudson before it
became available on the open-source market. Hudson was created by Kohsuke Kawaguchi
in 2004 while working at Sun Microsystems
 113

Continuation:
 Jenkins software’s helpful to track and monitor repetitive
activities that emerge throughout a project’s development. For
example, Jenkins will continually test your builds and alert you to
any mistakes early in the process, when a team is working on
that project.
 It is useful for deploying the code into production
 It helps in automating the workflows, for example automating the
installation upgradations in less time reducing the errors.
 The troubleshooting is easy since , when Jenkins tasks run, they
collect console output from stdout and stderr parameters.
 114

Managing build dependencies:

 Managing build dependencies is the key role of Jenkins.
 The dependencies are required for the code to function, which includes,
software packages, plugins, frameworks that are required to be
installed in a development environment. For example if we take Maven
plugin is used to handle Java dependencies. Similarly npm (node
package manager) or Yarn is used for Node.js applications. Pip for
python projects. Docker plugin for handling container dependencies.
 Jenkins helps in integrating these plugins and it also integrates to git or
BitBucket and fetches the code changes and its dependencies from
remote repository, ensuring that they are consistent across the system.
 115

Jenkins Plugins:
 Plugins are available on Jenkins that equips it with additional features or
functionalities you require in the SDLC.
 There are several plugins that helps in extending its efficacy by integrating
Jenkins with every tool present in the DevOps toolchain to ensure CI/CD.
 Jenkins plugins are written in Java and use the same Jenkins API. Each plugin runs
within Jenkins, extending its capabilities without modifying Jenkins core code.
 Plugins can be installed directly from the Jenkins UI (Manage Jenkins > Manage
Plugins)
Examples for Jenkins Plugins:
 Git: Git is one of the most installed Jenkins add-ons available on its repository. It
allows you to integrate Jenkins with GitHub and transfer GitHub repository data
to the Jenkins machine easily.
 116

Continuation:
 Build Tools Plugins: Example: Maven Integration Plugin is used to provide support
for building Java projects with Maven.
 Docker Plugin: The Docker plugin enables you to create Docker containers and
automatically run builds on them.
 Jira Plugin: The Jira plugin integrates Jenkins with Atlassian Jira Software. The Jira
plugin enables you to get better visibility into the development pipeline.
 Kubernetes: With this plugin, you can automatically scale the running process of
Jenkins agents in the Kubernetes environment.
 SonarQuobe: This offers necessary security feedback at the time of code review.
 And there are many like these around 1400 plugins.
 117

Continuation:
 In CI/CD , the continuous integration involves. Devolopment,Code
and Build and Test.
 While CD is the continuous delivery that involves steps like
building the docker image, push it to repository, adding to the
K8s file and Deploying
 With Jenkins we can build this pipelines to involve these and
automate the process, that speeds up development process.
 118

Host Server:
 In Jenkins, the host server (also referred to as the Jenkins Master or
Controller) is the main server that orchestrates the Jenkins pipeline,
manages jobs, schedules builds, and coordinates with build slaves (agents)
in a Jenkins environment.
 The host server is responsible for managing all Jenkins jobs and builds,
including creating, configuring, and monitoring them.
 The Jenkins Master schedules jobs to run on available agents (slaves) based
on job configurations, labels, and resources.
 When multiple builds are requested, the host server places them in a build
queue.
 It allocates builds from the queue to agents as they become available.
 119

Continuation
 The master server keeps track of the available resources on each agent and assigns builds
accordingly.
 The Jenkins master manages all plugins and their configurations, including installations, updates,
and removal.
 The Jenkins master server stores all job configurations, plugin configurations, user settings, and
security settings.
 The master is responsible for handling HTTP/HTTPS requests and acts as a web server for the
Jenkins UI.
 The Jenkins master handles authentication, authorization, and security settings.
 The Jenkins master is responsible primarily for job orchestration, job scheduling, and providing the
user interface.
 Therefore the host server i.e Jenkins Master is central to the Jenkins system, managing and
coordinating all CI/CD activities. It controls jobs, agents, build queues, security, and plugins,
providing a centralized environment for orchestrating and automating development workflows.
 120

Build Slaves
 A Slave is a Java executable that runs on a remote machine.
 It hears requests from the Jenkins Master instance.
 Slaves can run on a variety of operating systems.
 121

Continuation:
 Jenkins checks the Git repository at periodic intervals for any
changes made in the source code.
 Each builds requires a different testing environment which is
not possible for a single Jenkins server. In order to perform
testing in different environments, Jenkins uses various
Slaves as shown in the diagram.
 Jenkins Master requests these Slaves to perform testing and
to generate test reports.
 We can create the nodes on Jenkins and run the job on the
respective node
 122

Software on the host

 The core Jenkins software runs on the host and requires Java to
function. The Jenkins controller manages the pipeline logic, schedules
builds, and communicates with agents, so having a compatible Java
runtime (e.g., JDK) is crucial.
 In Jenkins, the software on the host refers to the tools and programs
installed on the machine where Jenkins operates. This machine can
either be the Jenkins controller (master) or a build agent (node).
 Jenkins leverages various build tools to compile and package software:
 Maven/Gradle for Java projects.
 npm/yarn for JavaScript.
 Docker for containerized applications.
 123

 Jenkins uses these tools to perform tasks like pulling code from Git, building it
with Maven, testing it with JUnit, and packaging it into deployable formats.
 If builds run on separate nodes, those nodes also need the required software
installed.
 Depending on the project, Jenkins requires appropriate runtimes to build and run
code:
 Java (JDK) for Java applications.
 Node.js for JavaScript-based projects.
 Python, Ruby, PHP, or other interpreters for specific language needs.
 Software for compiling and packaging applications:
 Maven/Gradle: For Java-based projects.
 npm/yarn: For JavaScript applications.
 Docker: If Jenkins is used to build and deploy Docker containers.
 Kubernetes CLI (kubectl): For deploying to Kubernetes clusters.
 Using Docker for builds ensures a clean, reproducible environment with all
dependencies included.
 124

Triggers
 There can be various kinds of triggers.
 A user manually starts the build by clicking the "Build Now" button
on the Jenkins dashboard.
 Jenkins periodically checks the version control system (e.g., Git or
SVN) for changes. If new changes are detected, a build is triggered.
 One job can trigger another job as part of a multi-step process in
Upstream/Downstream triggers.
 Useful for linking jobs in a pipeline or dependency chain.
 Builds can be initiated via an external script or system using an
HTTP API, these are called Remote Triggers.
 125

 In pipeline jobs, triggers can be scripted directly in the

Jenkinsfile. For example, you can use pollSCM or cron blocks
to define when the pipeline should start.
 Triggers play a vital role in automating workflows in Jenkins,
ensuring that jobs run seamlessly when needed.
 Jenkins responds to notifications (webhooks) from version
control systems like GitHub, GitLab, or Bitbucket when
changes are pushed.
 We can also combine triggers with parameters for dynamic
build configurations, allowing flexibility for environments or
branch-specific builds.
 126

Job chaining and build

pipelines:
 In Jenkins, Job Chaining and Build Pipelines are methods to link multiple
tasks or jobs together, ensuring they execute in a specific sequence or
as part of an automated workflow. These approaches help manage the
continuous integration and delivery processes effectively.
 Job chaining is the simplest way to connect multiple jobs in Jenkins,
where the completion of one job triggers another.
 It works in the following way:
 A job (let’s call it Job A) completes its task, and based on the outcome, it
triggers another job (Job B). This chain can continue with further jobs (e.g.,
Job C).
 Each job operates independently but communicates through triggers.
 127

 How to set up Job Chaining:

1. Post-Build Actions: In the configuration of a job, you can
specify "Post-build Actions" to trigger another job upon
success, failure, or always, for example, Job A compiles the
code, and upon success, it triggers Job B to run automated
tests.
2. Parameters such as branch names or environment
configurations can be passed from one job to another for
dynamic execution.
3. The downstream job (e.g., Job B) can be set to "Build after
other projects are built," linking it to the upstream job.
 128

Build Pipelines
 A build pipeline represents an entire workflow, connecting all
stages of the software lifecycle, from building to testing to
deploying, in one cohesive process.
 Pipelines are defined using a Jenkinsfile, which provides more
flexibility and better control
 Here, in all stages (build, test, deploy, etc.) are written in a
single script.
 Pipelines can run tasks sequentially or in parallel.
 This offers better error handling, retries, and visualization
compared to job chaining.
 129

Creating a Pipeline
 Declarative Pipeline: This is a more structured and simpler syntax
for writing pipelines.

pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building the code...'
sh 'mvn clean install'
}
}
 130

Continuation:
stage('Test') {
steps {
echo 'Running tests...'
sh 'mvn test'
}
}
stage('Deploy') {
steps {
echo 'Deploying application...'
sh './deploy.sh'
}
}
}
}
 131

Comparing both Job chaining and

Build Pipelines
In Job Chaining, it is easy to configure manually. But in Build
Pipelines it requires scripting.
The maintenance is Harder with many jobs in Job Chaining.
But it is easier with version-controlled scripts in Build
Pipelines
The Parallel Execution is limited or manual step in Job
Chaining but it is fully supported natively in Build pipelines.
The visualization is also basic in job chaining, but is having
advanced pipeline stage view in Build pipelines.
 132

Build Server and Infrastructure as

code:
 A build server is a specialized system that automates the processes involved
in software development, such as compiling code, running tests, and
deploying applications.
 It is a key component in modern development workflows, especially for
Continuous Integration (CI) and Continuous Deployment (CD).
The below is the way the Build Servers work:
1.Code Integration:
 Developers push their code changes to a shared repository (e.g., GitHub or
GitLab).
 The build server monitors the repository and automatically triggers a build
when changes are detected.
 133

2.Automated Builds:
 The server compiles the code into a functional application,
ensuring consistency in how the code is built across
environments.
3. Automated Testing:
 After the build, the server executes predefined tests (e.g.,
unit tests, integration tests) to validate the code.
4. Deployment:
 Upon successful builds and tests, the server can deploy the
application to staging or production environments.
Example: Jenkins is an open-source and highly customizable
build server.
 134

 Build servers help in reducing manual intervention in building

and testing code, that is automating the code.
 It also ensures the consistent builds across teams and
environments.
 It helps in Faster development that is automating repetitive
tasks, allowing developers to focus on coding.
 It catches issues early in the development cycle through testing
and integration.
 Applications :
i. CI/CD pipelines for modern software development.
ii. Automating deployment workflows.
iii. Ensuring code quality through continuous testing.
 135

Infrastructure as code:
 Infrastructure as a code is a method of managing and provisioning
computing resources like servers, networks, and database, using
machine-readable configuration files.
 Instead of manually setting up infrastructure, IaC allows developers to
automate the process, treating infrastructure like software code.
IaC Working as follows:
1. Configuration Files:
 Infrastructure settings are defined in text files using tools like YAML,
JSON, or HCL (HashiCorp Configuration Language).
 These files describe the desired state of resources, such as virtual
machines, storage, or networking.
 136

2. Execution:
 Tools like Terraform or Ansible interpret the configuration files and
automatically create or modify resources to match the desired
state.
3. Repeatability:
 The same configuration file can be used to recreate identical
environments, ensuring consistency.
IaC Tools Examples:
Terraform: Works with multiple cloud providers (AWS, Azure, GCP).
Ansible: Focuses on configuration management and application
deployment.
 137

Benefits of IaC:
 Automation: Eliminates manual configuration, saving time and
reducing errors.
 Version Control: Infrastructure code can be stored in Git
repositories, enabling collaboration and rollbacks.
 Scalability: Makes it easy to scale resources up or down as
needed.
Use Cases:
 Creating and managing cloud infrastructure for applications.
 Managing multi-cloud environments.
 Automating deployment alongside application code.
 138

Together as Build Servers

and IaC:
 Build servers and IaC tools work seamlessly together to streamline CI/CD processes.
They complement each other in the following way:
Workflow Example:
1. Code Commit: A developer commits code and infrastructure configuration (e.g,
Jenkinsfile and main.tf) to the repository.
2. Build Server Triggers: Jenkins detects the change and starts the pipeline.
3. IaC Execution: The build server uses IaC tools (e.g., Terraform) to provision or
update infrastructure (e.g., deploy a virtual machine or configure a database).
4. Application Deployment: Once infrastructure is ready, the build server deploys the
application.
5. Testing and Validation: Automated tests validate the infrastructure and
application, ensuring everything works as intended.
 139

Building by dependency
order:
 To build projects in dependency order in Jenkins, you need to
define a structure that respects the order in which dependent
modules or projects are built.
 The approach is as follows:
1. Analyze Dependencies
 Identify all project modules and their dependencies. For example,
if Module A is required by Module B and Module C, ensure Module
A builds first.
 Use tools like mvn dependency:tree or other dependency analysis
tools based on required stack.
 140

2. Implement Jenkins Pipeline:

Use a pipeline to define the order explicitly: In this setup, modules
setup sequentially repecting dependency chain.
pipeline { agent any }
stages { }
stage('Build }
Dependencies') { }
stages { stage('Build Final Module') {
steps {
stage('Build echo 'Building
Module A') { Module C’
steps { sh './build-
module-c.sh’
echo }
'Building Module A' }
}
sh './build- }
module-a.sh' // any build
command
}
}
 141

3. Using Upstream/Downstream Job Triggers:

 If projects are in separate Jenkins jobs:
 Use the "Build After Other Projects" or "Build Other
Projects" trigger to enforce the order.
 Use the Parameterized Trigger Plugin to pass variables
pipeline { stage('Trigger Module C') {
between
agent any jobs.
stages { steps {
stage('Trigger Module A') { build job: 'Module-C-Job'

steps { }
build job: 'Module-A-Job' }
}
} }
}
 142

3. Parallel Builds:
 Modules without dependencies can be built in parallel to save time.
4. Build tools for dependency management:
 Maven/Gradle: Configure dependencies in the build file to automate
the build order. Jenkins can invoke the build tool
 NPM/Yarn: Use dependency trees to determine build order in
monorepos.
5. Add Error Handling and Notifications
 Ensure any failure in the dependency chain is captured and
communicated.
6. Store Configuration in Jenkinsfile
Keep your pipeline code in a Jenkinsfile within your version control
system. This ensures consistency and versioning for build logic.
 143

 Therefore as a whole, we can say that ensure the pipeline or

job triggers reflect the dependency hierarchy.
 Use existing tools (Maven, Gradle, NPM) for dependency
management.
 And for optimization, we can leverage parallel stages for
independent modules.
 Add error handling and notifications for smooth CI/CD
processes.
 In this way we can ensure the proper dependency
management in Jenkins while keeping the builds organized
and efficient.
 144

Build Phases in Jenkins:

 There are several steps involved in Build Phases as follows:
 SCM checkout
 Build
 Test
 Deploy

Checkout phase:
 This purpose is to retrieve the source code from a version control system (e.g., Git,
Subversion).
 This process involves connecting to the repository using the configured credentials,
fetching the specified branch, tag, or commit. And then Place the source code into the
workspace for the build.
 The common tools we use in checkout phase are Git, SVN, Bitbucket, or GitHub plugins
etc.,
 145

 The checkout phase as a Jenkins pipeline:

pipeline { $class: 'GitSCM’,
agent any branches: [[name: '*/main']], //
stages { Specify the branch
stage('Checkout') { userRemoteConfigs: [[
steps { url:
script { 'https://github.com/your-repo/your-
echo 'Starting Checkout project.git’,
Phase...' credentialsId: 'your-credentials-id’
// Checkout code from ]]
Git repository ]) echo 'Source code has been checked
checkout([ out successfully!’
2.Build Phase: }}}}}
 This phase compiles the source code into executable programs or artifacts
 For this, run the build tools (e.g., Maven, Gradle, npm), Handle
dependencies, configurations, and compilation and Generate build artifacts
like JAR, WAR, binaries, or packages.
 146

pipeline { }
agent any }
environment { }
MAVEN_HOME = '/opt/maven' // post {
Path to Maven installation success {
PATH = "${MAVEN_HOME}/bin:$ echo 'Build completed
{env.PATH}" // Add Maven to PATH successfully!'
} }
stages { failure {
stage('Build') { echo 'Build failed. Check the
steps { logs for details.' }
script { }
echo 'Starting Build }
Process...'
sh 'mvn clean'
sh 'mvn package -
DskipTests'
archiveArtifacts
artifacts: 'target/*.jar', fingerprint:
true
}
 147

 Here in the Environment setup, the MAVEN_HOME and PATH variables

ensure the correct Maven version is used during the build process.
 In the Remove previous build files to ensure a fresh build environment
(mvn clean)
 Use mvn package to compile the source code and create a JAR/WAR
file, skipping tests to save time during this phase.
 Save the build artifact (e.g., JAR file) for use in subsequent stages, such
as testing or deployment.
 Echo statements log the progress and completion of the checkout
phase, making the pipeline easier to monitor.
3. Test: Finds Bugs and Potential Errors
 Purpose: Validates the quality of the code and identifies bugs through
automated tests.
 148

 Steps Involved:
 Run unit tests, integration tests, and functional tests.
 Capture and analyze test results.
 Generate and publish reports (e.g., JUnit, TestNG).
 Common Tools: JUnit, Selenium etc.,
pipeline {
stages {
stage('Test') {
steps {
sh './gradlew test'
}
}
}
}
 149

4. Deploy: Releases New Code into Production

 The purpose of deployment is to push the successfully built and
tested code to production or a staging environment.
Steps Involved:
 Deploy artifacts to a server, container registry, or cloud
environment.
 Common
pipeline { Tools: Docker, Kubernetes, Ansible, Jenkins
stages {
stage('Deploy') {
steps {
sh 'kubectl apply -f deployment.yaml' // Deploys to
Kubernetes
}
}
}
}
 150

Alternative build servers:

 There are different build servers other than Jenkins
1. GitHub Actions:
 GitHub Actions is GitHub’s native CI/CD solution that enables
automation directly from GitHub repositories.
 In GitHub Actions workflows are defined using YAML files in the
./github/workflows directory.
 We can able to trigger builds, tests and deployments on GitHub
events like pull requests, push and releases.
2. GitLab CI/CD:
GitLab CI/CD is an integrated part of the GitLab platform, enabling
you to define pipelines directly in your repository.
 151

 The way GitLab CI/CD works is that, Pipelines are configured using a .gitlab-
ci.yml file in the project. It integrates seamlessly with GitLab repositories and
triggers pipelines on push, merge, or tag events.
 The features are it is built-in CI/CD within GitLab (no need for external tools)
and also it has strong integration with GitLab's issue tracking and version
control and it supports Docker-based builds and deployment.
3. CircleCI:
 CircleCI is a cloud-based CI/CD platform that automates build, test, and
deployment workflows.
 This is configured using a config.yml file stored in the .circleci directory of the
repository. Both cloud-hosted and on-premises options are available, and a
vast ecosystem of plugins and integration points enables CircleCI to work with
any Git-based repository and integrate with a wide range of deployment
targets.
 152

4. Team City:
 TeamCity is a CI/CD tool developed by JetBrains, known for its
robust features and strong IDE integration.
 It provides a web-based interface to configure and manage build
pipelines.
 It integrates with popular version control systems and build tools.
5. Azure DevOps:
 Azure DevOps is a comprehensive suite of development tools
offered by Microsoft, including CI/CD pipelines.
 Pipelines are defined using YAML files or configured via the Azure
DevOps interface.
 This is best for enterprises using Microsoft technologies and
Azure cloud.
 153

6. Atlassian Bamboo
 Bamboo is Atlassian's highly customizable continuous integration tool
that is comparable to Jenkins, in that it is distributed as locally
installable software with which organizations can manage their CI/CD
builds on in-house servers.
 Here Projects are managed via an intuitive web interface.
 It can link builds to Jira issues and Bitbucket pull requests.
 Its key features are
 end-to-end visibility across the development lifecycle.
 Native integration with Atlassian tools.
 Strong support for branching and merging workflows.

7. CloudBees CI:
 This is an extended version of Jenkins tools to expand the CI efforts.
 154

 If an organization wants to use Jenkins but with professional

support,
 CloudBees CI might be the right choice. It builds upon and
extends Jenkins, the popular open-source CI/CD automation
server, offering enterprise-grade capabilities to streamline and
secure software delivery processes.
 It provides centralized governance and management for
Jenkins instances.
 It also supports hybrid deployments, allowing teams to build
and deploy across on-premise and cloud infrastructures.
 This is ideal for organizations with large teams and complex
CI/CD requirements.
 It has enhanced productivity with automation and
 155

Collating quality measures in Jenkins:

 Collating quality measures in Jenkins involves gathering and analyzing metrics
to ensure the software meets high standards of quality.
 This is achieved by integrating various tools and plugins that focus on specific
aspects of software quality, such as code analysis, test coverage, security, and
performance.
 There are some tools and techniques for Quality measurement
1.Using Static code analysis:
 That detects issues like bugs, code smells, and vulnerabilities early in the
development process.
 Example: SonarQube for Comprehensive tool for code quality and security
analysis.
 We implement this by adding a pipeline stage to run the static analysis tool.
 156

2. Testing:
 We can perform functionality through unit, integration, and system tests. And
the tools here we use are Junit/TestNG for Java testing frameworks and
selenium for automated testing for web applications.
3. Security Scanning:
 The objective is to identify vulnerabilities in the application and dependencies.
 The tools or plugins we use are like OWASP Dependency-Check which
scans for known vulnerabilities in libraries, Synk that helps in monitoring
dependencies for security issues. etc.,
 We implement this by adding a step to run security scans during the pipeline.
4. Performance Testing:
 This objective is to evaluate application responsiveness and stability under
load.
 157

The tools or Plugins we use in performance testing are

 Jmeter for Load and performance testing.
 Gatling for High-performance testing tool.
 By running this performance tests, we generate reports
5. Dashboards:
 Use plugins like Test Results Analyzer, SonarQube Plugin, or Build
Monitor to visualize trends and provide actionable insights for stakeholders.
6. Metrics tools:
 Use tools like SonarQube to define and enforce thresholds for metrics The
pipeline can fail if thresholds aren't met. Example: Quality Gates
Therefore by integrating these tools into your Jenkins pipeline, it can
systematically measure and maintain software quality, ensuring reliable,
secure, and performant releases.
 158

Unit 5: Testing tools and

Deployment
Testing is a systematic process used to evaluate the functionality,
performance, reliability, and security of a software application.
 It ensures the software meets the required standards and fulfills
the expectations of stakeholders.
 Various types of testing can be categorized based on the purpose
and stage of the software development lifecycle.
1. Functional Testing:
Functional testing ensures the software system performs according
to its specified requirements. This type of testing verifies that each
feature of the application functions as intended.
 159

There are different Functional testing as follows:

i. Unit Testing: This tests individual units or components of the software in
isolation. The objective of this is to validate that a specific section of the code
works correctly.
An example for this can be: testing a login function to ensure it accepts
correct credentials and rejects invalid ones.
ii. Integration Testing: This Examines the interaction between integrated
modules or components, to verify the interactions and data flow between
integrated components or modules of an application and ensures that the
combined parts of the system function as intended when they work together.
 This is mainly used to detect issues in the interaction between modules,
such as incorrect data handling, interface mismatches, or communication
errors
 To ensure that dependencies between modules, such as APIs, databases, or
external services, are handled correctly..
 160

iii. System Testing: This is used to evaluate the whole system. The
objective of this system testing is validate the software.
 For example testing an e-commerce application to ensure users can search
for products, add them to the cart, and complete the checkout process.
2. Non-Functional Testing:
Non-functional testing evaluates the performance, usability, and reliability of
the software rather than its specific functionalities.
i. Performance Testing: This assesses the speed, responsiveness, and
stability of the software under various conditions.
 The objective is to ensure the application can handle expected workloads
efficiently.
 For example measuring the time taken to load a webpage under normal
and peak user traffic.
 161

ii. Load Testing: The focus of this is to evaluates the application under expected
and maximum load conditions. This is used to determine the maximum number of
users the system can support.
 Example: Testing a web server to see how many concurrent users it can handle.
iii. Stress Testing: This Tests the system beyond its normal operational capacity.
This assess the system's robustness and its ability to recover from failure.
 Example: Simulating a sudden spike in user activity to observe system behavior.
iv. Scalability Testing: Here this determines the software's ability to scale up or
down as needed. It ensures the application can handle growth in user base or data
volume.
Example can be testing a cloud-based application for its capacity to add new
servers dynamically.
v. Security Testing: This help in identifying vulnerabilities in the software. The
objective is to protect data and ensure the application is resistant to attacks.
Example, testing for SQL injection vulnerabilities in a database.
 162

3. Maintenance Testing: Maintenance testing ensures that software

continues to operate correctly after updates or modifications. It focuses on
validating changes and preventing regressions.
i. Regression Testing: It ensures new changes do not negatively affect
existing functionality. It validates that updates or bug fixes maintain system
integrity.
 Example: Re-running test cases after adding a new feature to an existing
module.
ii. Retesting: It verifies that previously identified defects have been resolved.
The objective is to confirm that bug fixes work as intended.
Example: Testing a login feature again after resolving an issue with incorrect
password handling.
Therefore testing is helpful to address unique aspect of the software, from
verifying individual components in unit testing to evaluating overall
performance and usability in non-functional testing.
 163

Automation of Testing Pros

and Cons
 Automation in software testing is having its role in ensuring efficient,
accurate, and scalable software testing while considering the challenges
it poses.
There are several advantages of Test Automation as follows:
 Automated tests can be executed significantly faster than manual
testing, especially for repetitive tasks.
 Regression testing, which involves re-running tests for unchanged parts
of the software, can be performed efficiently with automation.
 It saves time and resources, allowing testers to focus on complex or
exploratory testing.
 164

 Automation reduces the likelihood of human error during test execution. It ensures
tests are executed exactly the same way every time.
 Test scripts, once created, can be reused across multiple versions of the software.
 Automation allows for broader coverage, including edge cases that manual testers
might overlook.
 Automation supports continuous integration pipelines, providing immediate
feedback on code changes.
 Although the initial costs are high, automation reduces long-term manual testing
expenses.
Disadvantages of Test Automation:
 The initial costs are high like buying licenses for enterprise testing tools like UFT or
training the team on Selenium.
 Automated scripts require frequent updates to align with changes in the application
or environment, for example a small feature change in UI needs reworking multiple
test cases
 165

 Writing complex test scripts for API testing may require advanced
programming skills, so sometimes automation demands a certain level
of expertise which is challenging for teams with limited experience.
 Creating automated test scripts can be time-consuming, delaying initial
testing efforts.
Selenium Framework:
 Selenium is a automation testing framework which is an open-source
that enables users to automate web applications across different
browsers and platforms.
 It supports multiple programming languages, including Java, Python,
C#, Ruby, JavaScript
 Selenium is highly extensible and forms the backbone for many test
automation frameworks.
166
 167

Selenium Core Components:

1. Selenium IDE (Integrated Development Environment):
 This is a browser extension for creating and running simple test scripts, primarily used for
record and playback testing and is suitable for beginners for a quick test case development
 This selenium is limited to browser testing and it lacks advanced programming capabilities
2. Selenium RC:
 RC stands for Remote Control. Selenium RC is a server, written in Java, that accepts
commands for HTTP-based browsers.
 It supports other programming languages such as Ruby, PHP, Python, Perl, C#, Java, and
Javascript for automation testing. It runs on almost ever web browser and operating
system.
 With the help of the other component, Selenium Grid, it supports the parallel execution of
various test cases.
 Executing a test program on Selenium RC requires starting the Selenium server manually.
To overcome this manual start, the Selenium WebDriver was launched.
 168

3. Selenium WebDriver:
 This WebDriver is a powerful and flexible api for creating, executing test scripts across
multiple browsers. Also this supports advanced programming features like loops,
conditions and other external configurations.
 It provides cross-platform and cross-browser testing capabilities.
 It can be able to work seamlessly with browsers such as Chrome, Firefox, Safari, and Edge.
 This Selenium webdriver is introduced as selenium 2.0 and is successor of Selenium
Remote Control, consisting of API, Library, Driver, and Framework that work together to
enable test execution.
This webdriver has four different components:
i. Selenium Language Bindings or Client Libraries:
 The language bindings or client libraries support multiple programming languages like
Java, Python, C#, and JavaScript. These libraries are jar files that contain Selenium
WebDriver classes and methods needed for automation test scripts.
 Selenium client libraries are not a testing framework, but rather an application
programming interface (API) for running Selenium commands from within the program.
 169

ii. JSON Wire Protocol: JSON means JavaScript Object Notation that supports
data interchange formats in languages like Java, C#, Python, Ruby, etc. It helps
to exchange data between the client and server on the web.
iii. Browser Drivers: This acts as a act as a bridge between Selenium
WebDriver and the web browser, because selenium WebDriver doesn’t
interact directly with the browser. Instead, it communicates through a browser-
specific driver, which translates the commands from WebDriver into browser-
understandable actions.
Using the wire protocol, WebDriver implements the appropriate operations for
each browser based on the requests received from language bindings.
 The communication between WebDriver and the browser driver happens using
the WebDriver Wire Protocol, where this protocol transmits JSON-formatted
commands over HTTP.
 The browser driver receives these commands, and performs the necessary
actions within the browser, such as clicking a button, navigating to a URL etc.,
 170

4. Selenium Grid
 The Selenium Grid is a tool used with Selenium RC to execute tests parallelly
on different machines and browsers. It reduces the execution time of the
testing suite because it runs parallel execution of tests across different
machines and browsers.
 It consists of two major components, a hub which is a central machine to
trigger test cases within selenium grid and other is Nodes, that execute test
cases loaded on a hub, launching on multiple machines, platforms, and
browsers.
 171

 For example, suppose you have a Gmail account. You have

entered a user id and password. After entering credentials, it will
send your request to the server and the server will send your
request to the data base.
 Then the server will take the response from that database and
the response will be given back to the application. Whatever test
cases we create, these test cases are converted into the test
script. After that, the test script is compiled and ran against the
selenium server (Selenium RC).
 Selenium RC uses a series of JavaScript libraries to perform the
actions such as clicking on a button, entering the text into the
textbox, selecting a value from drop-down, etc. against web
browsers.
 172

Steps to Setup Selenium Testing Environment:

https://www.selenium.dev/downloads/
 Install Java and configure Java Path
 Install IDE (Eclipse, IntelliJ IDEA)
 Create a new Java Project
 Download and configure selenium jars
 Download and configure browser driver
 Create first test script and execute.
 173

JavaScript Testing
 JavaScript testing ensures that JavaScript code behaves as expected in a web application.
 Since JavaScript drives the dynamic and interactive features of web applications, testing
verifies that these functionalities work correctly across browsers and devices.
Importance or Significance of JS Testing:
 JavaScript is responsible for tasks like form validation, real-time updates, and user
interactions. Testing ensures these tasks operate smoothly.
 Identifying bugs during development prevents issues from appearing in production,
saving time and costs.
 Properly tested JavaScript enhances the reliability and performance of web applications,
leading to better usability.
 JavaScript can behave differently across browsers, so testing helps maintain a consistent
user experience.
 174

There are different types of JS Testing:

 Unit Testing: Focuses on testing small, isolated pieces of JavaScript code, such
as individual functions.
 Functional Testing: Tests specific features to ensure they work as intended, for
example working of a button or navigation to another page in a web application.
 Integration Testing:Verifies that different parts of the application (like modules
or APIs) work together correctly, For example working of a different modules in
a e-commerce website like shopping, fetching items, checkouts
 Performance Testing: Measures how fast and efficiently JavaScript runs under
different scenarios. Testing how quickly a search feature returns results.
 Regression Testing: This ensures new changes don’t disrupt existing
functionality, for example adding new ribbon doesn’t affect the other
components in the application.
 175

 The JavaScript testing process includes: Writing the test cases,

Running test, analyzing the results, regressing testing that is
retest after making changes to ensure no functionality is broken
and cross browser testing that is running tests across multiple
browsers to ensure consistent behavior.
 But there are some challenges with JS Testing:
 In case of asynchronous code the usage of async/await and
callbacks can be complex
 Rapid changes in JavaScript codebases may require continuous
updating of test cases.
 JavaScript behavior may differ between browsers, requiring
extensive cross-browser testing.
 Some Popular JS testing frameworks and tools are: Jest, Mocha,
Cypress etc.,
 176

Test backend integration points

 Testing backend integration points is essential to ensure that your backend services are
working correctly with the frontend and any other external systems.
 This includes testing different components like application logic, APIs etc such that testing
whether these backend server-side components are meeting the requirements.
 The areas where the backend points tested are like: Databases, Api’s, application logic etc
 Databases: Testing database access, queries, schemas, connectivity, performance, and
integrity. Popular databases include MySQL, MongoDB, Cassandra, and PostgreSQL.
 Application Logic: Tetsing the workflow of a system, the core functions and algorithms
 Api’s: Testing application programming interfaces that connect the frontend user interface
with backend services and databases. This includes testing API endpoints,
 177

 This can also include testing of microservices, communication

between frontend and backend and other backend services.
 The network connections, load and speed and security are tested.
 This backend testing helps in ensuring reliable functioning of all
server-side operations.
 Therefore different testing strategies can be followed like Unit
Testing, Security testing, Performance testing, Database testing
(that is validating and evaluating whether the backend databases
queries, schemas and connections are intact)
 These testing can be again driven as manually or it can be
automated which has its own pros and cons.
The steps to perform testing backend points are like:
 178

 Step1: Identify the parts of backend system points that needs to

be tested.
 Step2: Identify the kind of testing strategy suitable for your
backend system like functional or non-functional and which
subtype is needed.
 Step3: Analyse the risks that can occur in the backend system
priorly that reduces the time to focus on required points on
backend.
 Step4: Select the testing tools suitable for your application. Like
selenium, Jmeter or SonaQube etc.,
Therefore this testing backend points helps in finding bugs early,
improving security and architecture.
 179

Test Driven Development:

 TDD test driven development is that, developers write the automation test scripts
before the actual code.
 It is an iterative approach combining Programming, Unit Test Creation, and
Refactoring.
 Before starting this, it is needed a clear understanding on requirements, then
writing for atomic parts of system has to be done
 The process follows a repetitive cycle known as Red-Green-Refactor.
 In the red phase: developer writes a test that defines a desired feature.
 In green phase: they write the minimum code necessary to pass the test
 Refactor: here the code is refactored for optimization while ensuring the test still
passes.
 180

 Therefore TDD helps ensure that the codebase remains reliable

and bug-free by catching errors early in the development process.
 TDD leads to higher code coverage and makes future
modifications or refactoring easier and safer, knowing that
existing functionality is thoroughly tested.
 The TDD approach originates from the Agile manifesto principles
and Extreme programming.
 In TDD, developers create small test cases for every feature
based on their initial understanding.
 TDD follows an iterative process, where small chunks of code are
developed, tested, and refined until they pass all tests. The code
is usually tested once and then refined based on the results in
traditional testing.
 181

The Test Driven Development has following steps:

 Create the tests: Unit tests has to be created for specific
features.
 Correcting the code: Once a test fails, developers must make the
minimal changes required to update the code to run successfully
when re-executed.
 Refactor the Code: check for redundancy or any possible code
optimizations to enhance overall performance. Also, refactoring
should not affect the other behaviour of program.
Examples: Junit, TestNG, PyUnit etc.,
Therefore by adopting TDD, teams can catch issues early in the
development cycle, reduce the likelihood of bugs, and create
maintainable, scalable codebases.
 182

REPL-driven development
 REPL-Driven Development is an interactive and exploratory approach to
programming where you actively use a REPL (Read-Eval-Print Loop) to write, test,
and refine code in real-time.
 Instead of writing a complete program and then running it to see the results, you
write snippets of code, evaluate them immediately, and get instant feedback.
 This feedback loop helps you quickly identify issues, experiment with ideas, and
understand how your code behaves.
 Read: The REPL reads the user’s input (a line of code or an expression).
 Eval: It evaluates the input to produce a result.
 Print: It displays the result of the evaluation to the user.
 Loop: The REPL waits for the next input and repeats the process.
 183

 The REPL acts as a live coding environment where you can type code line-
by-line, test ideas, and see instant results.
 REPL has Interactive coding, Instant feedback, live debugging and
incremental development that is building application step-by-step and
testing functionality.
 But there are some limitations as follows:
 It is not suitable for complex systems. For large applications, you need a
structured workflow with proper testing and version control.
 Without proper structure and automation, relying solely on REPL can lead to
missed edge cases. Therefore it is Error Prone.
 Works well for small functions or snippets but not for complex one’s therefore
there is scalability issue.
 We generally use REPL :
 when learning a new Language where A REPL is perfect for understanding the
syntax and behavior of a new programming language.
 184

 Try out new libraries or APIs without setting up a full project.

 Useful for developing small scripts or one-off tools.
 Languages like JS, Python, Ruby, Java, Go, Scala etc., support
REPL driven development.
 Therefore this helps in getting immediate results, Testing ideas
and iterate on them quickly without setting up a full project
structure. it excels at learning, prototyping, and exploratory
programming even though it’s not a substitute for structured
development in larger projects
 By combining REPL use with best practices like saving code and
transitioning to proper workflows, developers can significantly
enhance productivity.
 185

Deployment Systems:
 When you deploy an application, you’re essentially transferring all the components
(like code, configuration files, and dependencies) to an environment where it can
be used such as a production server, a container, or a cloud service.
 Deployment systems manage this process by automating repetitive tasks, handling
complex configurations, and monitoring the application to catch issues early.
 There are several tools and deployment platforms
 CI/CD Pipelines: These automate the entire deployment process example: Jenkins,
GitHubActions, GitLab CI/CD.
 Containers and Orchestration: Docker and Kubernetes.
 Serverless Platforms and Cloud platforms like AWS Lambda, Microsoft azure app service,
Google cloud functions
 IAC tools that automate and standardize like Terraform, Ansible, Puppet and chef etc.,
 186

 Generally Deployment systems will be working such that it is

used to automate everything to minimize manual steps and
reduce errors.
 It is used to track changes in configurations and deployment
scripts, also helps in incorporating automated tests into the
deployment pipeline.
 It also use monitoring tools to catch issues immediately, it has
plan for failures and have a rollback strategy ready.
 Use tools like Docker or Terraform to replicate environments
reliably.
 Therefore they combine tools for automation, testing, monitoring,
and rollback to create reliable, scalable, and efficient workflows.
Also choosing the right deployment strategy and tools is essential
 187

Virtualization stacks
 Virtualization stacks are systems or layers of technology that
enable multiple virtual environments to run on a single physical
machine.
 These environments can be virtual machines (VMs), containers,
or other isolated workloads.
 There are different kinds of Virtualization stacks like traditional
Virtual Machine Stacks like Hyper-V, Vmware
 Container based like docker, Kuberenetes,
 Cloud based and Serverless virtualization stacks etc.,
 188

 Generally it has a hardware abstraction, for example the

hypervisor abstracts physical hardware, dividing it into virtual
hardware units that can be allocated to virtual environments.
 Virtualization stacks provide tools for centralized deployment,
scaling, monitoring, and troubleshooting.
 Virtual environments are isolated from each other, ensuring
security and stability.
 Resources like CPU, memory, and storage can be dynamically
allocated to meet the needs of virtual machines or containers.
 There can be virtualization seen like hybrid virtualization which
combines both traditional vm’s and containers, Moving beyond
traditional VMs and containers to manage specific workloads
efficiently which is serverless computing etc.,
 189

Code Execution at client

 Client-side code execution refers to running code directly on a user's device, typically
within a web browser or application environment.
 Generally the server sends the code (usually JavaScript) to the client device as part
of a web page or application. The client’s browser or runtime environment
( example : Node.js ) interprets and executes the code. Tasks such as user interface
updates, animations, form validations, or even complex computations are carried out
locally.
 For example: Google Maps uses client-side execution to render maps dynamically.
 We use technologies like JS, HTML, CSS and other frameworks etc.,
 It helps in reducing load o server side, improves efficiency which enables faster
responses since much of the processing happens locally without waiting for server
communication.
 190

 But there are challenges like browser compatibility, security concerns and
device dependence also the load time because transferring and initializing
large client-side codebases can slow down the first load of an application.
Puppet Master and Agents:
 Puppet is a powerful automation tool designed to manage and configure
infrastructure. It follows a client-server model with two key components:
Puppet Master and Puppet Agents.
 This structure helps to automate the management of systems, ensuring
they remain in a desired state without manual intervention.
 Puppet Master: This is the central controller in the system, responsible
for managing configurations and distributing them.
 Puppet Agents: These are clients or nodes (typically servers) that receive
configuration instructions from the Puppet Master and apply them to their
respective systems.
 191

Puppet Master:
 The Puppet Master serves as the control center where configurations are
defined, stored, and compiled. It does not directly execute tasks but
coordinates and oversees the process.
 The Puppet Master holds manifests (configuration files) that define how each
system should be configured. Based on the requests from the Puppet Agents,
the Puppet Master creates a catalog that is a list of specific resources (files,
services, packages) that need to be configured on the Agent machine. And
once compiled, the catalog is sent to the Puppet Agents for execution.
 It allows system administrators to manage infrastructure from a single point,
ensuring consistency across all machines.
 The Puppet Master communicates with the Puppet Agents over a secure
HTTPS connection, sending compiled catalogs, while Agents report back their
status and changes.
 192

Puppet Agents:
 Puppet Agents are installed on individual nodes (servers or machines) that
need to be managed. They are the "clients" in this setup and handle
executing the configuration on each system.
 Periodically, the Puppet Agent contacts the Puppet Master to request the
latest configuration.
 Upon receiving the configuration catalog, the Puppet Agent applies the
required changes to the system—such as installing software, modifying
files, or restarting services.
 After the configuration is applied, the Puppet Agent reports back to the
Puppet Master, informing it of any changes or issues.
 Agents initiate communication with the Puppet Master, requesting
configuration updates. The agent checks in regularly (e.g., every 30
minutes) but can be manually triggered to request an update as needed.
 193

Advantages :
 It has centralized control where Puppet Master provides a single
point to manage configurations for all nodes, making system
administration simpler and more consistent.
 The Puppet Master can manage hundreds or even thousands of
nodes.
 Puppet automates configuration, reducing the need for manual
intervention and making infrastructure management scalable.
 The Puppet Agents run configuration tasks locally, reducing the
load on the Puppet Master and allowing faster application of
configurations.
 With Puppet, you can define a desired state for all your systems,
ensuring they remain consistent regardless of any changes or
 194

Ansible
 Ansible is an open-source automation tool used to simplify IT management tasks such
as configuring systems, deploying applications, and automating workflows.
 It allows system administrators to automate repetitive actions, streamlining
operations and ensuring that configurations are consistent and scalable across an
entire infrastructure.
 Ansible operates by connecting to remote machines via SSH (or WinRM for Windows)
and executing predefined tasks to configure and manage those systems.
 Ansible is agentless. This means it connects to systems directly via SSH eliminating
the need for additional software on the target machines.
 Ansible follows a declarative model, where you describe the desired state of a system
rather than specifying the exact steps, you tell Ansible what you want, and it ensures
the system reaches that state.
 195

 Ansible uses modules to carry out tasks on the target machines.

Modules perform specific operations, such as installing software,
managing services, modifying files, etc
 Ansible provides many built-in modules, but you can also create
custom ones when necessary.
 A playbook is written to define the desired actions. For example, a
playbook might specify that a specific package should be
installed or a service should be started on a target system. When
you run the playbook, Ansible makes sure these tasks are
executed in the correct order.
 One of the biggest benefits of Ansible is its idempotent nature.
This means that running the same playbook multiple times on the
same system won't cause any unintended changes or errors.
 196

 Ansible has several advantages like :

 It does not need any agents to be installed and manage them like
puppets
 The use of YAML makes playbooks easy to understand, even for
non-technical users. This lowers the learning curve and improves
team collaboration. Therefore it is human readable.
 Ansible works well in a variety of environments whether
managing on-premises infrastructure, cloud-based systems, or
hybrid environments.
 With Ansible, you can automate virtually every aspect of system
configuration, application deployment, and infrastructure
management, reducing manual work and ensuring consistency
across your environment.
 197

Chef
 Chef is an open-source automation tool used for configuring and managing
servers, applications, and services across a large-scale infrastructure.
 It automates tasks such as software installation, configuration management,
and system maintenance, ensuring consistency and efficiency.
 It has components like follows:
 Chef Server: The Chef Server acts as the central repository where
configurations are stored. It manages the cookbooks, recipes, and metadata
for each node in the infrastructure.
 The Chef Client is installed on each machine (node) that you want to manage.
It communicates with the Chef Server to retrieve and apply configurations on
the node, ensuring the system stays in the desired state.
 198

 The Chef Workstation is where administrators develop, test, and manage

Chef cookbooks and recipes. It provides the environment where you
create the automation scripts and configurations for nodes.
 Cookbooks are the primary unit of configuration in Chef. They contain
recipes, which define the actions to be performed on nodes, such as
installing packages or modifying system settings. Cookbooks can include
templates, attributes, and other components that support the recipes.
 Recipes within cookbooks define the desired system state and specify the
actions to take on the node, such as installing software or managing
services. Recipes are written in Ruby, allowing for complex configurations.
 Here, Chef Workstation is used for writing cookbooks and recipes. And
then Recipes are pushed to the Chef Server, which stores them for
access by nodes. Then Chef Clients periodically contact the Chef Server
to fetch configurations and apply them to the local systems.
 199

 We generally use chef for scalability, flexibility and in large ecosystem.

Because chef automates complex configurations across many systems and
it has a wealth of pre-built cookbooks and resources available through its
active community.
SaltStack:
 SaltStack (often referred to as Salt) is an open-source automation platform
designed to manage and configure systems.
 It provides features for infrastructure automation, configuration
management, and orchestration, making it ideal for large-scale
deployments.
Salt Stack has the following components:
 Salt Master: The Salt Master is the central server responsible for sending
commands and configurations to the managed nodes, known as Minions. It
controls the overall orchestration and automation tasks.
 200

Some of the components in salt works as follows:

 Salt Minions: These are the target systems that are managed by the Salt
Master. These agents receive commands from the Salt Master and
execute the tasks, such as applying configurations or installing packages.
 State files in Salt are written in YAML and define the system
configuration. They describe the desired state of the system, such as
which packages should be installed and which services should be
running. State files are comparable to Chef's recipes.
 Execution modules perform the actual tasks on the Minions. For example,
installing a package, restarting a service, or modifying a file. These
modules are invoked from the Master to be executed on the Minions.
The working of saltstack is as follows:
 The Salt Master sends configuration commands to the Salt Minions.
 201

 The Minions execute these commands, ensuring the system

meets the desired state as defined in the state files.
 Salt allows for real-time communication between the Master and
Minions, ensuring fast execution of tasks across many systems.
Advantages:
 Salt enables fast, real-time execution of tasks across multiple
nodes, making it ideal for high-demand environments.
 Salt can efficiently manage large infrastructures, handling
thousands of systems with minimal overhead.
 Salt supports both push and pull models for executing
configurations, providing flexibility in how tasks are applied.
 202

While we can see some differences between salt stack and chef, like
 Chef uses a client-server model, where the Chef Workstation, Chef Server, and
Chef Clients interact to automate tasks. Whereas SaltStack follows a master-
minion model, where the Salt Master manages the Salt Minions and sends them
commands to execute.
 Chef typically operates in a pull model, where clients request configurations from
the server. Whereas saltstack can use both a push and pull model, giving it more
flexibility in execution.
 But as part of performance SaltStack is known for its speed and can execute
commands across large numbers of systems quickly, making it suitable for real-
time automation. Whereas Chef may be slower compared to SaltStack when
managing a large number of systems due to its more complex configuration
process.
 As part of learning also SaltStack is eay to learn especially with its YAML-based
configuration, and is known for its straightforward setup. Because Chef uses ruby-
bases syntax having complex architecture.
 203

 Therefore we can say that both Chef and SaltStack are powerful
tools for automating infrastructure management having there
unique capabilities. Chef is ideal for complex environments, while
SaltStack is designed for speed and scalability, making it perfect
for real-time tasks and large infrastructures. And we can choose
anyone among according to the requirements.
Jeevitha
https://www.linkedin.com/in/kjeevitha771
[email protected]
9100090987

Thank You