4. Devops Unit 1 Theory Notes
4. Devops Unit 1 Theory Notes
Introduction
The DevOps is the combination of two words, one is Development and other is Operations. It
is a culture to promote the development and operation process collectively.
The DevOps tutorial will help you to learn DevOps basics and provide depth knowledge of
various DevOps tools such as Git, Ansible, Docker, Puppet, Jenkins, Chef, Nagios,
and Kubernetes.
What is DevOps?
Why DevOps?
Before going further, we need to understand why we need the DevOps over the other methods.
o In 2009, the first conference named DevOpsdays was held in Ghent Belgium. Belgian
consultant and Patrick Debois founded the conference.
o In 2012, the state of DevOps report was launched and conceived by Alanna Brown at
Puppet.
o In 2014, the annual State of DevOps report was published by Nicole Forsgren, Jez
Humble, Gene Kim, and others. They found DevOps adoption was accelerating in 2014
also.
o In 2015, Nicole Forsgren, Gene Kim, and Jez Humble founded DORA (DevOps Research
and Assignment).
In 2017, Nicole Forsgren, Gene Kim, and Jez Humble published "Accelerate:
Building and Scaling High Performing Technology Organizations".
A software life cycle model (also termed process model) is a pictorial and diagrammatic
representation of the software life cycle. A life cycle model represents all the methods required
to make a software product transit through its life cycle stages. It also captures the structure in
which these methods are to be undertaken.
The senior members of the team perform it with inputs from all the stakeholders and domain
experts or SMEs in the industry.
Planning for the quality assurance requirements and identifications of the risks associated with
the projects is also done at this stage.
Business analyst and Project organizer set up a meeting with the client to gather all the data like
what the customer wants to build, who will be the end user, what is the objective of the product.
Before creating a product, a core understanding or knowledge of the product is very necessary.
For Example, A client wants to have an application which concerns money transactions. In this
method, the requirement has to be precise like what kind of operations will be done, how it will
be done, in which currency it will be done, etc.
Once the required function is done, an analysis is complete with auditing the feasibility of the
growth of a product. In case of any ambiguity, a signal is set up for further discussion.
Once the requirement is understood, the SRS (Software Requirement Specification) document is
created. The developers should thoroughly follow this document and also should be reviewed by
the customer for future reference.
Once the requirement analysis is done, the next stage is to certainly represent and document the
software requirements and get them accepted from the project stakeholders.
The next phase is about to bring down all the knowledge of requirements, analysis, and design
of the software project. This phase is the product of the last two, like inputs from the customer
and requirement gathering.
In this phase of SDLC, the actual development begins, and the programming is built. The
implementation of design begins concerning writing code. Developers have to follow the coding
guidelines described by their management and programming tools like compilers, interpreters,
debuggers, etc. are used to develop and implement the code.
Stage5: Testing
After the code is generated, it is tested against the requirements to make sure that the products
are solving the needs addressed and gathered during the requirements stage.
During this stage, unit testing, integration testing, system testing, acceptance testing are done.
Stage6: Deployment
Once the software is certified, and no bugs or errors are stated, then it is deployed.
Then based on the assessment, the software may be released as it is or with suggested
enhancement in the object segment.
Stage7: Maintenance
Once when the client starts using the developed systems, then the real issues come up and
requirements to be solved from time to time.
This procedure where the care is taken for the developed product is known as maintenance.
Waterfall model
Winston Royce introduced the Waterfall Model in 1970.This model has five phases:
Requirements analysis and specification, design, implementation, and unit testing, integration
and system testing, and operation and maintenance. The steps always follow in this order and
do not overlap. The developer must complete every phase before the next phase begins. This
model is named "Waterfall Model", because its diagrammatic representation resembles a
cascade of waterfalls.
1. Requirements analysis and specification phase: The aim of this phase is to understand the
exact requirements of the customer and to document them properly. Both the customer and the
software developer work together so as to document all the functions, performance, and
interfacing requirement of the software. It describes the "what" of the system to be produced and
not "how."In this phase, a large document called Software Requirement Specification
(SRS) document is created which contained a detailed description of what the system will do in
the common language.
2. Design Phase: This phase aims to transform the requirements gathered in the SRS into a
suitable form which permits further coding in a programming language. It defines the overall
software architecture together with high level and detailed design. All this work is documented
as a Software Design Document (SDD).
3. Implementation and unit testing: During this phase, design is implemented. If the SDD is
complete, the implementation or coding phase proceeds smoothly, because all the information
needed by software developers is contained in the SDD.
During testing, the code is thoroughly examined and modified. Small modules are tested in
isolation initially. After that these modules are tested by writing some overhead code to check
the interaction between these modules and the flow of intermediate output.
4. Integration and System Testing: This phase is highly crucial as the quality of the end product
is determined by the effectiveness of the testing carried out. The better output will lead to
satisfied customers, lower maintenance costs, and accurate results. Unit testing determines the
efficiency of individual modules. However, in this phase, the modules are tested for their
interactions with each other and with the system.
5. Operation and maintenance phase: Maintenance is the task performed by every user once
the software has been delivered to the customer, installed, and operational.
Agile development
An agile methodology is an iterative approach to software development. Each iteration of agile
methodology takes a short time interval of 1 to 4 weeks. The agile development process is aligned
to deliver the changing business requirement. It distributes the software with faster and fewer
changes.
The single-phase software development takes 6 to 18 months. In single-phase development, all
the requirement gathering and risks management factors are predicted initially.
The agile software development process frequently takes the feedback of workable product. The
workable product is delivered within 1 to 4 weeks of iteration.
The Agile model first emerged in 2001 and has since become the de facto industry standard.
Some businesses value the Agile methodology so much that they apply it to other types of
projects, including nontech initiatives.
In the Agile model, fast failure is a good thing. This approach produces ongoing release cycles,
each featuring small, incremental changes from the previous release. At each iteration, the
product is tested. The Agile model helps teams identify and address small issues on projects
before they evolve into more significant problems, and it engages business stakeholders to give
feedback throughout the development process.
As part of their embrace of this methodology, many teams also apply an Agile framework known
as Scrum to help structure more complex development projects. Scrum teams work in sprints,
which usually last two to four weeks, to complete assigned tasks. Daily Scrum meetings help the
whole team monitor progress throughout the project. And the ScrumMaster is tasked with
keeping the team focused on its goal.
DevOps Life Cycle
The DevOps methodology is a relative newcomer to the SDLC scene. It emerged from two
trends: the application of Agile and Lean practices to operations work, and the general shift in
business toward seeing the value of collaboration between development and operations staff at
all stages of the SDLC process.
In a DevOps model, Developers and Operations teams work together closely — and sometimes
as one team — to accelerate innovation and the deployment of higher-quality and more reliable
software products and functionalities. Updates to products are small but frequent. Discipline,
continuous feedback and process improvement, and automation of manual development
processes are all hallmarks of the DevOps model.
Amazon Web Services describes DevOps as the combination of cultural philosophies, practices,
and tools that increases an organization’s ability to deliver applications and services at high
velocity, evolving and improving products at a faster pace than organizations using traditional
software development and infrastructure management processes. So like many SDLC models,
DevOps is not only an approach to planning and executing work, but also a philosophy that
demands a nontraditional mindset in an organization.
Choosing the right SDLC methodology for your software development project requires careful
thought. But keep in mind that a model for planning and guiding your project is only one
ingredient for success. Even more important is assembling a solid team of skilled talent
committed to moving the project forward through every unexpected challenge or setback.
DevOps Lifecycle
DevOps defines an agile relationship between operations and Development. It is a process that
is practiced by the development team and operational engineers together from beginning to the
final stage of the product.
Learning DevOps is not complete without understanding the DevOps lifecycle phases. The
DevOps lifecycle includes seven phases as given below:
1) Continuous Development
This phase involves the planning and coding of the software. The vision of the project is decided
during the planning phase. And the developers begin developing the code for the application.
There are no DevOps tools that are required for planning, but there are several tools for
maintaining the code.
2) Continuous Integration
This stage is the heart of the entire DevOps lifecycle. It is a software development practice in
which the developers require to commit changes to the source code more frequently. This may
be on a daily or weekly basis. Then every commit is built, and this allows early detection of
problems if they are present. Building code is not only involved compilation, but it also
includes unit testing, integration testing, code review, and packaging.
The code supporting new functionality is continuously integrated with the existing code.
Therefore, there is continuous development of software. The updated code needs to be integrated
continuously and smoothly with the systems to reflect changes to the end-users.
Jenkins is a popular tool used in this phase. Whenever there is a change in the Git repository,
then Jenkins fetches the updated code and prepares a build of that code, which is an executable
file in the form of war or jar. Then this build is forwarded to the test server or the production
server.
3) Continuous Testing
This phase, where the developed software is continuously testing for bugs. For constant testing,
automation testing tools such as TestNG, JUnit, Selenium, etc are used. These tools allow QAs
to test multiple code-bases thoroughly in parallel to ensure that there is no flaw in the
functionality. In this phase, Docker Containers can be used for simulating the test environment.
Selenium does the automation testing, and TestNG generates the reports. This entire testing
phase can automate with the help of a Continuous Integration tool called Jenkins.
Automation testing saves a lot of time and effort for executing the tests instead of doing this
manually. Apart from that, report generation is a big plus. The task of evaluating the test cases
that failed in a test suite gets simpler. Also, we can schedule the execution of the test cases at
predefined times. After testing, the code is continuously integrated with the existing code.
4) Continuous Monitoring
Monitoring is a phase that involves all the operational factors of the entire DevOps process,
where important information about the use of the software is recorded and carefully processed
to find out trends and identify problem areas. Usually, the monitoring is integrated within the
operational capabilities of the software application.
It may occur in the form of documentation files or maybe produce large-scale data about the
application parameters when it is in a continuous use position. The system errors such as server
not reachable, low memory, etc are resolved in this phase. It maintains the security and
availability of the service.
5) Continuous Feedback
The application development is consistently improved by analyzing the results from the
operations of the software. This is carried out by placing the critical phase of constant feedback
between the operations and the development of the next version of the current software
application.
The continuity is the essential factor in the DevOps as it removes the unnecessary steps which
are required to take a software application from development, using it to find out its issues and
then producing a better version. It kills the efficiency that may be possible with the app and
reduce the number of interested customers.
6) Continuous Deployment
In this phase, the code is deployed to the production servers. Also, it is essential to ensure that
the code is correctly used on all the servers.
The new code is deployed continuously, and configuration management tools play an essential
role in executing tasks frequently and quickly. Here are some popular tools which are used in
this phase, such as Chef, Puppet, Ansible, and SaltStack.
Containerization tools help to maintain consistency across the environments where the
application is tested, developed, and deployed. There is no chance of errors or failure in the
production environment as they package and replicate the same dependencies and packages used
in the testing, development, and staging environment. It makes the application easy to run on
different computers.
DevOps Model
The DevOps model goes through several phases governed by cross-discipline teams.
Those phases are as follows:
Planning,Identify,andTrack Using the latest in project management tools and agile practices,
track ideas and workflows visually. This gives all important stakeholders a clear pathway to
prioritization and better results. With better oversight, project managers can ensure teams are on
the right track and aware of potential obstacles and pitfalls. All applicable teams can better work
together to solve any problems in the development process.
Development Phase Version control systems help developers continuously code, ensuring one
patch connects seamlessly with the master branch. Each complete feature triggers the developer
to submit a request that, if approved, allows the changes to replace existing code. Development
is ongoing.
Testing Phase After a build is completed in development, it is sent to QA testing. Catching bugs
is important to the user experience, in DevOps bug testing happens early and often. Practices like
continuous integration allow developers to use automation to build and test as a cornerstone of
continuous development.
Deployment Phase In the deployment phase, most businesses strive to achieve continuous
delivery. This means enterprises have mastered the art of manual deployment. After bugs have
been detected and resolved, and the user experience has been perfected, a final team is
responsible for the manual deployment. By contrast, continuous deployment is a DevOps
approach that automates deployment after QA testing has been completed.
Benefits of DevOps
A properly implemented DevOps approach comes with a number of benefits. These include the
following that we selected to highlight:
Decrease Cost Of primary concern for businesses is operational cost, DevOps helps
organizations keep their costs low. Because efficiency gets a boost with DevOps practices,
software production increases and businesses see decreases in overall cost for production.
Customers are Served User experience, and by design, user feedback is important to the
DevOps process. By gathering information from clients and acting on it, those who practice
DevOps ensure that clients wants and needs get honored, and customer satisfaction reaches new
highs
.
It Gets More Efficient with TimeDevOps simplifies the development lifecycle, which in
previous iterations had been increasingly complex. This ensures greater efficiency throughout a
DevOps organization, as does the fact that gathering requirements also gets easier. In DevOps,
requirements gathering is a streamlined process, a culture of accountability, collaboration and
transparency makes requirements gathering a smooth going team effort where no stone is left
unturned.
DevOps Architecture
Development and operations both play essential roles in order to deliver applications. The
deployment comprises analyzing the requirements, designing, developing, and testing of the
software components or frameworks.
The operation consists of the administrative processes, services, and support for the software.
When both the development and operations are combined with collaborating, then the DevOps
architecture is the solution to fix the gap between deployment and operation terms; therefore,
delivery can be faster.
DevOps architecture is used for the applications hosted on the cloud platform and large
distributed applications. Agile Development is used in the DevOps architecture so that
integration and delivery can be contiguous. When the development and operations team works
separately from each other, then it is time-consuming to design, test, and deploy. And if the
terms are not in sync with each other, then it may cause a delay in the delivery. So DevOps
enables the teams to change their shortcomings and increases productivity.
Below are the various components that are used in the DevOps architecture
1) Build
Without DevOps, the cost of the consumption of the resources was evaluated based on the pre-
defined individual usage with fixed hardware allocation. And with DevOps, the usage of cloud,
sharing of resources comes into the picture, and the build is dependent upon the user's need,
which is a mechanism to control the usage of resources or capacity.
2) Code
Many good practices such as Git enables the code to be used, which ensures writing the code for
business, helps to track changes, getting notified about the reason behind the difference in the
actual and the expected output, and if necessary reverting to the original code developed. The
code can be appropriately arranged in files, folders, etc. And they can be reused.
3) Test
The application will be ready for production after testing. In the case of manual testing, it
consumes more time in testing and moving the code to the output. The testing can be automated,
which decreases the time for testing so that the time to deploy the code to production can be
reduced as automating the running of the scripts will remove many manual steps.
4) Plan
DevOps use Agile methodology to plan the development. With the operations and development
team in sync, it helps in organizing the work to plan accordingly to increase productivity.
5) Monitor
Continuous monitoring is used to identify any risk of failure. Also, it helps in tracking the system
accurately so that the health of the application can be checked. The monitoring becomes more
comfortable with services where the log data may get monitored through many third-party tools
such as Splunk.
6) Deploy
Many systems can support the scheduler for automated deployment. The cloud management
platform enables users to capture accurate insights and view the optimization scenario, analytics
on trends by the deployment of dashboards.
7) Operate
DevOps changes the way traditional approach of developing and testing separately. The teams
operate in a collaborative way where both the teams actively participate throughout the service
lifecycle. The operation team interacts with developers, and they come up with a monitoring plan
which serves the IT and business requirements.
8) Release
Deployment to an environment can be done by automation. But when the deployment is made to
the production environment, it is done by manual triggering. Many processes involved in release
management commonly used to do the deployment in the production environment manually to
lessen the impact on the customers.
DevOps resilience
DevOps resilience refers to the ability of a DevOps system to withstand and recover from failures
and disruptions. This means ensuring that the systems and processes used in DevOps are robust,
scalable, and able to adapt to changing conditions. Some of the key components of DevOps
resilience include:
4. Continuous testing: Continuously testing systems and applications can help identify and
fix issues before they become critical.
5. High availability: Designing systems for high availability helps to ensure that systems
remain up and running even in the event of failures or disruptions.
By focusing on these components, DevOps teams can create a resilient and adaptive DevOps
system that is able to deliver high-quality applications and services, even in the face of failures
and disruptions.
The Agile wheel of wheels
There are several different cycles in Agile development, from the Portfolio level
through to the Scrum and Kanban cycles and down to the Continuous Integration
cycle. The emphasis on which cadence work happens in is a bit different depending
on which Agile framework you are working with. Kanban emphasizes the 24-hour
cycle and is popular in operations teams. Scrum cycles can be between two to four
weeks and are often used by development teams using the Scrum Agile process.
Longer cycles are also common and are called Program Increments, which span
several Scrum Sprint cycles, in Scaled Agile Framework.
DevOps must be able to support all these cycles. This is quite natural given the central
theme of DevOps: cooperation between disciplines in an Agile organization.
The most obvious and measurably concrete benefits of DevOps occur in the shorter
cycles, which in turn make the longer cycles more efficient. Take care of the pennies,
and the pounds will take care of themselves, as the old adage goes.
Here are some examples of when DevOps can benefit Agile cycles:
The Kanban cycle is 24 hours, and it's therefore obvious that the deployment cycle
needs to be much faster than that if we are to succeed with Kanban.
Scrum
Scrum is a framework used by teams to manage work and solve problems collaboratively in short
cycles. Scrum implements the principles of Agile as a concrete set of artifacts, practices, and
roles.
The diagram below details the iterative Scrum lifecycle. The entire lifecycle is completed in fixed time
periods called sprints. A sprint is typically one-to-four weeks long
Scrum roles
There are three key roles in Scrum: the product owner, the Scrum master, and the Scrum team.
Product owner
The product owner is responsible for what the team builds, and why they build it. The product
owner is responsible for keeping the backlog of work up to date and in priority order.
Scrum master
The Scrum master ensures that the Scrum process is followed by the team. Scrum masters are
continually on the lookout for how the team can improve, while also resolving impediments and
other blocking issues that arise during the sprint. Scrum masters are part coach, part team
member, and part cheerleader.
Scrum team
The members of the Scrum team actually build the product. The team owns the engineering of
the product, and the quality that goes with it.
Product backlog
The product backlog is a prioritized list of work the team can deliver. The product owner is
responsible for adding, changing, and reprioritizing the backlog as needed. The items at the top
of the backlog should always be ready for the team to execute on.
In sprint planning, the team chooses backlog items to work on in the upcoming sprint. The team
chooses backlog items based on priority and what they believe they can complete in the sprint.
The sprint backlog is the list of items the team plans to deliver in the sprint. Often, each item on
the sprint backlog is broken down into tasks. Once all members agree the sprint backlog is
achievable, the sprint starts.
Once the sprint starts, the team executes on the sprint backlog. Scrum does not specify how the
team should execute. The team decides how to manage its own work.
Scrum defines a practice called a daily Scrum, often called the daily standup. The daily Scrum
is a daily meeting limited to fifteen minutes. Team members often stand during the meeting to
ensure it stays brief. Each team member briefly reports their progress since yesterday, the plans
for today, and anything impeding their progress.
Task board
The task board lists each backlog item the team is working on, broken down into the tasks
required to complete it. Tasks are placed in To do, In progress, and Done columns based on
their status. The board provides a visual way to track the progress of each backlog item.
.
The sprint burndown is a graph that plots the daily total of remaining work, typically shown in
hours. The burndown chart provides a visual way of showing whether the team is on track to
complete all the work by the end of the sprint.
Sprint review
The team demonstrates what they've accomplished to stakeholders. They demo the software and
show its value.
Sprint retrospective
The team takes time to reflect on what went well and which areas need improvement. The
outcome of the retrospective are actions for the next sprint.
Increment
The product of a sprint is called the increment or potentially shippable increment. Regardless of
the term, a sprint's output should be of shippable quality, even if it's part of something bigger and
can't ship by itself. It should meet all the quality criteria set by the team and product owner.
The entire cycle is repeated for the next sprint. Sprint planning selects the next items on the
product backlog and the cycle repeats. While the team executes the sprint, the product owner
ensures the items at the top of the backlog are ready to execute in the following sprint.
This shorter, iterative cycle provides the team with lots of opportunities to learn and improve. A
traditional project often has a long lifecycle, say 6-12 months. While a team can learn from a
traditional project, the opportunities are far less than a team who executes in two-week sprints,
for example.
Scrum is very popular because it provides just enough framework to guide teams while giving
them flexibility in how they execute. Its concepts are simple and easy to learn. Teams can get
started quickly and learn as they go. All of this makes Scrum a great choice for teams just starting
to implement Agile principles.
Kanban
Kanban is a Japanese term that means signboard or billboard. An industrial engineer named
Taiichi Ohno developed Kanban at Toyota Motor Corporation to improve manufacturing
efficiency.
Although Kanban was created for manufacturing, software development shares many of the same
goals, such as increasing flow and throughput. Software development teams can improve their
efficiency and deliver value to users faster by using Kanban guiding principles and methods.
Kanban principles
Adopting Kanban requires adherence to some fundamental practices that might vary from teams'
previous methods.
Visualize work
Understanding development team status and work progress can be challenging. Work progress
and current state is easier to understand when presented visually rather than as a list of work
items or a document.
Visualization of work is a key principle that Kanban addresses primarily through Kanban boards.
These boards use cards organized by progress to communicate overall status. Visualizing work
as cards in different states on a board helps to easily see the big picture of where a project
currently stands, as well as identify potential bottlenecks that could affect productivity.
Kanban focuses on maintaining an agreed-upon level of quality that must be met before
considering work done. To support this model, stakeholders don't push work on teams that are
already working at capacity. Instead, stakeholders add requests to a backlog that a team pulls
into their workflow as capacity becomes available.
Teams that try to work on too many things at once can suffer from reduced productivity due to
frequent and costly context switching. The team is busy, but work doesn't get done, resulting in
unacceptably high lead times. Limiting the number of backlog items a team can work on at a
time helps increase focus while reducing context switching. The items the team is currently
working on are called work in progress (WIP).
Teams decide on a WIP limit, or maximum number of items they can work on at one time. A
well-disciplined team makes sure not to exceed their WIP limit. If teams exceed their WIP limits,
they investigate the reason and work to address the root cause.
Kanban boards
The Kanban board is one of the tools teams use to implement Kanban practices. A Kanban board
can be a physical board or a software application that shows cards arranged into columns. Typical
column names are To-do, Doing, and Done, but teams can customize the names to match their
workflow states. For example, a team might prefer to use New, Development, Testing, UAT,
and Done.
Software development-based Kanban boards display cards that correspond to product backlog
items. The cards include links to other items, such as tasks and test cases. Teams can customize
the cards to include information relevant to their process.
On a Kanban board, the WIP limit applies to all in-progress columns. WIP limits don't apply to
the first and last columns, because those columns represent work that hasn't started or is
completed. Kanban boards help teams stay within WIP limits by drawing attention to columns
that exceed the limits. Teams can then determine a course of action to remove the bottleneck.
Cumulative flow diagrams
The CFD is particularly useful for identifying trends over time, including bottlenecks and other
disruptions to progress velocity. A good CFD shows a consistent upward trend while a team is
working on a project. The colored areas across the chart should be roughly parallel if the team is
working within their WIP limits.
A bulge in one or more of the colored areas usually indicates a bottleneck or impediment in the
team's flow. In the following CFD, the completed work in green is flat, while the testing state in
blue is growing, probably due to a bottleneck.
Kanban and Scrum in Agile development
While broadly fitting under the umbrella of Agile development, Scrum and Kanban are quite
different.
● Scrum focuses on fixed length sprints, while Kanban is a continuous flow model.
● Scrum has defined roles, while Kanban doesn't define any team roles.
● Scrum uses velocity as a key metric, while Kanban uses cycle time.
Teams commonly adopt aspects of both Scrum and Kanban to help them work most effectively.
Regardless of which characteristics they choose, teams can always review and adapt until they
find the best fit. Teams should start simple and not lose sight of the importance of delivering
value regularly to users.