Devops Labmanual (R20)
Devops Labmanual (R20)
Experiment: 01
In earlier days Iterative Waterfall model was very popular to complete a project. But
nowadays developers face various problems while using it to develop software. The
main difficulties included handling change requests from customers during project
development and the high cost and time required to incorporate these changes. To
overcome these drawbacks of Waterfall model, in the mid-1990s the Agile Software
Development model was proposed.
The Agile model was primarily designed to help a project to adapt to change requests
quickly. So, the main aim of the Agile model is to facilitate quick project completion.
To accomplish this task agility is required. Agility is achieved by fitting the process
to the project, removing activities that may not be essential for a specific project.
Also, anything that is waste of time and effort is avoided.
Actually, Agile model refers to a group of development processes. These processes
share some basic characteristics but do have certain subtle differences among
themselves. A few Agile SDLC models are given below:
• Crystal
• Atern
• Feature-driven development
• Scrum
• Extreme programming (XP)
• Lean development
• Unified process
Agile model is the combination of iterative and incremental process models. The
steps involve in agile SDLC models are:
• Requirement gathering
• Requirement Analysis
• Design
• Coding
• Unit testing
• Acceptance testing
• To establish close contact with the customer during development and to gain
a clear understanding of various requirements, each Agile project usually
includes a customer representative on the team. At the end of each iteration
Department of CSE 1
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
stakeholders and the customer representative review, the progress made and
re-evaluate the requirements.
• Agile model relies on working software deployment rather than
comprehensive documentation.
• Frequent delivery of incremental versions of the software to the customer
representative in intervals of few weeks.
• Requirement change requests from the customer are encouraged and
efficiently incorporated.
• It emphasizes on having efficient team members and enhancing
communications among them is given more importance. It is realized that
enhanced communication among the development team members can be
achieved through face-to-face communication rather than through the
exchange of formal documents.
• It is recommended that the development team size should be kept small (5
to 9 people) to help the team members meaningfully engage in face-to-face
communication and have collaborative work environment.
• Agile development process usually deploys Pair Programming. In Pair
programming, two programmers work together at one work-station. One
does code while the other reviews the code as it is typed in. The two
programmers switch their roles every hour or so.
Advantages:
Disadvantages:
Department of CSE 2
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
QUIZ-1
Department of CSE 3
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
9. The IT team management at pura vida company has decided to adopt Devops
and has drawn a roadmap for the journey. This information is communicated to all
the team members (developers, testers, architects, operations team, (includes
infrastructure, system administration and
deployment).
ANS: - * managers express that the buy –in should be there from business
(customers) and the top management in pure Vida
* IT management team members feel that additional roles will be
required (other than scrum master, product owner and Dev team) to execute
this
10. which of the following benefits does Agile NOT offer in comparison to
waterfall approach?
ANS: - * final product is visible at the end of the project only in Agile
method of software development
* There is a lot of focus on documentation in Agile method of
software development
Department of CSE 4
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Department of CSE 5
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
EXERCISE: - 02
Values
The five values of XP are communication, simplicity, feedback, courage, and respect and
are described in more detail below.
Communication
Software development is inherently a team sport that relies on communication to transfer
knowledge from one team member to everyone else on the team. XP stresses the
importance of the appropriate kind of communication – face to face discussion with the
aid of a white board or other drawing mechanism.
Simplicity
Simplicity means “what is the simplest thing that will work?” The purpose of this is to
avoid waste and do only absolutely necessary things such as keep the design of the system
as simple as possible so that it is easier to maintain, support, and revise. Simplicity also
means address only the requirements that you know about; don’t try to predict the future.
Feedback
Through constant feedback about their previous efforts, teams can identify areas for
improvement and revise their practices. Feedback also supports simple design. Your team
builds something, gathers feedback on your design and implementation, and then adjust
your product going forward.
Courage
Kent Beck defined courage as “effective action in the face of fear” (Extreme Programming
Explained P. 20). This definition shows a preference for action based on other principles
so that the results aren’t harmful to the team. You need courage to raise organizational
issues that reduce your team’s effectiveness. You need courage to stop doing something
Department of CSE 6
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
that doesn’t work and try something else. You need courage to accept and act on feedback,
even when it’s difficult to accept.
Respect
The members of your team need to respect each other in order to communicate with each
other, provide and accept feedback that honours your relationship, and to work together
to identify simple designs and solutions.
The XP Practices have changed a bit since they were initially introduced. The original
twelve practices are listed below.
QUIZ -1
Department of CSE 7
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
3. As per agile manifesto for software development, which is values more than
comprehensive documentation?
Self-assignment: -
ANS: - * steering
* Exploration
4. what are some benefits of pair programming?
5. which pair programming strategy involves one developer creating a test and the
other developer creating code to satisfy the test
Department of CSE 8
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
8.which statement best describes the importance of the customer role in xp?
ANS: - * only the customer knows what needs to be done and why
9.which statement best describes the difference between source control and version
control?
ANS: -*source control specification manage code, while version control includes
other types of files like binaries
10. which operations is used to merge code from one branch to another?
ANS: - * pull
11. which statements best describes the different between continuous integration (CI)
and continuous deployment (CD)?
ANS: - * CI manages compilation, testing and packaging while CD manages the
distribution
13. which category of coding standards can be left in via lotion if there are agreed up
on to do so?
ANS: - * Recommendation
14.what are some potential negative to collective code owner ship?
ANS: - * Decreased motivations
*Reliance on team expertise
Department of CSE 9
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Department of CSE 10
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
EXERCISE: -03
Progressive Collaboration
DevOps promises to bridge the gap between the two where both employ bottom-up
and top-down feedback from each other. With DevOps, when development seeks
operational help or when operations require immediate development, both remain
ready for each other at any given time. In such a scenario, the software development
culture brings in to focus combined development instead of individual goals; The
development environment becomes more progressive as all the team members work in
cohesion towards a common goal.
Processing Acceleration
With conjoined operational and developmental paradigms, the communication lag
betweenhood is reduced to null. Organizations continuously strive for a better edge
over their competing rivals, and if such acceleration is not achieved, the organization
will have to succumb to competing forces— innovation will be slower, and the
product market will decay.
Department of CSE 11
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
1. The IT term management at “pura vida” company has decided to adopt DevOps
and has drawn a roadmap for the journey. This information is communication to
all the term members (developers, testers, architects, operations team (include
infrastructure, system administration and deployment). Here is the first reaction
of the team
ANS: - * managers express that the buy-in should be there from the business
(customers) and the top management in: pura vida”.
* IT management team member feel that additional roles will be
required (other than scrum master, product owner and Dev team) to execute
this
Department of CSE 12
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
2. One of the project team of a portfolio for a customer account mention the
following which of the statement is TURE?
ANS: - *the tools and automation may cost my project more, but we get good
quality and speed as a benefits
2. choose the business drivers for adoption of DevOps (multiple response question)
ANS: - * Early time to market
* Quick deployment with good quality
Scenarios
1.A bank is introducing the online fixed deposit scheme. if this feature has to be
deployed in production, the account service module which provides the customer
account details online would need to be used and also updated. The updating would
disrupt the account service module. This cannot be afforded by the bank. However,
the new feature needs to be tested
Department of CSE 13
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
2. An online audio and video steaming company receive a million calla every day from
different types of devices for different services. They need an architectural style which
consists of lightweight components
3. A support team receives a ticket from the customer that a specific server is not
reachable. The support staff try out quick fixes, but is not working and the server
crashes. It needs to now be reconfigured. The support staff face this situation very
often and are wasting a lot of time doing reconfiguration manually all the time
4. Team A has completed working on a feature. However, they are waiting for same
related features from B and C so that deployment can be done together. Customer is
keen on having feature A urgently
5. A software services company brings all its stakeholders right from developers to
support teams together for effective execution of projects
ANS: - * a4, b2, c5, d1, e3
4. Match the stakeholder and what capabilities they need to build while embarking
on the Dev Ops Implementation journey.
a) Business
b) Dev Team
c) Testing Team
d) Infra team
e) Ops team
f) Organization
1. Continuous integration
2. Automated environment management
3. Progressive test automation
4. Policies to support merging of Dev and Ops teams
5. Big room planning
Department of CSE 14
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
QIS-3 :- CI-CD
1.The customer insists a Dev team to use Jenkins and construct an automated
continuous integration pipeline. The team accepts this request and constructs a Cl
pipeline orchestrated by Jenkins. They schedule daily integration. After a month of
implementation, the customer finds that the bugs that are released to production are
increasing. When they inspect the pipeline stages, they find the following stages
Version control-> build automation -> baseline in artifact repository
What is the team missing here?
ANS: - * The automated pipeline should have in-built quality with static code
analysis included with a good number of quality rules and gating conditions for
quality
* Unit tests should be automated and included so that they can be
repeatedly invoked
2.A development team which is implementing C using an orchestration tool are doing
the following activities Choose the ones which may not be good practices.
ANS: - * If the QA tests fail, the developers make the changes in the server
where the QA tests run, compile and run the tests again
* The team auto-trigger the pipeline whenever a team member completes
the work and push code to the central version control repository
* If the Cl pipeline is broken, the teams continue with the features they
planned during that day instead of fixing the pipeline as it might take a long
time to do it
3.Choose the statement(s) that are TRUE with respect to choosing tool stack for
automating the CICD pipeline.
ANS: - * Based on appetite of customer for automation of tasks
* Based on budget availability
* Based on domain and project requirements
* After consultation with a tool expert/coach
Department of CSE 15
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
4.Choose the statement(s) that are TRUE with respect to choosing tool stack for
automating the CICD pipeline
FINAL ASSIGNMENT: - C
Department of CSE 16
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
ANS: - * Microservices
6. From the options given, choose the stage(s) which are likely to be automated as
part of continuous integration [multiple response]
ANS: -Jenkins
13.A static code analysis tool helps ensure &blank.
ANS: - Teams keep separate backlogs but take each other’s user stories in their
backlogs
15.Which of the following is a definition of code coverage?
ANS: - Measure used to describe the degree to which the source code of a
program has been tested.
16.Which of the options represents the aspects to be taken care while choice of
tools is being made?
A. Repeatability
B. Reliability
C. End-end automation
D. Auto build quality in the pipeline
E. Customer’s appetite for tooling
Department of CSE 18
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
ANS: - * Ensure using an artifact repository and any change made to the code
or tests will go through the entire CICD pipeline
18.Which of the following can be used for releasing feature and enabling and
disabling them without changing the code?
Department of CSE 19
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
ANS: - * The right version of the build is used for QA and the completely tested
version goes for release.
25.When should a security testing be done?
Department of CSE 20
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Exercise 4:
Module name: Implementation of CICD with Java and open-source stack
Configure the web application and Version control using Git using Git commands and
version control operations.
What is GIT?
Git is a free open-source distributed version control system you can use to track
changes in your files. You can work on all types of projects in Git, from small to large.
With Git, you can add changes to your code and then commit them (or save them) when
you're ready. This means you can also go back to changes you made before.
What is GitHub?
GitHub is a web interface where you store your Git repositories and track and manage
your changes effectively. It gives access to the code to various developers working on
the same project. You can make your own changes to a project at the same time as
other developers are making theirs.
If you accidentally mess up some code in your project while making changes, you can
easily go back to the previous stage where the mess has not occurred yet.
There are so many reasons you should learn and use GitHub. Let's look at a few of
them now.
GitHub is a place where your Git repositories are stored. GitHub makes it easy for
developers working on the same project but in different locations to be on the same
page.
With GitHub, you can easily track and manage the changes you have made and check
on the progress you've made in your project.
Department of CSE 21
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
With GitHub, developers from all over the world can work together on a project
without having any problems.
Teams are able to stay on the same page while working on a project together and can
easily organize and manage the project effectively.
Open Source
GitHub is a free and open-source system. This means that developers can easily access
different types of code/projects which they can use in learning and developing their
skills.
Versatility
This attribute of GitHub is very important. GitHub is not a web interface for only
developers. It can be used by designers, writers, and anyone who wants to keep track
of the history of their projects.
To start using Git, you'll need to download it to your computer if you haven't already.
You can do this by going to their official website.
When Git opens, scroll down a bit and you should see a download button. Go ahead
and click on it.Download button on the Git website
Department of CSE 22
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
After the installation, you'll want to make sure that Git is successfully installed on your
system. Open your command prompt or Git bash (whichever one you choose to use)
and run the command:
Department of CSE 23
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
git --version
If Git was successfully installed on your computer, it should display the current version
of Git below the command you just ran. If the current version is being displayed,
congratulations!
To configure Git, we need to specify the name, email address, and branch by using
the git config --global command. For example:
From the image above, we used git config --global user.name to configure the
username. In my case I used my name “Derek Emmanuel”. The same applies for the git
config --global user.email.
Git comes with a default branch of master, so I changed it to be called the main branch
by using the git config --global init.default branch main command.
Now you're ready to start using Git.
Department of CSE 24
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
When the sign up form opens up, enter your email, create a password, enter your
username, and then verify your account before clicking on the create account button.
Department of CSE 25
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
• git config
• git init
• git add
• git commit
• git clone
• git push
• git rm
• git branch
Let's go through each of these briefly so you know how to use them.
For example:
You use the git init command to start Git in your project. This git command is used
when you are working on a project and would like to initialize git to the project in order
to keep track of the changes made in the project.
When you run this command, you should see a folder named. git being created
automatically in the current folder you are working on.
To add a file to the staging area, you use the git add command. It adds all the files in
the folder to the staging area.
Department of CSE 26
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
git add (file name) adds the name of the particular file you want to commit in the
staging area.
Use this command when you have made changes to a file and want to commit them to
your project.
For example:
You use the git clone command to copy an existing repository in another location to
the current location where you want it to be.
For example:
For example:
git rm (filename)
Department of CSE 27
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
You use this command only when you wish to get rid of an unwanted changes/file from
the Git repository.
You use this command to check the current branch you are working on,
either main or master.
For example:
git branch
This command helps you know the current branch you are working on.
Conclusion
In this tutorial you learned what version control systems are all about. You also learned
how to install and setup Git on your computer and setup a GitHub account. Lastly, we
went through some commonly used Git commands.
Department of CSE 28
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Exercise 5:
Module Name: Implementation of CICD with Java and open-source stack
Configure a static code analyser which will perform static analysis of the web
application code and identify the coding practices that are not appropriate. Configure
the profiles and dashboard of the static code analysis tool.
Static analysis is a method of analysing code for defects, bugs, or security issues prior
to pushing to production. Often referred to as “linters,” static analysis tools remove
the unnecessary fluff from your code and perform some automated checks to improve
code quality. Static analysis tools can check for:
Real-time tools: Instantaneous analysis tools are ideal for checking code in
development environments as it’s being written. Here, the trade-off is speed over
more thorough, time-consuming checks. Many of these are open source, which
allows for easier adoption and customization.
Deep analysis tools: On the other end of the spectrum, deep analysis tools can take
much longer and are likely to identify issues that a real-time tool would miss.
Enterprise-grade tools in this area often have hefty licensing fees and they may bring
Department of CSE 29
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
more issues to light than you have the bandwidth to address. Many of these tools may
be configured to report only the most important issues,
Compilers: Although not a dedicated static analysis tool, compilers may also be used
to improve the quality of your code. You can use configuration flags to adjust the
number of checks they perform.
• Among the many benefits of using static analysis tools, the one that is most
beneficial to organizations is the ability to discover bugs before they are released
into the wild (and when they are less costly to fix). Within the DevOps practice
of CI/CD, static analysis tools provide additional benefits.
• Tools that take a long time to run tend to be ignored during development. Even
if static analysis isn’t always a long process, it’s still not the best use of a
developer’s time.
• Integrating analysis tools within CI/CD ensures that they are used consistently
and automatically while offering an extra level of analysis to make sure that
nothing is able to sneak through.
• There are different options for how to integrate static analysis tools in your
environment. One approach is to run it early in the pipeline along with other
automated tests.
• At this point, you’ll be able to fix any issues before the peer code review and it
speeds up the overall process. In turn, developers spend less time reviewing and
have more time to develop new code.
• If you have large code bases, running a deep analysis on every commit may take
too much time. Instead, you can use a less thorough analysis configuration on
development branches and perform more expensive scans on a schedule or when
integrating into upstream branches.
• The goal is to discover bugs as early as practically possible and it’s up to you to
choose the system that works best for your team.
• Tools like Klocwork have fully embraced CI/CD workflows and can
incrementally analyse the code changes on each commit.
• Higher-end static analysis tools can also track bugs over time. This can help you
select which issues to work on in the current release cycle as source code is
continuously being integrated.
• Issues reported in longstanding legacy code that haven’t caused problems are
probably not worth the time investment to resolve them in the immediate term.
Instead, use precious developer time to focus on more recent issues.
• Another practical constraint is the budget available for static analysis. Rather
than obtaining a license for each developer, run the analysis tools on a set
number of build machines (or a single machine if possible).
Department of CSE 30
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Department of CSE 31
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Exercise 6:
Module Name: Implementation of CICD with Java and open-source stack
Write a build script to build the application using a build automation tool like Maven.
Create a folder structure that will run the build script and invoke the various software
development build stages. This script should invoke the static analysis tool and unit
test cases and deploy the application to a web application server like Tomcat.
Prerequisites
Note: If you use Linux, this tutorial assumes that you are not running
Docker commands as the root user, but instead with a single user account
that also has access to the other tools used throughout this tutorial.
To run Jenkins in Docker, follow the relevant instructions below for either macOS and
Linux or Windows.
You can read more about Docker container and image concepts in the Docker section
of the Installing Jenkins page.
Department of CSE 32
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
( Optional ) Specifies the Docker container name to use for running the
image. By default, Docker will generate a unique name for the container.
( Optional ) Runs the Docker container in the background. This instance can
be stopped later by running docker stop jenkins-docker.
Enables the use of TLS in the Docker server. Due to the use of a privileged
container, this is recommended, though it requires the use of the shared
Department of CSE 33
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
( Optional ) Exposes the Docker daemon port on the host machine. This is
useful for executing docker commands on the host machine to control this
inner Docker daemon.
Exposes ports 3000 and 5000 from the docker in docker container, used by
some of the tutorials.
The docker:dind image itself. This image can be downloaded before running
by using the command: docker image pull docker:dind.
The storage driver for the Docker volume. See "Docker storage drivers" for
supported options.
Note: If copying and pasting the command snippet above does not work, try
copying and pasting this annotation-free version here:
Department of CSE 35
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Connects this container to the jenkins network defined in the earlier step.
This makes the Docker daemon from the previous step available to this
Jenkins container through the hostname docker.
Maps (i.e. "publishes") port 8080 of the current container to port 8080 on the
host machine. The first number represents the port on the host while the last
represents the container’s port. Therefore, if you specified -p 49000:8080 for
this option, you would be accessing Jenkins on your host machine through
port 49000.
( Optional ) Maps port 50000 of the current container to port 50000 on the
host machine. This is only necessary if you have set up one or more inbound
Jenkins agents on other machines, which in turn interact with your jenkins-
blueocean container (the Jenkins "controller"). Inbound Jenkins agents
communicate with the Jenkins controller through TCP port 50000 by default.
You can change this port number on your Jenkins controller through
the Configure Global Security page. If you were to change the TCP port for
inbound Jenkins agents of your Jenkins controller to 51000 (for example),
then you would need to re-run Jenkins (via this docker run … command) and
specify this "publish" option with something like --publish 52000:51000,
where the last value matches this changed value on the Jenkins controller and
the first value is the port number on the machine hosting the Jenkins
controller. Inbound Jenkins agents communicate with the Jenkins controller
on that port (52000 in this example). Note that WebSocket agents do not need
this configuration.
Department of CSE 36
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Note that if you change the source volume or directory for this, the volume
from the docker:dind container above needs to be updated to match this.
Maps the $HOME directory on the host (i.e. your local) machine (usually
the /Users/<your-username> directory) to the /home directory in the
container. Used to access local changes to the tutorial repository.
Allow local checkout for the tutorial. See SECURITY-2478 for the reasons
why this argument should not be used on a production installation.
The name of the Docker image, which you built in the previous step.
Note: If copying and pasting the command snippet above does not work, try
copying and pasting this annotation-free version here:
On Windows
The Jenkins project provides a Linux container image, not a Windows container image.
Be sure that your Docker for Windows installation is configured to run Linux
Containers rather than Windows Containers. See the Docker documentation for
Department of CSE 37
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Department of CSE 38
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
If you have some experience with Docker and you wish or need to access your Docker
container through a terminal/command prompt using the docker exec command, you
can add an option like --name jenkins-tutorial to the docker exec command. That will
access the Jenkins Docker container named "jenkins-tutorial".
This means you could access your docker container (through a separate
terminal/command prompt window) with a docker exec command like:
There is a possibility you may need to access the Jenkins console log, for instance,
when Unlocking Jenkins as part of the Post-installation setup wizard.
The Jenkins console log is easily accessible through the terminal/command prompt
window from which you executed the docker run … command. In case if needed you
can also access the Jenkins console log through the Docker logs of your container using
the following command:
There is a possibility you may need to access the Jenkins home directory, for instance,
to check the details of a Jenkins build in the workspace subdirectory.
Department of CSE 39
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
access the contents of this directory through your machine’s usual terminal/command
prompt.
Setup wizard
Before you can access Jenkins, there are a few quick "one-off" steps you’ll need to
perform.
Unlocking Jenkins
When you first access a new Jenkins instance, you are asked to unlock it using an
automatically-generated password.
Department of CSE 40
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
4. On the Unlock Jenkins page, paste this password into the Administrator
password field and click Continue.
The setup wizard shows the progression of Jenkins being configured and the suggested
plugins being installed. This process may take a few minutes.
1. When the Create First Admin User page appears, specify your details in the
respective fields and click Save and Finish.
2. When the Jenkins is ready page appears, click Start using Jenkins.
Department of CSE 41
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Notes:
o This page may indicate Jenkins is almost ready! instead and if so,
click Restart.
o If the page doesn’t automatically refresh after a minute, use your web
browser to refresh the page manually.
3. If required, log in to Jenkins with the credentials of the user you just created
and you’re ready to start using Jenkins!
Throughout the remainder of this tutorial, you can stop your Docker container by
running:
1. Run the same docker run … commands you ran for macOS,
Linux or Windows above.
2. Browse to http://localhost:8080.
3. Wait until the log in page appears and log in.
Obtain the simple "Hello world!" Java application from GitHub, by forking the sample
repository of the application’s source code into your own GitHub account and then
cloning this fork locally.
1. Ensure you are signed in to your GitHub account. If you don’t yet have a GitHub
account, sign up for a free one on the GitHub website.
Department of CSE 42
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
▪ macOS is /Users/<your-
username>/Documents/GitHub/simple-java-maven-app
▪ Linux is /home/<your-username>/GitHub/simple-java-
maven-app
▪ Windows is C:\Users\<your-
username>\Documents\GitHub\simple-java-maven-app
o Otherwise:
a. Open up a terminal/command line prompt and cd to the
appropriate directory on:
▪ macOS - /Users/<your-username>/Documents/GitHub/
▪ Linux - /home/<your-username>/GitHub/
▪ Windows - C:\Users\<your-
username>\Documents\GitHub\ (although use a Git bash
command line window as opposed to the usual Microsoft
command prompt)
Note: If you don’t see this, click New Item at the top left.
2. In the Enter an item name field, specify the name for your new Pipeline
project (e.g. simple-java-maven-app).
3. Scroll down and click Pipeline, then click OK at the end of the page.
4. ( Optional ) On the next page, specify a brief description for your Pipeline in
the Description field (e.g. An entry-level Pipeline demonstrating how to use
Jenkins to build a simple Java application with Maven.)
Department of CSE 43
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
5. Click the Pipeline tab at the top of the page to scroll down to
the Pipeline section.
6. From the Definition field, choose the Pipeline script from SCM option. This
option instructs Jenkins to obtain your Pipeline from Source Control
Management (SCM), which will be your locally cloned Git repository.
8. In the Repository URL field, specify the directory path of your locally cloned
repository above, which is from your user account/home directory on your host
machine, mapped to the /home directory of the Jenkins container - i.e.
o For macOS - /home/Documents/GitHub/simple-java-maven-app
o For Linux - /home/GitHub/simple-java-maven-app
o For Windows - /home/Documents/GitHub/simple-java-maven-app
9. Click Save to save your new Pipeline project. You’re now ready to begin
creating your Jenkins file, which you’ll be checking into your locally cloned Git
repository.
You’re now ready to create your Pipeline that will automate building your Java
application with Maven in Jenkins. Your Pipeline will be created as a Jenkins file,
which will be committed to your locally cloned Git repository (simple-java-maven-
app).
First, create an initial Pipeline to download a Maven Docker image and run it as a
Docker container (which will build your simple Java application). Also add a "Build"
stage to the Pipeline that begins orchestrating this whole process.
1. Using your favourite text editor or IDE, create and save new text file with the
name Jenkins file at the root of your local simple-java-maven-app Git
repository.
2. Copy the following Declarative Pipeline code and paste it into your
empty Jenkins file:
Department of CSE 44
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
3. pipeline {
4. agent {
5. docker {
6. image 'maven:3.8.1-adoptopenjdk-11'
7. args '-v /root/.m2:/root/.m2'
8. }
9. }
10. stages {
11. stage('Build') {
12. steps {
13. sh 'mvn -B -DskipTests clean package'
14. }
15. }
16. }
}
This image parameter (of the agent section’s docker parameter) downloads
the maven:3.8.1-adoptopenjdk-11 Docker image (if it’s not already
available on your machine) and runs this image as a separate container.
This means that:
o You’ll have separate Jenkins and Maven containers running locally
in Docker.
o The Maven container becomes the agent that Jenkins uses to run your
Pipeline project. However, this container is short-lived - its lifespan
is only that of the duration of your Pipeline’s execution.
This args parameter creates a reciprocal mapping between the /root/.m2 (i.e.
Maven repository) directories in the short-lived Maven Docker container
and that of your Docker host’s filesystem.
Defines a stage (directive) called Build that appears on the Jenkins UI.
This sh step (of the steps section) runs the Maven command to cleanly build
your Java application (without running any tests).
17.Save your edited Jenkins file and commit it to your local simple-java-maven-
app Git repository. E.g., Within the simple-java-maven-app directory, run the
commands:
git add.
then
git commit -m "Add initial Jenkins file"
18.Go back to Jenkins again, log in again if necessary and click Open Blue
Ocean on the left to access Jenkins’s Blue Ocean interface.
Department of CSE 45
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
19.In the This job has not been run message box, click Run, then quickly click
the OPEN link which appears briefly at the lower-right to see Jenkins running
your Pipeline project. If you weren’t able to click the OPEN link, click the row
on the main Blue Ocean interface to access this feature.
Note: You may need to wait several minutes for this first run to complete.
After making a clone of your local simple-java-maven-app Git repository
itself, Jenkins:
o Initially queues the project to be run on the agent.
o Downloads the Maven Docker image and runs it in a container on
Docker.
Runs the Build stage (defined in the Jenkins file) on the Maven container.
During this time, Maven downloads many artifacts necessary to build your
Java application, which will ultimately be stored in Jenkins’s local Maven
repository (in the Docker host’s filesystem).
Department of CSE 46
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
20.The Blue Ocean interface turns green if Jenkins built your Java application
successfully.
21.Click the X at the top-right to return to the main Blue Ocean interface.
Department of CSE 47
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
1. Go back to your text editor/IDE and ensure your Jenkins file is open.
2. Copy and paste the following Declarative Pipeline syntax immediately under
the Build stage of your Jenkins file:
3. stage('Test') {
4. steps {
5. sh 'mvn test'
6. }
7. post {
8. always {
9. junit 'target/surefire-reports/*.xml'
10. }
11. }
}
so that you end up with:
pipeline {
agent {
docker {
image 'maven:3.8.1-adoptopenjdk-11'
args '-v /root/.m2:/root/.m2'
}
}
stages {
Department of CSE 48
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
stage('Build') {
steps {
sh 'mvn -B -DskipTests clean package'
}
}
stage('Test') {
steps {
sh 'mvn test'
}
post {
always {
junit 'target/surefire-reports/*.xml'
}
}
}
}
}
Defines a stage (directive) called Test that appears on the Jenkins UI.
This sh step (of the steps section) executes the Maven command to run the
unit test on your simple Java application. This command also generates a
JUnit XML report, which is saved to the target/sure-fire-reports directory
(within the /var/Jenkins home/workspace/simple-java-maven-app directory
in the Jenkins container).
This junit step (provided by the JUnit Plugin) archives the JUnit XML
report (generated by the mvn test command above) and exposes the results
through the Jenkins interface. In Blue Ocean, the results are accessible
through the Tests page of a Pipeline run.
The post section’s always condition that contains this junit step ensures that
the step is always executed at the completion of the Test stage, regardless of
the stage’s outcome.
12.Save your edited Jenkins file and commit it to your local simple-java-maven-
app Git repository. E.g., Within the simple-java-maven-app directory, run the
commands:
git stage.
then
git commit -m "Add 'Test' stage"
Department of CSE 49
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
13.Go back to Jenkins again, log in again if necessary and ensure you’ve accessed
Jenkins’s Blue Ocean interface.
14.Click Run at the top left, then quickly click the OPEN link which appears
briefly at the lower-right to see Jenkins running your amended Pipeline project.
If you weren’t able to click the OPEN link, click the top row on the Blue
Ocean interface to access this feature.
15.Click the X at the top-right to return to the main Blue Ocean interface.
Add a final deliver stage to your Pipeline
1. Go back to your text editor/IDE and ensure your Jenkins file is open.
2. Copy and paste the following Declarative Pipeline syntax immediately under
the Test stage of your Jenkins file:
3. stage('Deliver') {
4. steps {
5. sh './jenkins/scripts/deliver.sh'
6. }
}
and add a skipStagesAfterUnstable option so that you end up with:
pipeline {
agent {
docker {
image 'maven:3.8.1-adoptopenjdk-11'
Department of CSE 50
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
This sh step (of the steps section) runs the shell script deliver.sh located in
the Jenkins/scripts directory from the root of the simple-java-maven-
app repository. Explanations about what this script does are covered in
the deliver.sh file itself. As a general principle, it’s a good idea to keep your
Pipeline code (i.e. the Jenkins file) as tidy as possible and place more
complex build steps (particularly for stages consisting of 2 or more steps)
into separate shell script files like the deliver.sh file. This ultimately makes
maintaining your Pipeline code easier, especially if your Pipeline gains more
complexity.
Department of CSE 51
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
7. Save your edited Jenkins file and commit it to your local simple-java-maven-
app Git repository. E.g., Within the simple-java-maven-app directory, run the
commands:
git stage.
then
git commit -m "Add 'Deliver' stage"
8. Go back to Jenkins again, log in again if necessary and ensure you’ve accessed
Jenkins’s Blue Ocean interface.
9. Click Run at the top left, then quickly click the OPEN link which appears
briefly at the lower-right to see Jenkins running your amended Pipeline project.
If you weren’t able to click the OPEN link, click the top row on the Blue
Ocean interface to access this feature.
If your amended Pipeline ran successfully, here’s what the Blue Ocean
interface should look like. Notice the additional "Deliver" stage. Click on the
previous "Test" and "Build" stage circles to access the outputs from those
stages.
Here’s what the output of the "Deliver" stage should look like, showing you
the execution results of your Java application at the end.
Department of CSE 52
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
10.Click the X at the top-right to return to the main Blue Ocean interface, which
lists your previous Pipeline runs in reverse chronological order.
CONCLUSION:
The "Build", "Test" and "Deliver" stages you created above are the basis for building
more complex Java applications with Maven in Jenkins, as well as Java and Maven
applications that integrate with other technology stacks.
Department of CSE 53
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Exercise 7:
What is a pipeline?
A pipeline is a sequence of events or jobs that can be executed. The easiest way to
understand a pipeline is to visualize a sequence of stages, like this:
Here, you should see two familiar concepts: Stage and Step.
• Stage: A block that contains a series of steps. A stage block can be named
anything; it is used to visualize the pipeline process.
• Step: A task that says what to do. Steps are defined inside a stage block.
In the example diagram above, Stage 1 can be named "Build," "Gather Information,"
or whatever, and a similar idea is applied for the other stage blocks. "Step" simply says
what to execute, and this can be a simple print command (e.g., echo "Hello, World"),
a program-execution command (e.g., java HelloWorld), a shell-execution command
(e.g., chmod 755 Hello), or any other command—as long as it is recognized as an
executable command through the Jenkins environment.
Department of CSE 54
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
The Jenkins pipeline is provided as a codified script typically called a Jenkins file,
although the file name can be different. Here is an example of a simple Jenkins pipeline
file.
pipeline {
stages {
stage("Build") {
steps {
// Just print a Hello, Pipeline to the console
echo "Hello, Pipeline!"
// Compile a Java file. This requires JDKconfiguration from Jenkins
javac HelloWorld.java
// Execute the compiled Java binary called HelloWorld. This requires JDK
configuration from Jenkins
java HelloWorld
// Executes the Apache Maven commands, clean then package. This requires
Apache Maven configuration from Jenkins
mvn clean package ./HelloPackage
// List the files in current directory path by executing a default shell command
sh "ls -ltr"
}
}
// And next stages if you want to define further...
} // End of stages
} // End of pipeline
It's easy to see the structure of a Jenkins pipeline from this sample script. Note that
some commands, like java, javac, and mvn, are not available by default, and they need
to be installed and configured through Jenkins. Therefore:
A Jenkins pipeline is the way to execute a Jenkins job sequentially in a defined way
by codifying it and structuring it inside multiple blocks that can include multiple steps
containing tasks.
Department of CSE 55
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
• Java Development Kit: If you don't already have it, install a JDK and add it to
the environment path so a Java command (like java jar) can be executed through
a terminal. This is necessary to leverage the Java Web Archive (WAR) version
of Jenkins that is used in this tutorial (although you can use any other
distribution).
• Basic computer operations: You should know how to type some code, execute
basic Linux commands through the shell, and open a browser.
Navigate to the Jenkins download page. Scroll down to Generic Java package
(.war) and click on it to download the file; save it someplace where you can locate it
easily. (If you choose another Jenkins distribution, the rest of tutorial steps should be
pretty much the same, except for Step 2.) The reason to use the WAR file is that it is a
one-time executable file that is easily executable and removable.
Department of CSE 56
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Open a terminal window and enter the directory where you downloaded Jenkins
with cd <your path>. (Before you proceed, make sure JDK is installed and added to
the environment path.) Execute the following command, which will run the WAR file
as an executable binary:
Department of CSE 57
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Open a web browser and navigate to localhost:8080. Unless you have a previous
Jenkins installation, it should go straight to the Jenkins dashboard. Click Create New
Jobs. You can also click New Item on the left.
Department of CSE 58
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
You will see a Jenkins job configuration page. Scroll down to find Pipeline section.
There are two ways to execute a Jenkins pipeline. One way is by directly writing a
pipeline script on Jenkins, and the other way is by retrieving the Jenkins file from
SCM (source control management). We will go through both ways in the next two
steps.
To execute the pipeline with a direct script, begin by copying the contents of the sample
Jenkinsfile from GitHub. Choose Pipeline script as the Destination and paste
the Jenkins file contents in Script. Spend a little time studying how the Jenkins file is
structured. Notice that there are three Stages: Build, Test, and Deploy, which are
arbitrary and can be anything. Inside each Stage, there are Steps; in this example, they
just print some random messages.
Click Save to keep the changes, and it should automatically take you back to the Job
Overview.
Department of CSE 59
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
To start the process to build the pipeline, click Build Now. If everything works, you
will see your first pipeline (like the one below).
To see the output from the pipeline script build, click any of the Stages and
click Log. You will see a message like this.
Department of CSE 60
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Now, switch gears: In this step, you will Deploy the same Jenkins job by copying
the Jenkins file from a source-controlled GitHub. In the same GitHub repository,
pick up the repository URL by clicking Clone or download and copying its URL.
Click Configure to modify the existing job. Scroll to the Advanced Project
Options setting, but this time, select the Pipeline script from SCM option in
the Destination dropdown. Paste the GitHub repo's URL in the Repository URL, and
type Jenkins file in the Script Path. Save by clicking the Save button.
Department of CSE 61
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
To build the pipeline, once you are back to the Task Overview page, click Build
Now to execute the job again. The result will be the same as before, except you have
one additional stage called Declaration: Checkout SCM.
To see the pipeline's output from the SCM build, click the Stage and view the Log to
check how the source control cloning process went.
Department of CSE 62
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Manage Jenkins
There are many options available, including managing plugins, viewing the system
log, etc. Click Global Tool Configuration.
Department of CSE 63
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Here, you can add the JDK path, Git, Gradle, and so much more. After you configure
a tool, it is just a matter of adding the command into your Jenkinsfile or executing it
through your Jenkins script.
Department of CSE 64
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Exercise 10:
Code analysis in the agile product development cycle is one of the important and
necessary items to avoid possible failures and defects arising out of the continuous
changes in the source codes. There are few good reasons to include this in our
development lifecycle.
• It can help to find vulnerabilities in the distant corners of your application, which
are not even used, then also static analysis has a higher probability of finding
those vulnerabilities.
• You can define your project specific rules, and they will be ensured to follow
without any manual intervention.
• It can help to find the bug early in the development cycle, which means less cost
to fix them.
Quality Gates can be defined as a set of threshold measures set on your project like
Code Coverage, Technical Debt Measure, Number of Blocker/Critical issues, Security
Rating/ Unit Test Pass Rate and more.
Failing your build jobs when the code doesn’t meet criteria set in Quality Gates should
be the way to go. We were using Jenkins as our CI tool and therefore we wanted to
setup Jenkins job to fail if the code doesn’t meet quality gates.
Prerequisites
Department of CSE 65
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Here is the snapshot of the job that currently passing build before Quality Gates
setup.
Let’s setup Quality gate metrics in the SonarQube server. We are going to create quality
gate only for the metrics “Code coverage” for demo purpose. But there are more
metrics available that you should be selecting while creating quality gates.
(In this example, we selected Code Coverage) -> select operator along with warning
and error threshold.
Department of CSE 66
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Select the project from the available list to which you want to associate this quality
gate. We have selected sample map project for which we have set up Jenkins job.
Now go to the Jenkins job and configure the quality gate validation. Click on the job
and go to Post-build Actions and provide the project details you have associated with
Quality Gate created in the earlier steps.
Department of CSE 67
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Run the Jenkins job again and verify the build status post quality check enabled.
As we could see that code passed the build, however, it doesn't pass quality gate check.
Therefore, build fails in the end. We can verify the same with the project status in
SonarQube server.
Department of CSE 68
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Conclusion
In this, we have shown how can we enforce quality gates on code base using
SonarQube and Jenkins.
Department of CSE 69
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Exercise 11:
In the last article we configured a build on Jenkins to run unit tests and lint checks. We
now focus on triggering a build in Jenkins when you push code to a GitHub repository.
To know how to configure a Jenkins build for a project, please refer to Setting Up
Jenkins with GitHub.
Before you can start with the process given below, you will need to install the GitHub
plugin on your Jenkins server.
1. Configuring Jenkins
Jenkins listens for POST requests at a Hook URL. We need to give this URL to the
repository on GitHub. Then, whenever code is pushed to that repository, GitHub will
send a POST request to the Hook URL and Jenkins will run the build.
A textbox will appear with a hook URL. This is the Hook URL at which Jenkins will
listen for POST requests. Copy this URL and go to the next step.
We now have to provide the Hook URL we got from Jenkins in the previous step.
Click ‘Settings’ on the navigation bar on the right-hand side of the screen.
Department of CSE 71
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Click ‘Webhooks & services’ on the navigation bar on the left-hand side of the
screen.
Paste the URL you copied in the previous step as the ‘Payload URL’.
You can select the events for which you want the Jenkins build to be triggered. We will
select ‘Just the push event’ because we want to run the build when we push our code
to the repository.
Alternatively, you can click on ‘Let me select individual events’ to get a list of all the
events that you can select to trigger your Jenkins build.
You should now see the webhook you just added in the list of Webhooks for that
repository like this.
Department of CSE 72
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
We now have Jenkins configured to run builds automatically when code is pushed to
central repositories. However, Jenkins doesn’t run all builds for all projects. To specify
which project builds, need to run, we have to modify the project configuration.
In Jenkins, go to the project configuration of the project for which you want to run an
automated build.
In the ‘Build Triggers’ section, select ‘Build when a change is pushed to GitHub’.
Jenkins will now run the build when you push your code to the GitHub repository.
Conclusion
Department of CSE 73
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
Exercise 12:
Prerequisites
•
Click on create -> Add Condition -> Choose metrics
• (In this example, we selected Code Coverage) -> select operator along with
warning and error threshold.
Department of CSE 74
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
•
•
Select the project from the available list to which you want to associate this
quality gate. We have selected sample miqp project for which we have set up
Jenkins job.
•
•
Now go to the Jenkins job and configure the quality gate validation. Click on
the job and go to Post-build Actions and provide the project details you have
associated with Quality Gate created in the earlier steps.
Department of CSE 75
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
•
•
Run the Jenkins job again and verify the build status post quality check
enabled.
As we could see that code passed the build, however, it doesn't pass quality gate
check. Therefore, build fails in the end. We can verify the same with the project
status in SonarQube server.
Department of CSE 76
CONTINEOUS INTERATION & CONTINEOUS DELIVERY USING DevOps
• Conclusion
In this, we have shown how can we implement quality gates for code coverage
using SonarQube and Jenkins.
Department of CSE 77