SOFTWARE QUALITY

ASSURANCE AND
TESTING

Senedu G/mariam
ORIGINS OF SOFTWARE DEFECTS

Code
Other 7%
10% Design
28%

55%

Requirements
Software
Quality Assurance

Testing
• Transparency is very important.
• Be honest with your agents.
• They’ll respect that a lot more than a spin.
 WHAT IS SOFTWARE QUALITY?
• Simplistically, quality is an attribute of software that implies the software
meets its specification
• This definition is too simple for ensuring quality in software systems
• Software specifications are often incomplete or ambiguous
• Some quality attributes are difficult to specify
• Tension exists between some quality attributes, e.g. efficiency vs.
reliability
• Software requirements are the foundation from which quality is measured.
• Lack of conformance to requirements is lack of quality.
 CONT.…

• Specified standards define a set of development criteria that guide the

manner in which software is engineered.
• If the criteria are not met, lack of quality will almost surely
result.
• There is a set of implicit requirements that often goes unmentioned.
• If software conforms to its explicit requirements but fails to
meet its implicit requirements, software quality is suspect.
• Explicit requirements: is special requirements given by customer.
• Implicit requirements: that are added(analyzed) by the business analyst
which will increase the value of application without disturbing the original
customer requirement.
 SOFTWARE QUALITY
• Without quality, your company will not survive.
• Use a written quality program to ensure you can offer your customers
consistent products.
• Provide consistent products to keep production costs down and increase
revenue.
 WHAT MAKES QUALITY SOFTWARE?
• There is no clear-cut answer
• It depends on:
• Stakeholders
• Type of system
• Type of users
• …

Different ideas about a quality car

 SOFTWARE QUALITY ATTRIBUTES

• Modularity
• Safety • Complexity
• Security • Portability (compatible with different OS
• Reliability • Usability(Both Expert/beginner)
• Resilience(Flexibility) • Reusability(easily can be used for new one
• Robustness • Efficiency
• Understandability • Learnability
• Testability • Accuracy (of result)
• Correctness(According SRS)
• Maintainability (New Function easily added)
 AN EFFECTIVE QUALITY PROCESS MUST
FOCUS ON:
• Paying much attention to customer’s requirements
• Making efforts to continuously improve quality
• Integrating measurement processes with product design and
development
• Pushing the quality concept down to the lowest level of the organization
• Developing a system-level perspective with an emphasis on
methodology and process
• Eliminating waste through continuous improvement
DIFFERENT QUALITY SCENARIOS
• Online banking system
• Security
• Correctness
• Reliability
• Air Traffic Control System
• Robustness
• Real Time Responses
• Educational Game for Children
• User-friendliness
INTRODUCTION SOFTWARE QUALITY ASSURANCE

• Every organization can improve the processes it utilizes to provide products

and services.
• Such improvement will result in more satisfied customers, lower operating
costs, faster product delivery, etc..
• The question is how best to implement an effective, efficient, compliant
quality improvement system.
• With delighted (charmed/Happiness) customers, your organization will
successfully compete and grow.
• Without delighted customers, you organization will stagnate (idle) and
decline.
 CONT.….
• If daily business is done in a quality manner, work environment will
improve, staff will be motivated, new ideas will spring-up on how to
improve, etc. Therefore our organization will prosper.
• An organization that has an effective Quality Management System in
every aspect of its business, will grow, will retain its staff, will increase
its customer base and will become more profitable.
 QUALITY ASSURANCE
• Quality assurance (QA) is any systematic process of determining whether a
product or service meets specified requirements.
• QA establishes and maintains set requirements for developing or
manufacturing reliable products.
• A quality assurance system is meant to increase customer confidence and a
company's credibility (reliability/integrity), while also improving work
processes and efficiency, and it enables a company to better compete with
others.
• guidance for recognizing, defining, analysing, and improving the
production process.
 CONT.…
• The role of Quality Assurance may cover aspects such as the design
and monitoring of documentation systems,
• approval and monitoring of written procedures to produce the product,
• approval of written records of the processing operations,
• approval and monitoring of cleaning systems,
• regulatory control,
• batch or lot review,
• release of product. ..etc.
 QA VERSUS QC
• Some people may confuse the term quality assurance with quality control (QC).
• Although the two concepts share similarities, there are important distinctions
between them.
• In effect, QA provides the overall guidelines used anywhere, and
• QC is a production-focused process – for things such as inspections.
• QA is any systematic process for making sure a product meets specified
requirements,
• whereas QC addresses other issues, such as individual inspections or defects.
• In terms of software development, QA practices seek to prevent malfunctioning
code or products,
• while QC implements testing and troubleshooting and fixes code.
QUALITY ASSURANCE IN SOFTWARE
• Software quality assurance (SQA) systematically finds patterns and the
actions needed to improve development cycles.
• Finding and fixing coding errors can carry unintended consequences; it
is possible to fix one thing, yet break other features and functionality at
the same time.
• SQA has become important for developers as a means of avoiding errors
before they occur, saving development time and expenses.
• Even with SQA processes in place, an update to software can break other
features and cause defects -- commonly known as bugs.
• There have been numerous SQA strategies.
 CONT.….
• For example, Capability Maturity Model Integration
(CMMI) is a performance improvement-focused SQA
model.
• CMMI works by ranking maturity levels of areas within an
organization, and it identifies optimizations that can be used
for improvement.
• Rank levels range from being disorganized to being fully
optimal.
 CONT.….
• Software development methodologies have developed over time that rely on SQA,
such as Waterfall, Agile and Scrum.
• Each development process seeks to optimize work efficiency.
• Waterfall is the traditional linear approach to software development.
• It's a step-by-step process that typically involves gathering requirements, formalizing
a design, implementing code, code testing and remediation, and release.
• It is often seen as too slow, which is why alternative development methods were
constructed.
• Agile is a team-oriented software development methodology where each step in the
work process is approached as a sprint (race/run).
• Agile software development is highly adaptive, but it is less predictive because the
scope of the project can easily change.
• Scrum is a combination of both processes where developers are split into teams to
handle specific tasks, and each task is separated into multiple sprints.
 SOFTWARE QUALITY ASSURANCE

• To ensure quality in a software product, an organization must having

following approach to quality management:
• 1. Organization-wide policies, procedures and standards must be established.
• 2. Project-specific policies, procedures and standards must be tailored from
the organization-wide templates.
• 3. Quality must be controlled; that is, the organization must ensure that the
appropriate procedures are followed for each project
• Standards exist to help an organization draft an appropriate software quality
assurance plan.
• External entities can be contracted to verify that an organization is standard-
compliant.
 SQA ACTIVITIES
• Applying technical methods
• To help the analyst achieve a high quality specification and a high quality design

• Conducting formal technical reviews

• A stylized meeting conducted by technical staff with the sole purpose of uncovering
quality problems

• Testing Software
• A series of test case design methods that help ensure effective error detection

• Enforcing standards
 CONT.….
• Controlling change: Applied during software development and
maintenance
• Measurement
• Track software quality and asses the ability of methodological and
procedural changes to improve software quality
• Record keeping and reporting
• Provide procedures for the collection and distribution of SQA
information
 ADVANTAGES OF SQA
• Software will have fewer latent defects, resulting in reduced effort and time
spent during testing and maintenance
• Higher reliability will result in greater customer satisfaction
• Maintenance costs can be reduced
• Reduce Cost : Overall life cycle cost of software is reduced

• Reduces execution time - Quality processes reduce the cycle time to

complete orders and allows for more production time.

• functionally correct: i.e. performs all functions as specified

in the SRS document
 DISADVANTAGES OF SQA

• It is difficult to institute in small organizations, where available resources

to perform necessary activities are not available
• It represents cultural change - and change is never easy
• It requires the expenditure of dollars that would not otherwise be explicitly
budgeted to software engineering or QA
 SQA TOOLS
• Software testing is an integral part of software quality assurance.
• Testing saves time, effort and cost, and it enables a quality end product to be optimally
produced.
• There are numerous software tools and platforms that developers can employ to
automate and orchestrate (write/rewrite/coordinate/arrange) testing in order to facilitate
SQA goals.
• Selenium is an open source software testing program that can run tests in a variety of
popular software languages, such as C#, Java and Python.
• Another open source program, called Jenkins, enables developers and QA staff to run
and test code in real time.
• It's well-suited for a fast-paced environment because it automates tasks related to the
building and testing of software.
• For web apps or application program interfaces, Postman will automate and run tests.
• It is available for Mac, Windows and Linux, and it can support Swagger and RAML
formatting.
 WHAT IS TESTING?
 Testing is a process of executing a software application with the intent of
finding errors and to verify that it satisfies specified requirements .

 Testing is the process of exercising or evaluating a system or a system

component by manual or automated means to verify that it satisfies specified
requirements or to identify differences between expected and actual
results.

 Testing is a measurement of software quality in terms of defects found, for

both functional and non-functional software requirements and characteristics.
 TESTING

• Testers used to be relied upon to test everything, but as the whole software
delivery life cycle has evolved, code-specific tests (unit tests, which test
individual units of computer code, and integration tests, which test multiple
modules of code) have been taken on by the software developer and the
software tester has been allowed to focus on adding value into the process
with:
• Functional Testing - testing the function of a system, or part of an overall
system, to ensure that the specification is met, usually based on acceptance
criteria.
• Integration Testing - testing a number of software modules in an integrated
fashion as a group.
• User Experience Testing - testing the main graphical user interface between
the software system under development and the end user.
 CONT.….

• Exploratory Testing - manually exploring the developed system while

simultaneously learning about its functionality, designing tests as you go and
observing any testable or unexpected behavior.
• Regression Testing - testing designed to uncover any new bugs introduced to
existing functionality as a result of changes to the code currently being
deployed or environmental changes.
• Acceptance Testing - testing usually, although not always, performed on the
customer side to ensure that the code/system being delivered meets acceptable
standards and is as defined prior to commencing the project.
 CONT.….

• Smoke & Sanity(rational/stability) Testing - testing performed at a high level

to ensure that the main functionality of a system is as required prior to moving
onto more granular tests.
• Performance Testing - the process of determining the performance of a
system (speed, load, stress points, bottlenecks, etc.) based on a number of
metrics and quite often measured against a baseline measurement prior to
development.
• Security Testing - ensuring that the system security and data integrity are of
the highest possible standard and any weaknesses or vulnerabilities are spotted
prior to release.
• Internationalisation and Localisation Testing - testing that all local
languages, symbols, currencies and other regional variations are correctly
applied to each local site.
 IO MAPPING

Subset of inputs
causing erroneous
Input Set
Input Set outputs

Software
Software

Output Set
Output Set

Erroneous
outputs
 SOFTWARE FAULTS AND FAILURES
• A failure corresponds to erroneous/unexpected runtime behavior observed
by a user.
• A fault is a static software characteristic that can cause a failure to occur.
• The presence of a fault doesn’t necessarily imply the occurrence of a
failure.

Input Set

Erroneous
User A Inputs
Inputs

User B User C
Inputs Inputs
 RELIABILITY IMPROVEMENTS

• Software reliability improves when faults which are present in the most
frequently used portions of the software are removed.
• A removal of X% of faults doesn’t necessarily mean an X% improvement in
reliability.
• In a study by Mills et al. in 1987 removing 60% of faults resulted in a 3%
improvement in reliability.
• Removing faults with the most serious consequences is the primary objective.
 ERRORS, FAULTS AND FAILURES
• Error – a human action that produces an incorrect result
• Fault/defect/bug – an incorrect step, process or data definition in a
computer program, specification, documentation, etc.
• Failure – The deviation of the product from its expected behaviour.
• This is a manifestation(indicator) of one or more faults.
 COMMON ERROR CATEGORIES
• Boundary-Related
• Calculation/Algorithmic
• Control flow
• Errors in handling/interpreting data
• User Interface
• Exception handling errors
• Version control errors
 WHY DOES SOFTWARE CONTAIN FAULTS?
• We make mistakes:
• Unclear requirements
• Wrong assumptions
• Design errors
• Implementation errors

• Some aspects of a system are hard to predict:

• For a large system, no one understands the whole
• Some behaviours are hard to predict
• Sheer complexity

• Evidence (if any is needed!):

• Widely accepted failure of “n-version programming”
 THE NEED FOR INDEPENDENT QA

• Deep down, we want our software to succeed.

• We are generally not in the best position to prevent or detect errors in
our own products.
 FIRST THREE TERMS AS THEY ARE
UNDERSTOOD IN THE FAULT-TOLERANT
COMPUTING COMMUNITY:
• Failure: A failure is said to occur whenever the external behavior of a
system does not conform to that prescribed in the system specification.
• Error: An error is a state of the system. In the absence of any corrective
action by the system, an error state could lead to a failure which would not be
attributed to any event subsequent to the error.
• Fault: A fault is the adjudged cause of an error.
 DEBUGGING
• Debugging is the routine process of locating and removing computer
program bugs, errors or abnormalities, which is logically handled by
software programmers.
• Debugging checks, detects and corrects errors or bugs to allow proper
program operation according to set specifications.
?