Bodje N’kauh Nathan-Regis et al.

/ International Journal of Engineering Science and Technology (IJEST)

EVALUATION OF THE MOST USED

AGILE METHODS
(XP, LEAN, SCRUM)
RELATED TO THE CONCEPT OF TOYOTA QUALITY

BODJE N’KAUH NATHAN-REGIS

Research Scholar, PhD Candidate
Christ University
Bangalore, INDIA
[email protected]

V. BALAJI
Assoc. Professor, Research Guide
Christ University
Bangalore, INDIA
[email protected]

Abstract— With the emergence of Agile Software Development technologies, the software community is
expecting in a large impact on producing quality. The Agile Software Development ideas were presented
in the form of Agile Manifesto which consists of four values and twelve principles. Toyota System
production, set also a series of principles and practices which has lead Toyota Motor Corporation
commonly known simply as Toyota (multinational automaker) on the top of the car industry market by
achieving a reputation for the production of very high quality vehicles in all countries around the world.

The main question that each new technology must pass through with success is: How much it’s worth?

This paper propose a model for determining the significance worth of quality into the most used agile
methods (eXtreme programming, Scrum, Lean) inspired by the Toyota Production System(TPS).

Keywords : agile methods, software quality, Toyota production system, Extreme Programming, Scrum
Methodologies, Lean Software Development.

1. INTRODUCTION
Agile development methodology attempts to provide many opportunities to avoid the scenario, where a
company spent money and time to create software that no one wants because at the end of this; business realities
have changed. Quality development becomes important to meet the wide range of frequently changing customer
requirements.
The success rate of the software industry project has been improved. The Standish Chairman Jim Johnson
said, “Doing projects with iterative processing contrary to the waterfall method, (...) is a major step forward.” and
"The agile process is the universal remedy for software development project failure1.”
Lindvall et al. (2002) eWorkshop defines agile methodologies as a group of software development processes
that are iterative, incremental, self-organizing and adaptable. So, it can be presupposed that agile software
development increases the production of quality software.
To assess the worth of the software methodologies, researchers have built and define metrics. Common
software measurements work on cost estimation, quality assurance testing, software debugging, software
performance optimization, etc. But software development is a complex process, it is difficult to define or measure
Quality2.
This paper reports a design of a framework in order to show the balance between agile and TPs in regards of
their principles and practices. That Balance will be used as quantitative and qualitative measurements to show
how much agile method lead to produce a quality product.

ISSN : 0975-5462 Vol. 4 No.01 January 2012 23

 Bodje N’kauh Nathan-Regis et al. / International Journal of Engineering Science and Technology (IJEST)

2. AGILE METHODS
Agile methods are an approach for managing the development of software, which is based upon obtaining
early customer feedback on a large number of frequent software releases (Beck, 1999).
Called lightweight software development methods evolved in the mid-1990, and now typically referred to as
agile methodologies, after the Agile Manifesto published in 2001 proposed by 17 software developers met at the
Snowbird, Utah resort, to discuss lightweight development methods. Agile methodologies can be defined in (3)
three perspectives: a theoretical definition, a functional definition and a contextualized definition3.

a. Agile : Theorical Definition

After the first eWorkshop on agile methodologies in June 2002, Lindvall et al. (2002) summarized the
working definition of agile methodologies as a group of software development processes that are iterative,
incremental, self-organizing, and emergent. The word iterative is derived from iteration which carries with it
connotations of repetition. Incremental means each subsystem is developed new release. As the development
progresses, the usable functionalities increase until a full system is realized. The term Self-organizing introduces a
notion to the management. The emergent nature of agile methodologies means that agile software development is
in fact a learning experience for each project.

b. Agile : Functional Definition

According to the way some agile practitioners have understood agile methodologies as they used them in real
practice; the term “agile” carries with the connotations like: flexibility, quick delivery of the product, readiness
for motion, dexterity in motion, activity and adjustability4. Beck (1999) said that agile methodologies are a fun-
way because to the developers, agility provides a form of freedom to be creative and innovative , minimize
everything that has to be done in the development process; Beck also define Efficiency of agile methodologies by
doing only that work that will deliver the desired product with a little overhead as practically possible.

c. Agile : Contextual Definition

Contextual definition means description can derived in a terms of certain specific concepts. As agile
methodologies can be define according the concept of: software quality assurance, software process
improvement, software modeling and software project management.
Early implementations of agile methods include:
- The dynamic systems development methodology or DSDM has three broad phases that consist of
requirements prototypes, design prototypes, and then an implementation or production phase (Millington &
Stapleton, 1995).
- Scrum is a light weight software development process consisting of implementing a small number of
customer requirements in two to four week sprint cycles (Schwaber, 1995).
- Extreme programming or XP consists of collecting informal requirements from on-site customers,
organizing teams of pair programmers, developing simple designs, conducting rigorous unit testing, and
delivering small and simple software packages in short two-week intervals (Anderson et al., 1998).
- Crystal methods involve frequent delivery; reflective improvement; close communication; personal safety;
focus; easy access to expert users; and a technical environment with automated testing, configuration
management, and frequent integration (Cockburn, 2002a).
- Feature driven development or FDD involves developing an overall model, building a features list, planning
by feature, designing by feature, and building by feature (Palmer & Felsing, 2002).
The rational unified process or RUP involves a project management, business modeling, requirements,
analysis and design, implementation, test, configuration management, environment, and deployment workflow
(Kruchten, 2000).
- Adaptive software development or ASD involves product initiation, adaptive cycle planning, concurrent
feature development, quality review, and final quality assurance and release (Highsmith, 2000).
- Lean Software development or LEAN involves eliminating waste, amplifying learning, deciding as late as
possible, delivering as fast as possible, empowering the team, building integrity in, and seeing the whole
(Poppendieck & Poppendieck, 2003).

ISSN : 0975-5462 Vol. 4 No.01 January 2012 24

 Bodje N’kauh Nathan-Regis et al. / International Journal of Engineering Science and Technology (IJEST)

3. QUALITY IN SOFTWARE INDUSTRY

Quality is never an accident; it is always the result of intelligent effort. —John Ruskin
ISO 9000 defines quality as the totality of characteristics of an entity that bear on its ability to satisfy stated or
implied needs. According to Juran, quality is fitness for use, which means the following two things: “first, quality
consists of those product features that meet the needs of the customers and thereby provides product satisfaction.
Second, quality consists of freedom from deficiencies” (Juran & Gryna, 1988).
From an agile perspective, quality has been defined by some practitioners as follows:
(1) Ambler (2005) considers agile quality to be a result of practices such as effective collaborative work,
incremental development, and iterative development as implemented through techniques such as
refactoring, test-driven development, modeling, and effective communication techniques.

(2) STEPHEN MICHAUD about agile quality frames his answer by defining three types of testing, Quality
Control, Quality Assurance, and Quality Analysis.

- Quality control is concerned with checking and reviewing work that has been done; it is what a lot of people
think of as testing. Basically, quality control involves evaluating a product, activity, process, or service.
- Quality Assurance is designed to make sure processes are sufficient to meet objectives."Quality assurance
verifies that any customer offering, regardless if it is new or evolved, is produced and offered with the best
possible materials, in the most comprehensive way, with the highest standards5.
- Quality Analysis (also called quality control analysis) involves planning, testing for defects, capturing data,
and analyzing the results for improving the process in the future. In the case of software development, that can
mean efforts to reduce defects in the requirements phase.

4. CONCEPTUAL FRAMEWORK

a. Relative Work
To prove the robustness of agile methodologies, researchers have design lot of evaluation framework. General
framework was proposed by Huo et al’s technique by mapping software quality parameters to agile quality. The
contribution of this evaluation was the further identification of possible practices that can be done to improve on
the already high quality achievements by agile processes.
David F. Rico6 has built in his research a framework that link the quality effect with the use of agile
development in the building of website. By providing the major factor of agile methods, he built a framework for
each of these factors to demonstrate the link between website quality and the using of agile method.
MnKandla, 2006 presented an evaluation technique which permit to reveal the similarities among extreme
programming, lean development and scrum. The figure of the table below classifies the practices using the
superscript 1,2,3,4 and 5. The same superscript implements the same agile principle.

ISSN : 0975-5462 Vol. 4 No.01 January 2012 25

 Bodje N’kauh Nathan-Regis et al. / International Journal of Engineering Science and Technology (IJEST)

TABLE 3.1 IDENTIFYING SIMILARITIES AMONG THE PRACTICES

5. The Proposed Framework

As David F. Rico has done his is research, the work start first by building a table named “proposition of
translating principles" which bridge the principles and the criteria of the twelve (12) principles of agile and the
fourteen (14). After examining and comparing these principles, (4) four criteria was generated with (4) four
questions: Q1, Q2, Q3, Q4

1) Organization: Enclose all the rules in the organization

Q1 = how the decision is taken?

2) Teams: Enclose all the people in work and their interaction between each other.
Q2 = how they interact and improve their work?

3) Usage: Enclose all practice and techniques.

Q3 = what is the similarity or different in the work process?

4) Opinion: Enclose all the philosophy under the methodologies principle of agile and the Toyota way.
Q4 = there one improve another, is there any similarly?

Second, as the framework rises a certain question, according to the section “Reflection question” from the book
“The Toyota way field book: A practical guide for implementing Toyota’s 4Ps. Jeffrey K. Liker and David Meier,
Tata McGraw-Hill Edition 2008” and a certain degree of comprehension of agile methods; the framework Survey
Question have been created.
The Population to which the survey has been conducted was essentially agile practitioner find through forum,
website and by enterprises practitioners.

The next table (tab. 3.2) show the proposition of principles of Toyota production System et Agile Manifesto that
we suggest for built our framework.

ISSN : 0975-5462 Vol. 4 No.01 January 2012 26

 Bodje N’kauh Nathan-Regis et al. / International Journal of Engineering Science and Technology (IJEST)

TABLE 3.2 PROPOSITION OF TRANSLATING PRINCIPLES

The principles as numbered as mentionned in the appendix of the full research work.

The next figure (fig. 3.1) show the entire framework built.

FIGURE 3.1 PROPOSITION OF THE FRAMEWORK

The way of structuring the framework for agile methods (XP,LEAN,SCRUM) is to evalaute each method
against the principles described in the Toyota way by analysing the agile manifesto criteria. The framework
evaluated agile process and practice technique by measure how product is really built in IT industry (Using a
survey method) under the factor of of four factors: (a)Organisation, (b) Team, (c) Usage, and (d) Opinions
derived from the 14 principles of Toyota Production System.

ISSN : 0975-5462 Vol. 4 No.01 January 2012 27

 Bodje N’kauh Nathan-Regis et al. / International Journal of Engineering Science and Technology (IJEST)

6. RESULT OF EVALUATION

a. Descriptive Analysis
General view of data: The population was constitued of 43% of Team managerand 39% are programmer,
with 53% of them with the maturity on agile between 2 to 5 years, the method use in order are SCRUM (31%),
XP (23%) LEAN (14%) and OTHER (12%).
Summated Scales: The table below split in two gives the summated scaled for the four (4) factors.

(Negative) Disagreeing values

Factors Strongly Disagree Disagree

Organization 1,90% 6,67%
Usage 3,97% 6,35%
Team 4,63% 6,48%
Opinion 7,14% 12,70%

(Positive) Agreeing values

The table show that more than 50% of the respondent are agree with the questions under the each factor
(organization, usage, team and opinion). The result shows that the four (4) factors have really an impact on the
quality of production. This can mean that put in different side; agile methods are greater than TPS in producing
quality.

Neutral Agree Strongly Agree

36,19% 45,71% 9,52%
30,16% 40,08% 19,44%
37,50% 55,09% 12,96%
28,17% 33,73% 18,25%

b. Statistical Analysis
The statistics may used are non-parametric test . In this research two reliability test (Cronbach's α (alpha) and
Pearson-Brown coefficient of reliability) and one test of concordance (Kendall's coefficient of concordance (W))
was use.
The Test show the strong dependencies among all the variable and help to conclude that there are a link
between the production of quality in agile methods and the four factor of our framework.
Subburaj R.7 give this formula for the quality rate (Q) = Number of good parts produced / total produced. We
adapt this formula to our case to calculate the quality improvement factor (Qi) = mean observed / Maximum
expected
The tables below give the measurement of how much quality can be improve in agile depending of the factors.

For producing quality : ‘Agile Organization’ contributes at 71% , ‘Agile Usage’ contributes at 70% , ‘Agile
Team’ contribute at 72% , ‘Agile Opinion’ contribute at 69%.

ISSN : 0975-5462 Vol. 4 No.01 January 2012 28

 Bodje N’kauh Nathan-Regis et al. / International Journal of Engineering Science and Technology (IJEST)

7. CONCLUSION

a. Threat to validity
There are several major threats to the validity of this study. First, the instrument to measure agile methods is
new and untested, so its reliability cannot be fully determined in advance. Second, quality has only been tested on
perceptions of past product and service. Third, the respondents are self-selected, so there may be some bias
towards the use of agile methods. Fourth, survey research may or may not be the best research method to analyze
the impacts of agile methods, in lieu of qualitative methods, which yield richer experiences. Sixth, the statistical
analysis may not be sensitive enough to measure greatly the variations in the data we collect
b. Summary
This research has generated a numbers of contributions; the literature reviews, the framework developed and
the analysis of the survey result. In the literature review we provide the ideas inside lot of thesis done and also the
definition concept of the keywords of this research. The framework designed was specially built to this work and
can be modified for further more experimentation of agile methods. Using the framework, we are providing raw
data from the respond by many agile practitioners. The analysis gives a comfortable representation of the quality
reality in the IT industry.

REFERENCES
[1] http://www.softwaremag.com Agile, Agincourt, Adams, Alexander, and Edison: Something to be Said for “Failing Fast”
[2] Binstock, Andrew. "Integration Watch: Using metrics effectively" . SD Times. BZ Media. Retrieved 19 October 2010.
[3] Ernest Mnkandla and Barry Dwolatzky, Agile software methods : State of the art.
[4] Abrahamsson, Salo, Ronkainen, & Warsta, 2002
[5] Shannon Kietzman & Niki Foster, http://www.wisegeek.com/what-is-quality-assurance.htm
[6] effects of agile methods on website quality for electronic commerce; august 29, 2007
[7] Subburaj R., Total Quality Management. Tata McGraw-Hill Education Pvt Ltd, Noida, Uttar Pradesh, India..

ISSN : 0975-5462 Vol. 4 No.01 January 2012 29