Department of Software Engineering JIGJIGA University Software Process Management Hand-Out of Chapter 1
Department of Software Engineering JIGJIGA University Software Process Management Hand-Out of Chapter 1
JIGJIGA University
Software Process Management
Hand-out of Chapter 1
------------------------------------------------------------------
Introduction To Software Process
Process:
Provides a broad overview of software engineering, including a discussion of
requirements elicitation (to derive), software design, implementation, testing
and maintenance.
Reviews the activities and teams required to set up a CMMI improvement
initiative, describing the process as a continuous improvement cycle.
The Systems Development Life Cycle (SDLC), or Software Development Life
Cycle in systems engineering, information systems and software engineering, is
the process of creating or altering systems, and the models and methodologies
that people use to develop these systems.
Process and Process Models are considered, by many people, to be among the
most theoretical areas of software engineering.
Development models are simple ways of organizing development plans, and
effective representations for describing them to others.
Different models are not purely abstractions.
They are suited to different types of problems.
Choosing the right model sets reasonable expectations, and lays a foundation
for an effective attack on the problem.
Choosing the wrong model sets false expectations, and sketches out a plan for
failure.
Software Development Activities
In the readings on formal process models, and agile (responsive) process we
will see that there are a great many ways to structure the software
development process. All models, however, seem to agree that the work can
be divided into a few basic phases.
definition ... figuring out what to build.
This includes activities like creating the initial concept, developing product
descriptions, gathering requirements, and negotiating specifications.
planning ... figuring out how to build it.
This includes both technical planning (architecture, designs, modeling,
prototypes) and managerial planning (people, budgets, schedules) and the
validation of those plans.
construction ... building it.
This includes the coding of both the product and testing tools, unit testing, the
development of content and documentation, integration, whole system
testing, and acceptance testing.
deployment ... getting it to its users.
This includes activities like release packaging and product manufacturing, early
access programs, distribution, support, bug-fixing, patches, and on-going
maintenance and enhancement.
Exactly what each one of these activities implies may be different for every
project, but most of these activities are (in some form) applicable to most
software projects.
Where people differ is on how to structure these phases and activities into a
software project.
Software Process Models
There are a few basic models, into which the above phases are traditionally
organized.
1. The Waterfall Model
The first, and most primitive development model is the waterfall model so
named because its diagram resembles a succession of cascading waterfalls,
where each waterfall in the cascade feeds the pond that supplies the next. It is
a simple and intuitive model, where each phase is completed (and verified)
before the next phase is begun, and the whole sequence culminates in a
finished product.
Experience has shown this model to be a simpler solution than most
interesting software projects will admit of. We seldom know enough (about
the problem to be solved or the obstacles we will encounter) to enable us to
completely specify the solution up front. As a result, very few problems can
actually be solved in a simple one-pass waterfall project.
If we cannot solve our problem in one-pass, we will have to solve it with a
series of successive approximations. All of the other models represent
different ways of structuring that succession.
2. The Incremental Model
Often, we have a grand vision, but we clearly understand how to realize only a
small subset of it. If that initial subset would be a useful product in its own
right, there is no reason for us to even try to create the whole grand vision in a
single project.
Define, plan, build, and deploy the initial subset that we understand. In this
process we will learn valuable lessons about both the problems to be solved,
and how users use the software. Based on this new information, we can define
and plan the next release, with additional features.
This model of successive releases is employed in most major software projects.
Each successive release is much simpler than the grand vision, and we only
build the parts for which we have a good understanding of the requirements
and design. This approach has the potential to get us to our desired end point,
in a series of smaller, simpler, safer steps.
3. Iteratitive vs. Incremental (Research vs. Development)
It is important to recognize the difference between research and development:
A development model is appropriate when we are pretty sure that we know
what we want to build, and how to build it.
A development project is expected to culminate in a product..
If we are trying to figure out what we should build, or how to build it, those
are research problems.
A research project is expected to culminate in answers to questions, and
usually additional questions.
The difference between Incremental and Iterative models is the difference
between a cascading succession of development projects, and a cascading
succession of research projects.
If our goal is to figure out what to build (or how to build it), we cannot be
certain that, at the end of the next cycle of activities, we will have a product, or
that we will even have a clear definition and plan for a product. We may have
go through many iterations of definition, planning, construction, and
evaluation before we are prepared to define and plan a product cycle. Because
these models go through an unpredictable succession of development cycles,
they are often referred to as spiral models).
Choosing a Model
The decision of whether to use a waterfall, incremental, or iterative model is
not a matter of personal choice. It is a characterization of our goals and how
confident we are of them:
If we are building a one-time project with clear requirements, and no major
problems anticipated, a simple waterfall model is the right one.
If we are planning to deliver a succession of products, each with incremental
capabilities, and benefiting from lessons learned in prior releases, an
incremental model describes our approach.
If we are investigating potential products or ways of building them, and our
goal is to experiment with and validate alternative approaches, a spiral model
is the one that best suits the problem.
You are not forced to choose one model or the other. You are free to arrange
activities and to create a hybrid model that best fits your problem. For
instance, it might seem tautological that you can't (or at least shouldn't) build
something until after you have figured out what to build. But projects often
involve building many different things, some of which are much better
understood than others. Must we have defined everything before we can start
building anything?
If we started building a well-understood task A, and then later found that the
best solution to task B involved a redefinition of task A, we might find
ourselves having wasted a great deal of effort.
If we were sure that task A was truly independent of task B, but the same
people would be used for clarifying task B and implementing task A, doing task
A first might be accelerating a low-risk activity at the cost of delaying a higher-
risk activity. It is generally prudent to deal with higher risk problems sooner.
But if task A was truly independent of task B (in both requirements and
resources) applying our developers and testers to task A while the managers
and architects argued about task B might be a very good use of both time and
resources.
How safe it is to overlap project phases depends on how confident we are that
we have correctly understood the work that we propose to start sooner. The
benefits from such phase overlap depend on whether or not there are
different resources that can reasonably be working on the different phases in
parallel.
Processes Definitions
The high level models describe a general sequencing of activities, and set
expectations for what kind of a project we are undertaking.
They do not actually tell us what we have to do.
A list of activities for each phase exists for any given project, this list might
have inappropriate or be missing important activities.
Any one of those activities (e.g. gathering requirements) could be broken down
into a more detailed list of tasks:
o identifying potential stake holders
o preliminary requirements elicitation
o clarification of unclear requirements
o reconciliation of disparate priorities
o prioritization of requirements
o preparation of requirements specification
o validation of requirements specification
And for each of these tasks, we could, in turn further enumerate and refine
lists of sub-tasks. Most large software development organizations have such
process specifications.
They may spell out project phases activities that are expected to be completed
during each phase.
Tasks and sub-tasks that are included in each activity
what outputs are expected from each activity
project approval decision points, and how the decisions are to be made.
Many people consider these specifications to be very helpful, as they guide the
project team from one step to the next, and clearly spell out what is expected
of them at each point. Other people consider such specifications to be
meddlesome micro-management, focusing on forms and process rather than
working software. There is validity to both views. More enlightened processes
ensure that the important things are covered while giving projects wide
lattitude in the interpretation of requirements and means of demonstrating
satisfaction.
Checklists
It may be tempting to think of check-lists as mnemonic aids for people who
have not yet mastered the concepts. This would be a mistake.
For example, Jet Fighter pilots are some of the smartest, proudest, and highly
trained people in the world, and they depend heavily on check-lists.
They use check-lists because they perform complex tasks, where any mistake is
likely to result in disaster. In times of emergency, when response time is most
critical, they still adhere rigorously to their check-lists.
Check-lists are for people who can't afford to make mistakes.
Most software process definitions, and most software engineering texts,
contain numerous check-lists: things that you should make sure have been
covered. There is considerably more variation among software projects than
there is in flight operations, which means that these check-lists are not as
comprehensive, and may contain steps that are not applicable to all projects:
it is easy to say "not applicable" as you run your finger down the check list.
if you find important considerations that are missing from the standard check-
lists, create your own.
Don't gloss over those minute details, or resent the seemingly endless lists of
items to check. Every item on those lists is there because smart people have
forgotten about them, and come to bad ends as a result. Check lists represent
a great deal of collected wisdom. Using them will save you a great many
mistakes, and make you a much better engineer.
Process Taxonomy
Broad characterization of the evolutionary state of a process. Typical phase
names (e.g. definition, planning, construction, deployment) have been
discussed, but different organizations can define different sets of phases for a
project. The real meaning of a phase is best understood by the activities it
encompasses.
The transition from one phase to another is often accompanied by a review
and approval.
Task
A well-defined assignment, involving enumerated steps, to be carried out by a
specific person or group. A task might be the development and execution of a
unit test suite for a particular module.
Activity
General characterization of work that may encompass a great many tasks: unit
testing might be an activity within the construction phase.
Some people use the term task to refer to something that has a clear beginning
and end (e.g. reading this article), whereas an activity (e.g. learning software
engineering principles) may continue indefinitely.
Work-product
Any process-defined output of a task. They might be primary products (e.g.
code), key inputs to other tasks (designs), or evidence of compliance (review or
test reports).
Any work-product definition must specify the required content and qualities,
and many organizations specify standard forms for each standard work
product. Many people believe that all specified work products should be inputs
to subsequent process steps.
Don't specify it unless you plan to measure it.
Don't require it unless you plan to use it.
A Software Maturity Framework
Fundamentally, software development must be predictable. The software process is the set
of tools, methods, and practices we use to produce a software product. The objectives of
software process management are to produce products according to plan while
simultaneously improving the organization’s capability to produce better products.
The basic principles are those of statistical process control. A process is said to be stable or
under statistical control if its future performance is predictable within established statistical
limits. When a process is under statistical control, repeating the work in roughly the same
way will produce roughly the same result. To obtain consistently better results, it is neces-
sary to improve the process. If the process is not under statistical control, sustained
progress is not possible until it is.
1. Initial. Until the process is under statistical control, orderly progress in process
improvement is not possible. Must at least achieve rudimentary predictability of schedules
and costs.
2. Repeatable. The organization has achieved a stable process with a repeatable level of
statistical control by initiating rigorous project management of commitments, costs, sched-
ules, and changes.
3. Defined. The organization has defined the process as a basis for consistent
implementation and better understanding. At this point, advanced technology can be
introduced.
4. Managed. The organization has initiated comprehensive process measurements and
analysis. This is when the most significant quality improvements begin.
5. Optimizing. The organization now has a foundation for continuing improvement and
optimization of the process.
These levels are selected because they
•Represent the historical phases of evolutionary improvement of real software
organizations
•Represent a measure of improvement that is reasonable to achieve from a prior level
•Suggest improvement goals and progress measures
•Make obvious a set of intermediate improvement priorities once an organization’s
status in this framework is known
While organizations at this level may have formal procedures for planning and tracking
work, there is no management mechanism to insure they are used. Procedures are often
abandoned in a crisis in favor of coding and testing. Level 1 organizations don’t use design
and code inspections and other techniques not directly related to shipping a product.
Organizations at Level 1 can improve their performance by instituting basic project controls.
The most important ones are
•Project management
•Management oversight
•Quality assurance
•Change control
The key steps required to advance from the Defined Process to the next level are:
•Establish a minimum set of basic process measurements to identify the quality and cost
parameters of each process step. The objective is to quantify the relative costs and benefits
of each major process activity, such as the cost and yield of error detection and correction
methods.
•Establish a process database and the resources to manage and maintain it. Cost and yield
data should be maintained centrally to guard against loss, to make it available for all
projects, and to facilitate process quality and productivity analysis.
•Provide sufficient process resources to gather and maintain the process data and to advise
project members on its use. Assign skilled professionals to monitor the quality of the data
before entry into the database and to provide guidance on the analysis methods and
interpretation.
•Assess the relative quality of each product and inform management where quality targets
are not being met. Should be done by an independent quality assurance group.
When different groups gather data but do not use identical definitions, the results are not
comparable, even if it makes sense to compare them. It is rare when two processes are
comparable by simple measures. The variations in task complexity caused by different
product types can exceed five to one. Similarly, the cost per line of code for small
modifications is often two to three times that for new programs.
Process data must not be used to compare projects or individuals. Its purpose is too
illuminate the product being developed and to provide an informed basis for improving the
process. When such data are used by management to evaluate individuals or terms, the
reliability of the data itself will deteriorate.
The two fundamental requirements for advancing from the Managed Process to the next
level are:
•Support automatic gathering of process data. All data is subject to error and
omission, some data cannot be gathered by hand, and the accuracy of manually
gathered data is often poor.
•Use process data to analyze and to modify the process to prevent problems and
improve efficiency.
For example, many types of errors can be identified far more economically by design or
code inspections than by testing. However, some kinds of errors are either uneconomical to
detect or almost impossible to find except by machine. Examples are errors involving
interfaces, performance, human factors, and error recovery.
So, there are two aspects of testing: removal of defects and assessment of program quality.
To reduce the cost of removing defects, inspections should be emphasized. The role of
functional and system testing should then be changed to one of gathering quality data on
the program. This involves studying each bug to see if it is an isolated problem or if it
indicates design problems that require more comprehensive analysis.
With Level 5, the organization should identify the weakest elements of the process and fix
them. Data are available to justify the application of technology to various critical tasks, and
numerical evidence is available on the effectiveness with which the process has been
applied to any given product.
The Principles of Software Process Change
People:
•The best people are always in short supply
•You probably have about the best team you can get right now.
•With proper leadership and support, most people can do much better than they are
currently doing
Design:
•Superior products have superior design. Successful products are designed by people
who understand the application (domain engineer).
•A program should be viewed as executable knowledge. Program designers should
have application knowledge.
New methods must be carefully introduced and periodically monitored, or they to will
rapidly decay.
Human adoption of new process involves four stages:
•Installation - Initial training
•Practice - People learn to perform as instructed
•Proficiency - Traditional learning curve
•Naturalness - Method ingrained and performed without intellectual effort.
FIRM REQUIEMENTS - A software perversity law seems to be the more firm the
specifications, the more likely they are to be wrong. With rare exceptions, requirements
change as the software job progresses. Just by writing a program, we change our
perceptions of the task. Requirements cannot be firm because we cannot anticipate the
ways the tasks will change when they are automated.
For large-scale programs, the task of stating a complete requirement is not just difficult; it is
impossible. Generally, we must develop software incrementally.
However, we must have stability long enough to build and test a system. However, if we
freeze requirements too early, later retrofits are expensive.
SOFTWARE MANAGEMENT IS DIFFERENT - Management must not view it as black art. Must
insist on tracking plans and reviews.
Champions, Sponsors, and Agents
•Champions - Ones who initiate change. They bring management’s attention to the
subject, obtain the blessing of a sponsor, and establish the credibility to get the
change program launched. The champion maintains focus on the goal, strives to
overcome obstacles, and refuses to give up when the going gets tough.
•Sponsors - Senior manager who provides resources and official backing. Once a
sponsor is found, the champion’s job is done; it is time to launch the change process.
•Agents - Change agents lead the planning and implementation. Agents must be
enthusiastic, technically and politically savvy, respected by others, and have
management’s confidence and support.
Elements of Change
•Planning
•Implementation
•Communication
ASSESSMENT OVERVIEW
A software assessment is not an audit. Audit are conducted for senior managers who
suspect problems and send in experts to uncover them. A software process assessment is a
review of a software organization to advise its management and professionals on how they
can improve their operation.
The phases of assessment are:
•Preparation - Senior management agrees to participate in the process and to take
actions on the resulting recommendations or explain why not. Concludes with a
training program for the assessment team
•Assessment - The on-site assessment period. It takes several days to two or more
weeks. It concludes with a preliminary report to local management.
•Recommendations - Final recommendations are presented to local managers. A local
action team is then formed to plan and implement the recommendations.
Observe strict confidentiality - Otherwise, people will learn they cannot speak in confidence.
This means managers can’t be in interviews with their subordinates.
Involve senior management - The senior manager (called site manager here) sets the
organizations priorities. The site manager must be personally involved in the assessment
and its follow-up actions. Without this support, the assessment is a waste of time because
lasting improvement must survive periodic crises.
Respect the people in the assessed organization - They probably work hard and are trying to
improve. Do not appear arrogant; otherwise, they will not cooperate and may try to prove
the team is ineffective. The only source of real information is from the workers.
Assessment recommendations should highlight the three or four items of highest priority.
Don’t overwhelm the organization. The report must always be in writing.
While lack of a commitment discipline is the common reason for chaotic behavior, there are
other forces:
•Under extreme pressure, software managers make a guess instead of a plan. The
guess is usually low, so chaos develops.
•When things get rough, there may be a strong temptation to believe in magic. A
savior or new technology may be the answer.
•The scale of software projects normally follows an escalating cycle.
•Programs take more code than expected;
•As the programs become larger, new technical and management issues arise.
•Since these are unlike previous experience, they are a surprise.
•Even after a higher maturity level is reached, new management, increased
competition, or new technical challenges put pressure on processes; then, an
organization may revert to the Initial Process.
UNPLANNED COMMITMENTS
When a new feature seems simple, management may just commit to do it without planning.
Often, the change is far more complicated than thought. This results in confusion, changed
priorities, and stress. The users and senior management become frustrated and unwilling to
listen to excuses when this becomes a pattern.
GURUS
Gurus can make this worse because they run projects out of their heads. With nothing
written down, everyone comes to them for guidance. At the breaking point, the limits of this
intuitive approach have been reached and there is no simple recovery.
MAGIC
“No silver bullets” - Brooks
The senior manager should require evidence that the following work was done prior
to approving a commitment:
•The work has been defined between the developers and the customer;
•A documented plan has been produced, including a resource estimate, a schedule,
and a cost estimate;
•All directly involved parties have agreed to this plan in writing;
•Adequate planning has been done to ensure the commitment is a reasonable risk.
•An independent review ensures the planning work was done according to the
organization’s standards and procedures.
•The groups doing the work have or can acquire the resources needed.
Management Level:
Integrated Software Management
Intergroup Coordination
Organizational Level:
Organization Process Focus
Organization Process Definition
Training Program
Engineering Level:
Software Product Engineering
Peer (noble) reviews
Management Level:
Quantitative Process Management
Organizational Level:
--------
Engineering Level:
Software Quality Management