MODULE 4

Evaluation of Training and Development Programme

Training evaluation

Training evaluation refers to an attempt to obtain relevant information on the effects of a

training program. It is considered an essential aspect of a training event to be able to reflect,
analyze, and improve its effectiveness and efficiency.

The primary objective of evaluating any training program is to develop an understanding of

whether it has achieved its stated objectives. There are several types of training evaluation
methods and tools available that enterprises can use to evaluate and significantly improve the
outcome of future training as well.

Benefits of Evaluating Training and Development

 Accountability
 Transparency in Feedback
 Cost-Efficiency

Training evaluation is defined as a systematic approach where data and information is gathered to
ascertain the suitability and effectiveness of a training program. The evaluation process is beneficial as it
can assess the usefulness of the process, help in overall working and boost employee engagement.

Types of training evaluation

The different types of training evaluation program are-

 Formative evaluation – This type of training evaluation offers feedback to the developer and
designer of the program so that they can know whether the course meets the requirements of
its target audience.
 Process evaluation– This type of training evaluation deals in information related to events
occurred during training. It is about imparting and receiving feedback verbally.
 Outcome evaluation – This type of training evaluation determines whether results were
achieved after applying new skills and know-how.
 Impact evaluation – This type of training evaluation deals with the impact of the training on the
strategic goals of a company.

Training evaluation methods

Evaluation methods are conducted to know where the objectives were met and the impact of
training on performance levels. There are two types of training evaluation methods
Qualitative method

1. Focus groups
2. Case studies
3. Interviews

The quantitative method

1. Experiments
2. Surveys

Methods of Training

Training method refers to a way or technique for improving knowledge and skills of an
employee for doing assigned jobs perfectively. The organization has to consider the nature of
the job, size of the organization & workers, types of workers and cost for selecting a training
method.

There are different types of the Training method.

1. On-the-job training or internal training

2. Off-the-job training or external training

The classification of the different types of Training method is shown following chart and
explained below;
  On-the-job training or internal training

These methods are generally applied in the workplace while employees are actually working.

This form helps particularly to develop the occupational skills necessary to manage an
organization, to fully understand the organization’s products and services and how they are
developed and carried out.

Following are the on-the-job methods.

1. Apprenticeship programs.
2. Job instruction training (JIT).
3. Planned progression.
4. Job rotation.
5. Creation of assistant – to positions.
6. Temporary promotions.
7. Committees and junior boards.
8. Coaching.

Apprenticeship programs

People seeking to enter the skilled trades to become, for example, plumbers, electricians,
Ironworkers are often required to undergo apprenticeship training before they are accepted to
journeyman status.

Typically, this apprenticeship period is from two to five years. During this period, the trainee is
paid less than a qualified worker. These programs put the trainee under the guidance of a
master worker.

Job instruction training (JIT)

JIT consists of four basic steps;

1. preparing the trainees by telling them about the job and overcoming their uncertainties;
2. presenting the instruction, giving essential information in a clear manner;
3. having the trainees try out the job to demonstrate their understanding; and
4. Placing the workers into the job, on their own, with a designated resource person, who
is ready to provide the required assistance.
Planned progression

It is a technique that gives employees aclear idea of their path of development. They know
where they stand and where they are going. They must know the requirements for
advancement and the means of achieving it.

Job rotation

It involves periodically moving people from one job to another.The purpose of job rotation is to
broaden the knowledge of managers or potential managers. It also increases their experiences.
Trainees learn about the different enterprise functions by rotating into different positions.

They may rotate through;

(1) Non-supervisory work,

(2) observation assignments,
(3) various managerial training positions, and
(4) middle-level assistant positions

Such movement prevents stagnation. Other reasons for rotating people include compensating
for a labor shortage, safety and preventing fatigue.

Creation of assistant – to positions

Assistant-to positions are frequently created to broaden the viewpoints of trainees by allowing
them to work closely with experienced managers who can give special attention to the
development needs of trainees. Managers can give selected assignments to test the judgment
of trainees.This approach can be very effective when superiors are also qualified trainers who
can guide and develop trainees until they are ready to assume full responsibilities as managers.

Temporary promotions

Individuals are frequently appointed as acting managers when, for example, the permanent
manager is on vacation, is ill or is making an extended business trip or even when a position is
vacant.

When the acting manager makes decisions and assumes full responsibility, the experience can
be valuable. In this way, managerial people can be trained up well.

Committees and junior boards

These give trainee’s opportunities to interact with experienced managers.The trainees become
acquainted with a variety of issues that concern the whole organization. They learn about the
relationships among different departments and the problems created by the interaction of
these organizational units.

Trainees may be given the opportunity to submit reports and proposals to the committee or the
board and to demonstrate their analytical and conceptual abilities.

Coaching

On-the-job training is a never-ending process.A good example of on the job training is athletic
coaching. To be effective, which is the responsibility of every line manager, must be done in a
climate of confidence and trust between the superior and the trainees.

Patience and wisdom are required of superiors who must be able to delegate authority and give
recognition and praise for jobs well done.Effective coaching will develop the strengths and
potentials of subordinates and help them overcome their weakness.

Coaching requires time, but if done well, it will save time and money and will prevent costly
mistakes by subordinates; thus, in the long run, it will benefit all – superior, the subordinates,
and the enterprise.

 Off-the-job training or external training

Off-the-job training is sometimes necessary to get people away from the work environment to a
place where the frustrations and buzz of work are eliminated.

Training is generally given in the form of lectures, discussions, case studies, and
demonstrations. This enables the trainee to study theoretical information or be exposed to new
and innovative ideas.

Advantages of Off-the-Job Training

Off-the-job training has the following advantages:

1. It does not disrupt the normal operation.

2. Trainers are usually experienced enough to train,
3. It is systematically organized,
4. Efficiently created programs may add a lot of values.

Disadvantages of Off-the-Job Training

It is claimed that off-the-job training faces the following limitations:

1. It is not directly in the context of the job,

2. It is often formal,
 3. It may not be based on experience,
4. It is expensive,
5. Trainees may not be much motivated,
6. It is artificial in nature.

Methods of Off-the-Job Training

1. Lectures.
2. Straight lecture.
3. Discussion method.
4. Demonstrations.
5. Seminars and conferences.
6. Reading, television and video instructions.
7. Business Simulation.
8. Cases presentation.
9. Equipment simulators.
10. Business games.
11. Experimental exercise.
12. Role-playing.
13. Behaviour Modelling.
14. Computer modelling.
15. Vestibule training.
16. Sensitivity Training (T-groups).
17. Computer-based training.

Lectures

The lecture is one of the oldest forms of training, second to demonstrate. In the early days,
knowledge was transferred through demonstrations.Lecture may be printed or oral. It is the
best used to create an understanding of a topic or to influence attitudes through education or
training about a topic.The lecture is merely telling someone about something. There are
variations of a lecture format.

Straight lecture

It is an extensive presentation of information, which the trainee attempts to absorb. The lecture
is typically thought of in terms of a person (trainer) speaking to a group about a topic. It is a
shortversion of a lecture. It has the same features as the lecture but usually lasts less than
twenty minutes if done orally.

During a straight lecture, the trainee does little except listen, observe and perhaps take notes. It
is useful when a large number of people must be given a specified set of information. The oral
lecture should not contain too many learning points unless the printed text accompanies the
lecture.
Trainees will forget information provided orally. Short lectures are usually better. Longer
lectures can be effective if the length is due to examples and clarifying explanations. A major
concern about the straight lecture method is the inability to identify and correct
misunderstandings.

Discussion method

The discussion method uses a lecture to provide trainees with information that is supported,
reinforced and expanded on through interactions both among the trainees and between the
trainer and trainees.

It provides a two-way flow of communication. Knowledge is communicated from the trainer to

the trainees. Quick feedback is ensured. A better understanding is possible. Questioning can be
done by both the trainer and the trainees.

Demonstrations

A demonstration is a visual display of how to do something or how something works. To be

effective, a demonstration should, at a minimum, be accompanied by a lecture and preferably
by a discussion.

Demonstrations;

1. Break the tasks to be performed into smaller and easily learned parts;
2. Sequentially organize the parts of the tasks;
3. Complete each of the following steps for each part of the task;
4. Tell the trainees what trainer will be doing so they understand what he will be showing
them;
5. It serves to focus on trainee’s attention on the critical aspects of the task;
6. Demonstrate the task, describes what trainees are doing while the trainer is doing it;
7. After demonstrating each part of the task, the trainer explains why it should be
performed in that way.

Following steps will increase the value of the demonstration:

 Ask the trainee to talk through the task before actually doing it;
 Give the trainee opportunity to do the task and describe what he or she is doing;
 Provide feedback, both positive and negative;
 Let the learner practice.

Seminars and conferences

Conference programs may be used in internal or external training.During conference programs,

managers or potential managers are exposed to the ideas of speakers who are experts in their
fields. A careful selection of topics and speakers will increase the effectiveness of this training
device.

Conferences can be made more successful by including discussions. Two-way communications

allow participants to ask for clarification of specific topics that are particularly relevant to them.

Reading, television and video instructions

Another approach to training and development is the planned reading of relevant and current
management literature. This is essentially self-development.

A manager may be aided by the training department, which offer develops a reading list of
valuable books. This learning experience can be enhanced through discussion of articles and
books with other managers and the superior.

Management and other topics are featured in television programs. Moreover, videotapes on a
variety of subjects are available for the usage in the university or company classrooms.

Business simulation

Any training activity that explicitly places the trainee in an artificial environment that closely
mirrors actual working conditions can be considered a simulation.

Training games and simulations are designed to reproduce or simulate processes, events, and
circumstances that occur in the trainee’s job.

Trainees can experience these events in a controlled setting where they Can develop their skills
or discover concepts that will improve their performance. Simulation activities include case
exercises,

Equipment simulators, experiential exercises, complex computer modeling, role play, and
vestibule training.

Cases presentation

Case studies attempt to simulate decision-making situations that trainees might find on the job.
The trainee is usually presented with a written history, key elements and the problem of a real
or imaginary organization or subunit. A series of questions usually appears at the end of the
case.

Typically, trainees are given time to digest the information individually. If time permits, they are
also allowed to collect additional relevant information and integrate it into their solutions.
Once individuals arrive at their solutions, they may meet in small groups to discuss the different
diagnoses, alternatives, and solutions generated.

Then the trainees meet with the trainer, who facilitates and directs further discussions. The
trainer should convey that no single solution is right or wrong, but many solutions are possible.
The learning objective is to get trainees to apply known concepts and principles and discover
new ones.

Equipment simulators

Equipment simulators are mechanical devices that require the trainee to use the same
procedures, movements, or decision process, they would use with equipment back on the job.

Simulators train airline pilots, air traffic controllers, taxi drivers, etc. it is important that
simulators be designed to replicate, as closely as possible, the physical aspects of the
equipment operating environment trainees will find on their job site.

Business games

Business Games are simulations that attempt to represent the way industry, company, and a
subunit of a company function. They are based on a set of relationships, rules, and principles
derived from theory or research.

Business games involve an element of competition, either against other players or against the
game itself. Some of the purposes for which businesses games have been developed and used
are listed below:

 Strengthen executive and upper management skills

 Improve decision-making skills at all levels
 Demonstrate principles and concepts
 Integrate separate components of training into an integrated whole
 Develop leadership skills
 Explore and solve complex problems in a safe, simulated setting.
 Improve the application of total quality principles and develop skill in using quality tools.

Experimental exercise

Experimental exercises are usually short, structured learning experiences where individuals
learn by doing.

For example, an experimental exercise could be used to create a conflict situation where
employees have to experience a conflict personally and work out its resolution.
After completing the exercise, the trainer typically discusses what happened and introduces the
theoretical concepts to help explain the members’ behavior during the exercise.

Role-playing

Role-playing is a training technique in which trainees act out roles or parts in a realistic
management situation.

The aim is to develop trainees’ skills in areas like leadership and delegating. It is an enactment
or simulation of a scenario in which each participant is given a part to act out.

Trainees are provided with a description of the context-usually a topic area, a general
description of the situation, a description of their roles and the problem they face. Role plays
may be structured, and spontaneous.

(I) Structured role

Structured role plays provide trainees with more detail about the situation as well as more
detailed descriptions of each character’s attitudes, needs, opinions, and so on. This type of role
play is used primarily to develop interpersonal skills such as communication, conflict resolutions
and group decision making.

(II) Spontaneous role

Spontaneous role plays are loosely constructed interactions in which one of the participants
plays himself while the others play people with whom the first trainee interacted in the past.
This type of role play focuses on attitudes and is used to develop insight into one’s own
behavior and is impacting on others rather than to develop specific skills.

Behavior modeling

Behavior modeling uses the tendency for people to observe others learn how to do
something new. This technique is most frequently used in combination with some other
techniques. The modeled behavior is typically videotaped and then watched by the trainees.

The behavior modeling process can be summarized as follows:

 Define the key skill deficiencies;

 Provide a brief overview of relevant theories;
 Specify key learning points or critical behaviors to watch for;
 Use an expert to model the appropriate behaviors;
 Encourage trainees to practice the appropriate behaviors in a structured role play;
 Provide opportunities for the trainer and Other trainees to give reinforcement of
appropriate imitation of the model’s behavior;
  Ensure the trainee’s supervisor reinforces appropriate demonstration of behavior on the
job.’

Computer modeling

Complex computer modeling simulates the work environment by programming a computer to

imitate some of the realities of the job. It is widely used by airlines in the training of pilots.

The computer simulates the number of critical job dimensions and allows learning to take
place without the risk or high costs that would be incurred if a mistake were made in a real life-
flying situation.

An error during a simulation offers an opportunity to learn through one’s mistakes. A similar
error under real-life conditions might cost a number of lives and the loss of a multimillion-dollar
aircraft.

Vestibule training

In vestibule training, employees learn their jobs with the requirement they will be using but
the training is conducted away from the actual workplace. While expensive, vestibule training
allows employees to get a full feel for doing tasks without real-world pressures. It minimized
the problem of transferring learning to the job since vestibule training uses the same
equipment the trainee will use on the job.

Sensitivity Training (T-groups)

Sensitivity training, also called T-group, is basically a technique for management development.
It is concerned with the real problems existing within the group itself. It is not an imagined
problem existing outside the organization. It is not a program of teaching skills or improving the
understanding of participants.

In this program, an attempt is made to change the attitude and behavior of people in the group.
It is used in building team efforts. This is done by introspection, self-criticism, and open
arguments and through free and frank discussion so that one comes to know how others feel
about him and his behavior.

It is a means of providing a mirror in which one can see his mental makeup, attitude, and
behavior towards others. This will provide the best method of motivation for self-development.
The objective of this training includes:

 Better insight into one’s own behaviour and the way one appears to others;
 A better understanding of group processes;
 Members learn more about themselves, especially their weakness and emotional
stability;
  Development of skills in diagnosing and intervening group processes;
 Find a better method and means of behaviour for effective interpersonal relationships
without the aid of power over others.

T-group is a small discussion group without any leader. Trainer raises a question and
encourages open discussion, which is unstructured. The focus is about feeling and mutual
respect.

The T-group process may;

 Lead to personal anxieties and frustrations.

 Lead to a mental breakdown.
 It may make managers hypersensitive, which make them unable to take a hard decision
for fear of hurting another.

But if properly managed, it can result in collaborative and supportive behavior. The following
guidelines can help reduce potential harm and increase effectiveness:

 Participants in T-group should be voluntary;

 They should be screened and those who could be harmful should be expelled from this
experience.;
 Trainers should be carefully evaluated and their competence clearly established;
 Potential participants should be informed about the goals and the process before they
commit themselves to sensitivity training.

Computer-based training

Many companies are implementing computer-based training as an alternative to classroom

training to accomplish the goal. Some of the reasons for this shift are demonstrated in the
following beliefs, many companies hold about CBT:

 Reduces trainee learning time

 Reduces the cost of training
 Provides instructional consistency
 Affords privacy of learning
 Allows the trainee to master learning in a safe method for learning hazardous tasks
 Increases access to training.

Training Evaluation – Levels: Pre-Training Evaluation, Intermediate Training

Evaluation and Post-Training Evaluation

Training segment has to evolve criteria for evaluating the impact of training on employees.
Generally four different criteria are used to evaluate training programme namely reaction of
trainees, knowledge acquired, behaviour modification and other job performance parameters like
reduced accidents, increased productivity, lowered absenteeism leaping sales etc.

1. Pre-Training Evaluation:

In this stage, an evaluation is made in the beginning of the training programme in order to
understand the expectations of the trainees from the training programmes and the extent to which
they have understood its objectives. This step enables the training segment to modify the training
curricula in such a way that the objectives of the training programme are aligned to those of the
trainees.

2. Intermediate Training Evaluation:

Training and development segment wants to ensure that training is progressing as expected. Mid-
course corrections can be made in the event of deviation from the envisaged objectives. For
example, if trainees perceive that a training programme is aimed at building communication skill
is more theory-oriented, rather than practice-oriented, the feedback may be useful to modify the
instruction method. Thus, it serves as a verifying tool.

3. Post-Training Evaluation:

The criteria used for assessing the impact of training programme include Reaction, Learning,
Behaviour and Results (RLBR).

a. Reactions:

This measures the degree of satisfaction of trainees with the training programme, namely subject
matter and content of training programme, the environment, methods of training etc. The
outcome of evaluation of reaction may be useful in further strengthening the areas the
participants find it more useful and in modifying the areas they find it not useful. Negative
reactions may dampen the spirit of participation in future training programmes. However,
positive reactions may not provide complete information about the effectiveness of the
programme.

b. Learning:

It measures the degree to which trainees have acquired new knowledge, skill or competencies.
The trainer has to measure the knowledge and skill level of trainees in the beginning of the
programme. It is supposed to be the baseline or standard. Again the level of knowledge and skills
obtained at the end of training is measured and compared against the standard. Thus pre and post
training comparison helps to assess the improvement level.

c. Behaviour:

Similarly, a comparison of pre and post training behaviour may reveal the impact of training on
behaviour modification. Yet, unfortunately, much of what is learnt during training cannot be
used on the job owing to lack of resources or conducive environment. In such a case, one cannot
say that the training is ineffective. It follows that when training environment is similar to actual
work environment, such a climate facilitates transfer and application of learning.

d. Results:

Generally, it is difficult to measure precisely the impact of training on business performance

which depends on several other factors like economic climate, marketing, size of investment, etc.
However, certain measures like productivity, sales volume and profit, etc., may be compared
before and after the training episode. Any improvement may be partially attributed to the training
imparted. Besides, return on investment, cost benefit analysis and bench-marking are other
methods to assess the value of training.

Training Evaluation – 3 Steps in the Process of Evaluating Training: Before Training,

During Training and After Training

The process of training evaluation has been defined by A. C. Hamblin as “any attempt to obtain
information on the effects of training performance and to assess the value of training in the light
of that information”. Thus, evaluation of training effectiveness refers to the process of obtaining
information on the effects of a training programme and assessing the value of training in the light
of the information so obtained.

It involves controlling and correcting the training programme. The basis and mode of evaluation
are determined when the training programme is designed. According to Hamblin A. C., there are
five levels at which evaluation of training can take place.

i. Before Training:

Generally the HR manager or the employee’s supervisor appraises the employee’s skills and
knowledge before the training programme. Employee is asked to give his/her opinions on the
methods of the training used and whether those methods confirm to his/her preferences and
learning style.

ii. During Training:

This is the step which instruction is started. This step usually consists of short tests at regular
intervals.

iii. After Training:

This is the step when employee’s skills and knowledge are assessed again to measure the
effectiveness of the training. This phase is designed to determine whether training has had the
desired effect at individual department and organizational levels. There are various evaluation
techniques for this phase.
Evaluation helps in controlling and correcting the training programme. Hamblin(1974) suggested
five levels at which evaluation of training can take place.

These are:

1. Reactions- This measures participants’ reaction to the training at the time of training. Whether
they like or dislike the training programme? Trainee’s reaction to the overall usefulness of the
training includes coverage of the topics, the method of presentation, the techniques used to
clarify things and effectiveness of the programme.

2. Learning-This measures participants’ learning of the content of the training. The extent to
which the trainees have assimilated the knowledge offered and skills practiced in the training
programme. Does the participant score higher on tests after the training or development than
before?

3. Job Behaviour-This indicates participants’ use of their new skills and knowledge back on the
job. This includes a comparison of ratings; a participant receives before training and after
training.

4. Organization-This measures participants’ use of training, learning and change in the job
behaviour of the department/organization in the form of increased productivity, quality, morale,
sales turnover and the like.

5. Ultimate Value-This measures contributions of training programme to company goals like

survival, growth, profitability etc. and to the individual goals like development of personality and
social goals like maximizing social benefits.

Training Evaluation Methods:

Following are the methods of training evaluation:

1. Satisfaction and Participant reaction: Satisfaction evaluation is the most basic measure for
assessing the success rate of any training. For the purpose, the trainer, usually, hands out a
survey at the end of the course to test thereaction of the participants. Most of the time, it
covers common questions like whether the participants enjoyed the training or did they like
the trainer.
2. Knowledge Acquisition: Knowledge acquisition is the second level of the training
evaluation and involves the examination as the attachment of the training course to check
how much the participants have learned from the training course. It is a fact that most of
the participants take training seriously only if they know that they are required to
demonstrate the concepts that they have learned during the training.
 In this method, participants are supposed to take the exam, after the training. The instructors
or the trainers check and grade the responses, and share the results with the students as well
as the training managers. This is done so that any gaps in the expected and acquired knowledge
can be quickly sewn up.

A reliable and valid examination, as the training ends, can help in determining if the participant
has understood and learned the concept or not. It can point out the participants that did not
gain anything from training or it can highlight areas that might need additional coaching or
further training.

3. Behavioral Application: The third method of evaluation deals with the behavioral
application of their newly acquired skills. It also involves monitoring the changing behaviors
as the skills and knowledge are applied to the tasks. 

This method demonstrates the level to which the participants apply their newly acquired
knowledge in their real life and real-world problems. This provides crystal clear evidence of who
is applying the knowledge, where the knowledge is being applied and for what purposes. This
can assist the management to avoid any misapplications.

4. Measuring the Business Improvement: The primary objective of nearly all the
organizations arranging the training courses is to generate a particular business
improvement. So, it means that can assess the success level of a training program by the
improvement made in that particular field, once the training is complete and the
participants are ready to apply their knowledge for the cause of development of the brand.

5. Return on Investment (ROI): The final member of our list of training evaluation methods is
related to the measurement of return on investment. It deals with training regarding costs
and returns. Costs like those of the course fee, facility fee, staff management and their
wages, time used for the training the participants and returns like the business
improvement, increased number of conversions and financial gains, both short term and
long term net gains.

What are the key issues that should be addressed in the design, conduct, and evaluation of
training programs?

The following are the key issues to be addressed for a successful training program:

- Place (indoor/outdoor)
- Audio visual aids
- Relevant training materials
- Facilities
- Time schedule
- Non – visual aids
- Trainer

TRAINING EVALUATION MODEL

A. Kirkpatrick Model: This model has been introduced in 1959 by Donald Kirk Patrick.
This is a very popular model focuses on measuring four kinds of outcomes or it can be said that
outcomes in four levels that should result from a highly effective training programme.
Kirkpatrick (1977) divided the evaluation model into four parts: reaction; learning; behaviour
and results. Reaction would evaluate how participants feel about the programme they
attended. The learning would evaluate the extent to which the trainees learned the information
and skills, the behaviour would evaluate the extent to which their job behaviour had changed
as a result of attending the training. The results would evaluate the extent to which the results
have been affected by the training programme. According to a survey by the American Society
for training and development (ASTD), the Kirkpatrick four level evaluation approaches is still the
most commonly used evaluation framework among Benchmarking Forum Companies (Bassi&
Cheney, 1997). The main strength of the Kirkpatrick evaluation approach is the focus on
behavioural outcomes of the learners involved in the training (Mann & Robertson, 1996).

B. CIPP Evaluation model: CIPP model of programme evaluation is developed by Daniel L.

Stufflebeam (1983). It refers to the four phases of evaluation: context evaluation, input
evaluation, process evaluation and product evaluation. It is based upon the view that the most
important purpose of evaluation is to improve the functioning of a programme.

C. CIRO approach: In 1970, the CIRO model for the evaluation of managerial training was
proposed (Warr, Bird &Rackson, 1970). This model was based on the evaluation of four aspects
of training: context, input, reaction and outcomes. According to Tennant, Boonkrong and
Roberts (2002), the CIRO model focuses on measurement both before and after the training has
been carried out. The main strength of the CIRO model is that the objectives (context) and the
training equipment (input) are considered.

D. Phillip’s Evaluation approach: In the past decade, training professionals have been
challenged to provide evidence of how training financially contributes to business. Phillips
(1996) suggested adding another level to Kirk – Patrick‟s four level evaluation approach to
calculate the return on investment (ROI) generated by training. According to James and Roffe
(2000), Plillips‟s five level evaluation approaches translate the worth of training into monetary
value which, in effect addresses ROI. Philips‟ framework provides trainers a logical framework
to view ROI both from human performance and business outcomes perspectives. However, the
measurement goes further, comparing the monetary benefit from the programme with its
costs. Although the ROI can be expressed in several ways, it is usually presented as a percent or
cost/benefit ratio. While almost all HRD organisations conduct evaluations to measure
satisfaction, very few actually conduct evaluations at the ROI level, perhaps because ROI is
often characterised as a difficult and expensive process.

Training Evaluation – Kirkpatrick’s Model of Training Evaluation

Level 1 – Reaction:

Questionnaires, interviews, group discussion, or asking trainees to write a report can be used.
Care must be taken with all of these methods. Very often participants have enjoyed a course,
even if they learned very little. Factors such as the quality of the lunch provided, or the comfort
of the chairs, may influence the assessment of the training given. The other participants may
have spoilt a basically sound course, or conversely saved a basically poor course.

Trainees are not always in a position to know immediately whether what they have learned will
be useful and it may be best to wait some considerable time before asking for an opinion.
Sometimes a trainee may have felt unfairly criticised during a course, and so may ‘rubbish’ it in
retaliation. The more training a person receives, the more critical he or she is likely to become.
Standards and expectations rise with experience.

Using more than one technique can be helpful to gain a broader picture. Also look out for cues
such as an increase or decrease in demand for the training (where there is a choice), or if the line
managers start asking for one particular trainer in preference to another.

Level 2 – Learning:

Tests, examinations, workplace-based assessments of competence, projects, or attitude

questionnaires are the key techniques here. Some learning situations are easy to test for (e.g
typing ability), whereas others necessarily involve a good deal of subjectivity (e.g counselling
skills).

Yet other learning is so long-term in its nature that direct methods are frankly not appropriate.
For example, if a newly appointed supervisor attends a course, then an end test or examination
can only tell us if he or she has learned certain terms, concepts or models. It cannot tell us if he
or she will become a good supervisor by applying that learning in the work situation.

The processes used at level 2 are often termed validation.

Level 3 – Behaviour:

This level requires assessment of improved performance on the job. This is easiest in jobs where
before-and-after measures can easily be made (e.g the speed at which an insurance proposal form
can be processed).
It becomes more difficult to evaluate performance in jobs which are less prescribed and where
measurement is imprecise (e.g training design). There may be a time-lag between training and
the appearance of indicators of performance improvement.

For instance, upon returning to work after attending a course on leadership, a manager may
immediately practise what he or she has learned – but the results of this take two or three months
to become apparent.

During that time other factors in the situation may have changed – there may have been some
new staff recruited, or some redundancies have affected morale. If we were to instigate a long-
term assessment process, we would also find it difficult to separate out the influence of day-to-
day experience from the influence of the formal training course, it is often impossible to isolate
the precise influence of the training. Often the trainer has to resort to indirect performance
assessment measures to gauge the influence of the training.

Level 4 – Results:

Because departmental and organisational results depend upon many people and it is difficult to
attribute improvements to the efforts of specific individuals, evaluation at this level often has to
be conducted in a more general way.

Does the overall training programme result in greater efficiency, profitability, or whatever? If we
were to try to look at the impact of a large training programme on a part of a large organisation,
we can take an experimental approach.

Ideally, we take two identical units. One is given lots of training, the other is given none. Two
years later, the difference in performance is apparent!

Obviously such an approach is not one which can be easily advocated. If we really believe that
the training is likely to be of value, it is unfair, perhaps even unethical, to withhold it from one of
the units in order to conduct an experiment.

However, it is sometimes possible to obtain historical information which shows a correlation

between spending (or some other measure) on training and organisational performance. Perhaps
two similar units within the same organisation can be compared and the relationship between
past training activity and other measures can be assessed (e.g. accident rate, machine downtime,
customer complaints).

What is the CIRO Model?

The CIRO Model is a training evaluation model that is used to evaluate the effectiveness of
management training course. It was developed by Peter Warr, Michael Bird, and Neil Rackham,
who published the book, Evaluation of management training, in 1970. The CIRO Models offers
businesses an effective way of evaluating their management training needs and results.
‘CIRO’ is an acronym that stands for the four levels which make up this approach to learning
evaluation:

 Context
 Input
 Reaction
 Output

Stage 1: Context Evaluation

At this stage, the CIRO Model is used to assess the operational situation that a business or
organization finds itself in. This provides useful information that can be used to determine
the training needs and objectives.

During this first stage, a training-needs analysis is conducted, based upon the conditions in
the organization. The context evaluation helps to flag up any factors that may have an impact on
the effect of the training.

Stage 2: Input Evaluation

During the second stage of the CIRO Model, practitioners must gather information about
possible training techniques and methods. This is known as the ‘input evaluation’ and helps
identify the best choice of training intervention.

This stage also addresses design, planning, management and delivery of the training course. It
analyzes the organization’s resources and determines how these resources can best be used to
achieve the desired objectives.

Stage 3: Reaction Evaluation

The third stage of the CIRO Model involves gathering the views of the participants and
collecting suggestions about the training they received. The trainees are asked to give their
reactions to the following aspects:

 Program content
 Approach
 Value-added

Information gathered at this stage is used to find ways to improve the training program. As this
evaluation is subjective, it must be collected in an objective way.

Stage 4: Outcome
This stage of the CIRO Model involves presenting information about the results of the training.
The results are presented at three different levels;

 Immediate
 Intermediate
 Ultimate level
The immediate results include how the trainees got on and whether they managed to complete
the training successfully. The intermediate outcomes are those that may take some time to
implement, such as changes to the course design, or acquiring new training resource. Outcomes
at the ultimate level are the main goals for the organization, ones that have a far-reaching impact
on the organization.

As outcomes are evaluated in terms of what happened as a direct result of training, they can be
measured on the following four levels:

 The learner level

 The workplace level
 The team or department level
 The business level

What is cost benefit analysis?

Cost-benefit analysis (CBA) is a process or tool to support decision making in projects.

CBA evaluates the cost versus the benefit of a project to determine project feasibility as well as
provide a decision making metric when weighing up multiple options.

What is the process for cost benefit analysis?

1. Define the project

2. Quantify costs and benefits
3. Standardise the metrics in the calculations
4. Complete relevant calculations
5. Perform sensitivity analysis
6. Complete the business case inclusive of CBA outputs and engage stakeholders

What is training ROI?

Training ROI stands for training return on investment. It is a measure of training success in terms
of meeting your desired objectives. ROI is most often seen as a financial measure, accounting for
the training benefits relative to the money invested in an intervention. 

ROI gets to the bottom line, measuring business results that are a result of training. It does not
measure the lower level results that you would associate with a 'happy sheet' evaluation, such as
learner satisfaction and learning distance travelled.

What Is Transfer Of Training?

transfer of training refers to the ability of trainees to apply the information and skills that they've
learned from their training to practical scenarios.