Liddle offers an evaluation workout to help you measure the impact of your
Start with the business strategy
all know that people are fundamental to an organisation achieving its business
strategy. By increasing people’s performance, training can therefore help to
increase business performance. Evaluation allows you to quantify this impact.
If you consider how much your organisation spends on training, can you afford
not do evaluate?
Identify how performance needs to change
is much easier and more effective when you focus on the business objectives.
For each business objective, establish the required level of performance and
compare this to current performance. If there is a gap, think about how
people’s knowledge, skills or behaviour need to change.
is important to identify which group(s) of people need to be involved. It is
also important to ensure that they understand why changes in their knowledge,
skills or behaviour are required and how these changes will contribute to
business performance. By understanding why training is required, people are
more likely to be motivated to change.
Check for underlying issues
save yourself time, money and a lot of frustration check that there are no
underlying issues currently affecting people’s performance, for example:
Lack of motivation/reward,
may already have the necessary knowledge, skills or behaviour, but due to
underlying issues they are not motivated to perform to their full potential.
Where this is the case, even world-class training will have minimal impact on
Estimate the impact of training
is often required in order to justify the spend on training. You need to
estimate the anticipated changes in performance and the cost of training.
the cost is particularly useful when you are selecting from more than one
proposed training solution. A suggested framework includes: analysis time,
design and development time, materials, equipment/facilities, delivery time,
evaluation and participants costs.
the impact of training is expected to continue over a period of time, you can
spread the estimated training costs. This is known as a payback period.
an alternative perspective, calculating the cost to your organisation of not
training is a useful exercise. This can often be very revealing and create a
convincing business case for training.
Agree evaluation methodology
is important to agree with the relevant parties which evaluation methodology
you are going to use.
term “return on investment” (ROI) is originally an accounting term and measures
the anticipated profitability of an investment. Caution is required when using
ROI in training evaluation as managers may compare this methodology to ROI
calculations for capital expenditures.
following equation provides a simple but effective alternative:
in performance (A) – Cost of training (B) = Impact of training (C)
necessary, you can still put a pound sign on (A) by asking the question, “What
will changes in performance mean to your organisation in financial terms?”
Identity evaluation measures
fundamental part of this workout is identifying relevant evaluation measures to
describe changes in performance. This can be described as “firming up the vital
evaluation measures should include a mix of quantitative business data and
existing business measures as sources of quantitative data. This makes your
evaluation more credible, as the business areas will be able to identify with
their own facts and figures.
are a number of performance measurement systems used by organisations, eg The
Balanced Scorecard, The EFQM Excellence Model, Key Performance Indicators.
There is also a wide selection of data collected by organisations, such as
production time/costs, sales figures, customer data, human resource information,
data for industry specific standards. It is a good idea to find out how
performance is measured and recorded in your own organisation.
identified which business measures will provide you with relevant quantitative
data, it is useful to find out who is responsible for collating this data and
how frequently it is collected. You may find it helpful to describe these
business measures under broad categories e.g. quantity; quality; time; cost;
gather qualitative data/feedback you need to think about who will be affected
directly or indirectly by those who have been trained. For example: team
members, colleagues, line managers, customers, the individuals themselves.
managers also have a very important evaluation role to play, coaching and
reviewing on-the-job performance. This support motivates people to make
techniques for gathering qualitative data/feedback include questionnaires,
surveys, focus groups, interviews, observation, case studies. Where
appropriate, make use of your organisation’s technology to collect and manage
evaluation information – “E-evaluation”.
from a sample number of people will often be adequate, particularly in large
Plan the evaluation timescales
ensure that the business areas can make timely and full use of the evaluation
findings, identify any project deadlines which you need to accommodate in your
key element in gathering evaluation information is collecting base-line data,
prior to the training. This will allow you to do a before and after comparison.
initial reactions to the training provide a useful, early source of feedback.
Again, a sample number may be adequate.
most cases it is useful to do an interim review of how performance is changing.
Timescales for this will depend on the type of learning (shorter for
skills-based and longer for behavioural). Taking an interim “reading” of all
the business measures will not always be practical, you may have to select.
for the “final reading” of both the quantitative and qualitative measures
should coincide. This will provide an up-to-date picture of how performance has
changed. The final readings can then be compared to the base-line data.
you have followed the evaluation time scales, you will have a selection of
quantitative and qualitative data/feedback which describes how performance has
together, this provides the information for (A) in your final calculation.
Remember to put a pound sign on (A), if your methodology requires you to
describe the changes in performance in financial terms.
calculate (B), you need to revisit the costs. Remember to include a payback
period if necessary.
in performance (A) – Cost of training (B) = Impact of training (C)
a few moments to reflect on this evaluation workout and how it could increase
the impact of training in your organisation.
regularly, the exercises will help to ensure that training makes a noticeable
difference to your organisation’s bottom-line.
Liddle is an independent evaluation specialist. Tel 0131-447 0771 or e-mail [email protected]