Fiona
Liddle offers an evaluation workout to help you measure the impact of your
training
Warm-up
Start with the business strategy
We
all know that people are fundamental to an organisation achieving its business
strategy. By increasing people’s performance, training can therefore help to
increase business performance. Evaluation allows you to quantify this impact.
If you consider how much your organisation spends on training, can you afford
not do evaluate?
Exercise
1
Identify how performance needs to change
This
is much easier and more effective when you focus on the business objectives.
For each business objective, establish the required level of performance and
compare this to current performance. If there is a gap, think about how
people’s knowledge, skills or behaviour need to change.
It
is important to identify which group(s) of people need to be involved. It is
also important to ensure that they understand why changes in their knowledge,
skills or behaviour are required and how these changes will contribute to
business performance. By understanding why training is required, people are
more likely to be motivated to change.
Exercise
2
Check for underlying issues
To
save yourself time, money and a lot of frustration check that there are no
underlying issues currently affecting people’s performance, for example:
–
Poor management/leadership,
–
Lack of motivation/reward,
–
Inappropriate processes/technology.
People
may already have the necessary knowledge, skills or behaviour, but due to
underlying issues they are not motivated to perform to their full potential.
Where this is the case, even world-class training will have minimal impact on
performance.
Exercise
3
Estimate the impact of training
This
is often required in order to justify the spend on training. You need to
estimate the anticipated changes in performance and the cost of training.
Estimating
the cost is particularly useful when you are selecting from more than one
proposed training solution. A suggested framework includes: analysis time,
design and development time, materials, equipment/facilities, delivery time,
evaluation and participants costs.
Where
the impact of training is expected to continue over a period of time, you can
spread the estimated training costs. This is known as a payback period.
From
an alternative perspective, calculating the cost to your organisation of not
training is a useful exercise. This can often be very revealing and create a
convincing business case for training.
Exercise
4
Agree evaluation methodology
It
is important to agree with the relevant parties which evaluation methodology
you are going to use.
The
term “return on investment” (ROI) is originally an accounting term and measures
the anticipated profitability of an investment. Caution is required when using
ROI in training evaluation as managers may compare this methodology to ROI
calculations for capital expenditures.
The
following equation provides a simple but effective alternative:
Changes
in performance (A) – Cost of training (B) = Impact of training (C)
If
necessary, you can still put a pound sign on (A) by asking the question, “What
will changes in performance mean to your organisation in financial terms?”
Exercise
5
Identity evaluation measures
A
fundamental part of this workout is identifying relevant evaluation measures to
describe changes in performance. This can be described as “firming up the vital
statistics”.
Your
evaluation measures should include a mix of quantitative business data and
qualitative data/feedback.
Quantitative
Use
existing business measures as sources of quantitative data. This makes your
evaluation more credible, as the business areas will be able to identify with
their own facts and figures.
There
are a number of performance measurement systems used by organisations, eg The
Balanced Scorecard, The EFQM Excellence Model, Key Performance Indicators.
There is also a wide selection of data collected by organisations, such as
production time/costs, sales figures, customer data, human resource information,
data for industry specific standards. It is a good idea to find out how
performance is measured and recorded in your own organisation.
Having
identified which business measures will provide you with relevant quantitative
data, it is useful to find out who is responsible for collating this data and
how frequently it is collected. You may find it helpful to describe these
business measures under broad categories e.g. quantity; quality; time; cost;
margins.
Qualitative
To
gather qualitative data/feedback you need to think about who will be affected
directly or indirectly by those who have been trained. For example: team
members, colleagues, line managers, customers, the individuals themselves.
Line
managers also have a very important evaluation role to play, coaching and
reviewing on-the-job performance. This support motivates people to make
changes.
Evaluation
techniques for gathering qualitative data/feedback include questionnaires,
surveys, focus groups, interviews, observation, case studies. Where
appropriate, make use of your organisation’s technology to collect and manage
evaluation information – “E-evaluation”.
Feedback
from a sample number of people will often be adequate, particularly in large
organisations.
Exercise
6
Plan the evaluation timescales
To
ensure that the business areas can make timely and full use of the evaluation
findings, identify any project deadlines which you need to accommodate in your
evaluation timescales.
A
key element in gathering evaluation information is collecting base-line data,
prior to the training. This will allow you to do a before and after comparison.
People’s
initial reactions to the training provide a useful, early source of feedback.
Again, a sample number may be adequate.
In
most cases it is useful to do an interim review of how performance is changing.
Timescales for this will depend on the type of learning (shorter for
skills-based and longer for behavioural). Taking an interim “reading” of all
the business measures will not always be practical, you may have to select.
Timescales
for the “final reading” of both the quantitative and qualitative measures
should coincide. This will provide an up-to-date picture of how performance has
changed. The final readings can then be compared to the base-line data.
Exercise
7
Measure up!
If
you have followed the evaluation time scales, you will have a selection of
quantitative and qualitative data/feedback which describes how performance has
changed.
Pulled
together, this provides the information for (A) in your final calculation.
Remember to put a pound sign on (A), if your methodology requires you to
describe the changes in performance in financial terms.
To
calculate (B), you need to revisit the costs. Remember to include a payback
period if necessary.
Changes
in performance (A) – Cost of training (B) = Impact of training (C)
Cool
down
Take
a few moments to reflect on this evaluation workout and how it could increase
the impact of training in your organisation.
Sign up to our weekly round-up of HR news and guidance
Receive the Personnel Today Direct e-newsletter every Wednesday
Practised
regularly, the exercises will help to ensure that training makes a noticeable
difference to your organisation’s bottom-line.
Fiona
Liddle is an independent evaluation specialist. Tel 0131-447 0771 or e-mail [email protected]
