Online psychometric testing is gaining ground as a scientific and
cost-effective method for measuring a job applicant’s suitability – but it also
raises a number of concerns. How accurate is it? What are the safeguards
against fraud? Will it fall foul of data, race and sex discrimination rules?
Keith Rogers looks for some answers
Rick Woodward, European learning and development director at Kimberly-Clark,
is an enthusiastic advocate of online psychometric testing. Having introduced
the process in both its graduate and executive recruitment programmes, the
company has slashed its cost base, streamlined its administrative processes and
improved both internal communications and its feedback to job applicants. Best
of all, Woodward argues, the candidates themselves say they like it.
Kimberly-Clark’s experiences, based on services provided by SHL, fly in the
face of much recent research into the online testing market. To date, HR
managers have demonstrated a reluctance to go down the internet route at this
stage of the selection process, citing numerous fears ranging from the
difficulties of administering tests remotely, to concerns about their equal
opportunities responsibilities. Although the internet recruitment market is
expanding fast, psychometric testing seems to be running up against both
psychological and practical hurdles.
But given how widely the procedure is being deployed, particularly among
Times 1,000 companies, acceptance of online testing is becoming a major issue.
In a depressed economy where the volume of applications for each job grows, the
ability to automate parts of the recruitment process is becoming ever more
pressing – particularly as organisations expand the reach of their job
advertisements through internet advertising, generating a corresponding
increase in candidates. Although the employment market might favour the hirer
today, the conditions also create a growing administrative burden that becomes
costly and inefficient.
As a pre-interview screening process, psychometric testing is a powerful
tool, whether used online or carried out with pen and paper. Used to measure
both a candidate’s job-related abilities and to assess their personality, it
provides a relatively scientific context to the biographical data that’s
typically used as the basis for the interview process. Organisations such as
GlaxoSmithKline have already moved from paper-based tests to running the
process on PCs within their assessment centres, so speeding up data analysis
and report generation, removing the potential for human error in the
administrative process and allowing them to store candidate-related data
centrally. In theory, online testing merely takes that process one stage
further, retaining all the administrative benefits while allowing candidates to
carry out the tests from their own chosen location.
While the theory is good, in practice it raises a number of concerns. To
begin with, one advantage of bringing candidates to an assessment centre rather
than allowing them to take tests at home is that the employer can control the
environment in which they operate. As Mark Evans at GlaxoSmithKline points out:
"They are being supervised – it’s the same temperature in the room,
there’s no noise, no distractions – the kids aren’t running around, the guy
next door isn’t using a pneumatic drill. And there’s something good about the
fact that you can give similar verbal briefings. I think there’s something to
be said for having the human element there to offer reassurance."
But there’s a counterpoint to this argument, according to Dr Colin Selby of
Penna Consulting, which provided Glaxo-SmithKline with its PC-based testing.
Selby, who is also chairman of a British Psychological Society committee,
argues that while assessment centres can provide reassurance, they can also be
prejudicial – if the administrator is unfriendly, for example, that may inhibit
a candidate’s performance. The online testing experience of Woodward at
Kimberly-Clark supports that view: "It was a win from the candidates’
point of view – they liked it, and said it was better than sitting in an
assessment centre with other candidates, and us breathing down their
necks."
Managing the test environment is only one aspect of a wider control issue
that persistently crops up when HR managers express their reservations about
running the process online. John Hackston, managing consultant at test provider
OPP, draws a distinction between ability testing, which requires strict
controls, and personality tests – but even the latter can present problems
online. For one thing, some candidates will try to find out what the
"right" answer is – and given the speed with which organisations can
provide feedback to applicants, there’s little to stop the same person logging
in under a variety of pseudonyms to try different approaches. Tests that are
supposedly subject to strict time controls can also be hard to police.
There are some technical solutions to these issues. SHL’s testing program,
for example, drops a Java applet onto the applicant’s system that times them
out when the test period is complete. The company reinforces that approach by
distributing an "honesty contract", which points out that some
attributes will be tested again if the candidate progresses to the next stage
of assessment and any disparities will be investigated. Others take a more
pragmatic approach – Kimberly-Clark, for example, puts no time limit on the
tests it carries out online, partly in recognition of the fact that some
candidates (like those for whom English is a second language) will require more
time to complete the process than others.
Catering for these kind of cultural variances is a further area of concern
for potential adopters, both in ethical and legal terms. Will all candidates
have access to a PC – and will those that are unfamiliar with the PC
environment spend more time working out how to use a mouse than filling in the
online form? Is the online process liable to breach any key employment
legislation, particularly the Sex Discrimination, Race Relations and Disability
Discrimination Acts? What are the implications under the Data Protection Act?
In some areas, the issues are no different whether the tests are carried out
online or on paper. As the IRS Employment Review pointed out in January,
indirect discrimination can occur purely because of differences in the way that
men and women respond to psychometric questionnaires. Guidelines issued by the
Commission for Racial Equality advise companies on how to head off these kinds
of problems, ensuring, for example, that tests don’t contain irrelevant
questions in areas that may be unfamiliar to racial minority applicants. As
OPP’s Hackston points out, organisations should use the same criteria for
online testing service providers as they would for any other medium, looking
for reliability and validation in the testing procedures. Selby also suggests
that companies validate their PC or online procedures with existing employees
before they use it on candidates, measuring their 10 highest and lowest
performers and ensuring that the psychometric process successfully
distinguishes between the two.
Data protection issues are a little more specific to PC-based and online
testing, and processing of personal data is strictly regulated. As the IRS
report points out, employers must comply with all the data protection
principles enshrined in the 1998 Act, including ensuring that personal data is
relevant to the purposes for which it is processed, isn’t kept longer than
necessary and is protected against unauthorised or unlawful processing. In
technical terms, that means stored data must be secured against unauthorised
outside access, and controlled effectively internally. Some proponents of
PC-based or online testing argue that the automatic data collation is a benefit
in addressing all of these legal issues, since it allows for rapid collation of
data for analysis.
But the issue of candidates’ access to and familiarity with PCs is less
clear cut. GlaxoSmithKline’s Evans, who is keeping an open mind about moving
down the online route, says his only concern right now "is the fact that
you might be excluding sections of the population. There’s a danger of
excluding very good candidates by restricting testing to the internet."
Ultimately, the decision to go down the online route will be heavily influenced
by local environmental factors.
"Access to the web is still an issue for some people," says Selby.
"If you want to recruit a nurse in the Philippines, access issues could
prevent some candidates applying. You’ve got to accept that there are
inequalities in this respect. If somebody can’t be bothered to find access to
the web in the US, they’re not thought to be serious – that point has been
raised in the UK as well. While it’s okay for IT engineers – it’s not fair for
people who want to work in a children’s home." Woodward emphasises these
cultural differences apply across Europe – while psychometric testing is
increasingly common for graduates in Germany, for example, it raises eyebrows
when applied to more senior positions.
While these types of objections are all valid, few dispute the fact that
online testing offers three core benefits – better speed, accuracy and cost. As
well as quicker collation and analysis of electronic data, both PC and online
psychometric tests allow for faster dissemination of data within organisations,
speeding up the internal administration that accompanies selection and bringing
uniformity to the processes. In particular, candidate feedback – which Selby
describes as an applicant’s "right" – can be dramatically improved.
Administrative accuracy is also enhanced as processes are automated and the
re-keying of data is removed. That does not, however, necessarily improve the
accuracy of the data itself. While Selby points to BPS research indicating that
responses to personal questions tend to be more reliable online than on paper
or face-to-face, Woodward’s experience at Kimberly-Clark is that the
consistency of online responses is lower compared to previous testing
exercises. "Because people have got more time, they’re thinking ‘what’s
the politically correct answer?’" he says. "One engineer’s
consistency score was one (against a norm of five to seven) – when I met him I
did not believe a word he said."
From a cost perspective, however, the statistics are compelling. While there
is a cost associated with training testers to manage a new online environment,
Kimberly-Clark has seen its selection overheads drop significantly. The company
used to accommodate candidates for two nights in hotels while they went through
its assessment process – by running the psychometric tests offsite, the company
saved one night’s hotel accommodation, amounting to an overhead reduction of
some £7,000 a year. The tests themselves are also cheaper. "Every time we
did a test with paper and pencil and calculators it was costing us £25: this is
costing us £13.50," says Woodward.
But before organisations jump into online testing, they need to take into
account several key factors. To begin with, all the major software and service
providers insist that psychometric testing should be viewed as just one element
of the overall selection process. Steve Newhall, head of business development
at DDI, says: "We would not recommend that you use any level of
psychometric test as a tool on its own. It needs to be balanced with
behavioural information – what they can do and what they’ve done. We tend to
use psychometrics as a way to supplement the other information we get."
He adds: "It’s about the robustness of the assessment tools. You take a
decision in any other process about the point at which you switch to a human
interface. Are we bringing people through to that point, and have we weeded out
unsuitable people; gathered data that’s going to add significant value to that
human piece; and have I managed to get that data more cheaply and
efficiently?"
Also, organisations need to ensure that their existing recruitment processes
are suitable for the online environment. Roy Davis, head of communications at
SHL, points out that the internet is merely the transport mechanism for the
testing process – success depends on how effective the underlying HR processes
are. "If you’ve got a bad test to start with," he says, "putting
it on the internet won’t improve it." Because internet recruitment causes
potential overload problems as the organisation’s reach expands, Davis suggests
that companies should also give candidates an opportunity to opt out of the
process, perhaps by being more specific in describing what the advertised job
really entails. In that vein, Selby says companies need to be far clearer about
the nature of both the job and the personality specifications associated with
it before they go down the internet route. It’s no longer enough to ask for
"leadership potential" – rather, firms should be stressing the nature
of the leadership skills required and situations in which they apply.
Sign up to our weekly round-up of HR news and guidance
Receive the Personnel Today Direct e-newsletter every Wednesday
Right now, the market for online testing remains relatively immature, and
uptake is only likely to increase when the number of reference sites expands.
The emergence of more sophisticated techniques – such as adaptive testing,
where systems change the questions posed to candidates in relation to their
response – will also begin to swing the balance. Ultimately, however, it will
be the experiences of organisations like Kimberly-Clark that determine whether
other users follow suit.
"I’ve been surprised by the results of some of the research [into
online testing]," says Woodward. "We’ve got a cheap, sophisticated
test that we should be using for all executive appointments – it’s giving you a
quality second interview."
