The Cabinet Office, tax and audit consultancy KPMG, global manufacturer 3M and the NHS are among the growing numbers of employers turning to online testing to save both time and money in graduate and other large volume recruitment.
In 2003, only 6% of organisations were using online testing for selection, according to the 2004 Recruitment, Retention and Turnover survey by the Chartered Institute of Personnel and Development (CIPD). But there has been a rapid shift in the past year towards using this assessment method.
“More employers are implementing rigorously designed online testing to bring benefits of speed, cost, fairness and objectivity,” says Richard Alberg, chief executive of online test provider PSL.
3M introduced test provider SHL’s online ability tests three years ago to help sift through the large volume of graduate applications it receives each year. This year, it is recruiting 75 graduates for posts in IT, manufacturing, sales and marketing. Last year it received around 1,600 applications and expects a similar number this year.
Jenny Crespo, HR manager at 3M has reaped the benefits.
“We are seeing more applications since we have gone online,” she admits, “but because of the screening process, I get to see around 30 applicants instead of 200 or so. It is a good way of sifting candidates.”
She says that online testing is very easy to administer, saving time spent on administrative tasks such as scoring and putting together reports.
“The biggest saver is that we can send it off in advance and have the data in front of us for interview. We no longer have to bring people from all over the place to test them,” says Crespo.
But despite the obvious attractions of online testing, employers need to be careful which tests and providers they plump for, addressing issues such as security of tests and data, standards of administration, and control over the testing process. The CIPD raises these issues in its factsheet on psychological testing, updated this January, pointing out that the International Test Commission is currently working on a set of global guidelines on computer-based and internet-delivered testing.
John Hackston, managing consultant for research and development at test provider OPP, says: “There are so many online tests around including the kind saying they can tell your character by the colour of your lipstick and so on. Just because a test is online doesn’t mean it’s wonderful. Employers need to make sure the tests are valid, fair, reliable and consistent and all the things they would expect from written tests,” he says.
Traditionally, online testing has been mostly used in recruiting graduates, but employers are increasingly using it to recruit for other posts, including experienced hires and clerical staff. PSL is currently designing an online test to recruit traffic wardens for National Car Parks.
Beating the cheats
A major consideration for many employers is how to minimise the risk of cheating. The most common approach is the ‘item bank’ facility offered by suppliers such as PSL, SHL and Cubiks. The system draws questions randomly from a bank, minimising the chances of familiarity or cheating.
Paul Levett, SHL’s commercial director of products and training, says: “We have a unique test for each applicant making it very difficult for them to cheat. They could get a friend to sit the test, but often the final stage is supervised which takes away most chances of cheating.”
KPMG introduced item-bank technology from Cubiks about two years ago, along with random supervised re-testing. Keith Dugdale, director of recruitment and resourcing at KPMG says the firm has only encountered huge discrepancies between initial and follow-up tests twice over the past two or three years.
“The problem we gave a huge amount of thought to was that of user identification. We’ve got around it to an extent by randomly re-testing a percentage of candidates. It’s a bit like random drugs testing. If people know they might be tested, it makes them more cautious,” says Dugdale.
Rob Feltham, executive director of innovation, products and technology at Cubiks, says employers need to decide whether it is important that all candidates see the same test and should ask suppliers how widely exposed the tests are.
“They should ask about exposure to tests. Even with item bank systems, candidates can be very sophisticated to the extent that if you go into an internet chatroom, you find them asking each other ‘was it a Cubiks test or an SHL test?’, and so on,” he says.
Another approach to security, which is popular in the US and Asia Pacific but has been slow to take off in the UK, is supervised testing at online testing centres. The Psychometrics Centre at City University in London set up the UK’s first online testing centre towards the end of 2004.
Ian Florance, consultant to the Psychometrics Centre, says: “Certain tests are best done supervised in the employer’s offices or in online testing centres, such as large-scale recruitment ability tests, where there are wrong answers.”
Validation and tailoring
Validation of tests is another key issue. Psychometric tests, in particular, require a lot of trials to ensure they work, so it is vital to ask providers about test development and for details of trials, says Dugdale.
“The other big issue for us was validation, making sure there are robust samples on which to build norm tables [average test results for comparative purposes] and that the tests do what they should do,” he says. “It is important to continue to evaluate and see whether the tests are still delivering the right candidates.”
Employers also need to decide how tailored their tests need to be, and find out how much tailoring the provider can do. For many employers, generic tests are suitable, whereas others, such as the Cabinet Office, require bespoke solutions.
Whether the online tests can sit well with other technology is also a key issue. Employers need to ask about compatibility of the systems with those used by the employer and the supplier.
As 3M’s Crespo says: “Some tests can’t be done on some systems and people have to go to internet cafes, which can be a real stumbling block and can put people off.”
Employers need to consider how the tests will integrate with any existing HR workflow systems, whether provider systems can cope with large-scale recruitment, and what levels of IT support and infrastructure are available. It is a good idea to use an IT specialist to audit what is to be supplied.
PSL’s Alberg says:” You hear horror stories of employers sending out e-mails to people with a closing date, then everyone logging on and the system falling over. Getting the technology right is a real issue. If it causes problems, it is a nightmare.”
Employers also need to ask test providers about diversity, says Dugdale: “Evaluation is very important. For us, diversity was a big reason for using professional occupational psychologists. The danger in getting tests from the internet is that there is no guarantee they will deliver.”
Case study: KPMG
Tax and audit consultancy KPMG has slashed graduate hiring costs by 20% and the amount of time spent recruiting them by 40%, following an overhaul of its graduate recruitment process.
Graduate recruitment is very important for KPMG, winner of the 2004 Personnel Today Award for Excellence in Graduate Recruitment. It receives more than 9,000 applications every year for around 800 graduate positions. Following the far-reaching revamp, which included the introduction of online testing, KPMG spends much less time screening and interviewing candidates. But the percentage of candidates accepting job offers is now 89%.
Test provider Cubiks has developed a series of online tests for graduate recruit-ment, including a business-awareness questionnaire, which presents a range of scenarios they might encounter at KPMG.
“This builds up a profile of candidates which we can model against successful recruits, to give us an idea of whether they have the skills and competencies we’re looking for,” says Keith Dugdale, director of recruitment and resourcing at KPMG.
Cubiks also developed online verbal and numerical tests of 20 minutes, which have allowed about a third of candidates to be screened out.
The successful use of online testing in graduate recruitment has prompted KPMG to extend it to other recruitment. Later this year, it will launch an online in-tray assessment exercise to recruit experienced hires into its tax, audit and corporate functions. It hopes to go live with this in May and is also considering going online with graduate in-tray exercises.
The experienced hire exercise will consist of three 30-minute modules presenting can-didates with project management scenarios they might typically encounter if recruited. The first two modules will assess abilities relevant to the whole business. The third will be customised for particular functions such as tax, audit, or corporate finance.
Case study: NHS
The NHS Leadership Centre is using online numerical reasoning tests to cut down on the administrative and financial burden of processing thousands of graduate applications (Personnel Today, 1 January).
The centre trialled the new system for the 4,500 graduates in the February 2004 intake for 173 posts in general management, financial management and HR streams.
The competencies measured in the 20-minute tests, provided by PSL, are the same for all three streams, but with different cut-off scores.
Applicants are also required to complete online a 32-question de-selecting psychometric test designed for the NHS by occupational psychologists Pearn Kandola, measuring competencies in areas such as working with others, and managing, planning and organising.
Sign up to our weekly round-up of HR news and guidance
Receive the Personnel Today Direct e-newsletter every Wednesday
The NHS uses an applicant-tracking sys-tem, provided by Milkround Online, which it needed to be compatible with the online testing system, so that when applicants sit the test, the scores make their way easily into the NHS system. The centre narrowed its choice of providers down to SHL and PSL, but opted for the latter.
Fiona Mayslip, marketing and recruit-ment co-ordinator for the leadership centre, says: “It was very close, but the difference was the number of browsers. The number of tests that would run correctly was greater with PSL, which is important when it comes to giving candidates support.”
When to use a psychometric test
Only use tests if suppliers address the following issues satisfactorily:
- Test reliability
- Consistency as a measure
- Validity and whether the test truly identifies those skills or attributes claimed
- Evidence that tests do not unfairly disadvantage certain groups
- Ask whether it is appropriate in the eyes of those taking it
- Previous reactions to the test
- Previous effective use of the test in similar circumstances
- Question whether the norms provided for comparative purposes are up-to-date and appropriate
- Do norm results apply to a sufficiently diverse group for fair comparison with the user’s group?
- Source: CIPD factsheet Psychological Testing, updated January 2005
Questions to frighten the cowboys
How many occupational psychologists do you employ, are they full-time and what are their qualifications?
- How have the psychometric abilities been tested?
- Can I see the norm groups?
- How many people in the trial?
- Explain to me item response theory
- Does every candidate see the same test?
- How will this integrate with my existing workflow system?
- Can you create something bespoke if required?
- What IT infrastructure do you have in place, how many servers and server back-ups?
- How quickly do you turn around responses on the IT help desk?
