The BBC is to publish details of a ground-breaking method to evaluate the impact a new leadership training programme is having on its managers.
The method was devised following criticism from the national press that the £6m-a-year programme was wasting licence fee payers’ money.
Josie Barton, evaluation consultant for BBC training and development, said it helps to distinguish between the benefits gained from the programme from those gained elsewhere, including the leadership experience itself.
“Because it represented such an enormous investment for the BBC, the evaluation strategy had to be fairly robust,” said Barton.
“We did quite a lot of research into how other companies evaluated their training, but could not find a framework that was comprehensive enough to do everything we wanted it to do.”
Barton worked closely with organisational psychologists to devise the evaluation method, which involves comparing 100 managers who have completed the programme with 100 who have not.
At least three subordinates of each manager give feedback on their leadership competencies and behaviours, but Barton said direct questioning is avoided. Instead, the subordinates are presented with four workplace scenarios that demonstrate their manager’s reactive, responsive, pro-active and high-performing leadership in action.
“It’s not as black and white as saying their manager is good or bad at something,” said Barton. “You can look at the scenario and not necessarily know which is the high and low performing one.”
Barton describes the process as “hugely complex”, but added: “It could [be applied] in other businesses. It would be [most effective] where [training] programmes represent a large investment, and organisations need to isolate the impact they are having.”
Lancaster University is currently analysing the results but Barton says initial indications suggest managers who have been on the programme are more willing to experiment.
Details of the method will be published on the BBC training website later this month.
www.bbctraining.com
This month’s E-learning case study illustrates the detail of the BBC training programme on editorial policy
