Big claims are made for the return on investment of employee wellbeing programmes, but how reliable is the evidence? Dr Paul J Nicholson looks at research studies and analyses, and concludes it is misguided to allow profit alone to become the driving force behind employee wellbeing.
Workplace health and wellbeing has risen up the public health policy agenda in the past two decades. Apart from the need to respond to an ageing workforce, the main drivers behind the interest in employee wellbeing appears to be intangible benefits and, for governments, the wider costs of lifestyle-related ill health to society.
Insurer Aviva’s 2012 survey of 1,000 UK employers reported the main drivers to be: healthy, motivated employees are more productive (41% of respondents); staff recruitment and retention are improved (25%); and to be seen as a caring employer (21%).
In spite of such intangible drivers, it has become fashionable, in some quarters, to perform economic analyses and report return on investment (ROI) among other performance indicators. The work and health literature is deluged by studies, reports, systematic reviews and meta-analyses that report positive ROI from workplace wellbeing initiatives.
This might suggest that ROI is a proven economic model to apply to justify occupational health services and specific interventions. Indeed, the Council for Work and Health recommended recently that “it should be ensured that employers understand the ROI in occupational health”.
I think it is more apt to suggest that employers understand the value proposition. As Juniper notes: “ROI signifies how successful financially a particular venture has been” and “is problematic when examining the payback of wellbeing programmes” (Juniper, 2016).
There is additionally an infestation of papers debating which economic analyses to use. Reassuringly, more reports are appearing that critically appraise systematic reviews and meta-analyses, which otherwise might be accepted as gospel. Juniper noted that an often-cited Harvard meta-analysis was “retired” because the methodology lacked proper scientific rigour.
Websites such as those of the University of York’s Centre for Research and Dissemination (CRD) and McMaster University provide access to quality-rated systematic reviews. As an example, the CRD Database of Abstracts of Reviews of Effects (DARE) contains a review of Parks et al’s meta-anlaysis (2008), which suggested that participation in an organisational wellness programme was associated overall with lower absenteeism rates and higher job satisfaction. CRD concluded that “due to methodological limitations, paucity of study details and low methodological quality of the included studies, the authors’ conclusions may not be reliable”. This serves as a salient reminder of the weakness of using hierarchies of evidence that grade systematic reviews and meta-analyses as high-quality evidence irrespective of the quality of included studies.
The report likely to be most familiar to UK readers is PricewaterhouseCoopers’ (PwC) “Building the case for wellness” (PricewaterhouseCoopers, 2008). It stated that it reviewed eight systematic reviews (albeit one was a critical review and another only reviewed nine studies). Another systematic review included was that of Sockoll (2009), who had already covered four of the other reviews included by PwC, and, unlike PwC, noted that at least one of these reviews included studies of poor methodological quality.
PwC also reviewed 11 published case studies and analysed 55 UK case studies provided by the Department for Work and Pensions (DWP), which commissioned the report. PwC noted that seven case studies reported ROI (as benefit-cost ratio) for their wellness programmes. There was wide ranging ROI 1:1 to 34:1 – and studies were heterogeneous with regard to interventions studied, which included ergonomic improvements or support, influenza immunisation, physiotherapy, health and safety awareness, and physical wellbeing.
Herein lies a problem because many interventions have only been assessed in a small number of settings and much of the evidence on long-term costs and benefits relies on estimates (Merkur, 2013). Three separate reviewers (Burton, 2010; Martínez-Lemos, 2015; and Sockoll, 2009) note that: reviews often provide little detail regarding what exactly was done; that the original studies demonstrate huge variety in the methodologies and interventions studied; and there are frequent flaws in the research design of included studies.
As Tompa and colleagues (2010) described it as “a mixed bag” of methodology and quality, this makes it very difficult to summarise the main outcomes, such that the evidence base must be treated with caution (Merkur, 2013). It is “wise to take cost effectiveness data with a grain of salt unless exact details are known about the methodology” (Burton, 2010).
Can ROI claims for employee wellbeing programmes be justified?
Faced with these conclusions from expert analysts, how then can nine pages of the PwC report have the large header “Workplace wellness makes commercial sense”? While the sentiment may indeed be sound and principle based, it is even difficult to arrive at the same conclusion based on the scant UK evidence provided within the report, or the statement in the executive summary that “there is as yet no clear business case that demonstrates wellness programmes have a direct financial return or effects on tangible business benefits”.
PwC did describe concerns regarding the methodologies, albeit these were relegated to Appendix 4 and not mentioned in the executive summary or the main body of the text. As Juniper (2017) notes, it seems that PwC mistakenly took the information at face value and should have been more forensic in its approach. I share her disappointment that the report is still cited repeatedly without question or query. Reassuringly, a Department for Business Innovation and Skills report acknowledges that, as yet, there is relatively little empirical evidence relating to the relationship between employees’ subjective wellbeing and workplace performance (Bryson et al, 2014).
While the PwC report appeared to make the business case, others published shortly afterwards noted the very limited robust research on the relationship between workplace health promotion and prevention and individual job performance or cost-effectiveness (Bevan, 2010; Sockoll, 2009). More recently systematic reviews have reported that high-quality trials tend to report smaller effects than low-quality trials (Baxter, 2014; Feltner, 2016; Rongen, 2013).
Yet, in spite of the limited evidence, the business case for workplace health promotion appears largely to have been accepted and interest continues; in part, to promote workforce productivity and health as critical factors contributing to overall business performance (Martínez-Lemosm, 2014). An organisation’s occupational health and employee wellbeing programmes are not directed just by economic value or scientific evidence; less tangible variables influence the services provided. However, in recent years the focus appears to be directed toward demonstrating ROI from such programmes, leading to reports being published that show what people wish to demonstrate.
Presented with the volume of literature, many people will assume that the economic benefit of workplace health promotion is indisputable (Sockoll, 2009). The quality of such studies is generally poor, with flaws in study design, lack of assumption soundness, insufficient provisions for uncertainty, poor application of economic evaluation and overall poor research quality (Targoutzidis, 2014).
A systematic review of the methodological quality of 34 economic evaluations of occupational health interventions reported that less than half of the studies satisfied more than 50% of methodological quality criteria, and only three studies met more than 75% of the criteria (Uegaki, 2011). After all, workplace wellbeing programmes are conducted at work sites and not in controlled environments, hence several intervening factors might explain, to some extent, the results of an evaluation (PwC, 2008).
What then are the sources of error and bias?
Statistical issues – the types of statistical measures applied to evaluations often assume a normal distribution of the data when it may be skewed due to the voluntary nature of programmes (PwC, 2008).
No control group – many of the published evaluation studies fail to include control groups not exposed to the intervention (Bevan, 2010). Lack of randomisation weakens the study (PwC, 2008); consequently, the changes that occur may have happened anyway (Bevan, 2010; Burton, 2010).
Selection bias – people who volunteer to participate may already be highly motivated and interested in the outcome of the intervention. This means the results attained will overestimate the effects (Burton, 2010).
Attrition bias – participants who do not change behaviour and drop out of the intervention group will cause the impact as measured among surviving study participants to be overestimated (Burton, 2010).
The Hawthorne Effect – people being observed change their behaviour simply because they are being watched or studied. Although the validity of the Hawthorne Effect has been challenged, there is some evidence it may have an impact (Burton, 2010).
Diffusion of information – staff in the intervention group and control group often work in the same location. Health information might leak to the control group to reduce differences (PwC, 2008).
Subjective measures – many jobs do not have easily measurable output (Mattke, 2007; PwC, 2008) and the methods to measure and value productivity vary widely (Uegaki, 2011)). This contributes to varying estimates of productivity loss (Zhang, 2011) and hinders analysis (Uegaki, 2011). Many recent studies that make bold claims about productivity and performance improvements rely on self-reported and subjective measures of productivity (Bevan, 2010).
Duration of follow-up – many studies test the outcomes of interventions over too short a time frame (Bevan, 2010). Avoid assuming that health promotion interventions lead to long-term behaviour change as employees may only temporarily adopt them (Cancelliere, 2011; PwC, 2008). For some interventions, costs are incurred immediately, but the cost benefit may arrive much later (Burton, 2010; Targoutzidis, 2014).
Generalisability – economic analyses performed in one country may not be generalisable to others. Much of the economic evidence that supports workplace wellbeing programmes comes from the US, where employers are responsible for the healthcare costs of employees and retirees. A recent systematic review of 11 European randomised-controlled trials identified that the economic impact of workplace health promotion programmes was mostly negative (Martinez-Lemos, 2014). This conflicts with prior meta-analyses of mostly US studies; lack of universal health insurance is thought to be the key reason for the discordance.
Attribution of effect – it is impossible to attribute cost savings to specific measures within a broad programme, or when there are confounding factors – for example, a change of managers, organisational change, or changes in the state of the global economy that can impact on the health of the workplace, regardless of the intervention. These confounding factors make it difficult to draw any kind of reliable conclusion about the outcome, especially when there is no control group (Burton, 2010). Additionally, it might be that employers who are already profitable are more likely to afford such programmes.
Publication bias – studies demonstrating an economic loss as a result of an intervention may be less likely to be reported and/or published (Martínez-Lemos, 2014; Targoutzidis, 2014).
Taking note of the small, diverse body of evidence with many methodological limitations and risk of publication bias, the most that we may be able to say currently is that studies graded as having low strength of evidence support the effectiveness of wellbeing interventions for improving some health behaviours (reduced tobacco use, improved diet and reduced sedentary work behaviour); evidence is insufficient or lacking for other outcomes of interest (Feltner, 2016).
Conclusion: go back to basics
Economic evaluations help to determine how to allocate budgets, especially when resources are limited. This may be another factor that influences some departments to produce ROI data. More importantly, economic evaluations can help to improve efficiency by comparing alternative courses of action in terms of both costs and benefits.
Here it would be helpful to have national and/or international standards for employers and their OH services to follow, instead of everyone reinventing the wheel or wasting effort by generating dubious ROI data for the purpose of: proving one’s worth; justifying the department budget; getting a paper published; or self-aggrandisement.
That said, cost-effectiveness is not the only driving force for providing access to occupational health and wellbeing services. Other reasons to provide a healthy workplace include legal compliance and business ethics or moral reasons (Burton, 2010; Miller, 2009). Therefore, the business case for employee wellbeing should contain at least legal, financial, and moral justifications for taking action. Indeed, an indifferent employer is unlikely to be convinced by figures alone.
It is in the interests of employers to have healthy and productive employees; as such, staff wellbeing ought to follow from an organisation’s values. After all, many companies state that employees are their most important asset. We need to get back to basics and justify expenditure in terms of protecting and promoting the wellbeing of that asset. It would be misguided to allow ROI to become the driving force behind employee wellbeing, as this risks shifting a healthy workforce from being the desired goal to a being a means to an end (profit).
Furthermore, it would be misleading to report ROI data without emphasising the limitations and unreliability of the data. A good many reported ROIs are likely to show a false profit. The saying that “size isn’t everything” certainly holds true when your measures are unreliable. Competing over whose programme shows the biggest ROI is no better than boasting about who has the most expensive company car or the biggest annual bonus.
There is an area of principle where caution or a reminder is appropriate. The wellbeing of employees is influenced by their genes and lifestyle, but also by work and its organisation. Many wellbeing programmes focus on individual and lifestyle risk and appear to underplay the significance of work and the workplace. At worst, they may be doing no more that promoting resilience so that employees can withstand ongoing stress.
To be successful, workplace wellbeing activities must be comprehensive and address both individual and occupational risks, including physical and psychosocial environments at work (Goetzel, 2014; Kuoppala, 2008). Of course, it is the latter that are more amenable to workplace policy influence (Bryson, 2014).
Dr Paul J Nicholson is an occupational physician.
References
Aviva (2012). “The sixth health of the workplace report”. Norwich.
Baxter S, Sanderson K, Venn AJ et al (2014). “The relationship between return on investment and quality of study methodology in workplace health promotion programs”. American Journal of Health Promotion, vol.28, pp.347-363.
Bevan S (2010). “The business case for employees health and wellbeing”. The Work Foundation. London.
Bryson A, Forth J and Stokes L (2014). “Does worker wellbeing affect workplace performance?”. Department for Business Innovation and Skills. London.
Burton J (2010). “WHO healthy workplace framework and model: background document and supporting literature and practices”. WHO. Geneva.
Cancelliere C, Cassidy JD, Ammendolia C, et al (2011). “Are workplace health promotion programs effective at improving presenteeism in workers? A systematic review and best evidence synthesis of the literature”. BMC Public Health, vol.11, p.395.
Council for Work and Health (2016). “Planning the future: implications for occupational health; delivery and training”. London.
Feltner C, Peterson K, Palmieri, Weber R, et al (2016). “The effectiveness of total worker health interventions: A systematic review for a National Institutes of Health Pathways to Prevention workshop”. Annals of Internal Medicine, vol.165, pp.262-269.
Goetzel RZ, Henke RM, Tabrizi M et al (2014). “Do workplace health promotion (wellness) programs work?”. Journal of Occupational and Environmental Medicine, vol.56, pp.927-934.
Juniper B (2016). “Employee health and wellbeing programmes: time to look beyond cash returns?”. Occupational Health & Wellbeing. October 2017, vol.48 (10).
Juniper B. (2017). “Wellbeing programme providers – how to pick a winner”. Occupational Health & Wellbeing, vol.49(1), pp.14-15.
Kuoppala J, Lamminpaa A and Husman P (2008). “Work health promotion, job wellbeing, and sickness absences – a systematic review and meta-analysis”. Journal of Occupational and Environmental Medicine, vol.50, pp.1216-1227.
Martínez-Lemos RI (2015). “Economic impact of corporate wellness programs in Europe: A literature review”. Journal of Occupational Health, vol.57, pp.201-211.
Mattke S, Balakrishnan A, Bergamo G et al (2007). “A review of methods to measure health-related productivity loss”. American Journal of Managed Care, vol.13, p.211-217.
Merkur S, Sassi F, McDaid D (2013). “Promoting health, preventing disease: is there an economic case?”. Policy Summary 6. WHO. Copenhagen.
Miller P and Haslam C (2009). “Why employers spend money on employee health: Interviews with occupational health and safety professionals from British industry”. Safety Science, vol.47, pp.163-169.
Rongen A, Robroek SJ, van Lenthe FJ et al (2013). “Workplace health promotion: a meta-analysis of effectiveness”. American Journal of Preventive Medicine, vol.44, pp.406-415.
Parks KM and Steelman LA (2008). “Organizational wellness programs: a meta-analysis”. Journal of Occupational Health Psychology, vol.13, pp.58-68.
PricewaterhouseCoopers (2008). “Building the case for wellness”. London.
Sockoll I, Kramer I and Bödeker W (2009). “Effectiveness and economic benefits of workplace health promotion and prevention: summary of the scientific evidence 2000 to 2006”. Iga report 13e. Initiative Gesundheit & Arbeit: Essen.
Targoutzidis A, Koukoulaki T, Schmitz-Felten E et al (2014). “The business case for safety and health at work: Cost-benefit analyses of interventions in small and medium-sized enterprises”. Luxembourg: Publications Office of the European Union.
Tompa E, Verbeek J, van Tulder M et al (2010). “Developing guidelines for good practice in the economic evaluation of occupational safety and health interventions”. Scandinavian Journal of Work, Environment & Health, vol.36, pp.313-318.
Sign up to our weekly round-up of HR news and guidance
Receive the Personnel Today Direct e-newsletter every Wednesday
Uegaki K, de Bruijne MC, van der Beek AJ et al (2011). “Economic evaluations of occupational health interventions from a company’s perspective: a systematic review of methods to estimate the cost of health-related productivity loss”. Journal of Occupational Rehabilitation, vol.21, pp.90-99.
Zhang W, Bansback N and Anis AH (2011). “Measuring and valuing productivity loss due to poor health: A critical review”. Social Science & Medicine, vol.72, pp.185-192.
