Evaluating evidence – dirty hands and clean minds

Co-editor of the BOHRF review of occupational contact dermatitis defends claims made against it in Occupational Health in October 2010.

In a recent article relating to the British Occupational Health Research Foundation (BOHRF) occupational contact dermatitis guidelines (Nicholson, 2010), Chris Packham says: “The BOHRF study used the Scottish Intercollegiate Guideline Network (SIGN) guidelines to evaluate studies… it runs a risk of excluding much data, both scientific and experiential, that can add valuable information to any recommendations,” (Occupational Health, October 2010).

The value of SIGN grading is that it excludes studies where there is a high likelihood of bias and where findings would therefore be unreliable. Similarly, interpretation bias must be avoided ruthlessly when assimilating the evidence and making recommendations, putting aside one’s own experiential data.

The late OH physician Geoffrey Rose spoke of the need for a clean mind and dirty hands – dirty hands from collecting all of the data and clean minds with which to judge the evidence.

Testing situation

The problem with grading systems such as SIGN in the occupational health setting is well known (Petticrew, 2003; Concato, 2004; Vandenbroucke, 2004; Ogilvie, 2005; Glasziou, 2005; Nicholson, 2007). The problem is that they regard randomised controlled trials (RCTs) as providing the highest level of evidence.

RCTs do not apply in many areas of occupational health and so there is scarce level-one evidence as defined by these systems. To overcome this problem, as explained on page 9 of the BOHRF review, evidence statements and recommendations used both the SIGN system and the modified Royal College of General Practitioners’ three-star system.

Packham says: “The BOHRF article appears to suggest that irritant and allergic dermatitis can be considered as separate, discrete problems,” stating (and reiterated in bold) that “it is not uncommon to find that a hand dermatitis has both irritant and allergic components”.

In fact, the BOHRF review states on page 11: “The proportion of cases of occupational contact dermatitis that may be attributed as being allergic, irritant or mixed allergic/irritant in nature depends on the type of industry, the jobs that people have, the hazards to which they are exposed, the centres that report cases and differences in defining the disease and confirming diagnoses.”

Packham also takes issue with the evidence statement on page 17 in the BOHRF guidelines: “A temporal relationship between symptoms and work indicates that a person’s job has either contributed to or caused their occupational contact dermatitis or urticaria”, adding “this is not necessarily the case, for two reasons” and providing a lengthy argument for his case.

The word to emphasise is “indicates”. The BOHRF guidelines elaborate on page 5 that “a temporal relationship with work… only raise(s) suspicion of an occupational cause, and do(es) not necessarily confirm an occupational causation”, and on page 17 that “symptoms improving away from work can produce false positive diagnoses, so further validation of occupational contact dermatitis and urticaria is needed”.

As to the value of systematic reviews, no one would claim that they are perfect (see Data revision feature, p23). The BOHRF guidelines make a special effort to state on page 3 that “clinicians, employers and workers need to exercise their judgment, knowledge and expertise when deciding whether it is appropriate to apply guidelines, taking into account individual circumstances and patients’ wishes”.

“Clinical judgment is necessary when using evidence statements to guide decision-making. Limited recommendations on a particular issue or effect do not necessarily mean that it is untrue or unimportant but may simply reflect insufficient evidence,” it says.

It is an unwise person who would practise solely on the basis of an evidence review or solely on the basis of their own experience. The skill comes in marrying the two.

Dr Paul Nicholson, co-editor of the 2010 BOHRF review of occupational contact dermatitis and urticaria.

References

Packham C. Evaluating dermatitis studies. Occupational Health. October 2010, p.14.

Nicholson PJ & Llewellyn D (Editors). Occupational contact dermatitis and urticaria. British Occupational Health Research Foundation. London. 2010.

Petticrew M & Roberts H. Evidence, hierarchies and typologies: horses for courses. J Epidemiol Community Health, 2003; 57: 527-9.

Concato J. Observational versus experimental studies: what’s the evidence for a hierarchy? Neurorx, 2004; 1: 341–347.

Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet, 2004; 363: 1728-31.

Ogilvie D, Egan M, Hamilton V, Petticrew M. Systematic reviews of health effects of social interventions: 2. Best available evidence: how low should you go? Journal of Epidemiology and Community Health 2005; 59: 886-892.

Glasziou P, Vandenbroucke JP, Chalmers I. Assessing the quality of research. BMJ, 2004; 328: 39-41.

Nicholson PJ. How to undertake a systematic review in an occupational setting. Occupational and Environmental Medicine. 2007; 64: 353-8.

Comments are closed.