Gathering data is only part of the audit process; analysis can provide continued clinical improvements. Dr Siân Williams, a clinical director at the Royal College of Physicians, explains.
When clinical audit is done properly, it can be exciting, revealing and rewarding for the clinicians involved and the quality of the service they are providing. Clinical audit was defined in the publication Principles for best practice in clinical audit in 2002, endorsed by the National Institute for Health and Clinical Excellence (NICE), and this definition is still widely used today (see box).
History of clinical audit
Clinical audit is about an integrated programme of activities with cycles of measuring, reviewing and improving practice – it is not just a data-collecting exercise. Audit was formally introduced 20 years ago in a government White Paper, “Working for patients”. Initially it was called medical audit, was quite narrow in its focus and was for doctors to measure their practice.
Definition of Clinical Audit | |
“Clinical audit is a quality-improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria. Where indicated, changes are implemented at an individual, team, or service level and further monitoring is used to confirm improvement in healthcare delivery.” | |
In the early 1990s the emphasis on audit as a quality-improvement tool increased. Central Government provided ring-fenced money to support local audit committees and involvement of all clinicians, hence the name change from medical to clinical audit. The ring-fenced money ceased in 1995. Over the past 15 years it has been expected that audit will be carried out by all clinicians as part of clinical governance.
Does audit work?
Quality improvement, including audit, is becoming established as a science. Empirical research and theoretical modelling are informing how best to use audit data to effect change in clinical practice.
A systematic review of more than 100 studies has assessed the effect of audit data collection, with feedback to the participants, on the practice of healthcare professionals (Jamtvedt G et al, 2006). The results showed that there is great variation in the effectiveness of this process in improving clinical practice – overall the effects are generally small to moderate. The absolute effects of audit and feedback are likely to be larger when baseline adherence to recommended practice is low and intensity of audit and feedback is high.
There is a suggestion from qualitative studies – and theory of behaviour change – that audit feedback should be aimed at improving practice rather than aimed at the individual participant. Also the feedback should include action plans with specific targets. This emerging evidence may provide useful ways of optimising data obtained through audit.
The Healthcare Quality Improvement Partnership (HQIP) has produced useful guidance on factors to consider when developing an audit project. The guidance identifies markers of good-quality clinical audit, at both national and local level, conducted by both individuals and, more commonly, by teams.
The markers are described in detail within the document and are listed under the following headings:
1) Preparation and planning
- The topic for a clinical audit is a top priority.
- The clinical audit measures against standards.
- The organisation enables the conduct of the clinical audit.
- The clinical audit engages with clinical and non-clinical stakeholders.
- Patients or their representatives are involved in the clinical audit if appropriate.
2) Measuring performance
- The clinical audit method is described in a written protocol.
- The target sample should be appropriate to generate meaningful results.
- The data-collection process is robust.
- The data are analysed and the results reported in a way that maximises the impact of the clinical audit.
3) Implementing change
- An action plan is developed and then implemented to take forward any recommendations that are made.
4) Sustaining improvement
- The clinical audit is a cyclical process that demonstrates improvement has been achieved and sustained.
There is concern that clinical audit is sometimes performed without enough planning, resource or follow-through. Certainly audit is perceived by some as a rather dry data-collection and number-crunching exercise in which they have to participate. In these situations it is likely that stages one and two described above will not have been performed optimally, and stages three and four may have been neglected altogether.
A strength of clinical audit is that it gives a detailed insight into areas of practice that routinely collected data does not. A bespoke, focused data set can provide a rich picture of practice. This then directs improvement activities appropriately.
It is vital that those people involved in clinical audit understand the process and are enthused either to lead the process or be an active participant or supporter. Data collection and analysis provide the baseline, identify areas for improvement and measure change. Equally important are the improvement activities that take place between the rounds of data collection – these sometimes get overlooked in the excitement of reviewing the data.
The process of clinical audit has developed from the traditional two-dimensional audit cycle, to a three-dimensional structure that continues to spiral upwards. The two-dimensional circle represented repeated cycles of the same process; the upward spiral represents a more dynamic process, allowing developments to the data-collection tools in response to new results and priorities, and a higher level of quality on each turn of the spiral (see box 2). The quality-improvement spiral may include other activities not resulting directly from the audit, but contributing to the same area of clinical improvement.
Regional and national audits
At a national level, many specialties participate in clinical audits that are mainly run by medical royal colleges and specialist societies. Such audits have been very influential in improving the quality of care for several conditions, such as stroke, lung cancer and hip replacements. The national sentinel stroke audit, which has been running for more than a decade, now has 100% participation from acute trusts in England.
Clinical audit has been embraced by OH clinicians since its inception. Within the NHS, the Association of National Health Occupational Physicians (ANHOPS) set up regional audit groups 20 years ago. Some of these groups accessed the Government’s ring-fenced funding in the 1990s and have continued to flourish. Outside the NHS, OH clinicians have undertaken clinical audit within their employing organisations or as part of ad-hoc geographically based groups. Also, the Society of Occupational Medicine has recently renewed its advice to members on clinical audit in the context of appraisal and revalidation.
Optimising activity
In 2008 the Health and Work Development Unit (HWDU, previously known as the Occupational Health Clinical Effectiveness Unit) undertook the first national clinical audits in OH. The audits recruited more than 70% of NHS trusts in England and more than 80% of their OH providers. A national conference and regional implementation workshops were held in 2009 and data collection for one of these audits (depression detection and long-term sickness absence) was repeated in 2010, completing a full audit cycle.
The results of this audit show improvement from baseline in nearly all the criteria measured. In particular, participants that took part in the conference and workshops made more progress in the areas that these events concentrated on compared with those that did not attend the events.
Participants who were involved in the data extraction from clinical notes were more likely to report changing their practice prior to their audit result being known compared with those who were not directly involved in data collection. This suggests that using the audit tool can be educational. This is not surprising since many of the questions in audit tools are derived from recommendations in national evidence-based guidance.
All audits, regardless of their size, locality or topic, should be carefully designed and executed using accepted methodology. As data emerges from research into the most effective audit and feedback methodologies, this should be incorporated into practice.
National clinical audit is one way of optimising audit activity; benefits include an audit design that is robust and widely consulted on, centrally coordinated data collection and analysis, a rise in standards both locally and nationally, a reduction in variability in OH care and the opportunity to benchmark practice.
We should enjoy the audit process, celebrate successes and broadcast our activities. National clinical audit is a powerful demonstration of a specialty’s commitment to quality improvement.
Dr Siân Williams is clinical director of the Health and Work Development Unit of the Royal College of Physicians.
Reference
Sign up to our weekly round-up of HR news and guidance
Receive the Personnel Today Direct e-newsletter every Wednesday
Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. “Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback”. Qual Saf Health Care 2006;15:433-436.
