What is audit? Peter West Honorary Consultant Biochemist North Middlesex University Hospital NHS Trust Formerly Consultant Biochemist and Clinical Governance and Audit Lead for Pathology North Middlesex University Hospital NHS Trust Presentation for the ACB National Audit Meeting, Birmingham 22nd June 2016 What will be covered during my talk What clinical audit is
The history of clinical audit Why clinical audit is important Requirements for audit by UK doctors Types of audit which may be carried out The clinical audit cycle and the various stages of the cycle Who should be involved in the clinical audit project and general considerations Is service evaluation the same as clinical audit? The similarities and differences between clinical audit and research Does clinical audit need ethical approval? The role of the clinical audit lead The challenges in performing clinical audit
Clinical audit and clinical governance What is clinical audit? The current accepted definition of clinical audit endorsed by the National Institute of Clinical Excellence(NICE) in 2002 is a quality improvement process that seeks to improve patient care and outcomes through a systematic review of care against explicit criteria and the implementation of change The criteria are usually from evidence based documents such as NICE clinical guidelines, National Guidelines and National Service Frameworks. Clinical audit is now an established part of the NHS landscape and a key component of the clinical governance framework, forming part of the system for improving the standard of clinical
practice. Clinical audit is not a new concept. FLORENCE NIGHTINGALE ERNEST CODMAN The history of clinical audit-1 It is thought that clinical audit was first undertaken by Florence Nightingale during the Crimean war in the mid 19th century. The British Secretary for War at the time asked Florence Nightingale who was a mathematician at the
time to become a nursing administrator who was responsible for supervising the introduction of nurses into military hospitals. Florence accepted the position at the medical barracks in Scutari in 1854 but was appalled by the poor levels of hygiene and conditions which were partly responsible for the high mortality rates amongst the ill and wounded soldiers whom she cared for. She and her team of 38 nurses applied strict sanitary routines and standards of hygiene to the hospital and equipment. With her mathematician background she started to collect data which she analysed using statistics to calculate the mortality rates of wounded soldiers. It was estimated that soldiers were seven times more likely to die due to disease caught in the hospital such as cholera and typhus than if they were on the battlefield.
The history of clinical audit-2 Florence implemented changes following her discovery to improve hygiene on the wards and within one year the mortality rates had reduced from 60% to 42.7%. With continued work, the rates fell even lower to 2.2% by ensuring that only fresh clean water was used and the introduction of fresh fruit and vegetables into the patients diets and the results were instrumental in overcoming the resistance of the British doctors and officers to her procedures. Her methodological approach as well as the emphasis on uniformity and comparability of the results of health care is recognised as one of the earliest programmes of outcomes management. Another notable figure who advocated clinical audit was Ernest Codman, a distinguished Boston
surgeon who lived from 1869 to 1940. He became known as the first true medical auditor following his work in 1912 in monitoring surgical outcomes. The history of clinical audit-3 His end result idea was to follow every patients case history after surgery to identify errors made by individual surgeons on specific patients. His work anticipated contemporary approaches to quality monitoring and assurance, establishing accountability and allocating and managing resources efficiently. Whilst Codmans clinical approach was in contrast with Florence Nightingales more epidemiological audit, these two methods serve to highlight the different methodologies that
can be used in the process of improvement to patient outcome. Despite both their successes, clinical audit was slow to catch on and the situation was to remain so for the next 130 or so years with only a minority of healthcare staff embracing the process as a means of evaluating the quality of care delivered to patients. In 1989,the White Paper "Working for Patients saw the first move in the UK to standardise audit as part of professional health care and in that year, medical audit undertaken by doctors was first formalised. The history of clinical audit-4 A revised definition from the 1989 White Paper Working for Patients was as follows: Clinical audit is the systematic analysis of the quality of healthcare, including the procedures
used for diagnosis, treatment and care, the use of resources and the resulting outcome and quality of life for the patient. Prior to this, audit was isolated and infrequently undertaken. Four years later in 1993,Medical,Nursing and Therapy Audit were brought together to form the multi-disciplinary activity that we now recognise as clinical audit, medical audit having evolved into clinical audit. Therefore clinical audit now tends to be used as an umbrella term for any audit conducted by professionals in health care. The National Institute for Health and Clinical Excellence(NICE)endorsed the paper entitled: Principles for Best Practice in Clinical Audit in 2002 which defined clinical audit as:
The history of clinical audit-5 a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit standards and the implementation of change A 2007 Department of Health document concluded that clinical audit was falling short of its potential and since 2008 has made considerable amounts of funding available for the development of clinical audit. There are now many established national clinical audits that Trusts are expected to participate in and the National Advisory Group for Clinical Audit and Enquiries provides guidance to NHS England. Recent years have seen the drivers for clinical audit grow at an exponential rate and now clinical audit activity must be published via quality accounts and clinical audit reports made available to
the Care Quality Commission with clinical audit now an integrated part of NHLSA arrangements and NICE quality standards expected to be audited. The history of clinical audit-6 New commissioner/provider relationships have extended the remit of clinical audit and audit work is being linked to relatively new initiatives such as QIPP,CQUINs,PROMS etc QIPP-Quality,Innovation,Productivity,Prevention.There are a number of national work streams within QIPP designed to support the NHS to improve care and lower costs. CQUINs-Commission for Quality and Innovation payment framework which enables commissioners to reward excellence by linking a proportion of English healthcare providers income to the achievement of local quality improvement goals.
PROMs-Patient Reported Outcome Measures assesses the quality of care delivered to NHS patients from the patient perspective by calculating the health gains after surgical treatment using pre-and post-operative surveys. At a local level, clinical audit links into both clinical effectiveness and clinical governance. Why is clinical audit important?-1 There are a number of reasons why clinical audit is an important activity. The main reason is that it helps to improve the quality of the service being offered to users and to ensure that what should be done is being done. Without some form of clinical audit, it is very difficult to know whether one is practicing effectively and even more difficult to demonstrate this to others.
The benefits of clinical audit are that it: Identifies and promotes good practice and can lead to improvements in service delivery and outcomes for users Can provide the information needed to show others that your service is effective(and cost effective)and thus ensure its development Minimises error or harm to patients Provides opportunities for training and education Can lead to the development of local guidelines and protocols Why is clinical audit important?-2 Reduces the number of incidents and complaints
Helps to ensure better use of resources and therefore increased efficiency Can improve working relationships, communication and liaison between staff, staff service users and between agencies Requirements for audit by UK doctors In 2013 the General Medical Council stated that as part of good medical practice, maintaining and improving performance, doctors are required to: Take part in regular and systematic audit Take part in systems of quality assurance and quality improvement Respond constructively to the outcome of audit, appraisals and performance reviews, undertaking further training where necessary
Doctors in the UK are asked to perform an audit during their first two postgraduate years. Audit is an essential part of the revalidation process for GPs. It is part of the core evidence that a GP is reviewing their practice and undertaking quality improvement activity. If it is not possible for GPs to undertake an audit, case reviews and other projects which demonstrate evidence of quality improvement activity may be submitted. Types of clinical audit which may be carried out Standards based audit A cycle which involves defining standards, collecting data to measure current practice against those standards and implementing any changes deemed necessary.
Adverse occurrence screening and critical incident monitoring This is often used to peer review cases which have caused concern or from which there was an unexpected outcome. The multi-disciplinary team discusses individual anonymous cases to reflect upon the way the team functioned and to learn for the future. In the primary care setting this is described as a significant event audit. Peer review An assessment of the quality of care provided by a clinical team with a view to improving clinical care. Individual cases are discussed by peers to determine with the benefit of hindsight whether the best care was given. This is similar to the method above but might include interesting or unusual cases rather than problematic ones. Mortality and morbidity reporting is a specific peer review process that looks at specific, non-random cases with adverse outcomes such as death or injury to see what lessons can be drawn. Patient and service user surveys In terms of clinical audit,surveys can be a useful tool where measuring
compliance against your audit criteria requires information that can only be obtained from the patient or service user. Types of non-clinical audit which may be carried out Financial audit This looks at accounts to establish whether they provide a true and fair view of the organisations financial position at a given time. Internal audit This is an internal mechanism that traces non-clinical activities and systems along audit paths to see if things happened the way they should have. For example, tracing a patient complaint from the initial letter of complaint through to resolution in order to establish whether Trust guidelines were followed appropriately.
Organisational audit This is an external, independent and voluntary audit of the whole organisation based on a framework of explicit standards. Organisational audit looks at how well the organisation is set up and run on a daily basis. The Kings Fund is an example of an independent service that undertakes organisational audits. The clinical audit cycle Clinical audit can be described as a cycle. Within the cycle, there are stages that follow the systematic process of establishing best practice. The main components of the audit cycle are as follows: Choosing a topic to audit. Reviewing current standards or agreeing standards of best practice(audit criteria)
Collecting data on current practice Analysing the data against the standards Discussing possible changes Implementing agreed changes to make improvements-ie:an action plan needs to address what needs to be done to improve current practice and also to identify who needs to do what to create such an improvement. Allowing time for the changes to embed Re-auditing to make sure that the practice has improved Stages of the clinical audit cycle-1 Stage 1:Choosing a topic for audit
A topic selected for audit must have clear objectives and is likely to involve measuring adherence to healthcare processes that have been shown to produce best outcomes for patients. Selection of a topic is influence by factors such as the following: Where national standards and guidelines exist-where there is conclusive evidence about effective clinical practice ie:evidence based medicine Areas where problems have been encountered in practice Where there has been a patient complaint or an adverse incident Where there is a clear potential for improving service delivery Areas of high volume, high risk or high cost, in which improvements can be made Additionally, an audit topic may be recommended by national bodies such as NICE or the Healthcare Commission in which NHS Trusts may agree to participate.
It is important to design a clinical audit project which will produce meaningful data and which can be finished in the time available. Stages of the clinical audit cycle-2 Stage 2:Developing audit standards Clinical audit is by definition standards-based(sometimes referred to as criterion-based). Standards are more specific than objectives. They are quantifiable statements detailing what one intends to measure current practice against. If standards are available in the form of guidelines or protocols then one should base the audit on the most widely applicable guidelines available eg:national guidelines rather than regional or local guidelines. However, it is important to bear in mind that guidelines are only as good as the evidence they are based on.
If guidelines/protocols do not exist or if existing ones are out of date then one will need to undertake a literature search in order to identify best practice. It is important that there is agreement with the standards locally before one starts to conduct the audit. Standards should always be based on the strongest and most up to date evidence of what constitutes best practice. Stages of the clinical audit cycle-3 Finding the evidence Evidence can come from a range of sources: National guidelines such as NICE,NSFs,Royal Colleges Research findings, particularly systematic reviews
Local guidelines, protocols and procedures Local consensus-be careful about using this as it is not proven best practice Books Journal articles,reviews,letters,comments and editorials Reports from the Department of Health,Royal Colleges Databases that guide one to evidence include : the Cochrane Library of Systematic Reviews,MEDLINE,EMBASE,CANCERLIT Stages of the clinical audit cycle-4 Writing audit standards. Audit standards should be SMART.
Specific- clear,unambiguous,jargon free and covering one topic only Measurable- can be measured in a practical way Achievable-is something that is reasonable for staff to achieve Relevant-is an issue that is important to patients and staff Timescale- can be measured within a reasonable period of time Stages of the clinical audit cycle-5 Stage 3:Data collection Some Trusts have audit teams that can help with data collection. To ensure that the data collected are precise, and that only essential information is collected, certain details must be established from the outset of the audit and include:
The user(patient) group to be included with any exceptions noted The healthcare professionals who will be involved The period over which the criteria apply Sample size for data collection are often a compromise between the statistical validity of the results and pragmatic issues around data collection. Data to be collected may be available in a computerised information system or in other cases it may be appropriate to collect data manually or electronically using data capture systems depending on the outcome being measured. Stages of the clinical audit cycle-6 Consideration needs to be given to what data will be collected, where the data will be found and
who will do the data collection. Although clinical records are frequently used as the source of data, they are often incomplete. Collecting data from several sources such as clinical records, blood results from patient administration systems and imaging from picture archiving can help to overcome the problem. Retrospective data assesses data during a time period in the past with this type of data collection being usually faster but one should be aware of missing documentation. Concurrent data gives a more immediate feedback on its current performance. The data collected must only relate to the objectives of the audit. However one conducts a clinical audit, one must always be aware of maintaining confidentiality. Stages of the clinical audit cycle-7
The Data Protection Act of 1998 requires that all personal data held on computers should be secure from loss or unauthorised disclosure. One must also ensure that all print outs and results are stored in a secure place. Any potential sensitive topics should be discussed with the local Research Ethics Committee. Stages of the clinical audit cycle-8 Stage 4: Comparing performance with the standards This is the analysis stage, where the data collected is compared to the standards, concluding how well the standards were met and if they were not met, identifying the reasons for this. In theory, if the standard was not met 100% of the standard that was set, then there is a potential for improvement.
In practice,if the results are close to 100% of the standard one may decide that any further improvement may be difficult to achieve and that other standards with results further away from 100% are the priority targets for action. However,this decision also depends on the topic chosen for the audit. Stages of the clinical audit cycle-9 Stage 5:Implementing change Once the results of the audit have been published and discussed, an agreement must be reached about the recommendations for change. Using an action plan to record these recommendations is good practice and should include who has agreed to do what and by when.
Each point needs to be well defined with an individual named as responsible for it and an agreed timescale for its completion. Action plan development may involve refinement of the audit tool, particularly if measures used are found to be inappropriate or incorrectly assessed. In other instances, new processes or outcome measures may be needed or involve links to other departments or individuals. Too often, audit results in criticism from other organisations, departments or individuals without their knowledge or involvement. Joint audit is far more profitable in this situation. Stages of the clinical audit cycle-10
Compiling the audit report An audit report outlines the steps that have been taken to complete the audit. The report does not have to be lengthy or complex. One should always try to include information on the following areas: Background to the topic, research undertaken Aim,objectives,criteria and standards Target group and sample size Methodology-how the target group was identified and results collected Results Action plan Appendices-copy of audit proforma
References Stages of the clinical audit cycle-11 Stage 6: Re-audit: sustaining improvements Audit must be seen as a continuous cycle of activity. After an agreed period, the audit should be repeated. The same strategies for identifying the sample, methods and data analysis should be used in order to ensure comparability with the original audit. The re-audit should demonstrate that the changes have been implemented and that improvements have been made. Further changes may then be required leading to additional re-audits.
This stage is critical to the successful outcome of an audit process as it verifies whether the changes implemented have had an effect and to see if further improvements are required to achieve the standards of healthcare delivery identified in stage 2. Who should be involved in the clinical audit project and general considerations-1 Clinical and non-clinical staff providing the service should be involved. It is important to stress that very few health care procedures involve just one professional discipline and non clinical staff such as receptionists, porters etc play a vital role in the quality of the service provided. Service users and those whose support may be required to implement resultant changes in
practice eg: managers and Trust Board members should be involved. The roles and responsibilities within the team will need to be identified eg:the audit project lead, data collector. The key stakeholders should be involved in the clinical audit project from the start. The most time-consuming element of any clinical audit project is the implementation of required changes. It is suggested that projects be kept simple and cover areas in which changes can be achieved. Results of good audit should be disseminated locally and nationally where possible . Who should be involved in the clinical audit project and general considerations-2
Professional journals publish the findings of good quality audits, especially if the methodology is generalisable. Clinical audit projects should be registered with the Clinical Audit team and therefore will have been approved by the relevant Clinical Audit convenor. Is service evaluation the same as clinical audit? Service evaluation may be defined as a set of procedures to judge a services effectiveness or efficiency by providing a systematic assessment of its aims,objectives,activities,outputs,outcomes and costs(NHS Executive definition,1997). It addresses specific questions about the service concerned and results are specific and are local to a
particular team or service and may lead to service redesign and reconfiguration in that particular area. There are many different approaches to service evaluation. Whichever method is used, the process should provide practical information which helps to inform the future development of a service. Clinical audit may be one activity which takes place during a service evaluation, alongside other activities such as routine data gathering, incident reporting and interviews with staff and service users. In order to conduct an evaluation, services need to consider their aims, objectives and then identify their key evaluation questions. Clinical audit and research-what are
the differences?-1 research is concerned with discovering the right thing to do; audit with ensuring that is done rightSmith R. Audit and Research BMJ 1992 305:905-6 Research addresses clearly defined questions and hypotheses using systematic processes to generate new evidence to refute, support or develop a hypothesis by asking the question "what is best practice? as a result of which a new service or new practice may be developed. The methodology is designed so that it can be replicated and that the results can be generalised to other similar groups. Research may involve a completely new treatment or practice, the use of control groups or placebo treatment for purposes of comparison, or allocating service users randomly to different treatment groups. Patients should be involved in the design, implementation and analysis of the work. Research must comply with Research Governance and be registered with the Research and
Development Committee and has also to be submitted to the Research Ethics Committee(REC) for approval. Clinical audit and research-what are the differences?-2 Alternatively, clinical audit aims to improve the quality of local patient care and clinical outcomes through the peer-led review of practice against evidence-based standards, implementing change where necessary. It asks the questions "are we following best practice? and what is happening to patients as a result? Clinical audit is initiated by national bodies, commissioners or service providers including local healthcare staff and managers.
The methodology is designed to address clearly defined audit questions that establish whether a specific clinical standard is being met. Results are specific and local to a particular team or service although the audit tool may be used by more than one team or service. A clinical audit project will never involve a completely new treatment or practice, never involve the use of control groups or placebo treatments nor does it involve allocating patients randomly to different treatment groups but it may involve input from patients, service users or carers at a number of levels such as participation in surveys which help to determine whether standards have been met and involvement in the design of individual clinical audit projects. Ethical approval-1 Does audit need ethical approval?
All clinical audit needs to be conducted within an ethical framework. Clinical audit by definition does not involve anything being done to patients beyond their normal clinical management and therefore in practice does not require ethical approval. It aims to improve patient care through a systematic review of care against explicit criteria and the implementation of change. However, there are a few instances where an audit project may require ethical approval. These are considered under four headings: Is it really audit? Might you be doing research? Healthcare professionals sometimes confuse clinical audit with research.This is easily done because the two disciplines have much in common. The similarities and differences between audit and research have been outlined.
Ethical approval-2 Are you planning a patient survey? Patient surveys could be construed as: "doing something to patients beyond routine clinical management Surveys should be designed in such a manner as to cause minimum possible disruption to patients and may require specific ethics approval. Are you planning to publish? If the results of the project are generalised beyond the local setting,then one is probably doing research. Clinical audits are sometimes published but this is usually because the topic and/or methodology
may be of interest to a wider audience and not because the results are generalisable. Ethical approval-3 Staff who are not bound by a duty of confidentiality It is possible that some of the work for the audit project may be done by someone who is not an employee of the Trust ie:may be a university student A potential problem exists when people working on audit projects are volunteers or are not paid by the Trust. One must consider an Honorary Contract for such people if they have access to patients notes and a member of the Trust staff must take responsibility for how and where clinical information is accessed.
For more information one should speak to the Caldicott Guardian for the Trust. The role of the clinical audit lead-1 The clinical audit lead has a clear role in creating the strategy for embedding clinical audit within the organisation and should have a high profile within the organisation and must champion clinical audit both to colleagues and management alike. The clinical audit lead should develop a forward plan or programme of clinical audits to be carried out in a clinical service which reflects the organisations and the clinical service priorities and ensure that people working in the service carry out the designated audits. The clinical audit lead should monitor and manage progress on carrying out clinical audits in the plan.
The clinical audit lead should support colleagues to carry out individual clinical audits including junior doctors. The clinical audit lead should ensure that work being carried out related to clinical audits is communicated within the clinical service, to clinical service management and to the Trust. The role of the clinical audit lead-2 The clinical audit lead should ensure that any problems associated with clinical audit are addressed responsibly. The clinical audit lead should be actively involved in linkages to other aspects of clinical governance to allow for the dissemination of clinical audit information and the setting of local clinical audit priorities.
What are the challenges in performing clinical audit? The main challenge is how to make your audit a success. First, you need support-from the clinical audit department of the Trust, the clinical audit lead, the Consultant and non-Consultant staff. Secondly ,you need to have enough time to conduct the audit. Thirdly, you must be realistic when setting standards for the audit. Do not choose a standard of 100% if you know that you cannot possibly meet this ideal standard. Aim for an optimum rather than an ideal standard.
Summary Clinical audit is a quality improvement process that measures current practice against agreed standards of best practice with the key points of clinical audit being that: It aims to evaluate how close practice is to best practice It identifies ways of improving the quality of health care provided It aims to improve services It is usually led by service providers It is an ongoing process The collection of data which is not related to explicit audit criteria(standards) is not considered to be clinical audit.
Clinical audit v research v service evaluation Clinical audit is audit against agreed standards of best practice. Research aims to create new knowledge. Service evaluation assesses the effectiveness of a service.
Other Properties Polar Equation r(θ)= a(1-cosθ) where a is the radius of one of the two circles in the previous slide Area (3/2)*pi*a Length 8*a Trace of an Epicycloid Trace of an Epicycloid Trace of an Epicycloid The Evolute Cardioid...
Food and color additives: Only use additives approved by your local regulatory authority. NEVERuse more additives than are allowed by law . NEVERuse additives to alter the appearance of food . Do NOTsell produce treated with sulfites before it was...
What you need to know about important changes to Insurance Requirements in Contracts. FOA/PSSOA Costa Mesa. March 12, 2014. Presented by: ... Surplus and Conditional Reserve Funds (in Millions) FSC VII $50 to $100 - Minimum rec ... How Insurance...
The gap between clearance of accounts by audit committee and board meeting should be as narrow as possible to avoid leakage of UPSI. RELEVANT TERMINOLOGIES (3/4) ... Disclosure to be made by: promoter, Member of promoter group, designated persons, Director....
Europe and Japan in ruins . Devastation in Europe . Close to 40 million Europeans had died . Many displaced people were left homeless. Concentration camps, POW's, refugees ... Hess was found guilty- sentenced to life in prison. Goring committed...
Define objectives. Identify and inventory variables. Obtain baseline data on variables. Make statistical analysis of . relationships among variables. Determine significant interactions. Objectives Construct mathematical model . describing interactions among . variables. Run the model on a computer, with values...
Ready to download the document? Go ahead and hit continue!