Guidance for Medical Directors, Chief Executives, and local NCAP audit leads on how to respond to a negative NCAP individual operator ‘alert’ or ‘alarm’
The recommendations in this document are to offer guidance to a hospital following notification from NICOR of a negative individual operator alert or alarm from the NCAP.
Introduction: delivering the highest quality of care possible and ensuring patients are protected from avoidable harm is the highest priority. However, it is also fundamental that before reacting to potentially negative outcome notifications, that the quality of data is confirmed as being correct. Individuals with a suspected or confirmed alert need appropriate support and also to be managed in a constructive and sensitive way. Medical directors need to ensure that a transparent and objective action plan is developed that should focus on addressing concerns so that an individual operator can be expected to return to independent practice, accepting that on rare occasions this may not always be possible.
Identification of an individual consultant as a potential outlier is an extremely stressful period for the consultant involved. This event can have a major impact on the individual’s wellbeing as well as the direct impact professionally. Therefore it is imperative that individuals are appropriately supported and that a clear set of actions are followed.
If analysis of NCAP data suggests that an individual operator is a potential outlier that satisfies the definition of ‘alert’ or ‘alarm’ status, then NICOR will contact the Medical Director/ Chief Executive of the relevant trust.
Suspected outlier– initial analysis of NCAP audit data has potentially identified an individual consultant as a negative outlier but the quality of the audit data has not been reviewed and confirmed as correct.
Confirmed outlier– following review and revision of submitted audit data (if indicated), re-analysis demonstrates that the individual is a negative outlier
Handling of suspected Outliers at ‘alert’ or ‘alarm’ Level
Any identification of ‘outlier’ status indicates a statistically significant value and does not necessarily mean outlying performance by a consultant. Outlier status is a statistical warning and may be due to chance alone.
Risk of False Positive Outlier status and False Discovery rate: It is important that Medical Directors and Chief Executives appreciate that due to the large number of analysis that underpin the individual consultant NCAP analyses, false positive alerts will be expected to occur, even when an individual may have acceptable performance. This is especially relevant for ‘alert’ status and a statistical outlier may not equate with poor performance.
Judgements on performance can only be made after a full examination of all the issues involved in the delivery of care, and this may be multi-factorial and complex. Medical directors are advised not to solely react to the statistical data without appreciating the limitations of the statistical methodology and having considered all potential explanations for the outlier status.
A key priority for a hospital/ department with a clinician suspected of being an outlier is to undertaken a detailed and comprehensive review of the clinical audit data.
The medical director must ensure that adequate resources are allocated to the investigation and it is undertaken rapidly in order for the hospital to work towards the timelines laid out in the NCAP Outlier SOP.
The Medical Director/ Chief Executive is responsible for acknowledging receipt of the letter from NICOR.
Local National Clinical Audit Lead for the domain within the hospital is responsible for:
- Providing written confirmation of the accuracy of data underpinning the analysis following initial notification
- Amending the live data if the data underpinning the outlier analysis are inaccurate
- Discussing with the NCAP Domain Clinical Lead, NCAP Chief Operating Officer if a potential outlier is detected and there is a ‘case to answer’
- Providing NCAP with the reasons why the original data were inaccurate and outlining changes that have been made to prevent a recurrence
- Discussing the findings with the local clinical and governance leads
If the individual consultant with a confirmed alert or alarm has a position of responsibility relating to the management of the service (e.g. audit lead, clinical director, medical director) then their responsibilities with regard to the management of the notification, should be delegated to another appropriate individual. The Chief Executive should be responsible for this action.
Handling of Confirmed Outliers at ‘Alert’ Level
A confirmed alert from NICOR (after an initial review of data quality) should trigger a review of the whole of a consultant’s practice including relationships with colleagues and other aspects of team functioning.
A review should cover all sites at which the doctor practises, including any private practice.
The hospital/ trust must undertake a review and further analysis of local data, which may include a review of data prior to the time period included in the alert and also a review of more contemporary data that may not have been included in the analysis time period.
Hospitals must provide the support and facility for further analysis of locally held data, requesting support from NICOR and the specialist societies where appropriate.
Engagement and support of individual consultant
Medical directors (and the relevant Clinical Director):
- must support colleagues who are identified as a potential outlier
- should ensure good communication and engagement with the individual consultant
- should ensure that individual consultants are informed of any investigations and kept updated about progress of any investigations/ review of practice
- should allow individual clinicians to comment on any findings
- must support an individual colleague who requests assistance
Responsibility of the individual consultant with potential outlier status
- Individual doctors must engage with outlier status management and investigation
- Non-engagement should result in prompt referral to the Medical Director
Suggested (but not mandatory) areas for review include but are not limited to:
- Detailed review of audit data quality/ accuracy and completion
- Review of individual case notes of deceased patients
- Cross-checking of catheter laboratory or theatre records
- Review of MDT records
- Review of case load and case mix
- Consider subspecialist practice not adjusted for by the risk prediction model
- Consideration of institutional factors (e.g. referral practices, provision of intensive care, post-operative services)
- Undertake a time trend analysis of mortality to ascertain at what stage survival rates started to decline
- Team dynamics: investigation may include the assessment of an individual’s role within the wider clinical team
- Consider external review from relevant Royal College if appropriate
Actions to be considered with regard to a clinician’s current practice:
- mentoring of individual consultant (internal or external colleague or group of colleagues)
- temporary modification in practice
- restriction of cases to elective case for defined period
- review of waiting list and removal/ redistribution of higher risk cases to colleagues
- partial or complete supervision of practice
- having new high risk case referrals reviewed by colleague/s for second opinion
- joint consultant operating for all or high risk cases
- operative planning of individual cases (or cases or just high risk cases) with nominated colleague/ mentor
- case-mix adjustment
- further training
If further training felt to be helpful:
Consider a broad range of additional training opportunities
- sabbatical/ visiting other centres
- peer review and support
- recognised training courses
- utilising simulator training
Involvement of external bodies:
If internal investigation of outcomes raises concerns that cannot be dealt with internally then advice should be sought from the relevant professional society or GMC at an early stage.
For cardiac surgery this is the Society of Cardiothoracic Surgeons in collaboration with the Royal College of Surgeons:
For cardiology the British Cardiovascular Society in Collaboration with the British Cardiovascular Intervention Society and the Royal College of Physicians: https://www.rcplondon.ac.uk/invited-reviews
GMC’s Employer Liaison Service: https://www.gmc-uk.org/about/how-we-work/liaison-and-outreach/employer-liaison-service
Outcome of review:
The outcome of the review process and any actions should be agreed and signed off by the clinical director and documented in the individual consultant’s appraisal folder.
This documentation should also be presented as part of their evidence for revalidation.
The Responsible Officer must be fully informed of any concerns, reviews and remedial action.
Handling of Confirmed Outliers at ‘Alarm’ Level
NICOR will inform the Chief Executive and the Medical Director of the individual consultant outlier alarm status. The individual clinician and clinical director will also be contacted by NICOR.
Chief Executive will be required to:
- Acknowledging receipt of the letter in all cases
- Confirm that a local investigation will be undertaken and that the GMC Employment Liaison Advisor has been informed
- inform relevant bodies about the NCAP concerns
- Clinical Commissioning Groups
- NHS Improvement
- relevant professional society and Royal College
- the GMC Employment Liaison Advisor (ELA)
- Care Quality Commission.
- Consider initiating an external review of the consultant’s practice through the relevant Royal College and professional society (For cardiac surgery contact the Society for Cardiothoracic Surgery in collaboration with the Royal College of Surgeons: https://www.rcseng.ac.uk/standards-and-research/support-for-surgeons-and-services/irm/ For cardiology contact the British Cardiovascular Society in Collaboration with the Royal College of Physicians: https://www.rcplondon.ac.uk/invited-reviews)
- Needs to be responsible for the quality of audit data if alarm status is confirmed after initial review and investigation of local data
- Ensure individual consultant is adequately supported
- Undertake a risk assessment of individual clinicians continued practice
- Make a decision around continued practice, supervised practice or temporary or permanent exclusion
- Ensure current practice status of individual consultant is communicated to the clinical team and referring colleagues
- Ensure any problems and issues to be addressed are clearly defined and documented
- Document the remedial action plan (in collaboration with external review)
- Develop and document the mechanism of how patients or relatives can discuss concerns they may raise
- Develop and document the mechanism of how the organisation will respond to media enquiries
- Consider seeking advice from HQIP and appropriate professional organisations
- By the time outlier alarm results are published the organisation should have a good understanding of why the survival rates are not as expected
- Set an appropriate specified time period for when a full investigation and documented action plan will have been completed
- Consider all of the options for investigation and remedial actions described above under Handling of Confirmed Outliers at ‘alert’ Level
Where a problem has been identified and appropriately addressed
- Robust action plan of remedial actions must be documented and made available (including to patients and other external stakeholders if required)
- Ensure that both external review and validation of findings and any decisions related to practice has been undertaken
- Ensure that there is a comprehensive communication plan to stakeholders when an individual outlier status has been addressed/ resolved/ explained