The introduction of mandatory publication of consultant outcomes for surgical specialties by NHS England has focused more attention on surgical results. SCTS has traditionally defined 3 levels of ‘negative’ outlier; yellow (95% CL), amber (99% CL) and red (95% CL with adjustment for multiple comparisons).
In keeping with the other specialties publishing consultant level outcome data, and on the advice of the external statistical review of the audit last year, for 2015, we are moving to 2 levels; alert which represents the old yellow and amber levels and alarm which represents the old red. In 2015 we have used two tailed exact binomial confidence intervals at 95%and 99.8%.
There is a reasonable possibility that alert level alarms may occur due to chance alone. It is unlikely that alarm level outliers will occur solely due to chance. The methods for analysis are complex and have been subjected to external statistical review and are given in detail on the NICOR website.
Alarm level outliers (99.8%) will have their survival results published as ‘?’ lower than rather than worse than expected on the NHS Choices website. We believe that with improved internal governance procedures, it should be possible to avoid any alarm outliers in future years.
SCTS has 2 responsibilities in this process:
- To provide advice on understanding and explaining any lower than expected survival
- To provide support for members and units
Any surgeon or unit that triggers an alert or alarm has a duty to explain the divergence of their results.
Divergence is a cause for looking at the data in more detail and is not a sufficient reason in itself for restricting a surgeon’s practice unless there are clear concerns about the safety of patients. It is important that all investigations are reasonable and proportionate.
By the time any data is published it should have gone through a robust analysis to ensure that it accurate (with respect to the activity, mortality and risk factor data). We would then recommend:
- Analysis of the caseload to ensure that the risk stratification mechanism accurately reflects expected outcomes (e.g. is there any subspecialist practice which is not adjusted for by the risk prediction model).
- Analysis of institutional factors that may contribute to the divergence in clinical outcomes such as referral practices, the provision of intensive care, or other post-operative services.
- More detailed analysis of the surgeon’s performance.
It is important to look for trends in mortality over time to ascertain at what stage survival rates started to decline, and whether it is possible to identify any precipitants.
The SCTS believes that all intra-operative or post-operative cardiac surgical mortalities should be reviewed in detail, and that both the hospital and the individual have a responsibility here.
The hospital should be reviewing cases of mortality as part of their routine clinical governance meetings, to learn and feed-back to improve practice.
The surgeon should be reviewing all mortality through the process of reflective practice, and documenting this for their appraisal portfolio.
In addition to reviewing overall mortality rates and each death in detail, we would recommend a wider benchmarking of additional process and outcomes data.
All benchmarking of outcomes should be conducted in the full knowledge of the case mix and risk profiles. It may in this context be appropriate to benchmark complete practice and/or outcomes for specific operative groups.
It important that there is organisational engagement with these investigations to support the process. SCTS would suggest that this is supported by clear action plans with defined timescales and personal responsibilities.
Support for members
Any member who is identified as either an alert or alarm will be contacted by the President of SCTS or a nominated senior officer deputy.
This contact will take the form of a preliminary phone call.
This will be followed up by written contact from NICOR/SCTS.
The initial contact will:
- Explain the nature of the process
- Offer a choice of senior officers of SCTS to act as a supporter through the process
The colleague will:
- Offer personal support throughout the process
- Provide advice about other sources of support
- If necessary provide advice on the gathering of other sources of evidence to support good practice, such as colleague and patient 360 appraisal data.
Advice confined to area of expertise
|Other sources of support||IRM
Occupational Health Department
|Other sources of evidence||Appraisal
Other evidence to show that standards of
Good Medical Practice (GMC)
Good Surgical Practice (RCSEng) are maintained