Volume 7 Issue 5 December 2013 Welcome to Knowledge Matters It’s hard to believe how quickly this year has gone! Here at the Quality Observatory it’s certainly been a challenge with our move to new hosting arrangements and all the changes that have come to the wider NHS. But that doesn’t mean it hasn’t been rewarding, and we are all incredibly proud of what we have achieved in such a short space of time. 2013 has seen the demand for high quality information and intelligence put absolutely centre stage with the Francis Inquiry, the Keogh reviews and the Berwick report all highlighting how crucial it is that organisations analyse, review and understand their data in a truly intelligent way. Here at the QO we’ve met that challenge head on alongside our customers and in this edition of Knowledge Matters you can see some great examples of how we’ve achieved that. We have a review of the ‘Next Generation’ of Safety Thermometers and how the QO is working with one of its partners to help deliver the next chapter in the effective measurement of patient safety. Our regular guest Sam Riley provides us with an overview of all the latest goings on in the world of insight and the work we’re undertaking to ensure the voice of the patient is heard. On page 8 you can learn all about how the work of the AHSN is unfolding and read all about my busiest ever working fortnight on page 10. If your 2014 New Year’s resolution is to brush up on your analysis skills, check out Adam’s article on page 11 for a review of what exciting training courses we can offer. And because it’s Christmas, we have a festive crossword and the opportunity for you to vote on the award for the ‘Worst Chart of 2013’. It’s almost as packed as 2013 has been! It only remains to say thank you to all our customers, colleagues and partners for your continued support this year. Here’s to 2014 being even better!
Inside This Issue : Safety Thermometer: The Next Generation
2
CQC Maternity Survey
10
Ask An Analyst
16
Making the Million
4
A Fortnight in the Life...
12
News
18
Skills Builder
6
QO training Opportunities
13
Worst Chart 2013
19
AHSN: a briefing
8
Christmas Fun
14
A QO Christmas message
20
twitter.com/SECSHAQO issuu.com/SECQO http://www.networks.nhs.uk/nhs-networks/sec-qo www.QualityObservatory.nhs.uk
2
Safety Measurement: The Next Generation By Kurt Bramfitt The NHS Safety Thermometer is now a well established part of the safety culture of the NHS and is pretty much unprecedented in terms of the sheer volume of patients surveyed in a patient safety data collection. We’ve only scratched the surface of what we can do with the data, but it quickly became apparent that not all NHS services and settings were really getting the best out of the NHS Safety Thermometer because the nature of their business means that the measurement of pressure ulcers, falls etc. is not always giving them the information they need to focus on their patient safety priorities. Enter the next generation of Safety Thermometers. The ‘next generation’ uses the proven methodology from the Safety Thermometer ‘classic’ to allow the measurement of harm in a number of specialist areas. 8000 7000 6000 5000 4000 3000 2000 1000 0
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
Closest to completion is the Medication Safety Thermometer. Second only to patient accident, medication error is the biggest cause of harm, according to the National Learning and Reporting Service (NLRS). Have a look at the pareto chart for the Kent, Surrey & Sussex area to the left. But as yet there is no single measurement system that gives us everything we need to know in order to understand the burden of harm from medication errors and, crucially, whether our improvement efforts have actually made a difference. An expert group in tandem with the team from Haelo and the QO have developed a measurement instrument for medications that has a three step approach to the measurement of error and harm free care in relation to medications. The first step covers the basics of medications reconciliation and omissions, whilst the second step focuses on the impact of missing any high risk drugs, such as opioids or insulin. The final step is a multidisciplinary review with a focus on identifying harm and what steps can be taken to improve the problems that led to it happening. It may all sound complex but thankfully there is a handy webtool that guides you through the data entry side of things and allows you to save all your data and view your dashboard straight away. Ever so slightly behind the medications comes the Maternity Safety Thermometer, led by another expert group who advise on the focus required on basis of evidence and experience. Having a baby in the UK is the safest it has ever been. However human factors, working culture, communication and teamwork are key themes associated with avoidable harm in maternity care. This is borne out by the costs of CNST claims in obstetrics far outstripping the values of claims in other specialties.
info@quailtyobservatory.nhs.uk
www.QualityObservatory.nhs.uk
3 The Maternity Safety Thermometer allows maternity teams to take a temperature check on harm and records the proportion of mothers who have experienced harm free care, but also records the number of harm (s) associated with maternity care. The Maternity Safety Thermometer measures harm from ‘Perineal and/or Abdominal Trauma’, ‘Post-Partum Haemorrhage’, ‘Infection’, ‘Separation from Baby and Psychological Safety’. In addition we are identifying those babies with an ‘Apgar of less than Seven at Five Minutes’ and/or those who are ‘Admitted to a Neonatal Unit ‘. In the same way as the medications and ‘classic’ thermometers, this is a point of care survey that will be carried out on one day per month in each maternity service on all postnatal mothers and babies. Data are collected from postnatal wards, women’s homes and community postnatal clinics. Currently the Maternity Safety Thermometer is being piloted nationally with more than 30 organisations taking part in data collection and its development. You would be forgiven for thinking that this would be quite enough to be going on with, but there are two further developments in the pipeline in the form of mental health and paediatrics. There has been lots of work undertaken in these areas but it should be acknowledged that they are tricky areas to get just right. In mental health, an initial pilot allowed teams to measure harm and the proportion of patients that are 'harm free' from self-harm, falls, risk of violence and aggression, harm from violence and aggression, and omissions of medication. The paediatrics workstream is (excuse the pun) just in its infancy, but holds enormous promise, building on the work undertaken by colleagues in specialist children’s hospitals. All of the next generation Safety Thermometers as well as the established ‘Classic’ can be found on the NHS Safety Thermometer website. This has all the information you need about what’s going on with each development as well as all the guidance and analysis for the ‘Classic’. You can also submit data to the Medications Safety Thermometer pilot via the site so it really is a one stop shop! Do check it out and join in the conversation on the forums at: www.safetythermometer.nhs.uk
www.QualityObservatory.nhs.uk
info@qualityobservatory.nhs.uk
4
Friends and Family Test makes the million mark By Samantha Riley, Director Of Insight, NHS England Did you know that since the introduction of the Friends and Family Test (FFT) in April 2013, more than one million individual pieces of feedback have been collected on patients’ experiences of inpatient and A&E services?? That’s a huge wealth of feedback provided by patients to the NHS which can be used to support improvement. What an incredible achievement and great progress in giving every patient a voice. There is, though, much more to do over the coming year or so with FFT being rolled out to all NHS services by the end of March 2015. All women are now being asked to complete the test in NHS funded maternity services too and we are looking forward to seeing the first results of their feedback at the end January 2014. And of course we are gearing up for the roll out of the NHS staff FFT from April 2014 – more about this later. Over the past year, I’ve been out and about quite a bit to see for myself how FFT is working on the ground. I have witnessed, first hand, not only what a difference regular FFT feedback can make to both patients and their families, but also the difference it makes for staff, who can now see how much people appreciate what they do and where they can make improvements to services. There is no doubt that the introduction of FFT has made a significant (positive) impact on staff morale in many organisations where previously feedback may have only been received when care was not as good as it could have been. Where Trusts have most successfully implemented FFT, processes have been developed to enable front line staff to regularly review the comments made by patients. It is this free text which identifies areas for improvement. Examples of the types of improvements made as a result of comments include helping frontline teams to focus on better communications with patients, improving the quality of their local environment and of nutrition and caring. Currently, there is no requirement in inpatients and A&E for a free text box to be provided, however because of the reasons outlined about, we have mandated the inclusion of a free text box for maternity. Providers can expect that when guidance is issued for other FFT settings, a free text box will be mandated – this will also apply to inpatients and A&E. In the meantime, I would strongly encourage trusts to ensure that people can provide feedback on why they gave the answer they did. If you are not doing this already, my team can provide advice on how best to do it, so please feel free to contact us at england.friendsandfamilytest@nhs.net
Staff FFT From 1st April 2014, all NHS trusts providing acute, community, ambulance and mental health services in England will be required to implement the FFT for NHS staff. Why are we introducing FFT for staff as well as patients? Well, there is an increasing body of evidence which indicates an association between positively engaged staff and positive patient experiences. Research has shown a relationship between staff engagement and patient satisfaction, patient mortality, infection rates, Annual Health Check scores, as well as staff absenteeism and turnover.
info@qualityobservatory.nhs.uk
www.QualityObservatory.nhs.uk
5 NHS England has therefore adopted patient and staff FFT results as the first two measures in the high level scorecard in its first business plan, Putting Patients First. Much of this research has been undertaken by Professor Michael West (who developed and ran the staff survey for many years) - a leading expert in his field who has spent the greater part of his career undertaking research into the factors that determine the effectiveness and innovativeness of individuals, teams and organisations at work. Michael writes an informative monthly blog for NHS Employers which I would encourage Knowledge Matters readers to have a http://www.nhsemployers.org/PlanningYourWorkforce/Organisational-Development/ look at ProfessorMWDoODblog/Pages/ProfessorMichaelWest'sDoODblog.aspx The key principles of Staff FFT are summarised below. We do not expect these principles to change prior to publication of the final guidance which we plan to issue at the end of February: •
Feedback is collected and reported nationally on a quarterly basis;
•
All staff are included per quarter – a census approach;
•
Responses are anonymous;
•
Staff are asked two questions; about the care provided by the organisation and about the organisation as a place to work
•
The response will include the staff occupational group (using the high level groups currently used in the annual NHS staff survey);
•
The response options and scoring of staff FFT will mirror the patient FFT (“extremely likely” to “extremely unlikely” with a “don’t know” option);
•
Provider organisations can determine the data collection method (as with patient FFT);
•
Provider organisations can incorporate the staff FFT into existing staff engagement activities;
•
There will be a phased implementation to non-provider organisations in future years
Working draft guidance has already been shared with all Directors of HR in provider Trusts. We worked with various stakeholders, including NHS Employers, Unison, Unite, the Royal College of Physiotherapy, the Royal College of Nursing, trusts and subject matter experts such as Professor Michael West to develop the draft and are now encouraging as many Trusts as possible to test the guidance prior to publication of the final guidance planned for the end of February. For anyone interested in seeing the draft, wanting to learn more about staff FFT or keen to be a pilot, please email england.staff.friendsandfamilytest@nhs.net The final thing to say this time is that the annual staff survey will continue for 2014/15. We will be working with partners during 2014/15 to consider how FFT for staff and the annual staff survey can best work together to support local improvement. Merry Christmas everyone and see you next year!
www.QualityObservatory.nhs.uk
info@quailtyobservatory.nhs.uk
6
‘Tis the season to be jolly– more on death rates By Kate Cheema, Analyst Here at the QO we are often asked about the differences between HSMR and SHMI, the two high level measures of mortality often used by organisations as a key quality measure. As it seems to have become a bit of a trend in recent years to have something morbid (literally!) in the Christmas edition of KM, we thought it could be a good idea to review some of the core methodology that sits behind these important statistics. Hospital Standardised Mortality Ratio (HSMR) and Summary Hospital Mortality Index (SHMI) are both measures of in-hospital mortality designed for use in understanding whether or not there might be an issue with high death rates, as an outcome indicator of clinical quality. They are different measures from a methodological perspective but essentially are both indicators that serve a similar purpose, which is to help measure mortality at a hospital, allowing for differences between hospitals’ patient populations (such as deprivation, age, casemix etc.). The key idea behind such an adjustment is that if you have accounted for unavoidable factors then any differences in mortality between hospitals, or high death rates at a particular hospital will be due to variances in quality of care. To simply compare two hospitals, or highlight a high death rate, without taking these kinds of factors into account could give rise to misleading conclusions. For example, a hospital which specialises in cancer treatment is likely to have a relatively high number of deaths because of the nature of their casemix. Compare this to a small general hospital without such a specialism and the raw numbers would probably suggest a higher proportion of deaths at the first hospital, but we know that this difference could be simply due to the difference in the services offered. The temptation would be to ignore the difference in mortality rates, dismissing it as being due to differing casemix. But such an approach could mask a genuine problem and the HSMR and SHMI indicator are designed to be tools to help you understand the true picture. Both the HSMR and SHMI methodologies make an attempt to account for these kinds of differences statistically. Both the HSMR and the SHMI have a similar underlying methodology which is used to derive the measure. This involves comparing the observed (i.e. the actual) number of deaths at a hospital with the expected number of deaths to create a ‘ratio’ which is the number reported as the HSMR or SHMI indicator. This comparison takes the form of a simple calculation: HSMR/SHMI = Number of observed deaths / Number of expected deaths This means that if the hospital has a number of observed deaths greater than the number of expected deaths then a ratio greater than 1 will be seen (e.g. an observed number of 120 deaths, divided by an expected number of 110 will give a ratio of 1.09). If a hospital has a number of observed deaths less than the number of expected deaths the ratio will be less than 1 (e.g. an observed number of 100 deaths divided by an expected number of 110 will give a ratio of 0.91). Generally speaking, a number lower than 1 is seen as a ‘good thing’. The statistical aspect of both the HSMR and SHMI lies in the calculation of the ‘expected’ deaths. This is predicated on the idea that a predefined set of variables will, generally, dictate the likelihood of any given patient dying. For example, it is fairly easy to see that patient A, a 94 year old with end stage COPD who has just fallen over and broken their hip is more likely to have a hospital admission ending in death than patient B, a 23 year old who has a complex fracture of their wrist. In both HSMR and SHMI, the core methodology involves assigning a ‘risk of death’ to each patient seen by each hospital, based on a set of variables that have been shown to be significant predictors of death in England. This ‘risk of death’ calculation for each patient is then aggregated to provide a number of expected deaths, which is then used to derive the final HSMR or SHMI figure.
info@qualityobservatory.nhs.uk
www.QualityObservatory.nhs.uk
7 Obviously there will always be some factors that cannot be taken into account, either because the data doesn’t exist (or is of poor quality) or because it simply isn’t known. This might be a very specific issue linked to a single patient, such as certain social factor or part of their history. This means that no matter how good the construction of the HSMR or SHMI indicator there will always be a certain amount of error involved in calculating the expected deaths, simply because we don’t know everything about everybody. There are various pieces of statistical information that allow you to assess the degree to which an HSMR or SHMI statistic differs from ‘the norm’, which takes this kind of error into account. Perhaps the best way to view this is in the form of a funnel plot (see left), a statistical plot that indicates outliers by whether or not a hospital’s HSMR or SHMI value is outside of defined limits. You can see that Hospital Y is just outside the upper limit and therefore could be said to be an outlier. Note that we can’t say the same of Hospital X and Z even though they have slightly higher values than Hospital Y. Even now, there is a tiny degree of risk that hospitals are flagged as ‘poorly performing’ on the basis of chance rather than any systemic problem, but it is fair to say that this point that both HSMR and SHMI have controlled as far as they possibly can the risk of this happening. We have seen that the core methodology underlying both HSMR and SHMI is broadly speaking the same, but there are key differences between the two which means that they can tell you different things. The major, and fundamental, difference is that in its underlying dataset, HSMR includes deaths that occur in hospital only, whilst SHMI includes deaths that have occurred out of hospital within 30 days of discharge from the hospital in question. This naturally means that other organisations apart from the hospital in question need to be involved in the interpretation and use of the SHMI measure, for example local community trusts. The implications of this are primarily linked to how to use the measure which is addressed in the next section. Other differences between the two indicators can be seen in the table below: HSMR
SHMI
Spells with a palliative care code are excluded from the calculations Deprivation included as a risk adjustment variable Approximately 80% of spells included
No exclusion based on palliative care coding Deprivation not identified as a risk adjustment variable All spells included
Available monthly
Available quarterly
Sadly we don't have space here to really dig into the whys and wherefores of what this means in terms of usage of these measures, but there are some great resources available if you want to read more. Our favourite is the excellent publication from colleagues in Public Health England: Flowers, J., Abbas, J., Ferguson, B., Jacobson, B. and Fryers, P. (2010), Dying to know: How to interpret and investigate hospital mortality measures, Association of Public Health Observatories: http://www.apho.org.uk/ resource/view.aspx?RID=95780
But check out the independent review of SHMI as well as it has some interesting insights into the comparability of HSMR and SHMI as well: Campbell, M., Jacques, R., Fotheringham, J., Pearson, T., Maheswaran, R. and Nicholl, J. (2011), An evaluation of the Summary Hospital Mortality Index, University of Sheffield. http://www.sheffield.ac.uk/polopoly_fs/1.51777!/file/ SHMI_Final_Report.pdf
Just a final note though; whilst these indicators are an important part of the overall view of quality, nothing quite replaces having a good local understanding of crude death rates and numbers. Look at what Florence achieved with just a simple (yet attractive) plot of numbers of deaths.
www.QualityObservatory.nhs.uk
info@qualityobservatory.nhs.uk
8
Academic Health Science Networks – who are they and what do they do? By Fatai Ogunlayi There are 15 AHSNs establishing across England, bringing together NHS organisations, higher education, local government and business. They work with all parts of the NHS and healthcare delivery partners to accelerate the adoption and spread of innovation and best practice. They will develop solutions to healthcare problems and get the best existing solutions spread more quickly by building strong relationships with their regional scientific and academic communities and industry. •
the challenge to the United Kingdom for economic recovery and sustainable growth especially within the life sciences industries
•
the ageing population and associated challenges to rising healthcare costs and opportunities for leading development of innovative service and care models as well as novel therapies.
These challenges will need to be addressed by wider societal and UK infrastructure changes including the NHS. Emerging AHSNs will play an important role, sometimes leading and sometimes supporting in nature. AHSNs support UK PLC with a remit to improve health outcomes and create wealth. A set of core objectives are as follows: •
Focus on the needs of patients and local populations
•
Build a culture of partnership and collaboration
•
Speed up adoption of innovation into practice to improve clinical outcomes and patient experience support the identification and more rapid spread of research and innovation at pace and scale to improve patient care and local population health.
•
Create wealth through co-development, testing, evaluation and early adoption and spread of new products and services.
Academic Health Science Networks will strengthen collaboration between industry, academia and the NHS. They will create a more systematic delivery mechanism for diffusion and collaboration. Here in Kent, Surrey and Sussex AHSN, we have one of the most vibrant life sciences communities including a high concentration of health technology companies, nine universities and over 100,000 health and social care staff. To really get the most out of the wealth of resources we need to develop strategies to get them talking and working together to drive innovation across the patch, and implement sustainable solutions to the problems that we face. These problems are common to many parts of the country; an aging population with complex needs and variations in the quality of care across a complicated provider landscape. Of course in KSS AHSN we are very lucky to be able to build on the groundbreaking work of the Enhancing Quality and Recovery programme in helping us develop new innovations, spread best practice and, crucially, measure our impact in many key clinical outcomes. In addition, KSS AHSN has a focus on improving the pathways in dementia care and helping the whole system reduce unnecessary emergency admissions.
info@qualityobservatory.nhs.uk
www.QualityObservatory.nhs.uk
9 Through taking an holistic approach, bringing together partners in industry, academia and healthcare delivery, we can drive meaningful change at a rapid pace. Obviously driving continuous improvement is the name of the game but it’s not just about healthcare services. We also want to increase and improve our research culture and make KSS the go-to place for conducting exceptional research, attracting the best and the brightest to our corner of the UK. As this is Knowledge Matters we should really talk about the importance of data in the delivery and evaluation of the AHSN’s programme of work. We already know that the Enhancing Quality and Recovery programme has benefitted from having clear, reliable and regular measures associated with each workstream and the wider programme of AHSN work will be no different. In fact, our vision of realising the very best opportunities for improved health and wealth for the people of Kent, Surrey and Sussex will be underpinned by the very best analysis becoming the knowledge we need for success.
Healthcare Analyst of the Year– update Didn’t she do well! Those readers who follow the QO on Twitter will know that our very own Nikki Tizzard won the coveted Healthcare Analyst of the Year award at the Centre of Excellence in Healthcare Analysis awards following a lavish dinner (accompanied by a spot of interpretive dance (no we’re not kidding!)) in early November. A number of the team were in attendance and this photo evidences the fine celebrations undertaken by all!
www.QualityObservatory.nhs.uk
info@qualityobservatory.nhs.uk
10
CQC Maternity Survey 2013 Results By Samantha Riley, Director Of Insight, NHS England Did you know that there are almost 700,000 live births each year in England and having a baby is the most common reason for a hospital admission? On 12th December, the results of the 2013 maternity survey were published. Here’s a quick overview of what the results showed. There is evidence of improvements since the 2010 survey. There has been an increase in the proportion of women who said that they were always spoken to in a way they could understand during both their antenatal care, and during labour and birth. More women than the 2010 survey felt that they were always involved in their care, both antenatally and during labour and birth. Also more women than 2010 felt that they were treated with kindness and understanding and had confidence and trust in the staff caring for them during labour and birth. Many women felt listened to during their antenatal check-ups and a high proportion of women were asked how they were feeling emotionally as part of their postnatal care. More women were able to move around and find a position that made them most comfortable during labour and birth, and the majority of women felt that a partner or someone else close to them was able to be involved during labour and birth as much as they wanted. However, in other areas performance since 2010 has not improved and experiences fell short of expectations. The report highlights concerns around information or support being provided inconsistently, in some cases basic knowledge such as medical history was not known. While some women reported that they would have liked to see the same midwife or healthcare professional every time, many women reported seeing the same midwife or that they did not mind not seeing the same midwife. Additional analysis in relation to whether women saw the same midwife or not and continuity of care was completed as part of this report, and is detailed in the relevant sections. Both antenatally and postnatally, women that saw the same midwife and women that saw different midwives but did not mind, reported more positive experiences of aspects related to continuity of care than women who did not see the same midwife but had wanted to. The comments provided by women at the end of the questionnaire revealed problems with continuity of care and women reported having to repeat themselves to different health professionals. Information provision in some cases has definitely improved but is still inconsistent, especially in relation to primiparous women (first time mums). Information needed to make choices was not consistently provided and the choices themselves were not universally offered to women. In addition, both the survey responses and accompanying comments reveal some problems around communication and involvement in decisions made when women contacted the hospital or a midwife when they went into labour. It appears that information on feeding is not always being provided or is being inconsistently provided, and this is a particular problem for primiparous women; across all of the questions on feeding primiparous women felt less well informed and less supported than multiparous women. Fewer women than the 2010 survey reported
info@qualityobservatory.nhs.uk
www.QualityObservatory.nhs.uk
11 that they were not left alone at any time that worried them during labour and birth. The highest proportion that were left alone and worried was during early labour. The survey responses show that changes in pain relief are not always adequately explained, while the comments reveal instances of poor pain management or a lack of pain relief provided. Almost one in five women felt that their concerns during labour were not taken seriously and the comments provide some insight into this, as women reported instances when the quality of care was not what they felt it should be. The survey data shows that in the most part call buttons are being responded to, but not in all cases and the comments reflect this (especially when women had call buttons that did not work or were placed out of their reach). Some women felt that hospital wards, toilets and bathrooms are not clean enough, especially toilets and bathrooms and this is further supported by the comments that detail instances of unacceptable levels of cleanliness. A lot of negative comments were provided about postnatal facilities such as inadequate facilities during delivery or a lack of or poor equipment. Further details of the survey can be found here http://www.cqc.org.uk/public/publications/surveys/2013-nationalmaternity-survey
www.QualityObservatory.nhs.uk
info@quailtyobservatory.nhs.uk
12
A fortnight in the life=.. By Kate Cheema, Analyst Okay, it’s not exactly a typical fortnight in my working life, but the two weeks commencing the 19th November 2013 was a great example of two things: 1.
The huge range of work that the QO supports and is currently involved with
2.
How much work you can get done on a train
TUESDAY: Workshop with colleagues from CSCSU to review the mortality HSMR and SHMI measures and what they mean for local trusts using the Dr Foster mortality comparator tool. Included much discussion with regard to the veracity of these measures and how they can be used alongside local indicators to provide a full quality picture WEDNESDAY: Evening meeting as part of the Mary Seacole leadership programme in Reading followed byT.. THURSDAY: All day leadership workshop, part of the Mary Seacole leadership development programme FRIDAY: Team meeting, Excel training delivery and Safety Thermometer webtool and analysis discussions with colleagues in Haelo, based at Salford Royal Foundation Trust MONDAY: Evening meeting with colleagues from BAPEN to discuss development of nutrition measures TUESDAY: Morning presentation on measurement at BAPEN annual conference THURSDAY: Fly to Newquay to finalise the evidence section of a commissioning guide for frail elderly pathways with colleagues from NHS England South TUESDAY: Attend a GP learning event in Salisbury to discuss the possible development of an ‘early warning system’ but actually end up discussing a wide range of possibilities with regard to real time data and how this can help CCGs develop their commissioning priorities
info@quailtyobservatory.nhs.uk
www.QualityObservatory.nhs.uk
13
QO training programmes By Adam C. Cook Many people make a New Year resolution to do more physical training, but have you considered doing some training of a more cerebral bent? Have you ever wanted to be able to make a dashboard like the ones that the Quality Observatory does, or maybe even improve your skills in Excel or other software, well obviously you can read the Skills Builders and Ask An Analyst in this very publication, but that’s limited and slow – how about a bit of hands-on tutorial from our expert and experienced team. The QO are offering a variety of courses across a wide variety of subjects. We can provide techie courses for the geeks out there – so if you are already experienced in Excel and Access we can show you how to get more out of them, alternatively if you just need the basics we can cover that too. We have simple introduction courses, which will let you know what you need to know – we can tailor our courses for your specific needs to make it relevant to your role. It’s not just the technical stuff we cover either. We can deliver a programme of statistics courses, whether you don’t know your chi-squared from your odds ratio, or if you’ve got more experience and want to know more we can provide a package to suit your needs. Our well-received course Statistics for Clinicians pinpoints the measurement and analysis tools that best suit clinical needs. One of our main areas of expertise is in presenting data that is clear and understandable to all people, so maybe you’d be interested in our Presenting Data course which advises which charts and tables to use and when, and how best to deploy them. It also offers insight into why different visualisations are better for different purposes, and tackles the perennial presentation problem of style over substance. If that’s not enough, then we can offer you something a bit more meta-textual and teach you information about information. Our team has many years experience in the field of NHS data, we know where it comes from and where it goes to. We know what you can and can’t access, what information is publicly available, and what information is written on a piece of paper in a locked filing cabinet in the back room of a hospital. Our data flows courses can lead you through the minefield of what’s what in the NHS information field. One of the main advantages of our education programme is our flexibility. We are happy to talk through with you beforehand exactly what your needs are and then tailor our course to your needs. We’re happy to talk to packed lecture theatres, or go through the nitty-gritty of a problem with a handful of people. If you can supply a room we’re happy to travel to you to deliver the training, and very soon we’ll have our own fully equipped training room at QO Towers in the heart of Horley. We can even deliver a flexible and fun e-learning package that suits your needs which you can return to again and again. If you have any training needs that you think you want our help with, then please drop a line to info@qualityobservatory.nhs.uk
www.QualityObservatory.nhs.uk
info@qualityobservatory.nhs.uk
14
Christmas crossword! A particularly fiendish crossword this year from our puzzler in residence. No prizes sadly, except for the personal satisfaction of knowing you are a festive clever clogs! If you want to check your final solution, drop us a line at info@qualityobservatory.nhs.uk. Enjoy!
info@quailtyobservatory.nhs.uk
www.QualityObservatory.nhs.uk
15
YE OLDE PUB
Need a quick activity to keep the kids busy before Dr Who comes on? See if you can help the QO find their way to the pub through a snowy Horley. You could colour in the picture too!
www.QualityObservatory.nhs.uk
info@qualityobservatory.nhs.uk
16
IDENTIFYING PROVIDER ERROR=...RELIABLY Application: Excel Dear Quality Observatory We here at the National ‘Elf Service ™ Christmas Commissioning Group, the lead commissioner of Christmas toy deliveries, are deep in discussions with Mr S. Claus from our main provider,the Christmas Support Unit. with regard to the most appropriate way to measure and show delivery errors. Mr. Claus has made the excellent point that the burden of data collection involved in having a minimum data set for each of the several hundred million deliveries he makes each year would be prohibitive and involve a significant increase in the number of data-elves (a scarce enough resource) required, especially as his error rate is very low at an estimated 0.001%. Do you have any suggestions? Quality Control Dept., National Elf Service CCG™
Solution: Complexity 2/5 — Error Calculation rates Dear Quality Control Dept. It was of course only a matter of time until Mr Claus came under the same scrutiny we all come to expect when providing such a key public service; we’re very pleased to see that quality is at the heart of Christmas with the child experience key to all you do. However, naturally we understand how critical it is that Mr Claus isn’t overburdened with data collection. As data-elves ourselves, we can appreciate how distracting it can be to be wading through data that simply isn’t relevant to the task in hand. Given the highly specialist role your provider undertakes, with a single key deliverable, we need to ensure that we don’t fall into the trap of collecting lots of data just to answer a very simple question. So, the first thing to consider is what the question actually is. In this case we are looking at error rates so we presume that we are interested in both the rate itself but also how it changes over time to help us look at what influences increases in errors. So we need to count the errors, but we also need to see them relative to overall deliveries. There are approximately 80 million children under 15 years of age in the 27 EU countries so let us use this as our ‘eligible population’ for the purposes of this exercise. If we wanted to plot data over time, and see where a ‘real’ change has occurred, our go to methodology would be the SPC chart. It might be tempting to suggest that we plot the hourly error rate as a proportion of deliveries where errors were made. On the basis of our 80m population, that would translate to 6.67m deliveries per hour. With an error rate of 0.001% this would mean around 7 errors per hour. But given the size of the denominator here, it would be very difficult to interpret the chart. In addition, whilst we know that the overall error rate is very low,
info@qualityobservatory.nhs.uk
www.QualityObservatory.nhs.uk
17 this fluctuates throughout the night, with more errors towards the end of the evening. So we end up with a chart with a high proportion of zero values which adds to the difficulty in interpretation and would run the risk of drawing incorrect conclusions. Have a look at the chart below and to the left; time (in the 24 hour clock format) is shown in 45 minute intervals with the proportion of errors shown on the y-axis. Lots of zero values and all we can really see is that the last 15 minutes before 6 a.m. has the highest levels of error. For these reasons then we would recommend using a ‘rare events’ SPC approach to monitoring Mr Claus’ error rates. There are two to choose from; the first is a ‘T’ chart which looks at the time between errors. Given that this particular service only happens over a very short period of time, the unit of time used for the measurement of time between each error is likely to need to be very short and as such it will be hard to guarantee accuracy.
Deliveries between (millions)
We would instead recommend using a second approach, the ‘G’ chart which instead plots the number of ‘opportunities’ between each error. In this case, each delivery is an opportunity for error. As we will be counting the number of deliveries anyway, if we flag each delivery with an error we will easily be able to identify the number of opportunities (deliveries) between 18.00 each one. In our existing data set 16.00 we have 48 errors overall. So 14.00 instead of counting our errors, we 12.00 count the number of deliveries 10.00 carried out successfully between each error. 8.00 6.00 4.00 2.00 0.00
Deliveries between (millions)
The chart to the left shows what our data looks like if we do this. We can see clearly that there appear to be three periods where errors, whilst still rare, are occurring more 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 frequently in terms of how many Error number successful deliveries happen between each error. This already tells us much more about our errors than just plotting the proportion over time. But the real intelligence comes when we put our local understanding of the whole process alongside the data. This might be tricky from a commissioner perspective but a full and frank debrief over turkey sandwiches on Boxing Day might be invaluable to understanding where the problems may lie. It’s pure speculation, but these particular data-elves suspect the following chart might be a real possibility: 18.00 16.00 14.00 12.00 10.00 8.00 6.00 4.00 2.00 0.00
Flying over Germany; stollen is heavy going
SatNav loses signal over rural France Sherry kicks in
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 Error number
www.QualityObservatory.nhs.uk
info@qualityobservatory.nhs.uk
18
NEWS Nursing Technology Fund NHS England has announced funding of £100 million for which NHS organisations can bid to buy innovative technology to support nurses, midwives and care staff in improving patient care. A key technology the Nursing Technology Fund is focussed on is mobile technology such as notebooks or tablets. For the community healthcare worker, this helps with the accuracy of data and cuts down time travelling to and from the office to use IT systems, valuable time that can be spent with patients. Another focus for the fund is technology that provides staff with vital information at the point of care which can help staff to provide safer care. Funds will be split £30 million / £70 million between projects which can be delivered in 2013/14 and 2014/15. The deadline for first round applications is 15 January 2014 with decisions announced in February. See: www.england.nhs.uk/ourwork/tsd/sst/nursingtechnology-fund. HC2014 Safer hospitals, patient engagement and mobile technologies are just some of the topics that will feature at HC2014, the Health Informatics Congress. The conference, which will take place at Manchester Central 19 - 20 March, will focus on a number of themes in relation to the call for a paperless NHS by 2018. The two-day event will feature over 100 speakers including:
•
Professor David Haslam, Chair of NICE
More information www.hc2014.bcs.org.
and
registration:
NHS Innovation Expo 2014 More than 10,000 people from across health and care and the voluntary and community sector, including commissioners, clinicians, patient leaders and innovators will meet at Expo because they want to bring about change to improve NHS and care services. The theme is the ‘House of Care’ and through this Expo will showcase what patientcentred coordinated care looks and feels like. Expo is being held in Manchester on 3rd and 4th March 2014. Hopefully the QO will have a presence, so watch out for us! More information at: www.healthcareinnovationexpo.com/ Non-NHS RAs Initial approval has been given by the Health & Social Care Information Centre (HSCIC) for non-NHS organisations to run their own Registration Authorities. Applications are now being invited for NonNHS organisations to pilot their own Registration Authority. Applications are open until 12:00 on 6 Jan 2014. See: http://systems.hscic.gov.uk/ rasmartcards/nonnhsrapilot NHS Choices Indicators As part of NHS England’s drive for transparency and to support public participation in the design and quality of local health services, 38 new indicators have been added to the NHS Choices website in a new accountability area at http:// www.nhs.uk/Service-Search/Accountability
•
Justin Whatling, Chair BCS Health
•
Tim Kelsey, NHS England
•
Andrea Spryopolous, President of the Royal College of Nursing
•
Andrea Sutcliffe, Chief Inspector of Adult SC, Care Quality Commission
•
Mike Pringle, President, Royal College of GPs
•
Martin Murphy, Clinical Director, NHS Wales Informatics Service
Access it at: http://www.gp-patient.co.uk/results/
Professor Patrick Soon-Shiong, University of California at Los Angeles
The results of the CQC maternity survey have been published with results to trust level available at: http://www.nhssurveys.org/surveys/666
•
info@qualityobservatory.nhs.uk
GP patient survey The latest GP patient survey using aggregated data collected during January-March 2013 and JulySeptember 2013 has been published
CQC maternity survey
www.QualityObservatory.nhs.uk
19
WORST CHART OF 2013 One of our collective hobbies when we’re out and about is ‘bad chart’ spotting. 2013 has been a bumper year and we wanted to get our readers’ views on which should be awarded the accolade of ‘worst chart of 213’. This fortunate example will be used in our educational material to help educate future generations with regard to how not to display data. Follow the survey monkey link at the bottom of the page to choose your best worst chart of the year! CANDIDATE A: THE MANHATTAN SKYLINE
CANDIDATE C: POINTLESS PIE
CANDIDATE B: PICKUP STICKS
CANDIDATE D: THE VENN-PIE-A-GRAM
In the next issue we’ll give the considered reasons as to why we think these aren’t the best representations of data we’ve seen and announce the winner as voted by you!
https://www.surveymonkey.com/s/QO_Christmas
www.QualityObservatory.nhs.uk
info@qualityobservatory.nhs.uk
Working in a Data Wonderland Data flows, through the system,
QO CHRISTMAS MESSAGE
Bringing facts, bringing wisdom A beautiful sight, We're happy tonight. Working in a data wonderland. Charts that are trending, Showing performance is ascending
This year’s Christmas message comes courtesy of our poet in residence Adam Cook! Sing along with the QO using the lyrics to the left and the music at http://boo.fm/b1811057 Or scan the QR code:
Targets are met, No need to fret, Working in a data wonderland. In a spreadsheet we can build a dashboard, populated from a handy drop-down Select the CCG to be explored, Show results by LA or by town. Later on, we'll aspire, To make rates even higher
Fascinating Facts Norwegian scientists have hypothesised that Rudolph’s red nose is probably the result of a parasitic infection of his respiratory system
To face unafraid, The plans that we've made, Working in a data wonderland. In a spreadsheet we can build a dashboard, And fill it up with lots of different charts, Useful information won't be ignored, On the way to winning minds and winning hearts. When data flows, ain't it thrilling, To know it's fulfilling Its own little role, To make patients whole, Working in a data wonderland.
Simon says==. The word mistletoe is from the Anglo-Saxon word misteltan, which means “little dung twig” because the plant spreads through bird droppings.
Knowledge Matters is the newsletter of the Quality Observatory, to discuss any items raised in this publication, for further information or to be added to our distribution list, please contact us. Hosted by: Central Southern Commissioning Support Unit
E-mail: info@qualityobservatory.nhs.uk
To contact a team member: firstname.surname@qualityobservatory.nhs.uk