4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement?...
-
Upload
marvin-washington -
Category
Documents
-
view
214 -
download
0
Transcript of 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement?...
![Page 1: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/1.jpg)
4th National Patient Safety ConferenceFriday 7th November, 2014
Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new approach to acute hospitals
Professor Kieran Walshe
Email: [email protected]
Twitter: @kieran_walshe
![Page 2: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/2.jpg)
4th National Patient Safety ConferenceFriday 7th November, 2014
Or: Making improvement work: learning from research
Professor Kieran Walshe
Email: [email protected]
Twitter: @kieran_walshe
![Page 3: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/3.jpg)
Hieronymus Bosch. The Conjuror. c.1500
![Page 4: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/4.jpg)
What’s inside the “black box” of quality improvement• What do we know?
– Improvement projects work – sometimes…
– How/why they work is often quite hard to see/tell/predict
– Scaling up and transferring improvement is problematic
– Is it method, or implementation, or systems, or people, or…?
– Organisations vary in “improvement capability”
• Issues or problems with improvement research
– Terminology, taxonomy and description
– Theories and mechanisms
– Context and implementation
– Cumulation of knowledge or evidence
![Page 5: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/5.jpg)
![Page 6: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/6.jpg)
Diagram from James Albert Bonsack's patent application (U.S. patent 238,640, granted March 8, 1881)
![Page 7: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/7.jpg)
Case study: Care Quality Commission and regulating health and social care
• Established in 2009 to regulate health and social care in England
• Mission “to protect and promote” the health, safety and welfare of service users and the “general purpose of encouraging the improvement of health and social care services”
• Much criticised from all sides for its performance and effectiveness – several major high profile failures
• Complete change of board and senior executives from 2012, and review of strategy and approach in 2013
![Page 8: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/8.jpg)
• What is known about effective regulation?
• What can be learned from other settings?
• How does evidence from elsewhere apply to health and social care?
• What important knowledge gaps exist, and how can they be filled?
![Page 9: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/9.jpg)
Mapping the theories of regulation
![Page 10: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/10.jpg)
CQC’s new approach to hospital inspection• Much larger and more expert inspection teams
• More detailed and extensive set of inspection processes drawing on a wider range of data sources and fieldwork
• Focusing the inspections on eight defined core service areas
• Assessing and rating performance in five domains (safe, effective, caring, responsive and well-led) using a four-point rating scale (inadequate, requires improvement, good or outstanding)
• Much more detailed and comprehensive inspection report with a full narrative description of services in each of the core service areas alongside ratings.
![Page 11: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/11.jpg)
Researching CQC’s new approach• About 16 interviews with people in CQC and outside
about the new acute hospital regulatory model
• Six observed hospital inspections in late 2013 – about 48 days of non-participant observation, review of documents, attending QA group meetings, quality summits etc – and 4 followup observations in 2014
• About 65 1:1 telephone interviews with CQC inspection team members and NHS trust staff following inspections in 2013/14
• Surveys of CQC inspection team members and trust staff following inspections in 2014
![Page 12: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/12.jpg)
Evidence, KLOEs and information• Data pack really impressive – not well understood, so
not well used
• KLOEs – good starting point but mostly general, not specific to the organisation, quite high level
• Wider sources of data – up to 500+ documents from trust and others, some not even read, hard to sift/sort and synthesise
• Organisation’s own ability to assess/explain/use information on own performance only tangentially tested
![Page 13: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/13.jpg)
Inspection teams• Large teams maximise content expertise and “feet on
the ground”, at a cost to selectivity, coherence and manageability
• Minimal selection/training/matching, some skills gaps evident, prior experience largely shapes approach
• Team roles and management sometimes unclear and variable – best teams have clearly defined complementary roles
• Team members’ competence, confidence and credibility crucial – learning from experience
![Page 14: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/14.jpg)
Inspection processes• Actual fieldwork process – highly inductive,
experiential, intuitive, dependent on team skills, little structured or directed inquiry, variable across teams but scope for specialisation
• Some aspects like focus groups, corporate dimension, interviews, listening events need some rethinking to maximise value
• Corroboration sessions great idea but hard in large teams and become bilaterals under time pressure
• Relentless activity – no time to think/reflect/discuss
![Page 15: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/15.jpg)
Evidence, judgements, ratings, reports
• Inspector observation/experience tend to predominate
• Core service areas – good but don’t touch some clinical services, and lots of corporate systems/issues
• Domains have some ambiguities and hard to know what fits where; well-led an organisational cover-all
• Rating levels not really defined and so interpreted variably within teams, across teams, across inspections
• Process for integrating and weighting disparate evidence to form rating (and back it up) implicit
• A lot left to CQC team leads to write up afterwards
![Page 16: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/16.jpg)
Some ideas for CQC on improvement• Maximise the quality of inspection team members
through selection, training, deployment, experience and review/feedback
• Maximise validity, reliability, efficiency and utility of inspections through greater structure/process without constraining flexibility or losing scope for professional judgement
• Maximise validity/reliability of ratings through simplification, definition, training and testing
• Start to measure impact after the inspection to drive change and improvement
![Page 17: 4 th National Patient Safety Conference Friday 7 th November, 2014 Inspecting for improvement? Lessons from evaluating the Care Quality Commission's new.](https://reader035.fdocuments.in/reader035/viewer/2022062518/56649ea95503460f94bad868/html5/thumbnails/17.jpg)
Some lessons learned from improvement research• Build research/evaluation capacity in healthcare
organisations
• Make evaluation an integral part of all improvement programmes/projects – from design onwards
• Test taxonomy/description (what does it do?); theory/mechanism (how does it work?) and context/implementation (who does it and where?)
• Celebrate both successful and unsuccessful projects/programmes for different reasons
• Do not overclaim effectiveness and impact