Selecting An Intervention In A Complex Adaptive Health Care System
&Evaluating Its Impact
Malaz Boustani, MD, MPHIndianapolis Discovery Network for DementiaIndiana University center for Aging Research
Regenstrief Institute, Inc
www.indydiscoverynetwork.com
04/08/23 indydiscoverynetwork.com 2
The US Health Care System Report Card
• Sub-optimal quality– Dementia recognition:< 40%– FDA approved Trx: < 10%– Inappropriate Trx: > 25%
• Compromised safety– 90,000 death / yr– 1,000,000 injury / yr
• Waste – 30% - 40% health expenditure (overuse, underuse, misuse, duplication)
• Discovery to Delivery:– 17 yrs– $800 millions
• Generalizability of Research Findings:– < 1% of real world patients enrolled in AD studies
Boustani, 2007; IOM, 2001, Westfall, 2007; Faison et al, 2003;
04/08/23 indydiscoverynetwork.com 3
Independent Agent E
Other Local CASsStress
Feedback
Independent Agent D
Independent Agent C
Independent Agent B
Independent Agent A
Changing Environment
Connection
The health care CAS
Emergent Performance
04/08/23 indydiscoverynetwork.com 4
Complex Adaptive Health Care System • An open, dynamic, flexible, adaptive, and complex
network• Complex due to
– Numerous interconnected, semi-autonomous, competing, and collaborating members
• Adaptive due to– Its capability of learning from its prior experience– Its flexibility to change its members connecting
patterns to fit better with its surrounding environment
Holden, 2005; Litaker et al. 2006 Feb.
04/08/23 indydiscoverynetwork.com 5
Complex Adaptive Health Care System
• Emergent behaviors – NOT predetermined ones
• Self-organized controls – NOT central controls
Holden, 2005; Litaker et al. 2006 Feb.
04/08/23 indydiscoverynetwork.com 6
The Paradox of CAS
• Each CAS is unique in its – Members diversity– Members interactions– Surrounding environment– Previous history– Evolving and learning process.
• Health care research centers generate scientific data from group-based studies – Then regulators/guideline developers ask each unique CAS to
implement the findings.
Holden, 2005; Litaker et al. 2006 Feb; Stoebel et al, 2005.
04/08/23 indydiscoverynetwork.com 7
The Paradox of CAS
• The performance of a CAS fluctuates over time
• The connecting patterns of its members are dynamic and change over time in respond to the stress or the fitness requirements of its surrounding environment.
Holden, 2005; Litaker et al. 2006 Feb; Stoebel et al, 2005.
04/08/23 indydiscoverynetwork.com 8
How Can I Assess CAUSALITY?Can I Predict?
1. How can I identify or determine that a system is a CAS?2. How can a system with no bosses, no mandates and no
plans produce repeatedly complicated structures and perform difficult tasks?
3. Can I build a complex adaptive system model to study? What can I do with a simulator?
4. What data do I have & how can I analyze data in CAS?5. How can I manage the problems of uncertainty,
variability, and variable interdependency in a CAS?6. Is replication essential in science?
04/08/23 indydiscoverynetwork.com 9
7. How can I select a change in a CAS?8. How can I evaluate the success or the impact of my
proposed intervention or change in a CAS?9. How are experiments performed in a complex adaptive
system?10. Is a randomized controlled trial the best accurate study
design to evaluate an intervention in a CAHS?
How Can I Assess CAUSALITY?Can I Predict?
Selecting an Intervention
04/08/23 indydiscoverynetwork.com 11
The 9 Principles of Selecting a Change in a Complex Health Care System
1. View your system through the lens of complexity.
2. Build a good enough vision with minimum specifications
3. Lead from the edge by balancing data and intuition, planning and acting, safety and risk.
4. Foster the "right" degree of information flow, diversity and difference, connection among agents, power differential, and anxiety.
5. Uncover and work with paradox and tension.
Paul E Plsek. 1997.
04/08/23 indydiscoverynetwork.com 12
The 9 Principles of Selecting a Change in a Complex Health Care System
6. Go for multiple actions at the fringes, let direction arise.
7. Listen to the informal relationships, gossip, rumor, and hallway conversations that contribute significantly to the individuals’ perceptions about their surrounding environment and their subsequent actions.
8. Allow complex systems to emerge out of the links among simple systems that work well and are capable of operating independently.
9. Build a community of members who collaborate, create, learn and compete
Paul E Plsek. 1997.
04/08/23 indydiscoverynetwork.com 13
Reflective Adaptive Process
• Build a local “Think-Tank: the RAP Team” responsible of introducing an acceptable, locally matched, flexible and effective change in its CAS.
• External or internal facilitators who encourage the RAP Team to select, adopt or create local processes to – solve the CAS problem – enhance the CAS performance– guide the CAS respond to its surrounding environment
• RAP is the second generation of CQI
Stroebel et al.2005.
04/08/23 indydiscoverynetwork.com 14
The RAP Principles
• Vision, mission, and shared values are fundamental in guiding ongoing change processes in a CAS
• Creating time and space for learning and reflection is necessary for a CAS to adapt to and plan change
• Tension and discomfort are essential and normal during CAS change
• Improvement teams should include a variety of system’s agents with different perspectives of the system and its environment
• System change requires supportive leadership that is actively involved in the change process, ensuring full participation from all members and protecting time for reflection
McDaniel et al, 2003; Stroebel et al.2005; Boustani et al 2007.
04/08/23 indydiscoverynetwork.com 15
Evaluation of Interventions in CAS
• Selecting the content and the delivery of the intervention
A- The content based on SER of past research or guidelines
B- Develop a RAP team to • Localize the content• Localize / Invent the delivery process• Monitor the delivery process• Monitor the agents interactions• Detect emergent behaviors
Evaluating the Impact of the Selected Interventions
04/08/23 indydiscoverynetwork.com 17
Methodological Quality
“Extent to which a study’s design, conduct, and analysis has minimized selection, measurement, and confounding biases”
West et al, 2002
04/08/23 indydiscoverynetwork.com 18
• Random assignment against – both unconscious and deliberate human influence
on the assignment of subjects to different groups– unknown confounders
• Blind outcome assessment ensures – treatment and analysis of outcomes are not
colored by prejudice
The “Safeguards” Evidence
Guyatt et al, 1993; West et al, 2002
04/08/23 indydiscoverynetwork.com 19
The Hierarchical Strength of
Evidence in Treatment Decisions
• N-of-1 RCT • SER of RCT • Single RCT • SER of observational studies • Single observational study • Physiologic studies • Unsystematic clinical observations
Guyatt et al, 1993; West et al, 2002
04/08/23 indydiscoverynetwork.com 20
• Design:– N-OF-1 RCT
• System outcomes– Overall Cost– Overall Safety– Overall Quality– Overall Patient Satisfaction– Overall Staff Satisfaction
• Intervention specific outcomes
Proposed Evaluation Process
04/08/23 indydiscoverynetwork.com 21
Applying N-OF-1 RCT in CAS
• A CAS receives pairs of evaluation periods– Each pair including one period of active intervention
and another of usual care in random order.
• CAS members are kept masked to allocation • Outcomes are monitored blindly and
continuously
04/08/23 indydiscoverynetwork.com 22
0----------1------------2-----------3-----------4----------5-----------6------------7-----------8-------------9-----------10------------11------------12
Time
Outcome Data
Outcome Data
Outcome Data
Outcome Data
Outcome Data
Outcome Data
Outcome Data
R to Int R to Int R to Int
R to UC R to UCR to UC
N-OF-1 RCT in CAS
04/08/23 indydiscoverynetwork.com 23
Issues of –OF-1 RCT
1. Selecting the Intervention: – Is there enough data to suggest the efficacy of the
intervention? – Did the RAP team modify the intervention delivery
and content to match the local resources? 2. Determining the rapidity of onset of the intervention
effect: – How quickly does the intervention begin to act and
cease acting?3. Optimizing dose:
– What is the best dose? – Does it differ between systems?
Guyatt et al, 1986; Price & Evans, 2002
04/08/23 indydiscoverynetwork.com 24
Issues of –OF-1 RCT
4. Selecting the outcomes:– the overall common system outcomes not related specifically
to the intervention.– the outcomes that the intervention effect directly.– the process that mediates the effect of the interventions such
as the relationship and the new connection between the CAS members.
5. Interpreting the Results of N-of-1 RCTs: – What a priori criteria should be established for classifying a
trial as definitely negative or indefinite?
Guyatt et al, 1986; Price & Evans, 2002
04/08/23 indydiscoverynetwork.com 25
The Requirement for N-of-1 RCT:
1. The problem that requires intervention needs to be chronic.
2. The problem that requires intervention needs to be stable.
3. The effect of the new intervention begins to acts within days to weeks at most.
4. The effect of the new change ceases acting after discontinuing within days to weeks at most.
Guyatt et al, 1986; Price & Evans, 2002
04/08/23 indydiscoverynetwork.com 26
The Science of CAS
• “A general feeling of unease when contemplating complex systems dynamics. Its devotees are practicing fact-free science. A fact for them is, at best, the outcome of a computer simulation; it is rarely a fact about the world.”
» (Smith 1995, p. 30).
04/08/23 indydiscoverynetwork.com 27
The Science of CAS
“Prediction is our best means of distinguishing science from pseudo-science”
» Karl Popper “The history of 20th-century science should also give
complexologists pause. Complexity is simply the latest in a long line of highly mathematical ‘theories of almost everything’ that have gripped the imaginations of scientists in this century”
» (Horgan 1995, p. 104).
Top Related