Joining Minds Group Modeling to Link People, Process, Analysis, and Policy Design
description
Transcript of Joining Minds Group Modeling to Link People, Process, Analysis, and Policy Design
G. P. RichardsonJuly 2006
1
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Joining Minds Group Modeling to Link People, Process, Analysis, and
Policy Design
GP Richardson, DF Andersen, LF Luna-ReyesRockefeller College of Public Affairs and Policy
State University of New York at Albany
(Presented at the annual meeting of the Association for Public Policy Analysis and Management, 2004)
G. P. RichardsonJuly 2006
2
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Overview
• What is group modeling?• An extended example: Welfare Reform• Other cases• The Albany group modeling approach• Evaluating group model building efforts• Why does it work?
G. P. RichardsonJuly 2006
3
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Ancestry of GMB
• GDSS• Quinn, Nunamaker, Eden & Ackmann, DeSanctis & Gallupe, …
• Decision conferencing• Milter & Rohrbaugh, Schuman & Rohrbaugh, …
• System dynamics• Forrester, Richardson & Pugh, Sterman, …
• Mental models & systems thinking• Checkland, Senge, …
• For a rich history, see Zagonel
G. P. RichardsonJuly 2006
4
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
What is Group Modeling?
• A form of group decision support, involving a group of stakeholders with a complex problem• Group facilitation
• Model building and refinement in public
• Simulation of scenarios and options
• Extensive facilitated discussion and analysis
• Facilitated policy design and decisions
G. P. RichardsonJuly 2006
5
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
What is Group Modeling?
• Management team (10-20) with a Modeling/Facilitation team (2-4)
• Four full days over 3-to-4 months• Extensive between meeting work• Rapid prototyping of model with finished simulation
product• Facilitation of implementation plans
G. P. RichardsonJuly 2006
6
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Primary GMB references in the System Dynamics Community
• “Decision modeling”: Reagan-Cirincione et al.
• “Teamwork”: Richardson & Andersen
• “Scripts”: Andersen & Richardson
• “Group model building”: Vennix
• Special issue of the System Dynamics Review on GMB (1997)
G. P. RichardsonJuly 2006
7
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Why System Dynamics Modeling?
SystemConceptualization
Representation ofModel Structure
Perceptions ofSystem Structure
G. P. RichardsonJuly 2006
8
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Why System Dynamics Modeling?
SystemConceptualization
ModelFormulation
Representation ofModel Structure
Comparison andReconcilation
Perceptions ofSystem Structure
Diagramming andDescription Tools
G. P. RichardsonJuly 2006
9
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Why System Dynamics Modeling?
Empirical Evidence
SystemConceptualization
ModelFormulation
Representation ofModel Structure
Comparison andReconcilation
Perceptions ofSystem Structure
Alternative Models,Experience, Literature
Diagramming andDescription Tools
G. P. RichardsonJuly 2006
10
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Why System Dynamics Modeling?
Empirical Evidence
SystemConceptualization
ModelFormulation
Representation ofModel Structure
Comparison andReconcilation
Perceptions ofSystem Structure
Alternative Models,Experience, Literature
Empirical andInferred Time
Series
Comparison andReconciliation.
Deduction OfModel Behavior
Diagramming andDescription Tools
Computing Aids
G. P. RichardsonJuly 2006
11
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Why System Dynamics Modeling?
Empirical Evidence
SystemConceptualization
ModelFormulation
Representation ofModel Structure
Comparison andReconcilation
Perceptions ofSystem Structure
Alternative Models,Experience, Literature Literature
Empirical andInferred Time
Series
Comparison andReconciliation.
Deduction OfModel Behavior
Diagramming andDescription Tools
Computing Aids
G. P. RichardsonJuly 2006
12
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Why System Dynamics Modeling?
Empirical Evidence
SystemConceptualization
ModelFormulation
Representation ofModel Structure
Comparison andReconcilation
Perceptions ofSystem Structure
Alternative Models,Experience, Literature Literature
Empirical andInferred Time
Series
Comparison andReconciliation.
Deduction OfModel Behavior
Diagramming andDescription Tools
Computing Aids
StructureValidatingProcesses
BehaviorValidatingProcesses
G. P. RichardsonJuly 2006
13
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
The Albany Teamwork Approach
• Facilitator / Elicitor• Modeler / Reflector• Process coach• Recorder• Gatekeeper
G. P. RichardsonJuly 2006
14
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Components of the Process
• Problem definition meeting• Group modeling meeting• Formal model formulation• Reviewing model with model building team• Rolling out model with the community• Working with flight simulator• Making change happen
G. P. RichardsonJuly 2006
15
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
A Typical Room GMB Session
G. P. RichardsonJuly 2006
16
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
An Example: Welfare Reformin New York State Counties
• Initial interest within NYS Department of Social Services
• TANF model in Cortland County• Safety net model in Dutchess County• Joined TANF/SafetyNet model in Dutchess• Calibration in Cortland, Dutchess, & Nassau• Implementations in Cortland & Dutchess
G. P. RichardsonJuly 2006
17
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
First Group Model Building Meeting
• Introductions: Hopes and Fears• Stakeholders• Introduction to simulation: Concept models• Client flow elicitation• Policy resources and clusters• Mapping policy influences• Next steps for client group and modeling team
G. P. RichardsonJuly 2006
18
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Who Was in the Room?
• DSS Commissioner
• Deputy commissioner
• DSS director of medical services
• DSS director of administrative services
• DSS director of income maintenance
• NYS DSS representatives
• Health commissioner Mental health administrative manager
• Executive director of Catholic Charities
• Representative from the Department of Labor
• Minority leader of the county legislature
• Managed care coordinator
G. P. RichardsonJuly 2006
19
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Introduction to Simulation
• Concept models• Introduce the stock, flow, and causal link icons used
throughout the workshop
• Demonstrate there are links between feedback structure and dynamic behavior
• Initiate discussion about the structure and behavior of the real system
• Less than 30 minutes
G. P. RichardsonJuly 2006
20
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Concept Model Progression:“Models are ours to change and improve.”
Onassistance
At riskemployedJob finding
rate
Job loss rate
Avg length of stayon assistance
Fraction losing jobper year
Onassistance
At riskemployedJob finding
rate
Job loss rate
Avg length of stayon assistance
Fraction losing jobper year
Onassistance
At riskemployedJob finding
rate
Job loss rate
Avg length of stayon assistance
Fraction losing jobper year
Jobs
Employmentratio
Onassistance
At riskemployedJob finding
rate
Job loss rate
Avg length of stayon assistance
Fraction losing jobper year
Jobs
Employmentratio
Onassistance
At riskemployedJob finding
rate
Job loss rate
Avg length of stayon assistance
Fraction losing jobper year
Jobs
Employmentratio
Unemployedand
unassisted
Loss ofassistance rate
Avg total time onassistance
Fraction losingassistance per year
Onassistance
At riskemployedJob finding
rate
Job loss rate
Avg length of stayon assistance
Fraction losing jobper year
Jobs
Employmentratio
Unemployedand
unassisted
Loss ofassistance rate
Avg total time onassistance
Fraction losingassistance per year
G. P. RichardsonJuly 2006
21
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Concept Model Progression:“Behavior is a Consequence of Structure”
At Risk Populations4,000
3,000
2,000
1,000
0
0 2 4 6 8Time (Year)
On assistance : welf1At risk employed : welf1
At Risk Populations4,000
3,000
2,000
1,000
0
0 2 4 6 8Time (Year)
On assistance : welf1At risk employed : welf1
At Risk Populations4,000
3,000
2,000
1,000
0
0 2 4 6 8Time (Year)
On assistance : welf2At risk employed : welf2
At Risk Populations4,000
3,000
2,000
1,000
0
0 2 4 6 8Time (Year)
On assistance : welf2At risk employed : welf2
At Risk Populations4,000
3,000
2,000
1,000
0
0 2 4 6 8Time (Year)
On assistance : welf3At risk employed : welf3Unemployed and unassisted : welf3
At Risk Populations4,000
3,000
2,000
1,000
0
0 2 4 6 8Time (Year)
On assistance : welf3At risk employed : welf3Unemployed and unassisted : welf3
G. P. RichardsonJuly 2006
22
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Client Flows in the Resulting TANF Model
G. P. RichardsonJuly 2006
23
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Client Flows in the “Safety Net”
G. P. RichardsonJuly 2006
24
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Confidence building processes
• Structure of the model emerging from group process
• Parameters based on administrative data everywhere possible
• Parameter and table function group elicitations
• Group contributions to tests of model behavior
G. P. RichardsonJuly 2006
25
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Simulated vs Actual Caseload
G. P. RichardsonJuly 2006
26
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Three Policy Mixes
• Base run (for comparison)• Flat unemployment rate
• Historical client behaviors
• Investments in the “Middle”• Additional services to TANF families
• Increased TANF assessment & monitoring
• Safety net assessment & job services
• Investments on the “Edges” • Prevention
• Child support enforcement
• Self-sufficiency promotion
G. P. RichardsonJuly 2006
27
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Investing in the “Middle”
G. P. RichardsonJuly 2006
28
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Investing on the “Edges”
G. P. RichardsonJuly 2006
29
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Base, “Edges,” and “Middle” Compared:Populations on the Welfare Rolls
G. P. RichardsonJuly 2006
30
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Total Job-Finding Flows from TANF
G. P. RichardsonJuly 2006
31
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Program Expenditures
G. P. RichardsonJuly 2006
32
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Emerging Lessons
• Unemployment dominates system performance
• Loss of eligibility will shift the next economic cycle’s costs and caseloads
• Endogenous management makes a smaller difference
• Employment programs at the middle of the system are low leverage points
• Policies at the edges of the system have high leverage
• Community-wide partnerships are needed to implement “Edge” policies
• Performance measures continue to be problematic
G. P. RichardsonJuly 2006
33
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Resource allocation: Unpacking the Policy Resources for Implementation
• 43 participants about 30 agencies and organizations in the county
• Three stage process• 9 groups
• 6 larger groups
• 3 final groups
• Ending with five initiatives, costing about $675,000
G. P. RichardsonJuly 2006
34
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Final proposals implemented in Cortland
• Job center ($150K)• Centralized location for all referrals
• Resource center ($150K)• Coordination of community effort toward diversion
• Program to support employed self-sufficiency ($200K)• Job counselors, case managers, private sector
• Computer-based comprehensive assistance ($150K)• Link all providers and case managers, shared database
• Expansion of child-care services ($75K)
G. P. RichardsonJuly 2006
35
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Does It Work?
• Categories of evaluation data• Modeling team reflections
• Participant reflections
• Measurable system change
• Results• Methodological problems
• Implementation in about half of GMBs
• Positive measure of success in about half of the implemented interventions
G. P. RichardsonJuly 2006
36
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
Why Does It Work?
• Engagement• Mental models• Complexity• Alignment• Refutability• Empowerment
G. P. RichardsonJuly 2006
37
Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany
What are we really doing?
• Microworlds?• Data-based representations of a policy reality
• Tools for finding what options really work best to solve a complex dynamic problem
• Boundary objects?• Socially constructed representations of a negotiated world that
may not exist
• Tools for facilitating discussion and agreement in contentious environments