MWPAI Schoof of Art Community Art and Dance Summer 2013 catalog
Getting Public Health Research into Policy: Science, Art or Dance
-
Upload
professor-jim-mcmanus-afbpssffphcsci-frsb-cpsychol -
Category
Health & Medicine
-
view
237 -
download
2
Transcript of Getting Public Health Research into Policy: Science, Art or Dance
www.hertsdirect.org
Research to Policy:Science, Art or Dance?
Prof Jim McManus, OCDS, CPsychol, CSci, FFPH, FRSB, FRSPH, AFBPsS
Director of Public Health, Hertfordshire County [email protected]
July 14th 2016, UK Public Health Research Centres of Excellence Conference
www.hertfordshire.gov.uk
What I want to cover
• A slightly tongue-in-cheek journey• Poking fun at a relationship which doesn’t quite work,
because I care so much that it should• Seen policy making in various places over 25 years• Practical and real world view of research interface with policy• Things that work and things that don’t work• What not to do• Failings of Policy Makers
www.hertfordshire.gov.uk
Longer Version
• A range of case studies including some from criminology
• Some further tools for some of the steps
www.hertfordshire.gov.uk
A Thesis...• Public health is
inherently political• Because it affects
populations• It inevitably impacts on
or is impacted on by Policy
• Need to think much more about these issues and claims
www.hertfordshire.gov.uk
An average week for a chief officer/senior civil servant
• 200 - 500 or more emails a day
• Pile of FOIs, Surveys, Reports, MP letters, Briefings
• Politicians confused about media reporting
• Policymakers dissing experts and trying to assert integrity of expertise
• NGOs, Advocates, Government all want a piece
• Lots of people “with solutions” wanting time
• 63 meetings, many of them triple booked
• Series of conflicting deadlines• Reducing budgets• Things come up unpredictably
which need “solving”• Wanting to do the right thing for
the population• Trying to get inside the
Politicians and Partners heads
www.hertfordshire.gov.uk
Strategic Challenges1. Delivering core public health must dos for less2. Growing demand and need 3. The challenges of preventing cost and adverse outcome
(Falls going up in some areas)4. Levering whole system redesign and prevention using
everyone’s resources to save money and improve outcomes
5. Doing stuff which works6. Massive and confusing literature on implementing research
www.hertfordshire.gov.uk
The 3 or 4E’s Duty or Value for Money Duty
• Economy -value for money and delivering the required service on budget, on time and within other resource constraints
• Efficiency- getting an acceptable return on the money and resources invested in a service.
• Effectiveness - extent to which the organisation delivers what it is intended to deliver.
• Equity – not always in there even with legal frameworks
www.hertfordshire.gov.uk
What Policymakers think of researchers
Q. How many scientists does it take to reach a firm conclusion?
A.There are too many parameters to be definitive here
www.hertfordshire.gov.uk
What Politicians Think...• What is needed?• What is right?• What is doable?• What is politically and financially doable?• Who’s in my way? Who’s with me?• What do my voters want?• What do experts think?• What do I now think?............Conclusion
www.hertfordshire.gov.uk
The “knowing” and “interpretive” world of policymakers
• The legacy (good and bad) of New Public Management...measure everything, quantify
• Money, existing policy and law, influence• The desire to do the right thing or the desire to please
political masters. Why are they in the job?• Who’s got a view and how to I balance them?• There’s no money• What is the space for the possible when you take all this
into account
www.hertfordshire.gov.uk
The Epistemology of the Policy ProcessWhat they teach you at government school – the science
Source http://www.thisnation.com/textbook/processes-policyprocess.html
www.hertfordshire.gov.uk
The Policy ProcessWhat usually tends to happen – the art of influence
1001 issues and influencers, many competing
Not everyone happy
PolicyProposalrefinery
RefinementCompromise
PartialImplementation orEffectiveness
Repeat loop eternally
www.hertfordshire.gov.uk
More like a dance than art or science
• Requires instinct and “feel” for where you are in the dance and why you’re doing it
• Requires good evidence and knowing when to use it
• Requires relationship “heart speaks to heart”• Requires emotional engagement• Partnership with the other dancer(s)• Adapting steps to where you are
www.hertfordshire.gov.uk
The Policy Process Dance: Art & Science in relationshipThe Policy Networks and Influences Approach
Source: http://www.lancaster.ac.uk/fass/projects/hvp/newsletters/5findings.htm
www.hertfordshire.gov.uk
The Dance Floor - Policy Process in 5 DimensionsA model for dynamic tension
PolicyProcessQuadrilateral
“Evidence”The
Desirable,The
Achievable
Motivations
Influences
Multiple layers:1. Advocates2. Politician3.Officers 4. Scientists5. UsersAcross Time too
www.hertsdirect.org
Onto the dance steps....If that’s how it happens, what works for the Policy/Research relationship?
I have a personal proposal...
www.hertfordshire.gov.uk
Περιχώρησις - Research, Policy and Gregory Nazianzen
• Idea of περιχώρησις perikhōrēsis• Dancing round. Mutual Intimacy. • Interprenetration of understanding • Dynamic hermeneutic unity
but not identity• Psychology, Emotion, Mutuality• Perikhoresis as a model for research
into policy
www.hertfordshire.gov.uk
What to do – dance etiquette
• Build a Relationship!• Research on who you need to talk to• Focus on what you want to say and why they
should listen• Hone• The 60 Second Pitch to the right person made
personally• And the usual newsletters etc
www.hertfordshire.gov.uk
Dance Steps – 7/8 C’s of Consulting1st for me – What’s your Capital to offer?
1.Client – who, what do they want?2.Clarify – what is the dance?3.Create – work up the doable4.Change - what is to be done5.Confirm – the what and the doing6.Continue – keep doing it7.Close – thanks for the dance, happy?
Read one review of this here http://www.sweeneycomms.com/The-Seven-Cs-of-Consulting-g.asp
www.hertfordshire.gov.uk
What not to do..wrongfooting• Send a bulletin and expect anyone to read it• Write in scientific terms for policymakers• Set out what you’ve done without setting any
context• Send a 99 page report with eight appendices
and no executive summary• Send anything before you’ve worked out who to
talk to
www.hertfordshire.gov.uk
Research on Diet
Is your research clear and conclusive enough to give definitive guidance to people?
If yes – be clear and don’t be vague
If no – then shut up and continue researching until it is clear enoughA moral responsibility on scientists NOT to overstate their case and generalizability
www.hertfordshire.gov.uk
Family Nurse Partnership – applying the evaluation to public health decision making
• Evaluation report says doesn’t work not cost effective
• FNP says it does other valuable things
• Where do these contradictory views fit in policymaker decisions?
• Issues raised:– Appraisal of research– By products of
interventions and roundedness of evaluations
– Applicability
www.hertfordshire.gov.uk
Research on effects of e-cigarettes
• Most of it unappliable because of design flaws or even repeating existing badly done research
• In vitro studies which don’t reflect real life• Speculation and going beyond the research
doesn’t help
www.hertfordshire.gov.uk
You could interchange these headings..Policy is...• Messy and Spiral• Contested Space• “Oh God, someone’s had an
idea”• “We must do something”• “Any evidence”• “I want answers”• “What will it do”• Compromise, Influence and
Change
Research is..
• A trade off between what you want to know/do and resources to do it
• A methodological trade off• Hermeneutically situated• Contested
www.hertfordshire.gov.uk
Knowing your partner...feasibility
Option A Option B Option C
Effectiveness Very Positive Positive No impact
Flexibility Very Positive Positive Positive
Sustainability Positive Positive Negative
Political Feasibility High Medium Low
Administrative Feasibility
High Medium Low
Time Short Medium Long
Cost High Medium Low
Suggested criteria example
www.hertfordshire.gov.uk
Communication - Writing style as an exampleAcademic• Heavily referenced• Third person • Passive voice• Tentative conclusions
Local Govt• Active voice• Summaries• Recommendations• Costings• Policy options• Impact measurements• Feasibility• risk
www.hertfordshire.gov.uk
Two partners in the same dance – i) the research itself and ii) the organisation and its agility
www.hertfordshire.gov.uk
Soft skills for the dance....making the 8 Cs live• Contested space – competing demands• Learning and mutual understanding • influence, nuance• Understand what constitutes evidence• Understand each side’s concerns • Understand the pressure of being a policymaker and the
often limited analytical tools and apparatus available • Craft a working relationship in that space
www.hertfordshire.gov.uk
Formation for the Dance not Training for the Science
• Policy makers are formed – psychological, political, emotional, conceptual, skills and tools
• Researchers need to be formed not trained in how to work with one another
• A conscious relationship and working with one another
• Too little interpenetration of the two• Policy Networks
www.hertfordshire.gov.uk
Thank you
• [email protected] • Public Health Pages• www.hertsdirect.org/healthinherts • JSNA• http://jsna.hertslis.org/
www.hertsdirect.org
Slides for a longer session
www.hertfordshire.gov.uk
Tools for the road....• Contested space – competing demands• Learning and mutual understanding • Soft skills – influence, nuance• Understand what constitutes evidence• Understand each side’s concerns • Understand the pressure of being a policymaker and the
often limited analytical tools and apparatus available • Craft a working relationship in that space
www.hertfordshire.gov.uk
Some problems
• Massive literature on applying research in practice
• Lots of problems in the literature – Sometimes conceives receivers of research
as in deficit of knowledge mode– Focuses often on one side – the research
quality or the organisation – not synergistic enough
www.hertfordshire.gov.uk
Thesis
• Applying research needs a partnership between researchers and policymakers from the inception if possible
• Two sides of a coin : the quality of the research and the agility and leadership of the organisation
• Develop a roadmap jointly
www.hertfordshire.gov.uk
Context
• Cuts to public services• Increasing demand on NHS services• Financial unsustainability of current system• We need research which can help us keep
people independent and healthy, and reduce need for public services
www.hertfordshire.gov.uk
Nationally funded research
• http://www.nihr.ac.uk/funding/public-health-research-programme.htm
• http://www.nets.nihr.ac.uk/programmes/phr• http://www.nihr.ac.uk/funding/school-for-public-
health-research.htm• Great these are funded but we need more
emphasis on translatable research and interventions, and evaluating existing ones
www.hertfordshire.gov.uk
Case Studies from Criminology• Very policy directed field• Political and worldview issues in researchers and
politicians• Negotiating a sometimes difficult space• Government commissions a lot of research to
inform policy• Some ongoing controversies• Analogies for health work...
www.hertfordshire.gov.uk
Framing the debate can put your partner off…
Knowing your partner – Kenneth Clarke or Michael Gove would like this. How would it go down with Michael Howard or Theresa May?
Hulley et al, Brit J Crim, 2016
www.hertfordshire.gov.uk
Why are you writing? To Change Policy or just lay bare a problem?Feeling good about what you’ve written but what have you changed?
Raymen, Brit J Crim, 2016
www.hertfordshire.gov.uk
Actors in the System• Actors in
the system can derail the implementation, even when trying to do “the right thing”
www.hertfordshire.gov.uk
Study of Public Health Policy Making
Wilmot et al, Journal of Public Health (2016)
www.hertfordshire.gov.uk
For real world relevance: research cannot be static when life is dynamic
Limitation?Or actually a strength that it addresses dynamic real world decision making?
Wilmot et al, Journal of Public Health (2016)
www.hertfordshire.gov.uk
A good example of policy revelant work
• Frames a policy problem and how to respond to it relatively simply
Walker et al, JPH, 2016
www.hertfordshire.gov.uk
Research on Diet
Is your research clear and conclusive enough to give definitive guidance to people?
If yes – be clear and don’t be vague
If no – then shut up and continue researching until it is clear enoughA moral responsibility on scientists NOT to overstate their case and generalizability
www.hertfordshire.gov.uk
The Problem
• Outcomes Frameworks• Connecting evidence with outcomes• Connecting evidence with interventions• Interests of policymakers and interests of
researchers
www.hertfordshire.gov.uk
So how do you go about applying research into public health practice? • Lots of literature on this• Some very complex and complicated models• A Proposal from me
1. Treat it like a change management process2. Think about two major sides to a coin 3. Project and process manage it4. Fidelity and sensitivity test
www.hertfordshire.gov.uk
Change Management
• Lessons from ESRC evaluations of large scale IT programmes
• Lessons from NHS and Social Care research on applying research
www.hertfordshire.gov.uk
Levels and Phasing
• Which level is the intervention?– Societal– Environmental?– Biological?– Intrapersonal?
• Which phase of time?– 1 year, 2, 3?
www.hertfordshire.gov.uk
Return on Investment as part of the argument
• Variable response from commissioners to this• Too much use of poorly developed ROI models in
evidence• ROI is not always the answer and sometimes doesn’t
even help• Most ROI models aren’t real world enough• Depends on how robust it is• Who cashes the returned cheque? If agency A invests
but Agency Y gets the return, where is the incentive?
www.hertfordshire.gov.uk
Return on Investment Tools
• Helps if the evidence is also very clear • Helps if there is a timescale to the return and
clarity on who gets it• The policy/commissioning/research partnership
is crucial to making ROI useful
www.hertfordshire.gov.uk
What is evidence-based public health? Many definitions
◦ From Evidence-Based Medicine to Evidence-Based Public Health Evidence-based public health is defined as the development, implementation, and
evaluation of effective programs and policies in public health through application of principles of scientific reasoning, including systematic uses of data and information systems, and appropriate use of behavioral science theory and program planning models. (Brownson, Ross C., Elizabeth A. Baker, Terry L. Leet, and Kathleen N. Gillespie, Editors. Evidence-Based Public Health. New York: Oxford University Press, 2003.)
◦ E-Roadmap to Public Health Practice Concepts (New Hampshire Institute for Health Policy and Practice) Developing, implementing, and evaluating public health programs or public health
policies (in public health terms an "intervention") that have 1) data demonstrating their effectiveness and 2) a grounding in a health behavior theory or ecological model of health.
www.hertfordshire.gov.uk
What is evidence-based public health?
• Many definitions– “The process of integrating science-based
interventions with community preferences to improve the health of populations.” (Kohatsu ND, Robinson JG, Torner JC. Evidence-based public health: An evolving concept. Am J Prev Med. 2004 Dec;27(5):417-21.)
www.hertfordshire.gov.uk
Steps for EBPH
• Develop an initial statement of the issue• Gather data to quantify it• Use the research literature to determine what is
already known• Develop program or policy options• Create an implementation plan• Evaluate the program or policy plan
O'Neall, M. A., & Brownson, R. C. (2005). Teaching evidence-based public health to public health practitioners. Annals of Epidemiology, 15(7), 540-544.
www.hertfordshire.gov.uk
More on the 2 sides of the dance: Partner 1: the Research itself• Internal and external validity• Applicability• Clarity of methods and outcomes• Translatability• Intervention readiness• Engagement with policymakers
www.hertfordshire.gov.uk
Dance partner 2: the organization and its agility
www.hertfordshire.gov.uk
Example: Behavioural Sciences• Review of Research• Synthesis into policy paper• Applying resource• Articulating work and programme plan• Delivery and championing• Integration into strategy
– This is very high level and much more involved in this
www.hertfordshire.gov.uk
Develop an initial statement of the issue
Use PICO (from EBM) to begin define question◦ P: Population◦ I: Intervention◦ C: Comparison◦ O: Outcome
Use a logic model to begin your strategic planning◦ Inputs, activities, outputs, results (short/long term)◦ W.K. Kellogg Foundation Logic Model Development Guide
http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf◦ Logic model tutorials (CDC)
http://apps.nccd.cdc.gov/dashoet/logic_model_1/menu.html http://apps.nccd.cdc.gov/dashoet/logic_model_2/index.html
www.hertfordshire.gov.uk
Steps for EBPH
• Develop an initial statement of the issue• Gather data to quantify it• Use the research literature to determine what is
already known• Develop program or policy options• Create an implementation plan• Evaluate the program or policy plan
O'Neall, M. A., & Brownson, R. C. (2005). Teaching evidence-based public health to public health practitioners. Annals of Epidemiology, 15(7), 540-544.
www.hertfordshire.gov.uk
Mapping the Policy Process
• General Context issues – domestic and international.• Specific Policy Issues (i.e. the policy cycle)• Who are the Stakeholders? (Stakeholder analysis)
– Arena: government, parliament, civil society, judiciary, private sector.
– Level: local, national, international • Process matrix + political influence ratings• What is their Interest and Influence?
[Sources: M. Grindle / J. Court ]
www.hertfordshire.gov.uk
Ontario Drug Programme (Khan et al,2014)http://dx.doi.org/10.1016/j.healthpol.2014.06.007
• Policymakers have cited barriers to using evidence, including lack of research relevance and timeliness.
• .Although reports often demonstrate an increase in research relevance, rarely do they provide concrete methods of enhancing research timeliness
• . Additionally, the impact of researcher–policymaker collaborations is not well-discussed.
www.hertfordshire.gov.uk
Be clear about how feasible it is to do this
Option A Option B Option C
Effectiveness Very Positive Positive No impact
Flexibility Very Positive Positive Positive
Sustainability Positive Positive Negative
Political Feasibility High Medium Low
Administrative Feasibility
High Medium Low
Time Short Medium Long
Cost High Medium Low
Suggested criteria example
www.hertfordshire.gov.uk
The problems• Early and avoidable disability and disease• Early avoidable death• Smoking prevalence uneven• Obesity rising in adults, flattening in children in most places,
rising in some• Alcohol related disease on the increase• Preventable cost to public health• Multiple needs, worklessnessThe Tartan Rug and local profiles give much more info on this
www.hertfordshire.gov.uk
Thank you
• [email protected] • Public Health Pages• www.hertsdirect.org/healthinherts • JSNA• http://jsna.hertslis.org/