Improving the Quality of

Policy Decisions

The Center for Analysis of Science in Policy

www.scipolicy.org

2014

1

IMPROVING THE QUALITY OF POLICY DECISIONS

Today, there is no way to

forecast the outcome of a

policy before it is enacted.

True?2

IMPROVING THE QUALITY OF POLICY DECISIONS

FALSE !… because there now is a tool

that expresses numerically

the probability

that a policy will achieve its goal.

3

4

What if I showed you a new tool that could give

you a numerical index of the probability that a

policy will succeed?

"Impossible" you would say, but believe me, that

tool is here, and here’s what we can do with it.

IMPROVING THE QUALITY OF POLICY DECISIONS

What is that tool?

(Restless, anxious breathing, agitated movement.)

5

It is …

Integrative Propositional Analysis

WHAT ????????6

INTEGRATIVE PROPOSITIONAL

ANALYSIS

(IPA*)

Conceived and developed by

Steven E. Wallis, PhD

IPA DOES FORECAST THE PROBABILITY THAT A

POLICY WILL MEET ITS OBJECTIVES - OR NOT.

*IPA is NOT India Pale Ale.

7

APPLYING IPA 1.1

Through examining the logical structure of a

document, we can assign a numerical

value to the probability that the policy will

be successful - or not!

8

9

Applying IPA during policy development, we can

identify defects in the policy at that stage,

permitting us to step back, modify and improve the

result.

APPLYING IPA 1.2

10

When there are several policy alternatives, they

can be compared and the superior one selected for

completion.

This can obviate ideological gridlock, prompting

bi-partisan collaboration.

APPLYING IPA 1.3

GOALS &

OBJECTIVES

1. Further development of IPA theory

2. Automation of the now largely manual process

3. Perform analyses on documents submitted

4. Teach the use of IPA to policy makers

5. Demonstrate use of IPA during creation of policy

6. Make IPA an open platform for the common good

11

NEEDS

1. Hire postdoctoral fellows to perform analyses

2. Travel expenses to regional and national meetings

3. Part-time administrative assistant

4. Routine office and administrative costs

5. Marketing and publicity

6. Computer programmer to automate analyses

Estimated start up cost: $15,000

12

13

PUBLICATIONS UNDER SUBMISSION

Using Integrative Propositional Analysis (IPA) to Study the USAID/PERU Country Development Cooperation Strategy

Are theories of conflict improving? Using propositional analysis to determine the structure of conflict theories over the course of a century

Using Integrative Propositional Analysis to analyze the potential usefulness of “People sharing Resources” Theory and indicating

directions for advancement

Whole Scraps and Partial Maps: Investigating Some Underlying Assumptions of AQAL

PUBLICATIONS

Academic Publications

Wallis, S. (in press – publication anticipated in 2014) Propositional Analysis for Evaluating Explanations through their Conceptual Structures.

Chapter in “Modes of Explanation: Affordances for Action and Prediction” (edited by Michael Lissack and Abraham Graber).

Palgrave MacMillan. Wallis, S. (2014). Structures of logic in policy and theory: Identifying sub-systemic bricks for investigating, building, and

understanding conceptual systems. Foundations of Science. doi:10.1007/s10699-014-9360-4 Wallis, S. (2014). Existing and emerging methods

for integrating theories within and between disciplines. Journal for Organizational Transformation and Social Change. Guest Editor, Steven

Wallis, Vol. 11 (1).

Wallis, S. (2013). How to Choose Between Policy Proposals: A Simple Tool Based on Systems Thinking and Complexity Theory. E:CO –

Emergence: Complexity and Organizations. Vol. 15 (3).

Wallis, S. (2011). “Avoiding Policy Failure: A Workable Approach.” Litchfield Park, AZ: Emergent Publications, ISBN: 978-0-9842165-0-5.

Wallis, S. (2010). “Developing Effective Ethics for Effective Behavior.” Social Responsibility Journal, Vol. 6 (4).

Wallis, S. (2010). “Toward the Development of More Effective Policy Models.” Integral Review – special issue “Toward Development of Politics

and the Political.” Vol. 6 (1), p 153-177.

Wallis, S. (2010). “The structure of theory and the structure of scientific revolutions: What constitutes an advance in theory?” Chapter in

“Cybernetics and Systems Theory in Management: Tools, Views and Advancements.” Wallis, S. E. (Ed.) Hershey, PA: IGI Global.

IPA: Selected Publications

14

Franklin D. Nash, MD

Dr. Nash is an 82 year-old former Professor of Medicine at the Indiana University

School of Medicine, specializing in Nephrology (renal disease and hypertension), with

emphasis on research and teaching. In recent years he has been distressed by the abuse of

Science, Technology, Engineering, and Mathematics for partisan and ideological purposes in

creation of Congressional bills that have resulted in policies detrimental to the interests of

the American people. Attempting to see to an increase on the quality of policy

determinations, he founded the Center for Evaluation of Science in Policy

(www.scipolicy.org) that now serves as a vehicle for promoting the use of Integrative

Propositional Analysis (IPA), the first quantitative means of forecasting and improving the

effectiveness of policy during its origination.

In retirement, he has served as a life and health insurance company Medical

Director and Vice President for Academic Affairs at the RMP Institute of Biomechanics.

During his active years, Dr. Nash consulted for Social Security Disability, Medicare,

Medicaid, and for the USAF School of Aerospace Medicine among other organizations and

was a frequent Visiting Professor at various academic medical institutions.

A lover of the arts, Dr. Nash played double bass to finance his education and

continued playing professionally in his free time. His focus now is on Health Care Policy,

Biomedical Research Policy, and Water-Energy Nexus Policy, hence his passion for

Integrative Propositional Analysis.

Steven E. Wallis, PhD

Dr. Wallis is Director of Meta Analysis at Meaningful Evidence. He has a

decade of experience as an organization development consultant providing

collaborative workshop facilitation, leadership development, and training to

businesses and non-profits in Northern California. Building on that practical

experience, and a systems thinking perspective, he returned to school and earned

his PhD in 2006 at Fielding Graduate University.

Dr. Wallis also works at Capella University – mentoring doctoral candidates.

As director for the Foundation for the Advancement of Social Theory (FAST), he

supports emerging scholars working to identify rigorous paths for improving theory,

policy, and strategic knowledge. An interdisciplinary thinker, his publications cover a

range of fields including ethics, science, management, organizational learning,

entrepreneurship, and policy. His book, Avoiding Policy Failure, shows how a

systems view of policy models can be used to estimate the effectiveness of policies

before implementation, as well as improving policies for reducing cost and improving

results. Recently, Dr. Wallis was appointed to the Fulbright Specialist Roster. The

Fulbright Specialist program supports U.S. scholars in projects to help improve the

capacity of academic institutions outside the United States.

15

Bernadette Wright, PhD

I founded Meaningful Evidence, LLC to help leaders accumulate and order data so as

to disclose what effects they are having and why, so they can use that information to improve

their results. I support clients with survey design, data analysis, formative evaluation, impact

evaluation, and report writing. I also provide evaluation design and proposal writing for

grants.

My specialties include ability to solve tough evaluation challenges and to quickly

research and synthesize vast amounts of information. Believing in using whatever research

methods can best answer the questions, I have used a broad range of quantitative research

and qualitative research methods. Examples include survey research, focus groups, literature

reviews, case studies, interviews, and environmental scanning. This experience has covered

long-term care, caregiving, aging in place, Medicaid, assistive technology, health policy, and

many other topics.

Previously, as Consultant at The Lewin Group, I provided research, project

management, and report writing for government, corporate, and nonprofit clients. Before that,

I was at AARP, providing policy analysis and policy research. My PhD is in Public

Policy/Program Evaluation from the University of Maryland.

16

The Center for Analysis of Science in Policy

www.scipolicy.org

2014

17

We thank you for your attention