Nathan H. Danneman

c* -- A new way to communicate importance

Esarey and Danneman (forthcoming). A Quantitative Method for Substantive Robustness Assessment. Political Science Research and Methods.

The Point:

1. p-values don’t matter2. Substantive significance DOES matter3. Substantive significance is subjective4. Formalizing substantive significance can

help us communicate, and make decisions from evidence

But…in school we learned p-values are really really important!!!

Suppose my friend invented a magic weight loss pill...

...but, it *might* cause cancer:

How did you choose whether or not to take the drug?

Ostensibly, you weighed:1. The evidence that the drug causes cancer2. How bad it would be to get cancer3. The evidence that the drug is safe4. How good it would be to lose weight

How did you choose whether or not to take the drug?

Ostensibly, you weighed:1. The evidence that the drug causes cancer2. How bad it would be to get cancer3. The evidence that the drug is safe4. How good it would be to lose weight

Letting r denote the empirical change in cancer probability, you solved this:

Some of the inputs to your choice are subjective:

The evidence that the drug causes cancer-objective

How bad it would be to get cancer-subjective

The evidence that the drug is safe-objective

How good it would be to lose weight-subjective

Implication

It is totally unsatisfactory to say things like:

“As civil war becomes more prevalent in the state’s proximity, threatening the state with externalities that could make civil conflict more likely, the state increases its level of repression. Substantively, we find that moving from a conflict-free neighborhood (e.g., Switzerland in 2000) to one riddled with civil conflict (e.g., Swaziland in 1982) generates a change of four-tenths of a [repression measure]-unit.” Danneman and Ritter (2014).Contagious Rebellion and Preemptive Repression

Two problems

Problem 1: the utility acting on a false positive might be different than the utility of acting on a false negative.

Problem 2: the relative utility of these decisions may vary by person (that whole “subjective thing)

Solution: Part 1Pick a utility function that can capture the relative weighting the u(false positive) versus u(false negative).Our preferred choice: a kinked, linear utility function:

Utility=0

Solution: Part 2The amount of “kink” or “bend” in the utility function is governed by a single parameter. Let’s call it gamma, such that it’s square root represents the number of times a person values false positives versus false negatives.

Step 3: Plot gamma against EU

That assumes you are willing to get off the couch for E[u]=0

If the minimum amount of benefit you are willing to get off the couch for is, say, c, then we can plot gamma against c* that solves E[u] - c = 0.

Too much work?See our R package, {cstar} on CRAN.

Thanks.

Nathan Danneman@ndannemanndanneman@gmail.com

Esarey and Danneman (forthcoming). A Quantitative Method for Substantive Robustness Assessment. Political Science Research and Methods.