Experimental Methodology I

Inducing Values, Risk Attitudes and Controlling Other Characteristics

Induced Value Theory We have asserted that laboratory experimentation

affords greater control over economically relevant characteristics resource constraints, information, technologies and preferences than is possible in analyses of field data.

Resource constraints, information technologies are easy to control, but, how do we control subjectsinnate, hard-wired homegrown preferences?

Induced value theory (e.g. Smith (1976)) says that subjects can be induced to hold certain preferences (and other characteristics) by proper assignment of reward values to action choices.

A $25 Bottle of WineExample: Suppose I say to you:

1) Buy me any bottle of wine: I will pay you back.2) Buy me any bottle of wine: I will pay you $25 for it.

According to induced-value theory, in case 2 I have caused you to attach $25 to every bottle of wine you might purchase. In case 1, it is not so clear what value you attach to bottles of wine you might purchase.

Your choice of wine is likely to be quite different in case 1 as compared with case 2. In particular, in case 2, a rational, self-interested individual would seek to maximize $(25-p), where p is the price paid for the bottle of wine, or equivalently, to minimize $p. Case 1involves a quite different problem with a less clear objective; homegrown preferences are uncontrolled for.

Sufficient Conditions for Inducing Values

1. Nonsatiation (Monotonicity) Let V(m,) be the unobserved utility of the subject over the reward mechanism, m and everything else, x. It is asserted that Vm > 0 for all (m, ) combinations.

Money seems to be a good choice for m. Course grades do not.

2. Salience. m is determined by the individuals actions (and possibly those of others) in a precise fashion that is in accordance with the environment/institution being induced.

Example. Suppose in theory, U=ln(x). Suppose a choice of x yields a monetary reward of m(x). Salience requires that m(x)-m(x) ~ln(x)-ln(x).

Sufficient Conditions, continued Care must be taken to avoid excessive flatness in the reward

space. Duffys nickel rule for discrete choices (subject to change with US inflation): if, in theory, U(x)>U(x) then m(x)-m(x)$0.05.

3. Dominance. Changes in subjects utility depend primarily on changes in the reward mechanism m, and not other factors, .

Other factors: payments earned by other subjects, the experimenters preferences, transaction/decision costs. These must all be minimized, e.g. by not revealing others payoffs, the aims of the experiment, making decisions as clear/simple as possible. To the extent they cannot be minimized, increase m.

Controlling for Risk Attitudes When rewards are only stochastically related to subjects

action choices, various individual attitudes toward risk may affect decision- making. How to control?

Elicit subjects risk attitudes/preferences using pretests/surveys prior to the experiment. Conduct experiment using only the appropriate subset of subjects, e.g., those who are risk neutral.

Binary lotteries, originally developed by Roth and Malouf (1979) and generalized by Berg et al. (1986), are a means of inducing a particular attitude toward risk, regardless of the subjects innate preferences over uncertain payoffs. The original idea (Roth and Malouf (1979)) was to induce risk neutral

behavior by converting payoffs into points (or lottery tickets) of winning a large prize (as opposed to a small prize the binary lottery).

Binary Lottery Pick two monetary prize levels, x2 > x1. Suppose subjects payoffs are expressed in points

q [0,Q], higher payoffs = more points. Suppose that after points have been determined, the subject

earns monetary prize x2 with probability q/Q and x1 with probability (1-q/Q).

Expected utility is: q/Q(U(x2))+(1-q/Q)(U(x1)) where U is the subjects unknown utility function.

WLOG, we may normalize U(x2)=1, U(x1)=0. Expected utility reduces to q/Q, i.e., it is linear in points! A

subject with a concave U function who is paid in points (lottery tickets) should act as a risk neutral agent in decisions where payoffs are in points (lottery tickets).

Example From Duffy and Feltovich (2002, 2005) Prisoners dilemma game:

Suppose row plays C and column plays D. Row has a 10 percent chance of earning $1 (90 percent chance

of earning $0) while column has a 80 percent chance of earning $1 (and a 20 percent chance of earning 0) (Q max was set at 100, even though not attainable).

Draw a random number between 1 and 100 to determine earnings.

40, 4080, 10D

10, 8070, 70CDC

Berg et al. (1986): Any Preference Function G(q) Can be Induced

Problems with Binary Lotteries Little or no difference between decisions made by

subjects who earn monetary payoffs versus those who earn lottery points (lottery tickets). But see Prasnikar (2001).

Extra layer of complexity/time to describing implementing the binary lottery.

Lab gambles are often so small that, even for risk averse subjects, the curvature over payoff possibilities can be viewed as approximately linear. So many experimenters ignore homegrown risk attitudes

and assume subjects act as though they were risk neutral.

Inducing Discounting Suppose you want to implement an infinite horizon, e.g.,

consistent with theory. Obviously not possibly in the laboratory, but you can

implement the stationarity associated with an infinite horizon together with discounting.

Add a constant probability that a sequence of decision-making continues and it ends with probability (1-).

Use a randomization device at end of each decision to determine whether a sequence continues. Expected duration of a sequence is a constant 1/(1-)

decisions from each decision stage reached. Problem: the sequence could continue beyond the time

allocated for the experiment!

Eliciting Beliefs Suppose you want to obtain subjects subjective

probabilities with regard to certain outcomes before they choose actions. How can you get them to report these beliefs truthfully? E.g. the probability their opponent will choose a particular

action; the probability they will be pivotal in an election. Ask them to state their beliefs prior to action choices. Assign a certain (small) fraction of payoffs to belief

accuracy using a scoring rule-a means of assigning a numerical score to the subjects forecast-and explain this payoff function in the experimental instructions.

Quadratic Scoring Rule Most frequently used, e.g., in meteorology. Subject reports her discrete probability distribution

p=(p1,p2,,pn) that n mutually exclusive events i=1,2,..n occur, i pi =1.

Let Ii=1 if event i occurs, 0 otherwise.

Max = a, min =a-2b. So, e.g., we might set a=1,b=1/2 Binary case, n=2, p1=p, p2=(1-p). If event 1 occurs (I1=1), pay 2p-p2, otherwise if event 2

occurs, (I2=1), pay 1-p2. A proper scoring rule induces truthful revelation of

probabilities.

])(...)()[()( 22222

11 nn IpIpIpbapQSR +++=

Example: Suppose you think the probability of a binary event is

p=.25. If the event occurs, your payoff is 2p-p2 = $.44 and if it does not you earn 1-p2=$.94.

Alternatively, a p=.4 earns $.64 if the event occurs and $.84 otherwise.

If you think the true probability is .4, reporting p=.4 yields you .4($.64)+.6($84)=$.77. Similarly, reporting p=.25 when you think the true probability is .4 yields you .4($.44)+.6($.94) =$.74. So, if you are risk neutral, you would have the incentive to report your true belief that p=.4.

Eliciting Strategies Use Seltens (1967) strategy method: Ask subjects to

provide strategies instead of playing actions. A strategy is a complete contingent plan of action

detailing what decision is to be made at every information set.

With a player strategy, one is able to observe all possible choices, not just ones that arise at information sets actually reached.

Easy to incentivise: just tell subjects that, once their strategy is specified, you will play it for them and they will earn payoffs in accordance with the decisions their strategy makes.

Entry Game Example Consider a game involving a binary decision, enter or not enter. Payoffs:

1+2(c-m) if enter1 if dont enter.

m is the number of the n subjects choosing to enter.c is a capacity constraint variable, varied by the experimenter.

In equilibrium, m=c. In the strategy method adopted by Seale and Rapoport (Experimental

Economics 2000), in each of 50 periods, subjects were asked to state their decision, enter or not enter for various capacity values, i.e., c=1,3,5,19. n=20 subjects.

Then a random odd number c-value was drawn. Subjects strategies were used to determine their actions. The procedure of strategy elicitation was then repeated (50 periods total).

Aggregate results using strategy method are similar to those found using decisions, but strategy profiles reveal that randomization is not what subjects are doing; instead they use various cutoffs, e.g. enter if c>8.

Problems with Strategy Method?

Asking subjects to think about strategies may alter the way they play the game. Alternative approach: try to infer strategies from actions

(Duffy and Engle-Warnick (2003)).

Change in timing of decisions may affect the game, relative to sequential moves.

Hybrid approach: adaptive strategy method. Specify a strategy. See in real time what it prescribes but adjust strategy if not to your liking before the action is played. Adjusted strategy becomes new strategy.

Matching Protocols Partners (fixed matches) versus Strangers (random matches) is thought

to induce different behavior in repeated games. Partners matching protocol is thought to be more subject to

reputational/dynamic game considerations. At the same time, the stability of having the same partner might promote faster learning, e.g. of Nash equilibrium.

Strangers random matching protocol is thought to frustrate cooperative/dynamic game strategies, and implement one-shot games. This may not be true in finite populations (Kandori (1992)).

Turnpike protocol due to Cooper et al. (1996) avoids contagious interaction effects. Divide the subject pool into two equal sized groups A and B. Each member of group A plays each member of group B exactly once.

Experimental evidence on behavior under partners and strangers matching protocols in n-player public good games is mixed. (see Andreoni and Croson (2002)).

Stronger evidence that random matching works to frustrate cooperation is reported by Duffy and Ochs (2005)) for normal-form 2-player games.

Controlling for Heterogeneity: Between vs. Within Subject Designs

Between subject designs, different subjects are used in treatments A, B.

What if you have relatively more outlier subjects participating in treatment A, (e.g., subjects who got no sleep the night before the experiment) and these subjects behavior lead you to find no difference between treatments A and B.

Use a within-subjects design. Have all subjects play both treatments, e.g. first A, then B, or the reverse (avoid order effects).

In a within-subjects design, each subject serves as her own control group (Camerer (2003), p. 41); stronger statistical inferences are possible.

Problem: fewer observations per treatment in a within subjects design than in a between subjects design.

Controlling for the Other Factors Experimenter demand effects. Subjects try to satisfy

the experimenters demand for a treatment effect. Avoid context, use neutral language. Have uninformed

research assistants run your experiments. Use automated scripts or instructions. May be unavoidable.

Concerns about the payoffs of others, fairness. Avoid revealing payoffs earned by others pay subjects in

private. Use double-blind procedures for paying subjects. Often the payoffs of others is part of the information

revealed to subjects, as in game theory experiments. Still, one can make sure that subjects are anonymous to one another.