1

Nudge nudge and say no more:

How do reminders in the waiting room affect antibiotics use?

Linnea Wickström Östervall*†

October 2013

Prudent use of antibiotics is important to delay the spread of resistance. However, guidelines on prudent use are not adhered to. Patients tend to over-demand antibiotics; doctors overprescribe them. This paper assesses whether reminders to out-patients of the association between antibiotics use and resistance can reduce antibiotics use, and whether phrasing of the reminder matters. The intervention was carried out as an RCT across 31 clinics during the flu season in Stockholm, early March 2012. Two-level randomization was employed; the level of randomization was the clinic for distributing reminders – framed as questionnaires – and the individual patient for type of reminder. Two different phrasings of the reminder are used, in addition to a placebo questionnaire. At treated clinics antibiotics use was 12.6 percent lower than at control clinics. This is a substantial effect. There is, however, no difference in effect between the individual patients, regardless if they received any of the reminders or a placebo. This suggests doctors, not patients, were affected by the reminders. There are two possible channels: either the intervention lends support to doctors in arguing for prudent use of antibiotics, or doctors adhere more strictly to guidelines on prudent use when they know they are being monitored. A follow-up experiment was carried out fall 2012 to investigate this further. The results suggest monitoring is at least not the full story; interaction between doctor and patient seems highly important. Keywords field experiment, nudge, reminder, common resources, antibiotics use, resistance

* Stockholm University, linnea.wickstrom.ostervall@ne.su.se † Thanks to Astri Muren, Jakob Svensson, and Strama Stockholm, and to seminar participants at Stockholm as well as Lund University, and the 7th as well as 8th Nordic Conference on Behavioral and Experimental Economics. Funding from The Royal Swedish Academy of Sciences and from the Jan Wallander and Tom Hedelius Foundation is gratefully acknowledged. All mistakes are my own.

2

Introduction: Background, purpose, & previous literature Medical doctors as well as researchers in microbiology have long been aware of antibiotic

resistance. It’s not a matter of if; it’s a matter of when bacterial strains resistant to the

currently available antibiotics emerge. (Goossens et al 2005, OTA 1995) There is general

acknowledgment that prudent use of antibiotics may at least slow the development of

resistance (Phillips 2001). In their Threat Report 2013 the American Centers for Disease

Control and Prevention stress prudent antibiotics use as one of four core actions to prevent

antibiotic resistance (CDC 2013). In as much as 50 percent of the cases antibiotics may be

used when not needed or in the wrong – and hence inefficient – dose (CDC 2013). The

important question then is what can be done about this over-usage. Guidelines for doctors on

prudent use are not adhered to. General information to the public at large also does not seem

efficient in promoting prudent use. Across the county of Stockholm antibiotics use is

positively correlated with socioeconomic status of an area. (SMI & Strama 2009 & Strama

Stockholm 2011) A recent reform in Stockholm increased the competition for patients

between clinics; Fogelberg & Karlsson (2012) show that this led to an increase in antibiotics

prescriptions. Independent of the exact mechanism, it seems the prescription pattern might not

follow the health status but rather other characteristics among the patients in different areas.

This suggests that the patients’ capacity is important to antibiotics use.

The purpose of this study is to examine if antibiotics use could be reduced by reminders to

out-patients (or in the case of children: guardians’ of patients1) in the waiting room of the

association between antibiotics use and resistance, and to examine whether phrasing of the

reminder matters. Patients are given a short questionnaire, with a brief definition of resistance,

and simple questions on antibiotic resistance and effectiveness of treatment with antibiotics.

The reminder is given as a statement, inserted among the questions, to which the patients

respond whether they are aware of this fact or not. The reminder is framed in terms of either

the collective or the individual risk increase. One third of patients at treated clinics receive a

placebo questionnaire. The study is designed such that it is salient to doctors at treated clinics,

as doctors fill out the back page of each questionnaire. Since the results indicate that the

reminders have an effect at clinic level, but there is no differential effect if the patient

received a reminder or the placebo, a follow-up experiment is added, to shed some light on

1 No one under the age of 18 took part in the study.

3

whether monitoring or (perceived) support in the discussion with the patient is the channel at

work.

Antibiotics are by prescription only in Sweden. Thus, the doctor formally has the final say,

but in practice apparently the behavior of both doctors and patients jointly determine whether

antibiotics are prescribed or not. A reminder may affect the behavior of patients only, doctors

only, or both.

Nudges have been used to attempt at helping individuals overcome self-control problems, as

well as forgetfulness and inattentiveness (Altmann & Traxler 2012 and the references therein,

Thaler & Sunstein 2008). Here a nudge in the form of a reminder is used. This is in spirit

similar to Armstrong et al (2009) who study the effect of text message reminders to increase

sunscreen use, as well as Altmann & Traxler (2012) who look at reminders and dental care,

and also relates to Karlan et al (2011) who evaluate the effect of reminders on savings

behavior with and without commitment devises. Armstrong et al (2009) compare a group

receiving text message reminders to a control group without reminders. Sunscreen use of both

groups was monitored using an electronic adherence monitor. This monitor could potentially

act as a reminder in itself. Nonetheless, they find that text message reminders have a strong

and significant effect on sunscreen use. (Armstrong et al 2009) Altmann & Traxler (2012) use

three different reminders, which are sent to individuals who are due for a dental check-up:

neutral, and with either negatively or positively framed information on the benefits of

preventive dental care. The control group receives no reminder. They find the framing has no

effect, while reminders as such do. (Altmann & Traxler 2012) The results in both Armstrong

et al (2009) and Altmann & Traxler (2012) suggest that reminders could potentially be

important tools in improving behavior in the health domain. This study however differs from

these cases as antibiotics use has an important common resource component.

Each course of antibiotics taken contributes to an increase in the prevalence of resistance.

Basically, bacteria learn to defend themselves against antibiotics. Or rather, through the

evolutionary process of survival of the fittest, bacteria able to defend themselves against

antibiotics are favored when we take antibiotics. (See e.g. CDC 2013) As resistance develops

and spreads, fewer and fewer of the existing types of antibiotics remain effective against

common strains of bacteria. In this sense, effective antibiotics are a currently finite common

4

resource, as my use lessens your prospects of enjoying the benefits.2 In other words, I do

neither carry the full costs of my personal use, nor the full benefit of prudent use, leading to

the well-known problem with the “tragedy of the commons” (Hardin 1968, Fehr & Leibbrandt

2011).

Method: Experimental design & empirical strategy The causal effect of reminders on antibiotics use is assessed in a randomized field experiment.

Two-level randomization is employed. First participating clinics are randomly assigned to

treatment and control. Then, at the clinics in the treatment group, type of treatment or placebo

is randomly assigned to patients, with each patient receiving one of the three different

versions of the questionnaire. This could perhaps best be described as the field experiment

extension of randomization within sessions. To make the intervention possible, the clinics

must agree to participate. Therefore, clinics were first asked for consent to be included in the

study, and from those who accepted a treatment group was randomly drawn. Local emergency

wards may have special characteristics, why it is important these are represented in both

treatment and control. Hence, the sample is stratified according to type of clinic, in order to

make sure such observable characteristics which may affect outcomes are balanced in the

groups compared.

This study is carried out in collaboration with Strama Stockholm. Strama is short for the

Swedish Strategic Program against Antibiotic Resistance. It was founded in 1994, and

receives financial support from the Swedish government since 2000 (Mölstad et al 2008).

Strama is an advisory body with expertise on antibiotic use and resistance.

The intervention in the treatment group is a four page questionnaire (one folded sheet) to fill

out, with simple questions on antibiotic resistance and effectiveness of treatment with

antibiotics. The front page includes a brief definition of resistance. The second and third page

is to be filled in by the patient/guardian before seeing the health provider. The back page has

follow-up questions for the doctor to fill out after seeing the patient. Two different versions of

the treatment questionnaire are used. The reminder is given as a statement, inserted among the

questions, to which the patients respond whether they are aware of this fact or not. The

2 There is of course the possibility that new types of antibiotics can be developed. Prudent use of antibiotics aims at buying us time – hopefully enough time to develop such new types.

5

reminder is framed in terms of either the collective or the individual risk increase. The

phrasings used are:

- Are you aware that the risk of antibiotic resistance in society increases with the

amount of antibiotics used? (T1)

- Are you aware that the risk of you being affected by bacteria resistant to antibiotics

increases with each course of antibiotics you take? (T2)

Full questionnaires in English are included in Appendix I.3

The questionnaires serve as reminders not only to the patients, but also to the doctors, since

they are to fill out the back page. The intervention is hence salient also to the doctors. To

allow controlling for the effect of reminding the health providers, there is also a placebo

questionnaire. That is, all in all there are three versions of the questionnaire: two different

treatment questionnaires and one placebo. The placebo questionnaire has questions on

smoking habits to the patients but an identical back page for the doctor to fill out after seeing

the patient. It does not have the definition of resistance on the front page. That is, the placebo

questionnaire is a reminder to doctors only, not to the patients.4 5

The study was piloted in two rounds during spring 2011, to test the questionnaires. In the

pilot, the overall response rate was high; the few refusals that did occur were mainly because

the patient was so ill he/she had to be transferred to an ER. The experiment was carried out

during the flu-season in Stockholm, in early March 2012. Typically, antibiotics use is at its

highest during and just after flu-season.

The setting – waiting rooms at clinics – brings in particular the importance of anonymity to

the fore.6 Therefore, no information that would enable identification of any individual is

3 English versions of the questionnaires were not used in the study. Some clinics however reported having a large share of patients who only understands Arabic; hence questionnaires in Arabic were distributed to all participating clinics along with the Swedish. The same three versions of the questionnaires were used also in Arabic. The translation to Arabic was done by an authorized translator; the translation to English included in Appendix 1 was done by the researcher. Swedish versions are available on request. 4 Ideally, there should also be a treatment of patients only, but as individual data on antibiotics prescriptions then are not available (the doctor is the source of this information and asking for it is a reminder), adding this version of treatment demands an unfeasibly large sample for this study; there simply are not enough clinics in Stockholm to do this analysis at the clinic level. 5 Hypothetical questions to elicit time preferences and other-regarding preferences are included in all questionnaires. The responses to these questions are used in a companion paper: “Altruism, time preferences, and common resource conservation: A field study on preferences and antibiotics use”. 6 This project has been submitted to the Ethical Review Board in Stockholm. The board decided that the legislation concerning ethical review is not applicable to this project. In an advisory statement they gave an

6

asked. When the questionnaires reach the researcher, there is no way of identifying any of the

respondents. Special care is taken to minimize the possibility to identify individual

respondents already at the clinics. Thus only gender and approximate age is asked, no name

or personal number.7 Nonetheless, as the doctor is to fill out the back of the questionnaire, the

patients may feel there is a risk their doctor looks at their answers and makes some sort of

assessment or judgment based thereupon. Unfortunately there is no other way of obtaining

individual prescription data and in particular no other way of linking antibiotics use to the

questionnaire responses.

A first approach to examine the research question if antibiotics use is affected by reminders is

to look at clinic level data and estimate the aggregate effect of the reminders to the two-thirds

of the patients receiving either of the treatment questionnaires and all doctors. Aggregate data

on antibiotics prescriptions by clinic is provided by Strama Stockholm, and collected by

Stockholm county council, unit for assessment and development. At the aggregate level, that

is the clinic level, the study will be informative of the total effect of the reminders.

Using aggregate data and the information on whether patients and/or doctors at the clinic are

treated, the following equation is estimated:

������� = + �� � × ��� �� + � � + ���� � + ��� 1

where ��� is the number of antibiotics prescriptions at clinic c in time period t, T is a dummy

equal to one if the clinic c is in the treatment group, POST is a dummy equal to one from the

implementation week (week ten 2012). Clinic and weekly time fixed effects are subsequently

added to this base specification.

In a second approach the effect of reminders on antibiotics use is explored using the

individual patient data from the questionnaires. As explained above, the questionnaires serve

as reminders not only to the patients, but also to the doctors. Hence, using the patient data this

study will be informative on the difference in impact between reminding the patients as well

as doctors (treatment questionnaires) and reminding only the doctors (placebo

informal approval of the experimental design. It is unlikely that the project would have been approved had anonymity of participating patients not been carefully considered. 7 The question on age is to fill in year of birth, but with the additional comment that decade of birth is sufficient.

7

questionnaires). The effects of the two different phrasings as well as the placebo are

examined. Equations of the following type are estimated:

�� = + �� 1� + �� 2� + ��������� + ���ℎ ��� + ����!�� + �#�$ℎ$�%� +&� 2

where �� is a dummy equal to 1 if antibiotics are prescribed to the individual patient i; T1 and

T2 are dummy variables equal to one if the questionnaire was focusing on the collective risk

or the individual risk, respectively. The controls included are: gender, which is one for

women; child, which is one if taking a sick child to see the doctor; educ which is a discrete

variable with four possible values for the educational attainments representing only

compulsory school, senior high school, some college, and college degree; and cohort, which

are a set of dummy variables for decade of birth. Only patients who are seeing a doctor

because of an infection are included.

It is hypothesized that the reminders will reduce the use of antibiotics, and that the phrasing

focusing on the individual risk will have a greater impact, as it concerns the more immediate

connection between own behavior and health risks associated with resistant bacteria. Thus,

the coefficient on the interaction term in equation 1 and on T1 and T2 in equation 2 are

expected to be negative, with │T1│< │T2│.

Data and descriptive statistics 211 clinics, including 21 local emergency wards, in the county of Stockholm were invited to

participate in the study. The clinics were contacted by post and/or fax and in most cases also

by phone. 32 clinics agreed to participate. Questionnaires were distributed to patients at 19 of

these clinics. These were randomly drawn. To minimize any potential effect of treatment

status on prescriptions prior to the intervention, the clinics in the treatment group were

informed as late as possible; only a few days before the questionnaires were delivered to them

and less than a week before the intervention was to be carried out. This, however, potentially

increased the risk of last-minute drop-outs, why more than fifty percent of the participating

clinics were assigned to the treatment group. As it turned out, only one of them dropped out

during the week, leaving 18 clinics in the treatment group.8

8 Prescriptions at the clinic that dropped out are not in the data.

8

Two sets of data are used: weekly data on prescriptions by clinic, and individual patient data

from treated clinics. To estimate equation 1, the total effect of the reminders, data on

antibiotics prescriptions by clinic and week for participating clinics is used. This aggregate

clinic data is collected from all pharmacies by Stockholm county council, unit for assessment

and development, and provided to the researcher by Strama Stockholm. Antibiotics

prescriptions by clinic for week 1-13 for the years 2011 and 2012 are in the data.9.10). For one

clinic data is missing for weeks 3 and 4 2012. Hence, total number of observations is 804 for

the 31 clinics included: 18 clinics remain in the treatment group, after one of the 19 randomly

selected dropped out. Thirteen participating clinics form a control group. Mean number of

antibiotics prescriptions by clinic and week is 40.52 with a standard deviation of 22.35. Min is

1; max 128.

In the main specification three weeks before the intervention – weeks 6, 7 and 8 2012 are

included, as well as weeks 10 and 11. Week 10 was the main intervention week, but during

week 11 the researcher called, emailed and faxed the clinics in the treatment group and also

visited them to collect the questionnaires. That is, during week 11 the intervention was still

very much salient at the treated clinics. In addition data is for prescriptions filled. Thus, some

of the prescriptions written during week 10 might be filled week 11. Week 9 is when clinics

in the treatment group are informed about the outcome of the random selection to treatment,

and boxes with questionnaires are delivered to them. It is thus not obvious how this might

affect prescriptions. Data from week 9 is dropped in the main specifications, but included in a

robustness check.

Key to identifying a causal effect of the reminders on antibiotics use with data at the clinic

level, i.e. in estimating equation 1, is that the treatment group and the control group display

parallel trends in prescriptions prior to the intervention. This is tested by running the

following specification for each week six to eight 2012:

������� = + � � + ��� 3

9 This is the first thirteen full weeks of the years 2011 and 2012. A week starts Monday and ends Sunday in this data. For 2011, the first week began January 3rd; for 2012, January 2nd. 10 In accordance with the standard Strama generally follows antibiotics with ATC code J01 are included, except methenamine.

9

where ��� is the number of antibiotics prescriptions at clinic c in time period t, T is a dummy

equal to one if the clinic c is in the treatment group. As shown in Table 1 there is no

systematic difference in prescriptions the weeks prior to the intervention at clinics in

treatment and control; the coefficient on the dummy for being in the treatment group is

insignificant.

(1) (2) (3) VARIABLES Week 6 Week 7 Week 8 treatgroup -0.0770 -0.0785 -0.0982 (0.246) (0.231) (0.219) Observations 31 31 31 R-squared 0.004 0.005 0.008

Table 1 Dependent variable is ln(prescriptions). Treatgroup is a dummy equal to one for the clinics in the treatment group. Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1

As an additional test, a version of equation 1 is estimated, with POST switching on week 8.

This is estimated on data for weeks 6-8. This is a placebo test, and would capture if there was

a preexisting differential trend in the treatment group. The coefficient on the interaction

between the dummy for treatment group and a dummy equal to one week 8 2012 is

insignificant; that is, there is no evidence of any such differential trend. (See Table 2)

(1) VARIABLES Trend weeks 6-8 t_w8 -0.0204 (0.0846) week8_12 0.0671 (0.0589) treatgroup -0.0777 (0.234) Observations 93 R-squared 0.007

Table 2 Dependent variable is ln(prescriptions); t_w8 is an interaction between the dummy for treatment group and a dummy equal to one week 8 2012; treatgroup is a dummy equal to one for the clinics in the treatment group. Robust standard errors clustered by clinic in parentheses. *** p<0.01, ** p<0.05, * p<0.1

To estimate equation 2, data at the individual patient level is used. This patient data is

collected via the reminders/questionnaires from patients at those participating clinics which

were randomly selected to the treatment group. That is, individual data on antibiotics

prescriptions is available for those patients whose doctors have filled out the back page of a

10

questionnaire, i.e. from clinics in the treatment group. Data from 1362 respondents is

available; their doctors classify 453 of these as seeking medical attention due to an infection

(i.e. these are potential users of antibiotics). 142 respondents are prescribed antibiotics. The

response rate was acceptable at most clinics.

Clinics in treatment group

Variable Observations Mean Std. dev. Min Max

Poster11 14 0.286 0.469 0 1

Visits 18 325 162 85 700

Observations 18 75.7 59.0 11 213

Response rate 18 0.277 0.183 0.0354 0.534

Table 3 Descriptive statistics for clinics in treatment group.

The response rate varies slightly for the three different versions of the questionnaire, with the

lowest response rate for the placebo questionnaire (in total 438 respondents, compared to 474

and 450 respectively for the treatment questionnaires). The questionnaires were sorted, with

every third being of each type, first treatment one, followed by treatment two, then a placebo

questionnaire, followed again by a treatment one etc. It is unlikely that every third patient

entering the clinic would have special characteristics. More likely the response rate was

affected by something about the placebo questionnaire itself: Either the questions on smoking

habits were perceived as more personal or less important, or both, leading fewer patients to

care to fill the questionnaire out and/or want to give it to their doctors. This would be of

concern, as it could potentially bias the results. However, these differences in response rates

are not statistically significant.

In Table 4 means and differences in means for share of patients seeking medical attention due

to an infection as well as for a set of control variables for the group of patients who received a

placebo questionnaire or either of the treatment questionnaires are presented. All differences

in means are insignificant. Year of birth is measured with some error, as respondents have the

option to only state decade of birth (in order to enhance privacy). Means by decade of birth

are also not significantly different (not reported in table).

11 This is a dummy variable equal to one if the clinic had a poster from Strama in the waiting room when questionnaires were picked up on the Monday (i.e. the first day) after the intervention.

11

Individual questionnaires – All respondents

Variable Mean Placebo Mean Treatment Diff

Infection12

(395 & 841 obs)

0.342

(0.475)

0.378

0.485

-0.0363

(0.0292)

Child/children

(412 & 899 obs)

0.211

(0.0201)

0.188

(0.0138)

0.0232

(0.0244)

Woman

(412 & 879 obs)

0.590

(0.0243)

0.601

(0.0165)

-0.0109

(0.0294)

Compulsory schooling only

(417 & 887 obs)

0.125

(0.0162)

0.0981

(0.00999)

0.0266

(0.0190)

Year of birth

(422 & 898 obs)

1962

(0.848)

1963

(0.578)

-1.31

(1.03)

Table 4 Means by placebo/treatment. Two sample t-test with unequal variances **) Significant at least at 5% level Ha: diff ≠ 0 Pr(|T| > |t|) =<0.05 *) Significant at 10% level Ha: diff ≠ 0 Pr(|T| > |t|) =<0.1 Standard errors in parenthesis

Thus, to conclude this discussion, the development in treatment and control group is similar

prior to the intervention. While most clinics have an acceptable response rate, some clinics

have a low to very low response rate. Nonetheless, the data provides individual prescription

data and information on some potentially important characteristics.

Results With the first approach, the causal effect of reminders on prescriptions at clinic level is

estimated using data on prescriptions by clinic and week for weeks 6, 7 and 8 prior to the

intervention and intervention weeks 10 and 11. Data from week 9 is dropped in the main

specifications, but included in a robustness check. As mentioned above, week 9 is when

treated clinics are informed, and boxes with questionnaires are delivered to them.

Coefficients of main interest are those on treated, which is an interaction with a dummy for

after intervention begun and a dummy for being in treatment group. This is the estimated

aggregate causal effect of reminders on antibiotics use. The coefficient is unaltered when

clinic and time fixed effects are included. The significance level varies slightly, from the five

percent level in the base specification (p-value 0.035) to the ten percent level in columns two

12 A dummy variable equal to one if doctor responded the reason for patient’s visit is an infection.

12

and three, where first clinic fixed effects and then clinic and time fixed effects are included

(p-values 0.056 and 0.059, respectively). (See Table 5)

(1) (2) (3) VARIABLES Base spec Clinic FE Clinic & time FE treated -0.126** -0.126* -0.126* (0.0570) (0.0634) (0.0642) treatgroup -0.0845 (0.224) POST -0.0695* -0.0695 (0.0372) (0.0413) Clinic FE no yes yes Time FE no no yes Observations 155 155 155 R-squared 0.028 0.919 0.921

Table 5 Weekly clinic data. Dependent variable is ln(prescriptions). Treated is the variable of main interest. This is an interaction term with a dummy = 1 from week 10 2012 and a dummy for being in treatment group. Robust standard errors clustered by clinic in parentheses *** p<0.01, ** p<0.05, * p<0.1

The intervention is associated with substantially less antibiotics use in the treatment group.

These are estimates of the joint effect of reminding both the patients and the doctors. Or, to be

more specific, these are estimates of the effect of reminding the two-thirds of the patients who

receive either of the treatment questionnaires, and reminding all doctors. The effect is a

reduction by 12.6 percent. This is a large effect, in particular if it could be sustained over a

longer period of time. As will become clear below, this effect is by and large an effect of

reminding the doctors.

We now turn to the second approach in which the effect of the reminders is explored using the

individual patent data from the questionnaires. Neither of the reminders individually (Table 6,

base specification 1, column 1) nor when grouped together (Table 6, base specification 2,

column 2) have any significant effect on the likelihood the patient is prescribed antibiotics.

That is, the results at the aggregate level come from an effect on the doctors’ behavior. This

conclusion does not change when controls are included. The clinic being above average in

size, having a poster from Strama in the waiting room at the clinic, and the response rate at

the clinic all have no significant correlation to patients being prescribed antibiotics. Gender,

bringing a sick child to see the doctor, and cohort dummies are all insignificant. Education

increases the likelihood of being prescribed antibiotics, when included along with all controls.

This is significant at the ten percent level (p-value 0.086). If education is a proxy for

13

knowledge and/or awareness, this is unexpected. However, if education is a proxy for income,

and thus for the alternative cost of staying home sick, a positive sign for this coefficient is to

be anticipated. All these results are for patients who the doctor classified as seeing the doctor

because of an infection, that is, patients who potentially could be treated with antibiotics. (See

Table 6)

(1) (2) (3) (4) (5) VARIABLES Base 1 Base 2 Controls clinic Controls individual All controls treatment 1 0.0233 (0.0545) treatment 2 0.00755 (0.0548) treat 0.0157 -0.0119 0.0129 -0.0142 (0.0477) (0.0541) (0.0494) (0.0563) woman 0.0593 0.00453 (0.0486) (0.0567) child 0.00137 -0.0177 (0.0533) (0.0606) educ 0.0360 0.0482* (0.0236) (0.0280) poster 0.0680 0.0658 (0.0569) (0.0601) stor -0.0114 0.0123 (0.0496) (0.0525) responserate 0.0797 0.00163 (0.222) (0.233) cohort FE no no no yes yes Observations 445 445 354 420 335 R-squared 0.000 0.000 0.005 0.023 0.034

Table 6 Individual patient data. Dependent variable is a binary variable indicating whether the patient was prescribed antibiotics. Base specification 1 is for the three different types of questionnaires (two phrasings of the reminder and one placebo, with placebo as the reference category). Base specification 2 is with the two treatment questionnaires grouped together (treat = 1). Column 3 includes controls at clinic level, column 4 includes controls at individual level, and column 5 includes all controls. Dependent variable is being prescribed antibiotics. Only patients classified as seeking medical attention because of an infection are included. Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1

Robustness As a first test of the robustness of the results at clinic level (Table 5) the following

specification is estimated:

������� = + �� � × �'�� + �� � × �((�� + � � + )*��+� + ��� 4

14

where T is a dummy equal to one if clinic c is in the treatment group; ON is an interaction

between a dummy equal to one for weeks 10 and 11 and the dummy equal to one for clinics in

the treatment group; OFF similarly is an interaction between a dummy equal to one for weeks

12 and 13 and the dummy equal to one for clinics in the treatment group. Weekly time fixed

effects are included. This is estimated on data for weeks six, seven, eight, ten, eleven, twelve

and thirteen 2012.

The effect on antibiotics use is only there while the intervention is still ongoing and salient.

The effect when the intervention is active is again estimated to be a reduction by 12.6 percent

(p-value 0.037); as expected this estimate is indistinguishable from the coefficients in Table 5.

Once the intervention is no longer active, the effect is gone; the coefficient on “intervention

off” is insignificant (and small). (See Table 7)

(1) VARIABLES Base spec Intervention on -0.126** (0.0577) Intervention off -0.0111 (0.0696) treatgroup -0.0845 (0.227) Time FE yes Observations 217 R-squared 0.030

Table 7 Weekly clinic data. Dependent variable is ln(prescriptions). “Intervention on” is an interacti on between a dummy equal to one for weeks 10 and 11 and a dummy equal to one for clinics in the treatment group. “Intervention off” similarly is an interacti on between a dummy equal to one for weeks 12 and 13 and a dummy equal to one for clinics in the treatment group. Robust standard errors clustered by clinic in parentheses *** p<0.01, ** p<0.05, * p<0.1

Week 9 is “fussy” in the sense that it is the week during which the clinics in the treatment

group are informed and questionnaires are distributed to them. As the effect of reminders on

antibiotics use estimated in the main specifications (See Table 5) apparently is an effect of

reminding the doctors, rather than the patients, one might argue that already knowing that

they are in the treatment group might have an effect on prescriptions. Hence, week 9 might be

viewed as an intervention week.

In Table 8 the main specifications are rerun with week 9 defined as an intervention week. The

coefficients of main interest are those on informed. This is an interaction with a dummy equal

15

to one from week 9 2012 and a dummy for being in the treatment group. This is the estimated

aggregate causal effect on antibiotics use of doctors knowing their clinic is in the treatment

group, and from reminders to patients and/or doctors. The standard errors are larger; informed

is significant at the ten percent level in all three specifications (p-values are 0.054, 0.075, and

0.079, respectively). The coefficient is unaltered across all three specifications. At -11.1

percent it is somewhat lower than the estimated -12.6 percent in the main specification. This

is not surprising, as the clinics are informed mid-week week 9, and it is expected that if there

is any effect on antibiotics prescriptions it is only for the second half of the week.

(1) (2) (3) VARIABLES Base spec Clinic FE Clinic & time FE informed -0.111* -0.111* -0.111* (0.0552) (0.0602) (0.0610) treatgroup -0.0845 (0.224) AWARE -0.0458 -0.0458 (0.0381) (0.0415) Clinic FE no yes yes Time FE no no yes Observations 186 186 186 R-squared 0.023 0.898 0.902

Table 8 Weekly clinic data. Dependent variable is ln(prescriptions). Informed is the variable of main interest. This is an interaction term with a dummy for being in the treatment group and a dummy = 1 from week 9 2012. Robust standard errors clustered by clinic in parentheses *** p<0.01, ** p<0.05, * p<0.1

If data from week 9 is instead included in the pre-intervention period, the point estimate is not

statistically different from that in Table 8, but standard errors increase further, and the point

estimate is no longer significant at conventional levels when including clinic and time fixed

effects (p-values 0.078, 0.105, 0.110).

The main take-away from this is that week 9 is basically semi pre-intervention, semi

intervention week. If it were possible to split week nine in half, assign the first half to pre-

intervention period, and second half to intervention period, we could expect estimates in line

with those in the main specifications. As daily data is not available for this period, this

however cannot be implemented.

16

(1) (2) (3) VARIABLES Base spec Clinic FE Clinic & time FE treated -0.106* -0.106 -0.106 (0.0582) (0.0634) (0.0643) treatgroup -0.105 (0.224) POST -0.0698* -0.0698* (0.0347) (0.0379) Clinic FE no yes yes Time FE no no yes Observations 186 186 186 R-squared 0.025 0.900 0.902

Table 9 Weekly clinic data. Dependent variable is ln(prescriptions). Treated is the variable os main interest. This is an interaction term with a dummy for being in the treatment group and a dummy = 1 from week 10 2012. Data from week 9 is included in the pre-intervention period. Robust standard errors clustered by clinic in parentheses *** p<0.01, ** p<0.05, * p<0.1

As an additional test of the randomization at clinic level to treatment and control group, a

placebo test is carried out, using the weeks 6-8 and 10-11 from the year before the

intervention, i.e. 2011. Comparable data was fortunately available via Strama Stockholm.

Table 10 reports estimations of equation 1 on data from 2011. That we then find no effect is

further evidence that the effect found in 2012 is a causal effect of reminders on antibiotics

use.

(1) (2) VARIABLES Base spec Clinic & time FE placeboeffect -0.0882 -0.0882 (0.0823) (0.0928) treatgroup 0.0427 (0.306) PLACEBO 0.0737 (0.0765) Clinic FE no yes Time FE no yes Observations 155 155 R-squared 0.001 0.925

Table 10 Weekly clinic data. Dependent variable is ln(prescriptions). Placeboeffect is the variable of main interest. This is an interaction term with a dummy = 1 from week 10 2011 and a dummy for being in treatment group. Robust standard errors clustered by clinic in parentheses *** p<0.01, ** p<0.05, * p<0.1

The main conclusion remains: Reminders do have a substantial effect on antibiotics use.

17

Follow-up experiment A follow-up experiment was carried out in November 2012 to explore which of the two

channels – monitoring or facilitating the argumentation – is at work. If the effect of the

follow-up intervention is of similar magnitude as the effect found in the main study, this

would suggest that the channel at work is that doctors adhere more strictly to guidelines on

prudent use of antibiotics when they know they are being monitored. Else, it may be a

combination of the two channels – of monitoring and facilitating the argumentation – or only

that the reminders lend support to doctors in arguing for prudent use.

Treatment in this follow-up study is a letter informing the clinics that their antibiotics

prescriptions would be monitored during week 45, November 5th through November 11th,

2012.13 The letter is sent out just a few days before the stated monitoring, in order to avoid

affecting prescriptions prior to week 45. The week chosen for the follow-up experiment is

similar to week ten, when the main intervention was implemented, as both week ten and week

45 is immediately after a school break. Also week 45 is during the flu season, albeit at its

onset rather than near the peak.

In the follow-up study the clinics from the main study are included. In addition, 71 new

clinics, randomly drawn from the remaining clinics in the county of Stockholm, are included.

By remaining is meant those clinics not included in the main study. 35 of these new clinics

are randomly selected to be treated in the follow-up study, along with the 18 clinics from the

treatment group in the main study.

Follow-up experiment: Data Data on prescriptions is available for the 18 clinics that carried out the intervention in the

spring, and for all thirteen clinics in the control group in the spring. For the 71 new clinics

added, data is available for 33 out of the 35 in the treatment group, and 34 of the 36 in the

control group. That is, in total, with the clinics from the main study, the treatment group

includes 53 clinics (18 plus 35 clinics). Data on prescriptions is available for 51 of these

clinics. The control group includes 49 clinics (13 plus 36 clinics). Data is available for 47 of

these clinics.

13 As the letter is signed by the researcher, a professor, and a representative from Strama Stockholm it is safe to assume this will be understood as monitoring by Strama Stockholm, in collaboration with researcher(s).

18

Follow-up experiment: Results As evident below, the letter did not affect prescriptions week 45. This is true for all treated

clinics jointly as well as the clinics from the spring study only. Treated is again the variable of

main interest. This is an interaction with a dummy that takes the value one once the follow-up

intervention has begun and a dummy for being in the treatment group. Week 41, 42 & 43 are

pre-intervention weeks. In Table 11 all clinics with available data are included. In Table 12

the effect of the follow-up intervention is estimated separately for the clinics that were

included in the main intervention in the spring and the clinics that were only in the follow-up

study. There is no significant effect for either sample.

(1) (2) (3) VARIABLES Base spec Clinic FE Clinic & time FE treated 0.0220 0.0220 0.0220 (0.0455) (0.0507) (0.0509) treat -0.207 (0.135) POST -0.00612 -0.00612 (0.0342) (0.0382) Clinic FE no yes yes Time FE no no yes Observations 490 490 490 R-squared 0.020 0.910 0.910

Table 11 Weekly data. All clinics included. Dependent variable is ln(prescriptions). Treated is the variable of main interest. This is an interaction term with a dummy = 1 from week 45 2012 and a dummy for being in treatment group. Robust standard errors clustered at clinic level in parentheses *** p<0.01, ** p<0.05, * p<0.1

(1) (2) (3) (4) Spring clinics only New clinics only VARIABLES Base spec With clinic FE Base spec With clinic FE treated -0.0804 0.150 -0.213 -0.0267 (0.322) (0.0956) (0.158) (0.0570) Clinic FE no yes no yes Time FE yes yes yes yes Observations 155 155 335 335 R-squared 0.004 0.902 0.010 0.916

Table 12 Weekly clinic data. In columns 1 and 2 only the clinics that were also in the main intervention in the spring are included. In columns 3 and 4 only the new clinics are included. Dependent variable is ln(prescriptions). Treated is the variable of main interest. This is an interaction term with a dummy = 1 from week 45 2012 and a dummy for being in treatment group. Robust standard errors clustered at clinic level in parentheses *** p<0.01, ** p<0.05, * p<0.1

19

The main study in the spring required the consent of included clinics, while this follow-up did

not as no effort on the part of the clinics was needed. Some of the new clinics had declined

participating in the main study. As these potentially have special characteristics which might

affect their response to the letter, controlling for this might be important lest these might drive

the results. This is examined in the specifications presented in Table 13. Neither looking

separately at the clinics whose head declined participation in the spring (see columns 1 and 2),

nor excluding these clinics makes any difference to the over-all conclusion that the letter did

not affect prescriptions week 45 (see columns 3 and 4), as the large and significant effect

found in column 1 disappears once clinic fixed effects are included.

(1) (2) (3) (4) NO clinics only Without NO spring VARIABLES Base spec With clinic FE Base spec With clinic FE treated -0.659** -0.0179 -0.0953 0.0304 (0.302) (0.144) (0.155) (0.0550) Clinic FE no yes no yes Time FE yes yes yes yes Observations 85 85 405 405 R-squared 0.105 0.888 0.003 0.918

Table 13 Weekly clinic data. In columns 1 and 2 only the clinics whose heads’ said no to participating in the spring study. In columns 3 and 4 those clinics are excluded. Dependent variable is ln(prescriptions). Treated is the variable of main interest. This is an interaction term with a dummy = 1 from week 45 2012 and a dummy for being in treatment group. Robust standard errors clustered at clinic level in parentheses *** p<0.01, ** p<0.05, * p<0.1

The lack of an effect could indicate that the treatment in this follow-up study – that is the

letter – was not salient enough to affect weekly prescriptions. As it might have had a short

term impact, daily prescriptions for Monday and Tuesday week 45 are also explored. There is

no such short term impact. (Results not shown; available on request.) Thus the follow-up

experiment has strengthened the conclusion that the interaction between doctor and patient is

highly important to antibiotics use.

Concluding remarks The results from the analysis of clinic level data from the main study suggest that the

prescription pattern may be affected by reminders. Individual patients, however, seem to not

be affected by the reminders, as antibiotics use does not vary systematically with type of

questionnaire – neither with type of treatment or treatment vs. placebo questionnaire. Thus,

20

the effect found at the clinic level is due to reminders affecting doctors. The reminders pull

the doctors towards more conservative use of antibiotics. There are two possible channels:

either the intervention lends support to doctors in arguing for prudent use of antibiotics, or

doctors adhere more strictly to guidelines on prudent use when they know they are being

monitored.

The follow-up experiment was designed to shed some light on which channel is at work. The

intervention in the follow-up experiment had no significant effect on antibiotics prescriptions

at the treated clinics. This indicates that monitoring is at least not the full story; the interaction

between doctor and patient seems highly important. Reminders have the potential to tilt this

interaction to result in more prudent use of antibiotics, provided they are salient enough at

least to doctors.

21

References Altmann, S., and Traxler, C., (2012), “Nudges at the Dentist: Evidence from a Randomized

Field Experiment” (with C. Traxler), Working Paper, University of Bonn

Armstrong, A., Watson, A. Makredes, M., Frangos, J., Kimball, A., and Kvedar, J. (2009),

“Text-Message Reminders to Improve Sunscreen Use: A Randomized, Controlled Trial Using

Electronic Monitoring”, Arch Dermatol, 145:11, 1230-1236

CDC, (2013), “Threat Report 2013: Antibiotic resistance threats in the United States, 2013”,

US Department of health, available at

http://www.cdc.gov/drugresistance/threat-report-2013/pdf/ar-threats-2013-508.pdf

ECDC/EMEA Joint Technical Report (2009), “The bacterial challenge: time to react

A call to narrow the gap between multidrug-resistant bacteria in the EU and the development

of new antibacterial agents”, available at

http://www.ema.europa.eu/docs/en_GB/document_library/Report/2009/11/WC500008770.pdf

Fehr, E., & Leibbrandt, A., (2011), “A field study on cooperativeness and impatience in the

Tragedy of the Commons”, Journal of Public Economics, 95: 9-10, 1144-1155

Fogelberg, S., & Karlsson, J., (2012), “Competition and Antibiotics Prescription”, IFN

Working Paper No. 949

Goossens, H., Ferech, M., Vander Stichele, R. & Elsevier, M. (2005), “Outpatient antibiotic

use in Europe and association with resistance: a cross-national database study”, Lancet,

365:9459, 579-587

Hardin, G., (1968), “The tragedy of the commons”, Science 162:3859, 1243–1248

Herrmann, M. & Laxminarayan, R. (2010), “Antibiotic Effectiveness: New Challenges in

Natural Resource Management”, Annual Review of Resource Economics, 2:4, 1-14

Karlan, D. S., McConnell, M., Mullainathan, S., & Zinman, J. (2011), “Getting to the Top of

Mind: How Reminders Increase Saving”, unpublished, available at

22

http://karlan.yale.edu/p/Top%20of%20Mind%202011jan.pdf

Mölstad, S., Erntell, M., Hanberger, H., Melander, E., Norman, C., Skoog, G., Stålsby

Lundborg, C., Söderström, A., Torell, E., & Cars, O., (2008) “Sustained reduction of

antibiotic use and low bacterial resistance: 10-year follow-up of the Swedish Strama

programme”, Lancet Infectious Diseases 2008:8, 125–32

OTA, Office of Technical Assessment (1995), Impacts of Antibiotic-Resistant Bacteria,

Government Printing Office, Washington, DC: U.S., available at

http://www.princeton.edu/~ota/disk1/1995/9503_n.html

Phillips, I. (2001), “Prudent Use of Antibiotics: Are our Expectations justified?”, Clinical

Infectious Diseases, 33:3, 130-132

http://www.smittskyddsinstitutet.se/upload/Publikationer/swedres-2010.pdf

SMI & Strama (2009), “Swedres 2009: A Report on Swedish Antimicrobial Utilisation and

Resistance in Human Medicine”, available at

http://www.smi.se/upload/Publikationer/swedres-2009.pdf

Strama Stockholm (2011), “Antibiotikaförbrukning i SLL Kvartal 4 – 2010”, available at

http://www.janusinfo.se/Documents/Strama/rapport_q4_2010.pdf

Thaler, R. H., & Sunstein, C. R. (2008), Nudge: Improving Decisions about Health, Wealth,

and Happiness, New Haven and London

23

Appendix I: Questionnaires

24

All questions on this spread are to be filled out by the patient/accompanying guardian

1) To better our understanding of human reasoning, please check which alternatives you would prefer. Envision you’ll be given a sum of money and check the alternatives you prefer.

N.B. Make one spontaneous choice for each row/for each sub-question a-g.

a) □ 1000 SEK tomorrow or □ 850 SEK in a month

b) □ 1000 SEK tomorrow or □ 1100 SEK in a month

c) □ 1000 SEK tomorrow or □ 1250 SEK in a month

d) □ 1000 SEK tomorrow or □ 1500 SEK in a month

e) □ 1000 SEK in a year or □ 1100 SEK in a year and a month

f) □ 1000 SEK in a year or □ 1250 SEK in a year and a month

g) □ 1000 SEK in a year or □ 1500 SEK in a year and a month

2) Imagine you are given 500 SEK with the option to give some of the money to Save the Children. Fill out your choice: Keep to self: _________SEK Give to Save the Children: _________SEK 3) Are you aware that the risk of antibiotic resistance in society increases with the amount of antibiotics used?

□ Yes

□ No

□ Don’t know

4) Are antibiotics efficient against an infection caused by a virus?

□ Yes

□ No

□ Don’t know

5) How much faster would you think recovery is with a course of antibiotics, provided it is efficient? Number of sick days less: _________ Background characteristics

6) What is the cause of today’s visit?

□ Infection (e.g. sore throat, earache, cough, skin infection, or urinary tract infection)

□ Other

7) Are you here for your own part or with a child?

□ For own part

□ With one or more child/children -> Please fill out number of days the child/children have been ill on this occasion:

# of days: _____________ 8) Age of child: □ 0-2 yr □ 3-6 yr □ 7-17 yr If you are here with more than one child, check age of youngest child.

9) Indicate your gender: □ Male □ Female 10) Which is your highest completed level of education?

□ Compulsory school □ Some college/university

□ Senior high school/equiv. □ Degree from college/university 11) Your year of birth: _________ 12a) Today’s date: ___________ Indicating decade of birth is sufficient.

12b) Clinic/health center/local emergency ward: ________________ For today’s visit

Thanks for your participation! On the back there are a few questions for your doctor. Please fold the questionnaire and give it to your doctor during your visit, thanks.

25

Follow-up questions –to be filled out by the doctor after the visit 14) Is the patient seeking medical attention due to an infection? □ Yes -> Please fill out the remaining questions, thanks.

□ No -> End here

15) Was a quick test taken? □ Yes

□ No

16) Was a culture done? □ Yes

□ No

17) Were antibiotics prescribed? □ Yes (indicate type): ________________

□ No

T1

Health Economic Survey

Your participation is important!

This survey is about antibiotic resistance. Resistance implies that antibiotics are no longer effective on those particular bacteria.

The questionnaire is completely anonymous. Your responses will not be linked to you as a person. Participation is voluntary and does not

affect the care and treatment you receive.

This survey is undertaken in collaboration between Stockholm University and Strama Stockholm

Thank you for taking the time!

26

All questions on this spread are to be filled out by the patient/accompanying guardian

1) To better our understanding of human reasoning, please check which alternatives you would prefer. Envision you’ll be given a sum of money and check the alternatives you prefer. N.B. Make one spontaneous choice for each row/for each sub-question a-g.

a) □ 1000 SEK tomorrow or □ 850 SEK in a month

b) □ 1000 SEK tomorrow or □ 1100 SEK in a month

c) □ 1000 SEK tomorrow or □ 1250 SEK in a month

d) □ 1000 SEK tomorrow or □ 1500 SEK in a month

e) □ 1000 SEK in a year or □ 1100 SEK in a year and a month

f) □ 1000 SEK in a year or □ 1250 SEK in a year and a month

g) □ 1000 SEK in a year or □ 1500 SEK in a year and a month 2) Imagine you are given 500 SEK with the option to give some of the money to Save the Children. Fill out your choice: Keep to self: _________SEK Give to Save the Children: _________SEK 3) Are you aware that the risk of you being affected by bacteria resistant to antibiotics increases with each course of antibiotics you take?

□ Yes

□ No

□ Don’t know 4) Are antibiotics efficient against an infection caused by a virus?

□ Yes

□ No

□ Don’t know

5) How much faster would you think recovery is with a course of antibiotics, provided it is efficient? Number of sick days less: _________ Background characteristics

6) What is the cause of today’s visit?

□ Infection (e.g. sore throat, earache, cough, skin infection, or urinary tract infection)

□ Other

7) Are you here for your own part or with a child?

□ For own part

□ With one or more child/children -> Please fill out number of days the child/children have been ill on this occasion:

# of days: _____________ 8) Age of child: □ 0-2 yr □ 3-6 yr □ 7-17 yr If you are here with more than one child, check age of youngest child.

9) Indicate your gender: □ Male □ Female 10) Which is your highest completed level of education?

□ Compulsory school □ Some college/university

□ Senior high school/equiv. □ Degree from college/university 11) Your year of birth: _________ 12a) Today’s date: ___________ Indicating decade of birth is sufficient.

12b) Clinic/health center/local emergency ward: ________________ For today’s visit

Thanks for your participation! On the back there are a few questions for your doctor. Please fold the questionnaire and give it to your doctor during your visit, thanks.

27

Follow-up questions –to be filled out by the doctor after the visit 14) Is the patient seeking medical attention due to an infection? □ Yes -> Please fill out the remaining questions, thanks.

□ No -> End here

15) Was a quick test taken? □ Yes

□ No

16) Was a culture done? □ Yes

□ No

17) Were antibiotics prescribed? □ Yes (indicate type): ________________

□ No

T2

Health Economic Survey

Your participation is important!

This survey is about antibiotic resistance. Resistance implies that antibiotics are no longer effective on those particular bacteria.

The questionnaire is completely anonymous. Your responses will not be linked to you as a person. Participation is voluntary and does not

affect the care and treatment you receive.

This survey is undertaken in collaboration between Stockholm University and Strama Stockholm

Thank you for taking the time!

28

All questions on this spread are to be filled out by the patient/accompanying guardian

1) To better our understanding of human reasoning, please check which alternatives you would prefer. Envision you’ll be given a sum of money and check the alternatives you prefer.

N.B. Make one spontaneous choice for each row/for each sub-question a-g.

a) □ 1000 SEK tomorrow or □ 850 SEK in a month

b) □ 1000 SEK tomorrow or □ 1100 SEK in a month

c) □ 1000 SEK tomorrow or □ 1250 SEK in a month

d) □ 1000 SEK tomorrow or □ 1500 SEK in a month

e) □ 1000 SEK in a year or □ 1100 SEK in a year and a month

f) □ 1000 SEK in a year or □ 1250 SEK in a year and a month

g) □ 1000 SEK in a year or □ 1500 SEK in a year and a month

2) Imagine you are given 500 SEK with the option to give some of the money to Save the Children. Fill out your choice: Keep to self: _________SEK Give to Save the Children: _________SEK 3) Do you smoke?

□ No or quit at least ten years ago -> Skip to question 6.

□ Has smoked for some period the past ten years, but quit. -> Skip to question 6.

□ Occasionally

□ Yes

4) If your response to question 3 was yes or occasionally, approximately how much do you smoke a typical week?

□ Less than one pack/week □ 4-7 packs/week or more

□ 1-3 pack(s)/week □ Don’t know/ varies a lot

5) If your response to question 3 was yes or occasionally, have you attempted to quit smoking?

□ No

□ Yes, at least once -> Why did you not succeed in quitting?

□ Lack of motivation

□ Weight gain made me resume smoking

□ Quit temporarily due to pregnancy/baby in the family

□ Other: ______________________________________ Background characteristics

6) What is the cause of today’s visit? □ Infection (e.g. sore throat, earache, cough, skin infection, or UTI)

□ Other

7) Are you here for your own part or with a child? □ For own part

□ With one or more child/children -> Please fill out number of days the child/children have been ill on this occasion:

# of days: _____________ 8) Age of child: □ 0-2 yr □ 3-6 yr □ 7-17 yr If you are here with more than one child, check age of youngest child.

9) Indicate your gender: □ Male □ Female

10) Which is your highest completed level of education?

□ Compulsory school □ Some college/university

□ Senior high school/equiv. □ Degree from college/university

11) Your year of birth: _________ 12a) Today’s date: ___________ Indicating decade of birth is sufficient.

12b) Clinic/health center/local emergency ward: ________________ For today’s visit

Thanks for your participation! On the back there are a few questions for your doctor. Please fold the questionnaire and give it to your doctor during your visit, thanks.

29

Follow-up questions –to be filled out by the doctor after the visit 14) Is the patient seeking medical attention due to an infection? □ Yes -> Please fill out the remaining questions, thanks.

□ No -> End here

15) Was a quick test taken? □ Yes

□ No

16) Was a culture done? □ Yes

□ No

17) Were antibiotics prescribed? □ Yes (indicate type): ________________

□ No

P1

Health Economic Survey

Your participation is important!

The questionnaire is completely anonymous. Your responses will not be linked to you as a person. Participation is voluntary and does not

affect the care and treatment you receive.

This survey is undertaken in collaboration between Stockholm University and Strama Stockholm

Thank you for taking the time!