DEPARTMENT OF SOCIOLOGY

Building Simulations From Expert

Knowledge: Understanding Needle

Sharing Behaviour in Intravenous Drug

Use

Edmund Chattoe-Brown (ecb18@le.ac.uk)

Plan of the talk

• What is the issue?

• Some evolving thoughts on the novelty of research methods.

• The BBVsim case study.

What is the issue?

• Simulation is an innovative research method.

• I have been a “participant” in its acceptance in the social science community (SIMIAN).

• Reflexively, (how) can we understand the nature of social science innovation to improve the promotion of “deserving” innovations?

• What ideas do we need to talk about this issue?

What are novel methods?

• Usually, methods established in one discipline transferred to another (ethnography and the Chicago School?)

• Easy to take ones own methods for granted. Pretty much all methods (except “armchair theorising”) are novel to sociology within the last 100 years. Economics has almost no qualitative strand.

Ologyology?

• Thinking of populations of methods across social science disciplines like this is itself somewhat novel.

• Which methods succeed and fail (or are never even considered) in which disciplines?

• Which methods are “naturalised” in new disciplines? (Experiments.)

Two examples

• History: There will never be any “new” data (although old data may be rediscovered). How stable are social practices? Can we build models of open ended systems? Why no “theory” in history?

• Archaeology: Operates nearly always at an additional remove. Not even behaviour as reported rather than observed (like history via diaries) but via distribution and attributes of artefacts.

• Just social science with “special needs” rather than a whole other thing?

Some potentially interesting questions

• Do methods have to “grow up with” a discipline to make a major impact? Has a novel method ever changed the research trajectory of a discipline?

• Will the acceptance of novel methods be constrained by the methods currently dominant in a discipline?

• How many really distinct methods are there?

• Are double standards exercised (or revealed) in the (non) acceptance of novel methods? (Word count, background knowledge, existing data?)

Two hypotheses

• A radical innovation is one that adds significantly to the effective conduct of the “standard” social science research process.

• A radical innovation is one that allows “reuse” of existing data in a significantly different way. This claim is not trivial. Quants reuses its own data a lot. Quals doesn’t do this as much as you might expect (ethnographies of homelessness?) But quals and quants definitely don’t (can’t?) reuse each other’s data. What about Conversation Analysis?

Two claims

• By these criteria (and I am happy to consider others: New domains of study?), simulation is a radical innovation in research methods (RIRM).

• It contributes significantly to the systematic theory building component of research.

• By doing so, it draws attention to missing data, tacit assumptions and “biases” in existing research across methods.

Concerns about the “status quo”

• Substantive areas can become dominated by a particular method which “distorts” the area.

• Single methods don’t warn about their limitations of applicability, only in “contention” and not always then (the classic futile quals versus quants “row”).

• Problems can “vanish” between disciplines (space, place, networks and time use).

• Methods for solving real problems in one discipline can be “stuck” in another.

A “methodological” problem …

• How does simulation systematically make use of what is already known?

• How does it avoid reproducing the biases of existing research? (Or creating new ones?)

• Possible answers: Literature reviews, inductive coding of published research, use of bespoke “theory building” qualitative research, expert knowledge. Other? Help!

A little honesty

• Two recognised weaknesses of existing simulation from within the community.

• No references from the social sciences literature they claim to be “about”. (Social mobility example.) Does nobody good.

• “Heaps” of models rather than progressive research. Which model you build shouldn’t be so sensitive to which three articles you read!

The BBVsim case study

• Building a simulation using expert knowledge.

• Intended not (yet?) to explain or predict but to stop drug educators and policy makers thinking in a naïve “linear” way out of habit.

• Standard concerns also raised in sociology (i. e. critique of “general linear reality” by Andrew Abbott).

• Large effects don’t always have large causes. “Adjacent” effects don’t always have “adjacent” causes.

The Schelling model

• Agents live on square grid: Each has maximum 8 neighbours.

• Two “types” of agents (red and green) and some sites are unoccupied. Initially agents/vacancies distributed randomly.

• Each agent decides what to do in the same very simple way.

• Each agent has a preferred proportion (PP) of neighbours of its own kind (0.5 PP means that you want at least half your neighbours to be your own kind. Fractions are used because empty sites “don’t count” for satisfaction.)

• If an agent is in a position that satisfies PP then it does nothing.

• If it is in a position that does not satisfy its PP then it moves to an unoccupied position chosen at random.

Initial state (NetLogo)

Two questions• What is the smallest PP (between 0 and 1) that will produce clusters?

• What happens when the PP is 1?

Answer and explanation

• PP=0.3. Because PP is minimum, anyone “inside” a “tight” (no vacancies) cluster stays happy. Thus the stability of clusters depends on the ability of agents to form stable “edges”. This determines the shape (and thus somewhat size) of clusters.

• With PP=1 clusters break down (and a snap shot of the population is indistinguishable from PP=0.2 say). No agent can now be a satisfied “edge”.

• We see here, in a very concrete way, “structuration” by clusters.

Simple individuals but complex system

Individual Desires and Collective Outcomes

-20

0

20

40

60

80

100

120

0 50 100 150

% Similar Wanted (Individual)

% Similar Achieved (Social)

% similar% unhappy

This system is non-linear in

PP

Claims revisited

• Simulation allows a “formally complete” specification of a social process (like quantitative research) but can still represent the complexity stressed by qualitative research. Even in this very simplified case, it integrates space, movement, decision making, a “psychological disposition” (PP) and emergent structure.

• As output, it produces not only clusters which can be compared with real data but also simulated biographies ditto and (with the addition of heterogeneous PP) distributions of individual attributes across space and clusters.

What about data?

• Clearly this is a “toy” model. Now what?

• Individual level data likely to be collected by qualitative methods (ethnography, interviews, perhaps experiments). This forms a testable set of hypotheses.

• Aggregate level data likely to be collected quantitatively (surveys, GIS). The simulated outcome of the individual actions is falsified against similarity between simulated and real data of several kinds.

• “If I were you, I wouldn’t start from here at all.”

An unusual style of argument

• Even a simulation as simple as this shows it isn’t safe to assume linearity, to attribute micro behaviours from macro patterns and so on.

• How much more complicated the real world is likely to be than this!

• Perhaps (but it would be nice to see it), more complicated systems actually produce simpler behaviours. This is the quantitative “creed” but there is no direct evidence.

Back to BBVsim

• Modelling the impact of clean needle distribution on rates of hepatitis/HIV (BBV).

• A mixture of social, physiological and physical processes: How do people seek and agree sharing of “works”, how long does the virus stay live on the works, how likely is a “style” of injection with dirty works to infect?

• Based on expert interviews and iterative development.

The core of the model

• Friendship groups in “stylised” geographical space (some real data): Homeless/hostels, housing estates, clubs.

• Use level and need to use is fixed/exogenous.

• Clean works distributed daily by some process.

• Works are removed probabilistically depending on status (clean, dirty, used): Knowledge limit.

• Ethnographic “practices” of sharing.

Not tackled in current model

• Details of movement processes: Avoiding TOE.

• Special status of longer term sexual partnerships as “small” sharing groups.

• Other sources of infection: Selling sex.

• Economics of drug use.

• The physiological progress of illness/addiction.

• Other policy interventions (needle vending, harm reduction sharing practice change, methadone support, testing/cure.)

What we didn’t do

• Scour the academic literature for data. (This simulation is meant to illustrate principles like non-linearity in a context drug educators will recognise and not to predict the future. That may come later.)

• Collect any data of our own. (Ditto. Though we now have a much better/clearer idea of what is needed.)

• Guarantee that the expertise embodied in the simulation will agree with general expert consensus if that exists.

• Develop a “true” model.

The “front end”

Interviews and interactive development

• First interview, then draft specification, then second interview, then “version 0” program then repeated emails leading to later program versions and, later still, assessment by second expert. (Repeatable.)

• Based on what experts said, I tried to capture the important social processes involved in needle sharing, some in detail, some more approximately.

• This process continued to convergence with some additional discussion of visual “style” and what the target audience could “cope with”.

What did we discover broadly?

• There’s a lot of (not very high flown) stuff we don’t know about drug use behaviour, probably because of the “frames” imposed by existing research methods.

• We have to be rather careful about what it is we intend to explain. (The causes of epidemics versus the cause of the epidemic in city x. How many cases do we have available to study?)

Examples of “prior frames”

• Focus on people rather than needles. We don’t attend to how many needles people hold, what kind and how they are lost/disposed of.

• We don’t think about location histories even though we know these have a big impact on social circles.

• In the absence of integrative theory, we don’t really know “enough” about the ethnographic detail.

Ethnographic detail

• Want to use, check works, if none, try to share.

• What is exact preference order: Own dirty or other clean?

• What happens if sharing “fails?”

• Does sharing negotiation affect choice of works? (Is there a collective action failure?)

• What else goes on? (Accidental sharing.)

• What about heterogeneity and adaptation?

The “hazards” of (policy?) experts

• I managed to find at least some data the experts claimed didn’t exist.

• It is sometimes surprisingly hard work to explain what seem like rather basic social science/modelling issues (NICE criteria example).

• It is even more surprising what drug educators aren’t supposed to understand. (Mean and SD of use.)

• Is an expert just someone who is always prepared to venture an opinion?

An “ethical” problem

• How do we convey to users the limitations of a model that is supposedly stylised for their benefit?

• This involves some sort of “rating” for models at different degrees of realism (like sunscreen!)

• Do visuals actually convey “too much” credibility?

What now?

• Literature reviewing and ethnography could be a lot more focused to improve the model.

• Totally unstructured interviews won’t do to capture expert knowledge: The chief problem is that experts want both to specify the behavioural assumptions and the aggregate outcomes and these aren’t consistent! (No epidemic example.)

Conclusions

• Tentative (i. e. N=1!) confirmation for claim that simulation is a radical innovation (only according to a provisional definition) in research methods.

• A case study of the strengths and limitations of expert knowledge for building simulations.

• Lots of middle range theory still to develop i. e. time planning models.

• A “launching pad” for a substantive piece of research in “theory building ethnography”.