METHODOLOGICAL IMPLICATIONS OF CRITICAL REALISM … · MIS Quarterly Vol. 37 No. X, ... do little...

SPECIAL ISSUE: CRITICAL REALISM IN IS RESEARCH

METHODOLOGICAL IMPLICATIONS OF CRITICAL REALISMFOR MIXED-METHODS RESEARCH1

Markos ZachariadisWarwick Business School, The University of Warwick,

Coventry CV4 7AL, UNITED KINGDOM {[email protected]}

Susan ScottInformation Systems & Innovation Group, Department of Management, London School of Economics and Political Science,

Houghton Street, London WC2A 2AE UNITED KINGDOM {[email protected]}

Michael BarrettJudge Business School, University of Cambridge, Trumpington Street,

Cambridge CB2 1AG UNITED KINGDOM {[email protected]}

Building on recent developments in mixed methods, we discuss the methodological implications of criticalrealism and explore how these can guide dynamic mixed-methods research design in information systems.Specifically, we examine the core ontological assumptions of CR in order to gain some perspective on keyepistemological issues such as causation and validity, and illustrate how these shape our logic of inference inthe research process through what is known as retroduction. We demonstrate the value of a CR-led mixed-methods research approach by drawing on a study that examines the impact of ICT adoption in the financialservices sector. In doing so, we provide insight into the interplay between qualitative and quantitative methodsand the particular value of applying mixed methods guided by CR methodological principles. Our positioningof demi-regularities within the process of retroduction contributes a distinctive development in this regard. We argue that such a research design enables us to better address issues of validity and the development ofmore robust meta-inferences.

Keywords: IS research, critical realism, retroduction, mixed methods, qualitative and quantitative methods,econometric modeling, qualitative enquiry

Introduction1

The growing diversity in information systems (IS) researchand the increasing popularity of mixed methods approaches,involving the use of both qualitative and quantitative methodsin the same research endeavor, has motivated many scholarsto explore different research design strategies and alternative

theories to ground this new methodological movement (Ben-basat and Weber 1996; Landry and Banville 1992; Lee 1999;Mingers 2001; Ridenour and Newman 2008; Robey 1996;Venkatesh et al. 2013).2 Traditional paradigms of IS research

1John Mingers, Alistair Mutch, and Leslie Willcocks served as the senioreditors for this special issue and were responsible for accepting this paper.

2The terms mixed-methods, multimethods, and triangulation are often beingused interchangeably in IS and in social sciences literature generally. In thispaper we use the term mixed-methods to highlight the combination ofqualitative and quantitative methods in the same research project, in contrastto multimethod research which may only refer to the combination of two ormore qualitative methods (i.e., participant observation and interviews).Triangulation usually concerns various aspects of research and implies the

MIS Quarterly Vol. 37 No. X, pp. 1-XX/Forthcoming 2013 1

Zachariadis et al./Methodological Implications of CR for Mixed-Methods Research

do little to facilitate the use of multiple research methods fromdifferent methodological approaches, but as the fielddevelops, theoretical frameworks have emerged that arecapable of providing practical guidance for mixed-methods(MM) research design. Among the most prominent is that ofcritical realism (Mingers 2001, 2004; Venkatesh et al. 2013).

Critical realism (CR), which is largely based on the writingsof Bhaskar (1975, 1978, 1989) and others (Archer et al. 1998;Fleetwood 1999; Lawson 1997; Sayer 1992), is often seen asa middle way between empiricism/positivism on the one hand,and anti-naturalism/interpretivism on the other, thus intro-ducing a more nuanced version of realist ontology. As such,CR embraces various methodological approaches from dif-ferent philosophical positions by taking “a critical stancetowards the necessity and validity of current social arrange-ments” without following “the extant paradigms’ assumptionsat face value” (Mingers 2001, p. 248). This makes it parti-cularly attractive for the study of IS, which is primarily apractice-based research domain encompassing aspects of bothnatural science and social science (Carlsson 2003, 2005;Dobson 2001; Mingers 2004; Venkatesh et al. 2013).

Recent work has developed a set of methodological principlesfor conducting CR-based case study research in IS (Wynn andWilliams 2012) and highlighted the value of CR as an under-lying theoretical framework for mixed methods (Mingers2004; Venkatesh et al. 2013). Also of note is a small body ofwork that directly employs the distinctly CR (but rarely used)mode of analysis called retroduction (Strong and Volkoff2010; Volkoff et al. 2007). However, the latter rests on quali-tative case studies and therefore the implications of a retro-ductive CR methodology that introduces a thorough mixed-methods approach have been largely unexplored empirically(Mingers 2003).

Our purpose is to critically review how the methodologicalinheritance of IS tends to reinforce traditional dichotomiesbetween qualitative and quantitative methods, thereby inad-vertently reinforcing conventional and linear approaches toresearch (Creswell 2003; Law and Urry 2004; Tashakkori andTeddlie 1998; Teddlie and Tashakkori 2003, 2009). In con-trast, through our particular reading and use of the ontologicaland epistemological principles of CR, we emphasize a moreresponsive and dynamic mode of enquiry called for in pre-vious literature (Landry and Banville 1992; Morse et al. 2002; Myers 1997; Nandhakumar and Jones 1994; Walsham1995a). The paper is structured as follows: We start by

reviewing the distinctive conceptualization of causalityenabled by the ontological assumptions of CR and explainhow this affects the key epistemological principles of validityand generalization. We then describe the role of mixedmethods research from a CR standpoint and explore the pro-cess of retroduction as the core methodological principle ofcritical realism. In the next section, we demonstrate the valueof our approach to mixed methods CR research by drawing ona study of the impact of ICT, specifically SWIFT, adoption onbank performance. We conclude by discussing methodo-logical implications of CR for mixed methods research in IS.

Methodological Implications ofCritical Realism

It is argued that critical realism can add to IS research byopening up a particular methodological space that liesbetween empiricism and interpretivism (Mingers 2004).While the differences between philosophical paradigms andthe way CR confronts empiricism and interpretivism is aninteresting subject in itself, the aim of this paper is not to pro-vide an exhaustive account of CR but rather to highlight someof its key methodological implications. For critical realism,the link between the assumptions about the existence of theworld and society (ontology), the idea of how knowledge ispossible and of what (epistemology), and the choice ofmethodological approach is of major importance. Followingthis premise, we start by exploring the ontological basis of CRthat will help us make the ontology–methodology link. Bytaking this into account, we then discuss what is the CR viewof causality and what is the role of validity within that.

Ontology, Generative Mechanisms,and Causality

Being a realist philosophy, critical realism maintains a strongemphasis on ontology and supports the idea of a reality(intransitive domain) which exists independently of ourknowledge or perception of it (Archer et al. 1998; Bhaskar1975, 1978, 1989). In contrast, the generation of knowledgeis a human activity and depends upon the specific details andprocesses of its production (transitive domain) which can beestablished facts, theories, models, methods, and techniquesof study that are used by researchers at a certain time andplace. Hence, new knowledge is articulated in two dimen-sions: “it is a socially produced knowledge of a natural([hu]man-independent) thing” (Archer et al. 1998, p. 65).This useful analytical distinction indicates that, despite its

combination of different insights in an investigation (e.g., data triangulation,investigator triangulation, theoretical triangulation, etc.; see Downward andMearman 2006).

2 MIS Quarterly Vol. 37 No. X/Forthcoming 2013


ontological realism, CR allows for a degree of epistemo-logical relativism where the process of scientific knowledgeis viewed as historically emergent, political, and imperfect(Smith 2006). Even though these limitations signify the falli-bility or relativity of our knowledge (at least to some degree),it doesn’t necessarily mean that all knowledge is equallyfallible. CR consents that some researchers may have morevalid explanations or theories that approximate the intransitivedomain with more probabilistic accuracy than others.

Besides the distinction between the transitive and intransitivedimensions of knowledge, critical realism assumes a stratifiedontology divided into three domains: the real, the actual, andthe empirical (Bhaskar 1975, 1978, 1989). This importantillustration helps us understand that even though there is onereality it does not follow that we, as researchers, have imme-diate access to it or that we are able to observe and realize itsevery aspect. The domain of the real includes objects andstructures with inherent causal powers and liabilities whichresult in mechanisms that may not be visible. According toLawson (1997, p. 21),

A mechanism is basically the way of acting orworking of a structured thing....Structured things[physical objects or social processes] possess causal[or emergent] powers3 which, when triggered orreleased, act as generative mechanisms to determinethe actual phenomena of the world.

While generative mechanisms are not necessarily constantlyempirically observable, their potentialities may still existwhether they are exercised or unexercised (Bhaskar 1978,1998). Consequently, the actual is a subset of the real andincludes the events generated from both exercised and unexer-cised mechanisms. Finally, the empirical refers only to thesubclass of observable, experienced events and change (i.e.,bears witness to the powers and liabilities characterizingstructures that constitute generative mechanisms which in turncause the experienced events). Figure 1 presents an outline ofthe three ontological domains.

The assumption of a deeper level where the structures andpowers of things may or may not be observable is not only a

point of theoretical distinction but also has important method-ological implications. The main objective when pursuing CR-led research should be to “use perceptions of empirical events[those that can be observed or experienced] to identify themechanisms that give rise to those events” (Volkoff et al.2007, p. 835). As a result, the critical realist view oncausality should not be about a relationship among distinctevents (e.g., the fact that event “A” by and large has beenfollowed by event “B”) but about realizing the process andconditions under which “A” causes “B,” if at all.4 Whilegenerative mechanisms form a key interest (Collier 1994), itis important to appreciate that “when…one writes that amechanism has a tendency to x, one is, in reality, referring tothe ensemble of structures, powers and relations: it is, strictlyspeaking, the ensemble that has a tendency to x” (Fleetwood2001, p. 211). Accordingly, the existence of social lawsassumes an assertion about the activity of some mechanismand not about the conditions under which the mechanismoperates; hence, it is difficult to make statements about theresults of its activity (Bhaskar 1975). This is because CRadopts a view of reality as an open and complex system whereother mechanisms and conditions also exist. As a result, apartfrom the ensemble of structures, powers, and liabilities weshould be attempting to identify the conditions in whichgenerative mechanisms are experienced. This presents chal-lenges not only in terms of our analysis of causation but alsorevises our methodological requirements and expectations forvalidity and generalization in research.

From a constructivist point of view, CR finds some commonground with interpretivism in that social phenomena areconcept-dependent and need interpretive understanding(Giddens 1979). However, unlike interpretivism, CR does notexclude the existence of a relational intransitive domain insocial structures. This idea is based on the notion of humanattributes and social activities (e.g., physical features, or astructured activity like speaking a language) that result incausally effective social structures (Mingers 2004; Sayer2000). Consistent with this line of “horizontal stratification”(Strong and Volkoff 2010), critical realists oppose interpreti-vists who do not succeed in relating discourses to these under-

3According to critical realists, these emergent powers or liabilities cannot bereduced further to those of their constituents. For example, we cannot dis-aggregate the power of people to think by reference “to the cells thatconstitute them, as if the cells possessed this power too” or the power ofwater to that of its constituents (oxygen and hydrogen) as these belong to adifferent stratum (Sayer 1992, pp. 118-119). For a further discussion on thisstratification and the distinction between internal and external relations seeChapter 4 in Sayer (1992).

4Since causal powers and liabilities of objects are independent of any specificpattern of events it can be inferred that under particular circumstances theactualizations of the same causal powers may not lead “A” to generate “B.” In some circumstances, events derive from social structures, which by theirnature could be influenced by other events. In this case, “the modification ofprevious structural properties and the introduction of new ones or the rein-forcing of existing structures” is called reproduction (Volkoff et al. 2007, p.835). However, CR argues that social structures are historically specific(contingent upon both space and time) and difficult to transform so that, atone point in time, agents interact with structures that they did not create(Sayer 1992).

MIS Quarterly Vol. 37 No. X/Forthcoming 2013 3


Figure 1. The Stratified Ontology of Critical Realism (adapted from Bhaskar 1975)

lying social structures which influence the behavior of agentsor groups in which agents are embedded (Granovetter 1985).

In practical terms, the CR position on causality means that adifferent mode of inference needs to be adopted in order toexplain events “by postulating (and identifying) mechanisms[liabilities and powers] which are capable of producing them”(Sayer 1992, p. 107). This logic of inference, which could bebetter described as “thought operation” (Danermark et al.2002), is called retroduction (Bhaskar 1975). Retroductionallows researchers to move between the knowledge of empi-rical phenomena as expressed through events to the creationof explanations (or hypothesizing) in ways that hold “onto-logical depth” and can potentially give some indications onthe existence of unobservable entities (Downward and Mear-man 2006). This makes it possible to understand how thingswould have been different, for example, if those mechanismsdid not interact the way they did. Under these circumstances,patterns of empirical events can provide inadequate informa-tion about the activity of the mechanisms in play.

It has been widely argued that this retroductive approach toresearch embraces a wide variety of methods (Downward andMearman 2006; McEvoy and Richards 2006; Mingers 2000,2001, 2003, 2004, 2005; Mingers and Gill 1997; Venkateshet al. 2013; Wynn and Williams 2012) where qualitative andquantitative approaches can be integrated in order to hypothe-size and identify the generative mechanisms that cause theevents we experience. However, regardless of whichevermethod is used, there is a common principle, namely that “thefoundation for our knowledge [produced in the transitivedomain] is the empirical domain” (Danermark et al. 2002, p.

155). Figure 2 illustrates the process of knowledge creationin CR and the place of retroduction within that (looselyadapted from Downward and Mearman 2006). The value ofretroduction in combining different methods will be furtherelaborated later.

Validity and Quality of Inferences

Having briefly described causal mechanisms and reviewed theretroductive approach to knowledge creation, we now explorethe implications of using retroduction for the validity and thequality of inferences produced. Specifically, we need toexamine how the conventional concepts of validity describedin the mixed-methods literature (Creswell and Clark 2007;Lee and Hubona 2009; Maxwell 1992; Ridenour and Newman2008; Tashakkori and Teddlie 1998; Venkatesh et al. 2013)change in the face of critical realism. Validity often signifiesthe level of quality and rigor of research and can have a signi-ficant impact on the quality of inferences that are generatedfrom a study. In order to avoid any confusion due to the vari-ability between connotations of the terms used for validity indifferent textbooks and papers (Lincoln and Guba 1985, 2000;Morse et.al. 2002), we follow the classification proposed inVenkatesh et al. (2013) and Teddlie and Tashakkori (2003,2009). The use of that terminology contributes to our discus-sion later on when we talk about validation within MM ISresearch.

By synthesizing the literature on different validity types,Venkatesh et al. generally outline three distinctive categoriesfor validity which are widely used and are somewhat common



Figure 2. The Retroductive Approach of Critical Realism for Knowledge Creation

for quantitative and qualitative research. These are designvalidity, measurement (or analytical) validity, and inferentialvalidity. In quantitative research, design validity broadlyrefers to internal (that the correlation observed is causation)and external (results can be generalized) validity. Measure-ment validity includes the reliability of the data (i.e., if thereis measurement error) and construct validity that describes thedegree to which the variables used in the model capture whatthey intend to measure. Finally, inferential validity refers tothe validity of the statistical conclusions and whether they aresufficient in order to make inferences.

In qualitative research, despite the different points of view onthe need for validity (Denzin and Lincoln 1994; Guba andLincoln 2005), there is agreement that validation is crucial tothe development of a common scientific body of knowledge(Maxwell 1992; Morse et.al. 2002; Venkatesh et al. 2013).Hence, scholars have established similar terms and definitionsfor validity focusing on how well the qualitative study wasdesigned and executed (design validity); how well data werecollected and analyzed, in order to get dependable, consistent,and plausible findings (analytical validity); and, finally, howto assess the overall quality of interpretation and inferences(inferential validity). The first two columns in Tables 1 and2, which are loosely adapted from Venkatesh et al. and fromJohnston and Smith (2010), provide a summary of the typesof validity in qualitative and quantitative research.

We can now compare some of these conventional views ofdifferent types of validity with that of critical realism. As

mentioned before, CR moves the center of attention fromempirical events to underlying causal mechanisms by arguingthat these could be independent and non-related. What thismeans in practical terms is that while an empiricist is con-cerned with whether correlated empirical phenomena arecausally linked (internal validity), a critical realist will belooking to establish whether the generative mechanismhypothesized or uncovered is involved in the observed eventsin the field (Johnston and Smith 2010; Modell 2009; Wynnand Williams 2012). Similarly, for CR, external validitywould be concerned with the generalizability of the knowl-edge claims about one or more causal mechanisms, that is, thebelief that the generative mechanism or mechanisms thatcaused the observable events (in our specific research settingand under certain circumstances) also caused similar (or evendifferent) outcomes in other domains. Finally, constructvalidity would address whether “empirical traces give infor-mation about the actual events…that are purportedly causedby the generative mechanisms” (Johnston and Smith 2010, p.33). The difference of these three validity types between theconventional view (dashed lines) and the CR view (con-tinuous lines) is illustrated in Figure 3 (loosely adapted fromJohnston and Smith 2010).

As can be seen in Figure 3, the conventional interpretation ofconstruct validity focuses on the relationship between thetheoretical concepts (construct 1 or 2) and their operationaldefinitions (observed variables) existing in the empiricaldomain (measurement 1 or 2). In contrast, according to CR,construct validity should be concerned with the correspon-



Table 1. Validity in Quantitative Research

Validity Type Conventional Description Critical Realism

DesignValidity

Internal validity: The observed correlation betweenvariables of interest in a causal effect.

Actual events are manifestations of the particulargenerative mechanism in the context of the field.

External validity: The cause–effect relationshipgoes beyond the variation of samples, settings, andtreatment variables, thus results can be generalized.

The likelihood that similar or related events thatoccur (or might occur) in other settings are causedby the generative mechanism that caused theactual events in the field.

MeasurementValidity

Reliablity: The variables used in the model do nothave measurement error.

The measurements used in the extensive methodsdo not have measurement error.

Construct validity: The variables indicate what theyare intended to measure (consistent with thetheoretical description).

Whether data that is empirically available givesvalid knowledge about the actual manifestation ofthe alleged generative mechanism in the field.

InferentialValidity

Statistical conclusion validity: Statistics are appro-priate and findings from the statistical analysis areadequate to construct a narrative.

Findings from statistics can provide informationabout the relationships of events observed in theempirical domain (not causal assumptions).

Table 2. Validity in Qualitative Research

Validity Type Conventional Description Critical Realism

DesignValidity

Descriptive validity: Accuracy of events, objects,behaviors, and settings reported.

Explanations of mechanisms in action and theconditions with which they are interacting;appreciation of the field by identifying, prioritizing,and scoping boundaries of the study.

Credibility: Results are believable from theparticipants of the research.

Transferability: Results can be generalized andtransferred to other settings.

The idea that similar or related events that occur(or might occur) in other settings are caused by thegenerative mechanism that caused the actualevents in the field.

AnalyticalValidity

Theoretical validity: Theoretical explanationdeveloped fits the data.

Theory is used to help hypothesize about themechanisms and provide explanations for theevents that have occurred.

Dependability: Researchers describe changes inthe research setting and its effects on the researchapproach of the study.

This is an essential part of the retroductive processand identification of contingent factors.

Consistency: Verifying the steps of qualitativeresearch process.

Challenge and inform the terms of (quasi-)closureand process of ongoing inquiry in retroductiveanalysis.

Plausbility: Findings of the study fit the data fromwhich they are derived.

Whether data that is empirically available givesvalid knowledge about the actual manifestation ofthe alleged generative mechanism in the field.

InferentialValidity

Interpretive validity: Interpretation of participants’views are accurate.

Findings from qualitative research can provideinformation about the mechanisms that cause theevents at the empirical level.Confirmability: The results are confirmed by others.

dence between empirical traces of events (E1 and E2) and theinformation they give us about the actual events in the fieldwe are studying (C1 and C2), which in turn are manifestationsof the mechanisms we seek to uncover. In parallel, internalvalidity in Figure 3 demonstrates the different notions ofcausation, which on the one hand is perceived as the corre-lation between empirical observations (neasurement 1 and 2),

and on the other hand focuses on the generative mechanismsin the domain of the real that cause the actual events weexperience in the empirical domain.

Finally, external validity, which represents the attempt togeneralize in the empirical domain by inferring that thepresumed causal relationship between two observable events



Figure 3. Validity Based on a Critical Realist Approach

(e.g., measurement 1 and 2) potentially applies across dif-ferent contexts (persons, organizations, locations, and times)and alternative measures, is highly problematic from theperspective of CR (Johnston and Smith 2010; Tsoukas 1989).This is because it combines and confuses the empirical traceswith the actual events that are produced through on-goinginteraction of mechanisms. Since empirical events are parti-cular manifestations of the mechanisms that caused them, itcannot be assumed that because a relationship between eventsappears under one set of circumstances it will appear again inexactly the same form in a different context, thus, causal lawsshould be ontologically distinct from patterns of events(Tsoukas 1989). The same relationship may appear but notinvolve exactly the same mechanisms, or may not appear, butthis does not imply that the specific mechanisms were absentbecause they might have been counterbalanced by thepresence of other mechanisms. Instead, according to CR,generalizations are valid when we are confident that similaror other events that arise (or may arise) in other contexts arecaused by the same generative mechanisms that led to theactual events in our research domain.

The emphasis on studying multiple, dynamic, and shiftingrelationships in context would seem to favor qualitativeapproaches capable of producing situated analytical explana-tions that might help reveal the potential mechanisms in-

volved in observable events. In the interpretive school of ISresearch, generalizing from a single case study is a commonlyaccepted practice (Lee and Baskerville 2003; Markus et al.2006; Walsham 1995b). Researchers leverage existing state-ments of causal mechanisms to explain events in differentsettings and hence refine their theories by validating “theexplications of causal tendencies and the interplay of mech-anisms and context” (Wynn and Williams 2012, p. 805).Hence, the degree of external validity is subject to discerningbetween the contingent factors from the necessary ones(Sayer 1992). This is to say that it is not logical to talk aboutthe impact of IT, for example, when it is not possible toseparate the external or contextual conditions within theorganization that partially influence the effects we see, fromthe internal (necessary) properties of the technology we study(Smith 2006). This well-reasoned and systematic conceptualabstraction of the object characteristics from external condi-tions is an essential part of theorizing which leads to ourknowledge of things5 (Danermark et al. 2002) and needs to be

5This position of external realism is based on the ontology of CR and is whatdifferentiates it from interpretivism whose lack of an external reality andacceptance of subjectivity does not allow us to discern between competingtheories and thus generalize (Walsham 1993). The CR process of theorizingentails the continuous restructuring and adjustment of abstractions thatdetermine the nature of the research object (Danermark et al. 2002).



combined with a description of the empirically researchedcontingencies, within which causal powers have taken place(Tsoukas 1989).

Accepting a critical theorist view of generalization changesthe role of quantitative approaches which usually measureevents in order to theorize social laws based on the regularityand degree of correlation between variables (Lawson 1994). However, from a CR perspective, even though structures arecomprised by situated interactions which are better under-stood qualitatively, it is also likely to be useful to employquantitative measures to quantify certain characteristics of astructure or object. In that respect, statistical descriptions areregarded as helpful simplifications, which serve as “a quan-titative measure of the numbers of objects belonging to someclass or a statement about certain common properties ofobjects” (Sayer 1992, p. 100). For example, one might saythat 80 percent of organizations with high diversity are multi-nationals or that larger firms tend to invest more in tech-nology. The important characteristic of these overviews isthat, even though they suggest a necessary relationship orcorrelation, they do not say anything about the causal statusof the relationships (if any) and, thus, should be seen asdescriptive summaries rather than predictive tools (Fleetwood1999; Lawson 1997; Sayer 1992).

In the next section, we will describe in more detail the role ofdifferent methods in CR including that of quantitative tech-niques and the extent that they can be used to shape ourknowledge about the world. More importantly, we willdiscuss how qualitative and quantitative techniques can notonly complement each other but be reciprocally responsive ina MM research design and produce robust meta-inferencesthat would be difficult to produce using single methods.

Mixed-Methods and the Processof Retroduction

The Role of Quantitative Methods

The role of quantitative or extensive methods within CR islargely viewed as descriptive, since quantitative summariesand correlations between variables alone cannot uncover evi-dence on the causal mechanisms that generate the actualevents we observe or predict future incidents. On that basis,a lot of the empirical work in IS journals that uses quantitativemethods to generalize knowledge claims has been describedas unsatisfactory and problematic (Seddon and Scheepers2012), underlying further the need to reevaluate the role ofextensive methods within not only IS, but social sciences asa whole. Most of the CR critique on quantitative methods is

directed toward the field of economics and the use of econo-metrics (Bache 2003; Cartwright 1989; Downward andMearman 2002; Fleetwood 1999; Hands 1999; Hoover 1997;Lawson 1994, 1997). Econometric modeling is based on thedevelopment of statistical methods (most often regressionanalysis) for testing economic theories, estimating relation-ships that emerge from patterns identified in the data, andforecasting economic situations (Wooldridge 2006). In IS,econometrics is often used to identify relationships and pat-terns that surface from the use of technology in organizationsand to estimate the impact of innovation adoption fromindividuals. As such, it has achieved considerable promi-nence among IS researchers, policy makers, and seniormanagers (Aral et al. 2006; Brynjolfsson and Hitt 1995, 1996;Draca et al. 2007).

Focusing exclusively on the identification of strictly definedpatterns of observable events (regularities) in order to makecausal statements in support of the argument that correlationis causation would indeed be at odds with the stratifiedontology of CR. For example, by suggesting that a correla-tion between the use of IT by policemen and an increase incrime rates means that IT causes more crime may be aninappropriate leap of causation when in fact IT has simplyimproved the recording of crime levels. Nevertheless, it isreasonable to argue that we can escape this deductive logicused in mainstream economics, which, according to CR, oftenleads to epistemic fallacy (Lawson 1997), and use statistics inline with CR ontological assumptions. The extent to whicheconometrics or other statistical techniques can be helpfulwithin CR research relates to “the role played by demi-regularities” (Bache 2003, p. 14). Demi-regularities can beunderstood as the partial event regularities that indicate “theoccasional, but less than universal, actualization of a mech-anism or tendency, over a definite region of time-space”(Lawson 1997, p. 204). For example, when exploring theimpact of social media on company reputation and revenue,a large number of mechanisms can influence the wide rangeof results that have been identified by scholars. However,when the research is restricted to the impact of a particularsocial media travel web-site, on five-star hotels, in a specificregion, between 2000 and 2002, then it begins to becomepossible to focus selectively on the particular mechanismsinvolved in this relationship. In such contexts, the role of ademi-regularity is twofold: first, it can play a significant partin focusing the research design and developing propositionsabout existing causal mechanisms, and, secondly, it can helpto assess and explain the results in the analysis phase. Itachieves this by focusing us on indicative patterns in the datawhich are thus raised up for further inquiry of a qualitativeand comparative kind. It can also inform research design byconfirming or challenging the boundaries of inquiry.



As mentioned, CR fundamentally accepts that social scientistsdeal with complex and open social systems where causalmechanisms are subject to external conditions (includingother mechanisms) depending on the context, and that theobjects of research are also likely to change. This means thatthe boundaries of the inquiry may have to be revised as theresearch process advances which may help define them.Consequently, many critical realists argue that demi-regularities are rarely to be found in the social realm and canonly exist when specific mechanisms come to dominatewithin restricted contexts. In other words, when we scope ourstudies and limit the conditions of possibility by establishingextrinsic conditions of closure that serve as analyticalboundaries (e.g., focusing on a particular sector, historicalperiod, or geographical region) while simultaneously settingup intrinsic conditions for closure in the object under study(e.g., an organization or a specific technology) in an effort toensure that it does not undergo any qualitative variation (e.g.,culture or technological change).6 However, characterizing asocial system as closed is problematic from a critical realistperspective which is founded on the premise of interactingopen systems (Bhaskar 1975). But what does this imply forresearch methods that, in their conventional use, depend onconditions of closure?

Sayer (1992) maintains that while most social systems weencounter violate these conditions of extrinsic and intrinsicclosure, partially closed systems that enjoy quasi-closure canbe studied separately if we restrict our research spatially andtemporally based on the specific contextual factors that affectour conditions. Hence, it is up to the social scientist to con-struct the conditions of closure in order to get closer to thereal mechanisms that cause the events in the domain of theactual (Tsoukas 1989). In this case, demi-regularities canplay a significant role as we are going to illustrate laterthrough our example. Consequently, the degrees of closureand the specific nature of quasi-closure are important toepistemological (if not ontological) claims (Downward andMearman 2006).

Revising the scope of a study does not have to be a singularact but may form part of the process of analysis as eachrevision to the boundaries of quasi-closure may enable theparticular use of research methods to recover different infor-mation about specific mechanisms (Cartwright 1989; Fleet-wood 1999; Lawson 1994, 1997). Although demi-regularitiesare a point of debate among critical realists, the argument that

they might reveal patterns in conditions of quasi-closurewould make the contribution of quantitative methods muchmore significant than previously assumed by Bhaskar,Fleetwood, and Lawson (see Bache 2003; Downward andMearman 2002; Hands 1999; Hoover 1997). Whatever thecase, we believe that the legitimacy of quantitative methodswithin critical realism lies more with the interpretation ofstatistics and the process of revising the parameters of ourmodels to achieve appropriate degrees of closure (boundaries)in our research design than any inherent quality of themethods themselves. Henceforth we argue that by focusingon these two important aspects of extensive methods, namelydemi-regularities and degrees of closure (quasi-closure), wecan demonstrate the value of their role at various stages of theretroductive methodology7 proposed by CR (Mingers 2000,2005).

The Role of Qualitative Methods

In contrast to quantitative methods, the role of qualitative orintensive methods within CR is more profound. This isbecause intensive methods (e.g., interviews, ethnography,case studies, historical narratives, etc.) are “epistemologicallyvalid” (Tsoukas 1989, p. 556) and more capable of describinga phenomenon, constructing propositions (or hypothesizing),and identifying structures and interactions between complexmechanisms (Layder 1990; Sayer 2000; Volkoff et al. 2007).We should also remind ourselves that social phenomena orstructures are concept-dependent and thus are not independentfrom the agents’ notion of them or the apparatus throughwhich they became observable. Consequently, the need forinterpretation and understanding of their actions is quite signi-ficant. In addition, social structures are fundamentally subjectto geographical and historical conditions (contingent status ofsocial structures) as the social systems they belong to areinherently interactive and open (Mingers 2004). Therefore,it becomes easier to use a theory for explanatory or descrip-tive purposes than to test a theory using empirical effects.This is because theories help to conceptualize objects andstructures “at the abstract level about necessary or internalrelations” while they “remain agnostic about relations whichare contingent” (external) that rely on empirical questions andon observing actual cases at the concrete level (Sayer 1992,p. 143).

Considering the importance of the existence of contextualfactors (external and contingent circumstances that are bothspace and time dependent) in knowledge generation andgeneralization within CR, the role of the empirical qualitative

6 This idea that social laws might exist within limited domains is also referredto as local realism (Cartwright 1989). In spite of that, some critical realistsargue that even when these demi-regularities exist, they would still beinadequate to meet the requirements of econometric analysis and hencecannot be recovered with conventional quantitative methods (Lawson 1997;Mingers 2005).

7Here we use the idiom retroductive methodology to refer to the methodologythat is driven by the retroductive approach described previously.



Figure 4. Types of Research and Methods (adapted from Figure 12 in A. Sayer, Methods in SocialScience Research: A Realist Approach (2nd ed.), London: Routledge, p. 237)

methods is to uncover these conditions and distinguish themfrom the necessary (internal) aspects of objects and structures.In that respect, historical narratives, archival research, andsemi-structured interviews can play a significant part inbuilding informative narratives. Figure 4 illustrates the dif-ferent types of methods that can be used in a retroductivemethodology and their potential to be synthesized (extensive,intensive, and abstract).

Designing Mixed-Methods Research

From the discussion above, it should be apparent that CR doesnot commit to a single type of research but rather endorses avariety of quantitative and qualitative research methods. Thiscritical methodological pluralism is not taken lightly but isgrounded in the ontological and epistemological assumptionsof CR, thus preserving a strong link between meta-theory andmethod (Danermark et al. 2002). As described earlier, a majorpremise that leads to this kind of multiplicity is that socialphenomena are contextually defined (i.e., they are subject toother mechanisms and causal powers in the system) andfurthermore that mechanisms cannot always manifest them-selves empirically since they can be suppressed in a complexinteraction. So the choice of methodology then depends on thecapability and complementarity of different methods to conveydifferent kinds of knowledge about generative mechanisms.

The value and purpose of MM research in social science andin IS particularly has been discussed extensively elsewhere(Teddlie and Tashakkori 2003, 2009; Venkatesh et al. 2013);however, their role within CR is quite different. For example,the main purpose of quantitative methods in conventional ISresearch is theory testing and so they are often used in con-firmatory studies. Critical realists object to that view because,as we have seen, statistical generalizations at the empiricallevel (described as conjunctions of events) cannot makemeaningful connections with and validate the qualitativeresults (see “corroboration/confirmation” in Table 3) whosepurpose is to identify active mechanisms (Downward andMearman 2006). For the same reason, the use of qualitativework as preparation for quantitative work would also not beconsistent with the CR approach unless the latter is followed,again, by further qualitative work in a way that informs theretroductive process.

Generally speaking, the most profound and widely recognizedapproach to mixed methods, in line with CR’s retroductivemethodology, is to use extensive methods to identify andestablish demi-regularities with data patterns, which are thenused to guide intensive research that will uncover themechanisms, agencies, and social structures that produce thebehavior observed (see “complementarity” in Table 3). How-ever, further ways of combining MM research can be iden-tified. For example, MM design can be used to ensure that a



Table 3. Purposes of Mixing Methods in Critical Realism

Purpose ofCombination Description Implication from CR

Complementarity MMs are used in order to gain complementary viewsabout the same phenomena or events

Different levels of abstraction of a multi-layered world demand different methods

Completeness MM research design is used to ensure a completepicture (as detailed as possible) of the phenomenonunder study

Requires meta-theoretical considerations(i.e., angle of approach)

Developmental Inferences of one type of research are being used asquestions for another type of research

This being part of the retroductive approachof CR, inferences need to hypothesize aboutthe causal mechanisms whose recovery willthen inspire additional research

Expansion MMs are being implemented in order to provideexplanations or expand the understanding obtained inprevious research

Quantitative methods can be used to guidequalitative research which (subject to thecontext) is more capable of uncoveringgenerative mechanisms

Corroboration/Confirmation

MMs are used in order to confirm the findings fromanother study

Epistemic fallacy occurs when trying tovalidate qualitative results with quantitativemethods

Compensation The weakness of one method can be compensated bythe use of another

The weaknesses of different methods arerecognized so alternative methods can beused to compensate

Diversity MMs are used in order to obtain divergent views onthe same phenomena

Different levels of abstraction of a multi-layered world demand different methods

systematic picture of a phenomenon is obtained (see “com-pleteness” and “diversity” in Table 3). This is particularlyfeasible when a specific object has quantifiable properties orwe want to measure the number of objects that belong to aclass while at the same time we are trying to improve ourqualitative understanding. In such cases, though, it is impor-tant to consider the angle of approach and the level of abstrac-tion beforehand. Another case could be that of “compensa-tion” (see Table 3), where the limitations of one method canbe compensated by the use of another, or that of “develop-mental” and “expansion,” where previous research raises newquestions that demand further explanations in terms ofgenerative mechanisms. While prior literature has generallyacknowledged that the use of different methods may overlap(see Downward and Mearman 2006; Venkatesh et al. 2013),we provide a more detailed account of their interplay in CRand the value that this adds to validity, the development ofmeta-inferences and generalization of research (see Table 3).

Following the previous discussion, CR accepts that differentresearch methods have complementary strengths and limita-tions, and that in general, the insights from extensive methodsneed to be moderated (Sayer 2000). For this reason, it hasbeen argued that “the validity of the qualitative analysis ofcases [should] not rely upon broad quantitative evidence”

(Downward and Mearman 2006, p. 89). Even though thischaracterization seems to suggest that different methods areultimately focusing on different ontological paradigms (i.e.,supporting conventional dichotomies between quantitativeand qualitative research), this is not the intent. Rather,methods are seen to be redescriptive devices uncoveringalternative views of objects of analysis in order to comparetheir relative standing (and therefore the validity of thefindings produced) as well as allowing them to mutuallyinform each other. Consequently, the appropriateness ofmethods will be ultimately determined by the research ques-tions under consideration and the nature of object (e.g., thelevel of abstraction, emphasis, and measurement scales), aswell as the degree of closure that is assumed8 (Downward andMearman 2006). For example, while a study can addressmore than one research question, all of the questions mightfocus on the same phenomenon for which they will stress

8In this regard, it could be argued that qualitative methods are more powerfulbecause they might involve less closure than quantitative methods. Havingsaid that, qualitative methods (e.g., interview data) also require some level ofclosure and coding of responses that will ultimately have an effect on thecapability to mirror the direction and magnitude of variation in responses,and, thus, affect the inferences that can be made (Downward and Mearman2006).



different aspects of abstraction and uncover alternativecharacteristics of the same layered reality. Holding that inmind, the logic of retroduction not only suggests the recog-nition of MM research but at some level it implies itsnecessity as well as the need to articulate its deployment as asystematic process.

The Retroductive Process in CR

While most textbooks and papers on MM research proposevarious guidelines and research strategies based on thesequence and implementation of method (Creswell 2003;Tashakkori and Teddlie 1998), the retroductive methodologyfocuses on research and intervention not as a discrete eventbut as a creative process with different phases that involvedifferent types of activities (Bhaskar 1978; Mingers 2005). Its main objective is to link the structures and causal powersof the objects under study to the events we want to explainthrough the notion of causal mechanisms (Wynn andWilliams 2012). In doing so, four main phases are identified.

The first phase involves the description or appreciation of theresearch situation and focuses on the identification of thecomposite events or phenomena under study. As it is not pos-sible to examine all possible distinctive constituents of a studyor phenomenon, a decision needs to be made about whichcomponents to select. The acknowledgment of situatednessplays a key role here and serves as an important resource(Runde and de Rond 2010). In working through the feasi-bility of a study, CR encourages the researcher to regard alldata as situated in a point of view (i.e., focusing on one oranother aspect of some event) which helps to devise the initialdesign and consider gaps in the corpus of data that needs to becollected in order for the research to be systematic. Thispertains not only to the identification of data sources that willenable comparative analysis (e.g., interviews, quantitativedata, historical documents, surveys, etc.), but also motivatesdeeper examination into the relative insights afforded by eachdata type and method of analysis. Finally, another centralachievement of this stage is to consider extant theoreticalschemes (in the academic literature and in the domain understudy) and consider how these can help to shape the existingdescriptions of the phenomena.

The second phase, that of the actual retroductive analysis ofthe data, involves hypothesizing about the possible mech-anisms or structures capable of generating the phenomena thathave been observed, measured, or experienced. An importantpart in this process involves abstracting and analyzing objectsin terms of their constitutive structures and causal powers.This enables the identification of the conditions and properties

necessary to generate the event under study. This mayinvolve revising the boundaries or scope of focus in the study.During these iterative phases of analysis, we engage in aprocess of abstraction where propositions are developed foruse in subsequent phases of investigation. We then considerthe context of the study and investigate how these necessarypowers and the liabilities recovered are at work in specificconditions in the concrete domain. The analytical dynamic atwork in the process of abstraction is resonant of “constantcomparison” in grounded theory (Strong and Volkoff 2010)in that it involves iterative cycles of reflection between aca-demic literature (original theories), data, and propositions inan effort to achieve analytical stability about the mechanisms(activated or unactivated) characterizing an event or outcome.

Following that, the third phase focuses on the criticalassessment and elimination of the alternative explanationsthat have been produced. When using MM research, thisentails comparison between the findings or inferences pro-duced by the combination of methods—or what Venkatesh etal. (2013) refer to as “meta-inferences.” It also involves theuse of complementary theoretical interpretations and abstrac-tions to explain how different mechanisms interact undercertain conditions and how they contribute to concrete socialphenomena.

Finally, action is required in order to circulate the researchfindings and, where applicable, see if the causal explanationsuncovered so far are satisfactory to an “intended audience”with background knowledge and expertise (Runde 1998). Inaddition, action stage often involves the development of pro-grams of change appropriate to the situation based on theresponses. To sum up, the above phases generally address thefollowing questions: “What is happening? Why is it hap-pening? How could the explanation be different? And, sowhat?” (Mingers 2001, p. 246). This retroductive process,variations of which have been described elsewhere (Daner-mark et al. 2002), should not necessarily dominate theresearch design of a mixed-methods research but, rather, canbe usefully drawn upon to implicitly guide activities as thestudy progresses.

An Example of Applying CriticalRealism to IS Research: A Mixed-Methods Study on ICT Adoptionand Firm Performance

In this section we provide an example of mixed-methodsresearch in IS using critical realism as the underlyingphilosophy. We draw on the retroductive process discussed



Figure 5. Mixed-Method Retroductive Research Design

in the previous section and demonstrate the methodologicalimplications of CR in action. Using an in-depth field studyconducted by two of the authors of this paper, we illustratehow MM research informed by a critical realist approach canreveal underlying mechanisms, validate findings, and gener-alize results in an interactive, responsivem and often iterativeprocess of inquiry. Our goal is to illustrate the value of MMresearch and how it can be guided by a CR perspective touncover robust meta-inferences in IS (Venkatesh et al. 2013).

The Effect of SWIFT Adoptionon Bank Performance

In this research we focus on one of the most hotly debatedtopics in IS research: the effect of ICT adoption on firm per-formance (Brynjolfsson 1993; Brynjolfsson and Hitt 1995,1996, 1997; Dewan and Min 1997; Lichtenberg 1995; Triplett1999). To address this matter, we conducted a longitudinal

study to understand the impact of a specific financial tele-communications innovation (SWIFT) on the performance ofbanks. Founded in Brussels in 1973, the Society for World-wide Interbank Financial Telecommunication (S.W.I.F.T.) isa cooperative, member-owned, international, organizationserving as a shared global communications platform and amessaging standard for cross-border financial transactions.SWIFT was originally founded with the objective of auto-mating and potentially replacing the Telex as a mean of com-munication between banks (Scott and Zachariadis 2012).Consequently, the nature of SWIFT and the benefits that itoffers are deeply rooted in the history of correspondentbanking and the development of ICTs since the 1970s. Con-sistent with the critical realist approach, in order to study theeffects of SWIFT adoption on bank performance, we designeda MM retroductive methodology with the purpose of identi-fying the mechanisms (liabilities and powers lying in the do-main of the real) that are capable of producing the events weobserve in the empirical domain (see Figure 5 for a summary).



Table 4. Main Steps During the Retroductive Process

Method Key Findings Assessment & Next StepsRetroductive

Emphasis

1 Exploratoryinterviews

“SWIFT is the plumbing of financial services,”hence effect on bank performance assumed

Helped identify the debate in the sector andguide the next steps of the research study;further the nature of SWIFT

Appreciation

2 Econometricmodeling(generalization)

SWIFT adoption is correlated with bank per-formance; used the whole population of SWIFTand non-SWIFT adopters in 29 countriesbetween 1977 and 2006

Identified a conflict between findings of explor-atory interviews and QUANT analysis; specifySWIFT adoption; identify structure and neces-sary relations of SWIFT

Appreciation/RetroductiveAnalysis

3 Interviews/His-torical narrativeon the role andbenefits of bigbanks (abstract& concreteresearch)

The impact of SWIFT is subject to historicaldevelopments in the banking sector and thedifferent context of adoption; technological,business standards, and other factors affectthe nature of SWIFT itself

Theorized on the structure of SWIFT (powers,capabilities, and liabilities–abstract level) andhow these are affected by the external mech-anisms in different context, whereby producingdifferent results (concrete research); need toconfine further QUANT analysis and take intoaccount QUAL findings

RetroductiveAnalysis

4 Econometricmodeling ofsmall banks

SWIFT adoption is correlated with small bankperformance (revenue increase); used asample of small, retail banks that adoptedSWIFT at a later satage (between 1998 and2006)

Identified a demi-regularity that was in line withprevious QUAL results; using previous findingsrestricted analysis to specific samples in orderto create closure; investigate further smallretail banks and explore value-generatingmechanisms


5 Small bank casestudy (concreteresearch)

SWIFT creates value through giving access tonew clients and products; helps increaserevenues (and not decrease costs as firstinstance); economies of scale through auto-mation and aligning standards

Identified a particular mechanism in action andhow the specific context (that of a small retailbank that adopted SWIFT late) influencedrealization of SWIFT capabilities and benefits;consider theories and eliminate competingexplanations


Appreciation Phase

As outlined in the first two steps of the retroductive processin Table 4, the “appreciation” phase provided the basis for ourfirst round of analytical boundary-making to scope theresearch. As discussed earlier, establishing the terms ofquasi-closure offers an opportunity to focus and in so doingprompts us to examine the possibility for data collection andmodes of analysis within these boundaries. Thus we set theboundaries of our study in recognition that SWIFT forms partof an identifiable sector offering a range of opportunities foraccess to statistical databases that enables the matching offirm data (accounts showing profit over time) and, as anindustry cooperative, it has a defined membership (withrecords of adoption). When related to the skill sets andresources available in our research team, these characteristicsinspired our choice of mixed-methods and provided a pointfor the process of retroductive analysis to begin.

As discussed in the section on designing MM research, in aCR approach qualitative methods are not used to establish anagenda that is then tested. Instead, these methods serve as auseful way to produce insights into new phenomena that will

inform subsequent phases of the retroductive process, some ofwhich may involve quantitative techniques. In our study ofSWIFT, exploratory open interviews were used in the appre-ciation phase to identify the distinguishing features of itscontext and begin the process of revealing complex dynamicsat work, as well as exploring the opportunities for data collec-tion (see step 1 in Table 4).

The insights from exploratory interviews helped us get asense of the issues that experts in the field regarded as impor-tant, thus confirming or revising the questions in our quali-tative fieldwork protocol. We were struck by what seemedlike a contradiction in terms: on the one hand, professionalsin the financial sector asserted that SWIFT is merely the“plumbing” of the banking sector and thus they would “ques-tion the rise in profits [attributed] to SWIFT,” while on theother they drew our attention to an on-going controversyregarding terms of membership (who gets to be a member andwhat kind of control this gave them over the priority given tothe SWIFT network and standards agenda). These situatedaccounts of SWIFT’s contribution to firm performance trig-gered further research in order to gather additional views andto investigate further opinions as well as motivating us to



reconsider the boundaries of research from situated narrativesby individuals to statistical descriptions of the SWIFT popula-tion. Were there sector-wide relational dynamics at work thatwere not observable in the average functional role? We setabout examining what the interplay between intensive and ex-tensive research methods would show (see step 2 in Table 4).

In parallel to the initial interview process, we had beenexploring various possibilities for using the quantitative datasets that were available (e.g., SWIFT adoption data and firmlevel data detailing bank revenues, expenses, size, etc.). Wedecided to engage in extensive research methods using formalor established statistical descriptions in an effort to produceinformation about the relationship between measures ofSWIFT adoption and that of bank performance (profitmargin). In so doing, we revised the scope of our study froma relatively small number of personal narratives to statisticson the entire SWIFT membership. The quantitative exercisewe performed included a panel dataset of 6,848 adopter andnon-adopter banks from 29 (mostly OECD) countries andcovered 10 years of financial performance data and 30 yearsof SWIFT adoption data. To identify the relationship ofinterest, we used a linear model of the form Πit = α +βSWIFTit + Tt + ηi + εit and performed a statistical analysisusing panel data econometrics.

The main finding from our analysis was that SWIFT adoptionwas indeed correlated with significantly higher bank perfor-mance (using different performance measures like ROS,ROA, and ROE and getting statistically significant results).This effect was persistent among different countries andregions and robust to different forms of estimation such asfixed-effects and first differences. In our case, sample selec-tion was not an issue because we used the whole populationof SWIFT adopters and matched it to a rich database thatincluded nearly every bank worldwide. We also tried dif-ferent techniques to exclude the possibility of reversecausality (the notion that more profitable banks would adoptSWIFT anyway, so the direction of any hypothetical causaleffect would be the opposite). This mode of analysis showedthat our variable of SWIFT adoption did not correlate withany measures of performance prior to SWIFT adoption. Also,by constructing lags, we were able to uncover patterns ofperformance drop in the first two years following SWIFTadoption by banks and subsequent increase in profit marginsleading to a significant effect that lasted up to an average of10 years.9

Holding both qualitative and quantitative data in mind, weattempted to develop our appreciation of the issues raised intopropositions that would inspire the next phase of analysis.This involved relating the preliminary analysis from ourfeasibility study to theories in this topic area: the capabilityof ICT to boost firm performance (also known as theproductivity paradox debate10); organizational capital and ITcomplementarities; the commoditization of technology; andnetwork externalities. This cycle of reflection with the litera-ture informed our hypothesizing about the adoption of SWIFTon firm performance. As a result, we set the received practi-tioner wisdom that “SWIFT is the plumbing of financialservices” alongside a competing theoretically driven propo-sition that “SWIFT’s impact on firm performance has beenoverlooked.”

Retroductive Analysis

As highlighted in step 3 of Table 4, an important element ofthe retroductive process is to abstract and analyze objects interms of their constitutive structures and causal powers. Inthe next phase of the study, we drew up a fieldwork designinvolving further interviews and analysis of historical archivedata with the aim of identifying the structures characterizingSWIFT (our object of research) in terms of its powers andliabilities in order to further understand the different viewsabout its impact on firm performance.

We began by identifying the powers characterizing the deepstructuring of SWIFT. Archive documents revealed that thefounders of SWIFT represented the most significant largebanks in the sector and their financial accounts showed thatthey provided a stable source of funding during its develop-ment. A combination of qualitative interview data, annualreports, and conference proceedings (video and text) revealedthat the founding of SWIFT centered on a single mission (toreplace Telex) so that prominent correspondent banks in the1970s could reduce costs by automating (error prone) manualtransaction processing. As a result of this highly focused stra-tegy, SWIFT’s resources and organizational agenda weresystematically organized around its realization. Technicalspecifications showed that the technology used at its foundingwas not state-of-the-art but rather provided a tried-and-testedinfrastructure which meant that their goal of exceeding expec-tations in terms of reliability and security was soon achieved,

9For more detail on the statistical data and econometric analysis see Scott etal. (2010).

10While the productivity paradox is considered resolved by many contem-porary scholars, the mechanisms and conditions under which ICT can bebeneficial (or not) are still questioned (Brynjolfsson 1993). Hence, thereseems to be a shift in the literature from the if to the how, making our MMretroductive approach more attractive.



Figure 6. Big Bank Scenario and Mechanisms in Action

which encouraged others to join. There were, however, signi-ficant liabilities during its development, including on-goingconcerns about whether a critical mass would join the net-work; the cost of integrating SWIFT with existing systems;dependence on telecom infrastructure dominated by postoffice monopolies in Europe; the absence of multi-jurisdictionregulation to govern cross-border transactions; and the factthat the organization and priority of SWIFT were subject tothe interests of its founders as expressed through the board ofdirectors and its terms of governance. The significant body ofdata gathered on this last point was summed up in the wordsof various interviewees as follows: “SWIFT was built by bigbanks, for big banks,” “Don’t forget your mother,” “Don’t doa VISA” (i.e., be a private company rather than a coopera-tive).

As our analysis became more refined, we began to appreciatethat these powers and liabilities were exercised to varyingdegrees at different points in the historical evolution ofSWIFT’s organization and network (Scott and Zachariadis2012). At the time of the founding of SWIFT (1970s), thecompany’s strategy was centered on “twenty banks getting ridof their problems.” As one of our interviewees pointed out, “We had about 60 to 70 people to process [600 to 700 tradesper day], now they do most likely 10,000 to 15,000 and haveabout 30 people.” Gaining a historical perspective drew our

attention to two important points: first, it became apparentthat the costs and benefits incurred by founder banks weredifferent from those experienced by late adopters; and second,that these late adopters were often much smaller banks com-pared to SWIFT’s founders (the big banks). Figure 6 depictsthe (causal) powers and liabilities underlying the mechanismsin action for large banks, which have been highlighted in step3 of the retroductive process shown in Table 4.

With further interviews and archival work, we were able touncover evidence that showed significant changes in the termsof governance affecting the benefits that different memberswould realize from their use of the SWIFT network andinvolvement in standards development.

The notion of [the SWIFT] cooperative changedover time. When I joined in ’88, there was one pricefor a message. Whoever sent it, and whatever thedestinations were....That was the interpretation of acooperative at that point in time: everybody pays thesame thing. It was only later that we started tointroduce segmentation by volume….That was amajor evolution.

These changes in the structure and capabilities of SWIFT aswell as potential contextual factors (such as sector develop-



ments) had significant implications for our understanding ofthe generative mechanisms in particular historical situationsand their effect on bank performance.

These insights into the changes that had happened over timealso meant that the intrinsic and extrinsic conditions of clo-sure anticipated in the study had to be rethought. Forexample, we realized that in our earlier econometric modeling(step 2 in Table 4) we had averaged the effect of SWIFT over30 years and across the entire SWIFT membership (more than3,000 diverse banks), which in light of the qualitative findingssuggested that changing benefits for different members overtime was not appropriate. We therefore needed to redraw theboundaries of possibility and create revised conditions ofquasi-closure; in other words, this phase of our study led us tochallenge our quantitative findings and revisit our model inorder to apply a new set of controls and restrict our analysisto specific samples that would give us results in chronologicalcategories as well as distinguish between bank size and type.Therefore, taking into account these qualitative insights on thecontext as well as SWIFT itself would create more focuseddemi-regularities.

In order to move forward with this further investigation, wehad to isolate the effect of the contingently related and histori-cally specific phenomena from the necessary relations(relying on the properties of SWIFT structures and liabilities)that produced these findings. More practically, by scopingour research, we were trying to satisfy some of the conditionsfor closure. On the one hand, intrinsic conditions for closurecould be achieved by ensuring as little variation as possible“in the object possessing the causal powers”—in this case,SWIFT (Sayer 1992, p. 122). If these conditions were nottaken into account, we would be comparing the effects causedfrom the SWIFT of the past (different standards, technologies,access to knowledge, one price fits all, etc.) with those relatedto the SWIFT of the future (advanced networking capabilities,business-oriented standards, value of the community, tieredand bundled pricing, etc.). On the other hand, achieving someof the extrinsic conditions for closure would ensure thedecrease in external mechanisms or situations that wouldaffect or differentiate the action of particular powers andliabilities in generative mechanisms of SWIFT. For example,the adoption of SWIFT from a smaller bank versus a largebank or a retail bank versus an investment bank could triggerdifferent mechanisms (see Figures 6 and 7).

With these revised conditions of closure in mind, we broke upour sample into different, more homogenous, subsamples andreexamined our initial relationship between SWIFT adoptionand bank performance. This further round of analysis (sum-marized in step 4 of Table 4) at a different level of granularity

uncovered some key demi-regularities. First of all, by differ-entiating between small banks and bigger banks we were ableto identify that smaller banks were, on average, correlatedwith substantial boosts in performance, whereas larger bankswere very slow in realizing significantly lower and somewhatambiguous performance results. In addition, most of theeffects on performance ratios were correlated with revenuestreams instead of expense cuts, something that was counter-intuitive and not consistent with the majority of our interviewdata. Finally, there was some indication that these patternswere primarily driven by retail banks and late adopters in oursample. These demi-regularities, achieved by assuming aconsiderable degree of closure, revealed some nonobviousfindings, which provided additional information and droveanother round of qualitative enquiry that would dig evenfurther beneath the surface in order to uncover the necessaryrelations (excluding any contextual relationships) and causalpowers that SWIFT holds at a certain point in time, and howthey cause the increased performance of banks (if at all).Through this process we were able to further assess theinternal validity and robustness of our findings, as qualitativedata enrich and substantiate causal explanations coming fromstatistical covariations (Modell 2009).

Having identified these demi-regularities, we sought toconduct a case study in a small retail bank (step 5 of Table 4)and supplement it with other sources in order to investigatethe mechanism under which SWIFT can potentially add valueto financial institutions in this context. One of the key inter-views with the head of a bank in the UK, who was personallyresponsible for the adoption of SWIFT, produced narrativesthat were consistent with the relationships illustrated in theextensive research and provided significant insights intoSWIFT implementation and the specific benefits that itgenerated:

There was an appreciation that [installation] ofSWIFT would improve our service, but it would alsodrive up our costs…there was feedback from ourmarketing effort, which said that we would be ableto increase our business, increase our volume, if wehave SWIFT….a number of relationships that wehad been trying for, visiting those banks, our cor-responding banking department, marketing efforts,etc., they actually crystallized after we installedSWIFT.

The above description provides useful information on thevalue-generating mechanism activated by the adoption ofSWIFT and underlines the necessary components and lia-bilities that trigger these mechanisms (see Figure 7). Forexample, SWIFT, as a common standard, and network tech-



Figure 7. Small Bank Scenario and Mechanisms in Action

nology that provides connectivity and access to other banks(necessary and internal relations) enabled the small retail bankat one point in time (contingent conditions) to increaserevenues and eventually boost its performance. Specifically,SWIFT was instrumental in adding value by attracting newclients and providing SWIFT-related products: “We are pro-cessing and providing service to our correspondent banks thatwe previously were not able to provide.” On the contrary,SWIFT-related costs seemed to be a negative factor for sucha small bank; one of our interviewees pointed out that “if wecould relate costs to SWIFT, we would have gone [to SWIFT]many years ago.” This difficulty was also confirmed as anissue from other SWIFT employees who raised concerns forthe high total cost of ownership (TCO) for smaller banks. Itis important to note that while the dynamics involved inrealizing SWIFT benefits were similar to the demi-regularitiesuncovered, the mechanism that drove these dynamics couldhave not been uncovered without additional qualitativeinsights and explanations:

I’d say that over the first year, definitely over thefirst 12 months, our profit margins came down. Andthen, the two things that we did, one, we were ableto increase our transactions, and two, we were ableto rationalize all the interface building that we did,which further increased our capacity.

The evidence from the case study suggests that, under dif-ferent circumstances (e.g., of a small retail bank whichadopted SWIFT late in the diffusion curve), the same orsimilar causal powers and liabilities can be exercised (orremain unexercised), leading to different mechanisms andgenerating different results on how a bank realizes the bene-fits from SWIFT. Figure 7 depicts the (causal) powers andliabilities underlying the mechanisms in action for small retailbanks, which are also highlighted in step 5 of the retroductiveprocess shown in Table 4.

The combination of quantitative and qualitative methodshelped us gather evidence about the characteristics of banksand the powers, capabilities, and liabilities of SWIFT indifferent periods and allowed us to create (quasi-)closure inorder to capture robust demi-regularities. This gave usconfidence to generalize and suggest that the generativemechanisms observed in our case study will also apply toother small retail banks in our sample and thus will drive thebank performance effect observed in the demi-regularities(see Figure 7). The establishment of external validity doesnot just demonstrate the premise that our assumptions for theexisting mechanisms will apply to other similar contexts, butalso denotes a causal statement that links SWIFT adoptionwith bank performance through the realization of the genera-tive mechanisms identified. This causal link is the main



property of internal validity, which ensures that the events weobserve are manifestations of particular generative mech-anisms in a particular context (see Table 1 and Figure 3).

Apart from abstracting the powers and liabilities of SWIFTfrom their context, a critical element of the retroductiveprocess is to engage in concrete research and investigate howthese powers and liabilities play out in the real world whilebeing influenced by the external and contingent factors(external mechanisms). In this regard, the interview dataproved informative because it offered situated insights andinterpretations of how these contingent factors influence thechange that is generated from the mechanisms of SWIFT. Forexample, we found that some of the issues influencing theimpact of SWIFT on firm performance were external andbound to broader developments both within the sector andoutside it (for example, the invention of the Internet Mes-saging Protocol). As the SWIFT network grew and thedevelopment of standards extended from international pay-ments to other parts of the financial transaction life cycle,more diverse institutions began to request membership andconnectivity to SWIFT (e.g., broker-dealers in securities,other sector infrastructures such as CREST). Thus, the impor-tance of the context (global financial sector and the needs ofthe banks at the time) was immensely critical to how thepowers and liabilities were realized and hence to the effect ofSWIFT on bank performance.

The above description, shows not only the value of mixed-methods in providing closure, and hence abstract explanationsof how the events we observe are produced, but also the rolethat the critical realist ontology and retroductive methodologyplays in facilitating a critical methodological pluralismclosely linked to its meta-theoretical assumptions (Danermarket al. 2002). As Table 4 outlines, the main steps in our retro-ductive process and the key findings and contributions fromeach method led us to the use of different methods thatenabled us to construct our abstractions, perform research atthe concrete level, and revise the conditions of closure toachieve more robust validity. In prior literature, the emphasishas been on combining the output of qualitative and quanti-tative studies at the end of a study to form end-game meta-inferences (Venkatesh et al. 2013). In this study, the CRapproach guided the dynamic interplay and responsive modeof analyses that not only mutually informed our findings butincreased their validity by systematically challenging andrevising the conditions of closure and by asking “why” ques-tions in order to assess the causal explanations (Runde 1998).

A further way in which our CR mixed-methods approachimproved the quality of contribution made by our study wasto ensure construct validity in both our qualitative and quan-

titative studies. This was apparent throughout the study butwas of particular valence when the differences in findingsfrom the intensive and extensive methods motivated us todeconstruct the implementation of SWIFT into historicalphases in order to get a deeper understanding of the timing ofSWIFT adoption. Through the combination of methods, wewere able to assess whether the variable used in our quan-titative study provided robust findings about events at thelevel of the actual. In other words, was our SWIFT adoptionvariable an empirical trace of the actual events and did itcorrespond with what happened in reality regarding thephysical implementation of SWIFT and also with what thecommunity agrees to be the norm? The latter is importantbecause while the physical implementation of SWIFT canindeed be accurately measured, the realization of SWIFTadoption also implies its usage. While we did not have accessto representative firm-level usage data, we uncovered quali-tative evidence suggesting that due to the costs involved whenbanks adopt SWIFT, they tend to take advantage of its capa-bilities as much as possible and justify its use, thus makingthe SWIFT adoption variable a good proxy of SWIFT usage. In parallel, due to the diverse environment and governmentalcomplexities of SWIFT, which were recovered from ourarchival research, we felt that we had to complement ourstudy with a variety of interviews (see Table 5 for the numberand type of interviews conducted), which would furtherenhance our construct validity.

Assessment and Elimination Stage

As the research progressed, the analysis of data from inten-sive/extensive methods provided possible narratives for therelationships and the empirical events observed whichenabled us to eliminate alternative explanations for the gener-ative mechanisms causing the phenomena we encountered. Inthis regard, the activity of elimination implicitly took placethroughout the retroductive process as we assessed the differ-ent results coming from the intensive/extensive methods anddecided on next steps to further investigate more probableexplanations for the empirical phenomena. This correspondswith the column “assessment and next steps” in Table 4,where the main implications of the different retroductive stepsand their role in guiding the way forward are described.Eventually, in the elimination stage at the conclusion of theanalysis (see step 5 in Table 4), we considered the differenttheories that could provide explanations and complement theempirical findings.

Overall, even though it was pointed out to us early on that “abank without SWIFT is like an office without a phone”implying the commoditization of payment technologies in



Table 5. Number and Type of Interviews

Number of Interviews Positions

SWIFT 49 9 Executives; 13 Directors; 27 Managers

Other financial services infrastructures 4 DTCC; Euroclear; SEPA

Banks 14 3 Executives; 5 Directors; 6 Managers

Technology and consulting companies 3 3 Directors

TOTAL 70

banking and the inability to create competitive advantage, wewere able to construct more powerful narratives that describedthe mechanisms through which members achieved valuethrough SWIFT membership. Having said that, these distinc-tive mechanisms proved to be historically specific since thenature of SWIFT adopters and the organization of SWIFT hasshifted over time. If the initial findings had not been placedin the context of a historical narrative the results would havebeen very misleading and may likely have imposed a trans-historical explanation to time-bound objects. Similar robust-ness checks were applied to our use of other methods; forexample, without the quantitative analysis that uncoveredimportant demi-regularities for small retail banks (step 3 inTable 4), it would have been very difficult to further explorethe role of the type and size of banks as well as the timing ofbenefits realized from SWIFT adoption. Conversely, withquantitative methods alone, it would have been impossible tohypothesize about the generative mechanisms that led to therelationship between SWIFT adoption and bankperformance.11

The above illustration of our MM IS research examplehighlights the importance of both the abstract knowledge ofmechanisms and the empirical knowledge of contingentrelations (concrete). It is thus assumed that the incomplete-ness of the explanations of mechanisms and the lower qualityof meta-inferences can come from the lack of either of thesetwo, since they are very closely related (Sayer 1992). It is,therefore, the aim of the retroductive framework to ensure thatabstract research (analyzing objects in terms of their consti-tutive structures and causal powers) as well as concreteresearch (study the consequences of combining these withother contingent conditions) are performed in this process.Ignoring this stratified ontology and relying only on obser-vations (e.g., that “X is generally followed by Y”) can be very

misleading, as we saw that the same mechanism can some-times generate different events and vice versa.

In addition to the empirically discovered contingently relatedphenomena, theoretical propositions must be combined inorder to move back from abstract concepts to the concrete(Sayer 1992). In our case, for example, IT and productivitytheory argue that additional investments in technology andimplementation of supplementary business processes coulddelay the streamlining of IT with the rest of the business buteventually add significantly to the overall bank performance. Similarly, the theory of network externalities highlights thepower of networks to create value through connectivity to anincreasing user base of a specific network technology. Thesetheoretical claims inspired a set of research questions in orderto identify similar effects: Does SWIFT adoption follow thestandard S-shaped curve? Does SWIFT add more value to itsmembers as the number of adopters grows? Eventually, theoverall impression we got from our fieldwork interviews andarchival work was that complementarities related to SWIFTadoption also played an important role for the actualizationand timing of the benefits. This relationship (which wasconfirmed with our qualitative work) and the notion of net-work effects could also provide explanations about the largecoefficients of SWIFT adoption and the long lags to perfor-mance.

Discussion

Recent methodological developments in IS research haveestablished guidelines for mixed methods (Tashakkori andTeddlie 2003; Venkatesh et al. 2013), and the value of criticalrealism as a theoretical and philosophical approach in the useof case studies (Wynn and Williams 2012) and other methods(Mingers 2000, 2001, 2004) has been noted. However, therehas been little exposition as to how the retroductive approachin CR can usefully act as a guide empirically in deploying amixed-methods study. In this paper, we illustrate the retro-ductive process in CR by drawing on a large-scale study of

11Even if by chance we discovered the effect of SWIFT on the total revenuesof small banks, we could never be sure that we are observing a causalrelationship, not to mention the significance of complementary IT invest-ments and business processes in realizing the benefits from SWIFT.



SWIFT to uncover the generative mechanisms that helpedunderstand the effect of SWIFT adoption on bank perfor-mance. Within this retroductive MM process, we demon-strated how the notions of demi-regularities and quasi-closurecan lead to improvements in validity and support the devel-opment of robust meta-inferences. Forming different abstrac-tions from SWIFT data we managed to identify its constituentfactors and theorize about the causal mechanisms that gener-ate the events we observe (e.g., SWIFT automates processesand creates economies of scale that help banks decreaseexpenses and increase revenues). However, we realized that,depending on contingent factors (e.g., timing of SWIFTadoption and characteristics of adopters), these effects weredistorted, leading to different results (e.g., smaller retail banksdidn’t benefit from expense cuts but were able to increasetheir sales, etc.). Two key points emerged from this retroduc-tive approach. First, our study emphasized the importance ofabstract and concrete research and their relationship in theresearch process, which leads to the identification of causalmechanisms and the formation of meta-inferences. Second, itillustrated the key role of demi-regularities, and of extrinsicand intrinsic closure in ensuring validity and allowing forgeneralization. We now discuss each of these in turn.

The Role of Abstraction in IdentifyingGenerative Mechanisms and FormingMeta-Inferences

Abstractions are a crucial part of the retroductive analysisbecause they allow us to strip out and separate the constitutivestructures and necessary (internal) properties of the objects weare studying from the contingent (external) ones that affect theevents we observe (Sayer 1992; Tsoukas 1989). Within thisprocess, mixed-methods played a key role in identifyinggenerative mechanisms (revealing why things are as theyappear) and forming robust meta-inferences that can guide ourthinking in different research contexts. In our case, under-standing the components and structure (as a set of internallyrelated objects) of SWIFT (e.g., its technological capabilitiesand network infrastructure, message standards, communityand governance, etc.) helped us to theorize about what causalpowers and liabilities SWIFT might hold. For example, theability of SWIFT to transmit payment messages quickly andsecurely, or the availability of knowledge and expertise fromthe SWIFT community, provides key information about itspotential impact on organizations. On the other hand, con-crete research helped us understand what happened whenthese powers and liabilities were exercised under certaincontingent conditions. In our case, these were recovered byemploying qualitative methods, including a historical study,alongside econometric analyses to understand the differences

between adopters in the early and late years and the differentcharacteristics of financial institutions and the banking sectorin the 1970s and the 21st century.

This use of quantitative and qualitative methods was vital inorder to identify both internal (necessary) and external (con-tingent) relations. In the early stage of our research inter-views, historical narratives, and quantitative analysis un-covered constituents of SWIFT such as the size of its net-work, its technological capabilities, governance structure, etc.,while, in the later phases, further interviews, archival work,quantitative analysis, and case study enquiry provided infor-mation about the contextual factors such as the types ofSWIFT adopters, and regional and temporal influences. Theinterplay between methods not only helped to ensure com-pleteness and complementarity in our study but also advancedour research strategy by enabling us to build on the resultsfrom different methods and develop a systematic process ofinquiry through retroduction. Finally, different methods wereused to compensate for the limitations of the other methodsand thus provide diversity by looking at different levels ofabstraction as shown in Table 3.

The identification of causal mechanisms through abstractionand concrete research has commonalities with meta-inferences in MM research as both concepts effectively usequantitative and qualitative methods (Tashakkori and Teddlie2003; Venkatesh et al. 2013). However, there are some keydifferences. Meta-inferences are often described as a mergebetween the quantitative and qualitative inferences once aresearch enquiry has come to an end. However, our studyhighlights the identification of generative mechanisms as aniterative and nonlinear retroductive process where quantitativeand qualitative methods work in conjunction and feed intoeach other until a robust mechanism that can explain thephenomena is recovered. Further, the main route of devel-oping meta-inferences is principally an inductive one(Venkatesh et al. 2013), while the identification of causalmechanisms is driven by a creative retroductive process(Mingers 2004, 2005). Had we used a linear inductive ap-proach to meta-inferences that combined findings from thetwo different types of research (extensive and intensive), wewould have been confronted by conflicting views on theimpact of SWIFT on organizations. For example, our initialinterviews supported the argument that SWIFT is a neutralplumbing of the banking sector that did not add to the bottomline performance of banks, whereas statistical findingsdemonstrated a robust correlation between SWIFT adoptionand measures of financial performance. The retroductivelogic allowed us to go beyond the conventional synthesizingof the results and question the assumptions and conditions(i.e., degree of closure and contingent factors) under which



they were produced in order to uncover the generativemechanisms.

Demi-Regularities and ValidityThrough Closure

In our paper we also demonstrate how demi-regularities notonly focus the research design but also facilitate the hypothe-sizing and generalizing about existing causal mechanisms. Inaddition, we show that they can also help to assess andexplain the results in the analysis phase. By acknowledgingthat social systems are inherently open, and thus observable,events are a result of complex interactions between severalcausal mechanisms that may offset each other. We highlightthat conditions of closure are imperative to establish thesedemi-regularities and produce valid generalizations and causalarguments. For that reason, we show how intrinsic andextrinsic quasi-closure was important in our study in estab-lishing validity, and in particular providing some possiblegeneralized statements about the real mechanisms in place bymerging them with actual events (Tsoukas 1989).

For example, intrinsic closure in our study was achieved whenwe restricted our research to reflect no qualitative variation inSWIFT, that is to say, when the necessary constituents ofSWIFT remained largely unchanged during a specific periodof time. In parallel, extrinsic closure, which required that theexternal mechanisms in the environment of SWIFT (e.g., thecountry or region of use, the timeframe, the type of adopterorganizations, etc.) are constant, was achieved when werestricted our sample to a particular set of banks of specifictype and size. Once these two conditions were applied wecould argue that the demi-regularities observed in terms ofevent correlations could be a manifestation of a causal mech-anism recovered during the interplay between the abstract andconcrete research.

In our example, we demonstrated how with the use of MMswe managed to narrow our study down to a specific sub-sample where the two closure conditions could be met. Forexample, through the interviews we conducted, we realizedthat certain intrinsic aspects of SWIFT changed over time, sowe had to confine our quantitative study to particular periodswhere there wouldn’t be much qualitative variation of theseaspects (intrinsic closure). As Runde and de Rond (2010)note, not only is it necessary to embrace that the questionsasked and answers given by people depend on their interestand situation, but more importantly

this kind of situatedness, given the impossibility ofciting all the causes of an event, is in fact a resource

that makes causal explanations feasible at all. Thereason for this has to do with our why-questionsalways being from a point of view, thereby alwaysfocusing on one or other aspect of some event….That people always come to things and ask aboutthem from a particular (subjective or situated) pointof view, rather than from what Nagel (1986) callsthe “view from nowhere,” provides the necessaryrestriction on the cases we need to cite in any par-ticular explanation (p. 436).

The qualitative methods were boundary-making, helping usto narrow the range of issues in our inquiry, revise the scopeof (quasi-)closure at work, as well as forming part of theelimination process by which we moved the study forward.

Meanwhile, through our statistical analysis, we uncovered acorrelation between smaller retail banks and SWIFT adoption.As a result, we restricted our qualitative study to that domainof small retails banks and we were able to isolate and uncoverthe mechanisms that lead to increased bank performancewithout the influence of other objects and mechanisms (i.e.,extrinsic closure). Meeting these conditions allowed us togeneralize our findings and argue that within that specificdomain (that of a small, late-adopter retail bank) the applica-tion of SWIFT led to increased performance. This generali-zation in the form of internal (establishing a causal linkbetween events) and external (generalizing the same mech-anism between different banks) validity was achieved throughclosure and the recovery of robust demi-regularities in theprocess of our retroductive approach to mixed methods.

Theory generation in CR lies on the development ofabstractions that hypothesize about and identify generativemechanisms (rather than regularities) that are observed indifferent contexts and are expressed by means of empiricalcategories—empirical generalizations (Danermark et al.2002). In that respect, generalizing means to identifyuniversal structures or mechanisms that are theorized to existin different contexts but not necessarily cause the same events(Wynn and Williams 2012). Therefore, in our study, theprocess of generalizing the findings was not focused aroundreplication of the correlation in a different context but rather,through the use of MM, it examined the links betweengenerative mechanisms in different contexts, for example, inother small retail banks. Our study also identifies differentlevels of generalization. For example, going from a specificcase study to a small sample (Seddon and Scheepers 2012),and from there to the wider population, so as to provideexplanations in other contexts (Lee and Baskerville 2003; Leeand Hubona 2009; Seddon and Scheepers 2012; Venkatesh etal. 2013).



Conclusions

In this paper, we contribute a deeper understanding of the roleof causality and validity in the retroductive mixed-methodsapproach of critical realism by leveraging CR’s rich onto-logical assumptions (Danermark et al. 2002; Sayer 1992) andby adopting its “attitude of epistemological modesty” (Rundeand de Rond 2010, p. 433). The stratified ontology of CR andthe notion of generative mechanisms as the causal linkbetween the real and the empirical motivated us to considerthe use of diverse strategies for mixing intensive and exten-sive methods in our study of the impact of SWIFT adoptionon bank performance. We illustrate how the logic of retro-duction can guide a productive and dynamic interplaybetween methods involving constant comparison in our CR-inspired, mixed-methods approach. By exploring the inter-action between situated narrative and statistical descriptionsof populations we are not only better able to uncovergenerative mechanisms but also to have stronger validityclaims, which help us to develop more robust meta-inferences. In the process, we demonstrate the significance ofabstract and concrete research in identifying internal (neces-sary) properties of the exercised and unexercised mechanismsbased on the external or contextual conditions and how, bysystematically challenging and revising conditions of closure,we were able to uncover unobvious demi-regularities whileenhancing the validity of our study.

We believe that the methodological implications of criticalrealism we have advanced in this paper go beyond the singleCR-led case study methodology that has characterizedprevious studies in this literature (Wynn and Williams 2012). In so doing, they may provide useful insights for others topursue retroductive analyses of in-depth mixed-methods ISresearch. We suggest that future research could usefullyexamine the critical dimension of CR by exploring theexercised and unexercised mechanisms influencing ICT inno-vation in particular sectors. Using a CR-led mixed methodsapproach in IS research can fruitfully challenge acceptedwisdom in the industry and allow for effective perspective-making with stakeholders by moving between intensive andextensive methodologies that mutually inform each other.

References

Aral, S., Brynjolfsson, E., and Wu, D. J. 2006. “Which Came First,IT of Productivity? The Virtuous Cycle of Investment and Usein Enterprise Systems,” in Proceedings of the 27th InternationalConference on Information Systems, Milwaukee, WI.

Archer, M., Bhaskar, R., Collier, A., Lawson, T., and Norrie, A.1998. Critical Realism: Essential Readings, London: Routledge.

Bache, I. W. 2003. “Critical Realism and Econometrics,” WorkingPaper, Central Bank of Norway.

Benbasat, I., and Weber, R. 1996. “Research Commentary:Rethinking ‘Diversity’ in Information Systems Research,”Information Systems Research (7:4), pp. 389-400.

Bhaskar, R. 1975. A Realist Theory of Science, Leeds, UK: LeedsBooks.

Bhaskar, R. 1978. The Possibility of Naturalism, Sussex, UK: Harvester Press.

Bhaskar, R. 1989. Reclaiming reality, London: Verso.Bhaskar, R. 1998. “Philosophy and Scientific Realism,” in Critical

Realism: Essential Readings, M. Archer, R. Bhaskar, A. Collier,T. Lawson, and A. Norrie (eds.), London: Routledge, pp. 16-47.

Brynjolfsson, E. 1993. “The Productivity Paradox of InformationTechnology,” Communications of the ACM (36:12), pp. 66-77.

Brynjolfsson, E., and Hitt, L. M. 1995. “Computers as a Factor ofProduction: The Role of Differences Among Firms,” Economicsof Innovation and New Technology (3:3), pp. 183-199.

Brynjolfsson, E., and Hitt, L. M. 1996. “Paradox Lost? Firm-LevelEvidence on the Returns to Information Systems Spending,”Management Science (42:4), pp. 541-558.

Brynjolfsson, E., and Hitt, L. M. 1997. “Computing Productivity:Are Computers Pulling Their Weight?,” Working Paper,Massachusetts Institute of Technology and Wharton School ofBusiness, University of Pennsylvania.

Carlsson, S. A. 2003. “Advancing Information Systems Evaluation(Research): A Critical Realist Approach,” Electronic Journal ofInformation Systems Evaluation (6:2), pp. 11-20.

Carlsson, S. A. 2005. “Developing Information Systems DesignKnowledge: A Critical Realist Perspective,” The ElectronicJournal of Business Research Methodology (3:2), pp. 93-102.

Cartwright, N. 1989. Nature’s Capacities and Their Measurement,Oxford: Clarendon Press.

Collier, A. 1994. Critical Realism, London: Verso.Creswell, J. W. 2003. Research Design: Qualitative, Quantitative,

and Mixed Methods Approaches, London: Sage Publications.Creswell, J. W., and Clark, V. L. P. 2007. Designing and Con-

ducting Mixed Methods Research, Thousand Oaks, CA: SagePublications.

Danermark, B., Ekstrom, M., Jakobsen, L., and Karlsson, J. C.2002. Explaining Society: Critical Realism in the SocialSciences, London: Routledge.

Denzin, N. K., and Lincoln, Y. S. 1994. Handbook of QualitativeResearch, N. K. Denzin and Y. S. Lincoln, (eds.), ThousandOaks, CA: Sage Publications.

Dewan, S., and Min, C. 1997. “The Substitution of InformationTechnology for Other Factors of Production: A Firm LevelAnalysis,” Management Science (43:12), pp. 1660-1675.

Dobson, P. J. 2001. “The Philosophy of Critical Realism—AnOpportunity for Information Systems Research,” InformationSystems Frontiers (3:2), pp. 199-210.

Downward, P., and Mearman, A. 2002. “Critical Realism andEconometrics: Constructive Dialogue with Post-KeynesianEconomics,” Metroeconomica (53:4), pp. 391-415.

Downward, P., and Mearman, A. 2006. “Retroduction as Mixed-Methods Triangulation in Economic Research: ReorientingEconomics into Social Science,” Cambridge Journal ofEconomics (31:1), pp. 77-99.



Draca, M., Sadun, R., and Van Reenen, J. 2007. “Productivity andICT: A Review of the Evidence,” in The Oxford Handbook ofInformation and Communication Technologies, R. Mansell, C.Avgerou, D. Quah, and R. Silverstone (eds.), Oxford, UK: Oxford University Press, pp. 100-147.

Fleetwood, S. 1999. “Situating Critical Realism in Economics,” inCritical Realism in Economics, S. Fleetwood (ed.), London: Routledge, pp. 127-135.

Fleetwood, S. 2001. “Causal Laws, Functional Relations andTendencies,” Review of Political Economy (13:2), pp. 201-220.

Giddens, A. 1979. Central Problems in Social Theory: ActionStructure and Contradiction in Social Analysis, London: Macmillan.

Granovetter, M. 1985. “Economic Action and Social Structure:The Problem of Embeddedness,” American Journal of Sociology(91:3), pp. 481.

Guba, E. G., and Lincoln, Y. S. 2005. “Paradigmatic Contro-versies, Contradictions, and Emerging Confluences,” in Hand-book of Qualitative Research, N. K. Denzin and Y. S. Lincoln(eds.), Thousand Oaks, CA: Sage Publications, pp. 191-215.

Hands, W. D. 1999. “Empirical Realism as Meta-Method: TonyLawson on Neoclassical Economics,” in Critical Realism inEconomics, S. Fleetwood (ed.), London: Routledge, pp. 169-186.

Hoover, K. D. 1997. “Econometrics and Reality,” Working Paper, Department of Economics, University of California, Davis.

Johnston, R. B., and Smith, S. P. 2010. “How Critical RealismClarifies Validity Issues in Theory-Testing Research: Analysisand Case,” in Information Systems Foundations: The Role ofDesign Science, D. N. Hart and S. D. Gregor (eds.), Acton,Australia: Australian National University Press.

Landry, M., and Banville, C. 1992. “A Disciplined MethodologicalPluralism for MIS Research,” Accounting, Management andInformation Technology (2), pp. 77-97.

Law, J., and Urry, J. 2004. “Enacting the Social,” Economy andSociety (33:3), pp. 390-410.

Lawson, T. 1994. “A Realist Theory for Economics,” in NewDirections in Economic Methodology, R. Backhouse (ed.),London: Routledge, pp. 257-285.

Lawson, T. 1997. Economics and Reality, London: Routledge.Layder, D. 1990. The Realist Image in Social Science, New York:

St. Martin’s Press, pp. 1-189.Lee, A. S. 1999. “Rigor and Relevance in MIS Research: Beyond

the Approach of Positivism Alone,” MIS Quarterly (23:1), pp.29-34.

Lee, A. S., and Baskerville, R. L. 2003. “Generalizing Generali-zability in Information Systems Research,” Information SystemsResearch (14:3), pp. 221-243.

Lee, A. S., and Hubona, G. S. 2009. “A Scientific Basis for Rigorin Information Systems Research,” MIS Quarterly (33:2), pp.237-262.

Lichtenberg, F. R. 1995. “The Output Contributions of ComputerEquipment and Personnel: A Firm-Level Analysis.,” Economicsof Innovation and New Technology (3:3), pp. 201-217.

Lincoln, Y. S., and Guba, E. G. 1985. Naturalistic Enquiry,Thousand Oaks, CA: Sage Publications.

Lincoln, Y. S., and Guba, E. G. 2000. “Paradigmatic Contro-versies, Contradictions, and Emerging Confluences,” in Hand-book of Qualitative Research, N. K. Denzin and Y. S. Lincoln(eds.), Thousand Oaks, CA: Sage Publications, pp. 163-188.

Markus, M. L., Steinfield, C. W., Wigand, R. T., and Minton, G.2006. “Industry-Wide Information Systems Standardization asCollective Action: The Case of the U.S. Residential MortgageIndustry,” MIS Quarterly (30:Special Issue), pp. 439-466.

Maxwell, J. A. 1992. “Understanding and Validity in QualitativeResearch,” Harvard Educational Review (62:3), pp. 279-300.

McEvoy, P., and Richards, D. 2006. “A Critical Realist Rationalefor Using a Combination of Quantitative and QualitativeMethods,” Journal of Research in Nursing (11:1), pp. 66-78.

Mingers, J. 2000. “The Contribution of Critical Realism as anUnderpinning Philosophy for OR/MS and Systems,” The Journalof the Operational Research Society (51:11), pp. 1256-1270.

Mingers, J. 2001. “Combining IS Research Methods: Towards aPluralist Methodology,” Information Systems Research (12:3),pp. 240-259.

Mingers, J. 2003. “The Paucity of Multimethod Research: AReview of the Information Systems Literature,” InformationSystems Journal (13:3), pp. 233-249.

Mingers, J. 2004. “Real-izing Information Systems: Critical Real-ism as an Underpinning Philosophy for Information Systems,”Information and Organization (14:2), pp. 87-103.

Mingers, J. 2005. “A Critique of Statistical Modelling in Manage-ment Science from a Critical Realist Perspective: Its Role WithinMultimethodology,” Journal of the Operational Research Society(57:2), pp. 202- 219.

Mingers, J., and Gill, A. (Eds.). 1997. Multimethodology,Chichester, UK: John Wiley & Sons Ltd.

Modell, S. 2009. “In Defence of Triangulation: A Critical RealistApproach to Mixed Methods Research in ManagementAccounting,” Management Accounting Research (20:3), pp.208-221.

Morse, J. M., Barrett, M., Mayan, M., Olson, K., and Spiers, J.2002. “Verification Strategies for Establishing Reliability andValidity in Qualitative Research,” International Journal ofQualitative Methods (1:2), pp. 13-22.

Myers, M. 1997. “Qualitative Research in Information Systems,”MIS Quarterly (21:2), pp. 241-243.

Nandhakumar, J., and Jones, M. 1997. “Too Close for Comfort?Distance and Engagement in Interpretive Information SystemsResearch,” Information Systems Journal (7), pp. 109-131.

Nagel, T. 1986. The View from Nowhere, New York: OxfordUniversity Press.

Ridenour, C. S., and Newman, I. 2008. Mixed Methods Research:Exploring the Interactive Continuum, Carbondale, IL: SouthernIllinois University Press.

Robey, D. 1996. “Research Commentary: Diversity in InformationSystems Research: Threat, Promise, and Responsibility,” Infor-mation Systems Research (7:4), pp. 400-408.

Runde, J. 1998. “Assessing Causal Economic Explanations,”Oxford Economic Papers (50:2), pp. 151-172.

Runde, J., and de Rond, M. 2010. “Evaluating Causal Explanationsof Specific Events,” Organization Studies (31:4), pp. 431-450.



Sayer, A. 1992. Method in Social Science: A Realist Approach,London: Routledge.

Sayer, A. 2000. Realism and Social Science, London: Sage Publi-cations, pp. 1-301.

Scott, S. V., Van Reenen, J., and Zachariadis, M. 2010. “TheImpact of the Diffusion of a Financial Innovation on CompanyPerformance: An Analysis of SWIFT Adoption,” WorkingPaper, Centre for Economic Performance, London School ofEconomics.

Scott, S. V., and Zachariadis, M. 2012. “Origins and Developmentof SWIFT, 1973-2009,” Business History (54:3), pp. 462-482.

Seddon, P. B., and Scheepers, R. 2011. “Towards the ImprovedTreatment of Generalization of Knowledge Claims in ISResearch: Drawing General Conclusions from Samples,” Euro-pean Journal of Information Systems (21:1), pp. 6-21.

Smith, M. 2006. “Overcoming Theory–Practice Inconsistencies:Critical Realism and Information Systems Research,” Infor-mation and Organization (16:3), pp. 191-211.

Strong, D. M., and Volkoff, O. 2010. “UnderstandingOrganization–Enterprise System Fit: A Path to Theorizing theInformation Technology Artifact,” MIS Quarterly (34:4), pp.731-756.

Tashakkori, A., and Teddlie, C. 1998. Mixed Methodology: Com-bining Qualitative and Quantitative Approaches, London: SagePublications, pp. 1-185.

Teddlie, C., and Tashakkori, A. 2003. “Major Issues and Contro-versies in the Use of Mixed Methods in Social and BehavioralScience,” in Handbook of Mixed Methods in Social andBehavioral Science, A. Tashakkori and C. Teddlie (eds.),Thousand Oaks, CA: Sage Publications, pp. 3-50.

Teddlie, C., and Tashakkori, A. 2009. Foundations of MixedMethods Research, Thousand Oaks, CA: Sage Publications.

Triplett, J. E. 1999. “The Solow Productivity Paradox: What DoComputers Do to Productivity?,” The Canadian Journal ofEconomics (32:2), pp. 309-334.

Tsoukas, H. 1989. “The Validity of Idiographic Research Explana-tions,” Academy of Management Review (14:4), pp. 551-561.

Venkatesh, V., Brown, S. A., and Bala, H. 2013. “Bridging theQualitative–Quantitative Divide: Guidelines for ConductingMixed Methods Research in Information Systems,” MISQuarterly (37:1), pp. 21-54.

Volkoff, O., Strong, D. M., and Elmes, M. B. 2007. “TechnologicalEmbeddedness and Organizational Change,” OrganizationScience (18:5), pp. 832-848.

Walsham, G. 1993. Interpreting Information Systems inOrganizations, West Sussex, UK: John Wiley & Sons.

Walsham, G. 1995a. “The Emergence of Interpretivism in ISResearch,” Information Systems Research (6:4), pp. 376-394.

Walsham, G. 1995b. “Interpretive Case Studies in IS Research:Nature and Method,” European Journal of Information Systems(4:2), pp. 74-81.

Wooldridge, J. M. 2006. Introductory Econometrics: A ModernApproach, Mason, OH: South-Western.

Wynn, D., and Williams, C. K. 2012. “Principles for ConductingCritical Realist Case Study Research in Information Systems,”MIS Quarterly (36:3), pp. 787-810.

Yin, R. K. 2003. Case Study Research: Design and Methods,Thousand Oaks, CA: Sage Publications.

About the Authors

Markos Zachariadis is an assistant professor of InformationSystems and Management at the Warwick Business School,University of Warwick. He received his Ph.D. from the InformationSystems and Innovation Group, Department of Management, at TheLondon School of Economics and Political Science, and did hispostdoctoral research at the Judge Business School, CambridgeUniversity. His research interests focus on the circulation ofknowledge and dissemination of information in social and digitalnetworks. He has also conducted research on the economics ofnetworks; productivity effects of ICT; and the diffusion of innova-tion in the financial services sector for which he received the NETInstitute award.

Susan V. Scott is a senior lecturer in the Information Systems andInnovation Group, Department of Management, at The LondonSchool of Economics and Political Science. She received aPh.D. from the Judge Business School at the University ofCambridge. She serves on the editorial boards of Informationand Organization, Information Systems Research, andOrganization Science. Her primary research interests focus onthe entanglement of technology, work and organizing. Prior tothis, she conducted research on the implementation of riskmanagement decision support systems; electronic trading; thestrategic organization of post-trade services; enterprise resourceplanning and organizational best practice; organizationalreputation risk; and service innovation. Susan is currentlyworking on a study exploring the materiality of rating, rankingand review mechanisms.

Michael Barrett is Professor of Information Systems and Inno-vation Studies at the Judge Business School, CambridgeUniversity. His research interests focus on the role of socialtheory and discourse in understanding technology-enabledchange, knowledge translation and service innovation, and theuse of mobile infrastructures to enable social innovation inemerging economies. He serves as a senior editor for MISQuarterly and Information & Organization and is on theeditorial board of Organization Science. His research has beenpublished in journals such as Information Systems Research,MIS Quarterly, Academy of Management Journal, andOrganization Science.


METHODOLOGICAL IMPLICATIONS OF CRITICAL REALISM … · MIS Quarterly Vol. 37 No. X, ... do little...

Documents

Transcript of METHODOLOGICAL IMPLICATIONS OF CRITICAL REALISM … · MIS Quarterly Vol. 37 No. X, ... do little...