THE QUALITATIVE CODING MATRIX: IMPROVING THE RIGOUR OF ...
Transcript of THE QUALITATIVE CODING MATRIX: IMPROVING THE RIGOUR OF ...
Vol. 2: Work-in-progress papers f|ffe 347
THE QUALITATIVE CODING MATRIX: IMPROVINGTHE RIGOUR OF PHENOMENOLOGICAL RESEARCH
IN BUSINESS PURCHASING
Graham, Gary 1 University of Salford
ABSTRACTThe main objective of this research study was to develop understanding by using the IMP approach the ways in which buyer-supplier relations in the UK defence industry are adapting as buyer (monopsonistic) power strengthens, the threat of new entrants increases and the associated competitive rivalry intensifies. The predominant research tradition in business purchasing has been of a quantitative, empirical nature (Knox and White, 1991; Wilson, 1995). However the limitations in explaining the more normative processes of buyer-supplier interaction and interdependence is evident (Ford, 1980, 1982, 1990, 1997; Turnbull and Cunningham, 1981; Hakansson, 1982; Gummesson, 1995; Gronroos, 1997). The lacunae of work at the operational level (Smith and Smith, 1992), and the opportunity for access led to the use of the case study for research purposes (Yin, 1985). Lowe's (1991) "Qualitative Coding Matrix" was adapted to ensure that the qualitative data collected was analysed in a rigorous manner.
1 Dr Gary Graham, Department of Business Studies, The University of Salford, The Crescent, Salford, England, M5 4WT (Phone +44 161 745 55336, Fax +44 161 745 55556, E-mail [email protected] or [email protected]).
348 IjVJ P 14* IMP Annual Conference
1. INTRODUCTION
The Ministry of Defence (MoD) is UK industry's largest single customer.
On the supply side of the market, the UK defence industry comprises of
companies involved in R and D, production, repair and maintenance of
defence equipment for the MoD and overseas governments. The industry
has the capability to design, develop and produce a complete range of
defence equipment including high-technology equipment such as missiles,
combat aircraft, helicopters, nuclear-powered submarines, aircraft carriers
and tanks.
There is a consensus view of opinion among academic researchers, civil
servants, research institutes, defence industrialists and pressure groups that
equipment procurement spending is set to continue falling as a percentage
of defence spending and a recognition of the need to review defence
spending constantly. However, the problem of defence contraction did not
just arrive with the fall of the Berlin Wall in 1990 and the end of the Cold
War. The fall in defence spending can be traced back to 1983 and the
MoD's "value for money" initiatives, which emphasised the increased use
of fixed price contracts and competitive tendering by defence contractors.
In the interests of efficient project management, the MoD has also
tended to place most large defence contracts in the hands of a single prime
contractor. In such cases, the Ministry's role in the day-to-day management
of the contract is a more limited one, the rule being that it is for the prime
contractor to choose its own sub-contractors. Given this devolution of
power to the defence primes and the non-commercial policy of the MoD, it
is largely the purchasing policies of the primes which control the
distribution of defence sub-contracts.
1.1 Industry Analysis, Business Strategy and Buyer-Supplier RelationsIn order to capture both the external and internal dimensions of strategy the
research initially drew heavily on industry analysis (Porter, 1990),
Vol. 2: Work-in-progress papers l||p 349
competitive analysis (Porter, 1980) and value chain analysis (Porter,
1985). For the empirical work supporting Porter's generic strategies (Dess
and Davies, 1984; Miller and Friesen, 1986; Kotha and Vadlamani, 1995),
there has been equally valid work criticising and refining Porter's work. In
particular the works of White (1986), Wright (1987), Speed (1989), Sharp
(1991) and Wright et al., (1991) have all questioned Porter's rigid
typology.
However, as a first step in exploring a possible relationship between
industry structure and strategy in the UK defence industry, it made sense to
proceed by selecting a simple business strategy concept which incorporates
a few critical dimensions, yet has strong theoretical underpinnings (White,
1986). Moreover, the success of the defence firm is significantly influenced
by its relations with the environment in which it functions 1 .
1.2 Methodology
The researcher takes the view of phenomenology that reality is socially
constructed rather than objectively determined (Hunt, 1983). Within this
framework there are many epistemological variations. The four main
versions are interpretative sociology (Habermas, 1974), naturalistic enquiry
(Berger and Luckman, 1966), qualitative methodology (Van Maanen,
1979) and "new paradigm" enquiry (Reason, 1988). This work adopted the
qualitative methodological approach. Van Maanen defines qualitative
research in the following words:
"The label qualitative research has no precise meaning in any of the social sciences. It is best an umbrella term of an array of interpretative techniques which seeks to describe, decode, translate, and otherwise come to terms with the meaning, not the frequency, of certain naturally occurring phenomena in the social world."
(1979: p. 9)
Please refer to the comments of Kenny (1982).
350 Jim 14* IMP Annual Conference
Qualitative investigators tend to describe the unfolding of social
processes rather than the social structures that tend to be the focus of
quantitative researchers (Yin, 1985). These methods range from
ethnography (Stockport and Kakabadse, 1991), participant observation
(Levine, 1980), direct observation (McCall and Simmons, 1969) to semi-
structured and unstructured interviews (Easterby-Smith et al., 1991).
Many qualitative researchers after collecting case study data spend a
great deal of time turning it into numbers or other attempts at quantification
(Silverman, 1985; Miles and Huberman, 1989; Patton, 1995). This tends to
fail to give a holistic view that is so important in case study research
(Hosking, 1991). It was my aim to stay close to the data in order to produce
common or contradictory themes or patterns that would act as a basis for
interpretation (Glaser and Strauss, 1967; Strauss and Curbin, 1995).
1.3 Data Sources
The aim of the information gathering was to identify the purchasing
strategies that companies are following, assessing the effects that the
defence industry's structure and environment have on strategic choice.
Case study research was carried out in five suppliers with which the
collaborating institution 1 helped provide access for my inquiry. The main
method of data collection came from in depth interviews; discussing
purchasing strategies, policies and activities in operations, distribution and
R and D. These interviews were supplemented with data collected from a
combination of sources which included reports and accounts, company
brochures and internal purchasing strategy documentation (Bonoma, 1985).
The next section presents background information on the company selected
to illustrate the "Qualitative Coding Matrix" in operation. The basis of
1 The Defence Manufacturers Association. This is a trade association of approximately 300 registered members which supply defence equipment indirectly or directly to the UK MoD.
Vol. 2: Work-in-progress papers 351
selection was the high quality of access to internal processes of
decision making, implementation, and change in the company
(Gummesson, 1991: pp. 21).
2. CASE STUDY SYSTEMCO 1 : PRODUCER OF DEFENCE ELECTRONICS SYSTEMS
Systemco has developed an extensive range of military products in static or
transportable ground based air defence systems, custom designed systems
to support general military operations and a wide range of system
components including universal work stations, high definition colour
display; digital entry and digilux touch entry devices. From Table 1 it can
be seen that the total value of turnover is 323 million pounds and that only
2 million pounds of Systemco's turnover is accounted for by commercial
business.
TABLE 1: Company information in September 1996
Employees (OOO's)
Civil DefenceTotalTurnover (millions)
Civil DefenceTotalCompany ReformationNumber of Years in:
Civil Work Defence Work
0.5 15
15.5
2 3213231991
8 70
1 For reasons of confidentiality the true identity of the firm is hidden by the pseudonym Systemco.
352 I|P 14* IMP Annual Conference
The firm prior to its acquisition in 1983 had a long history in military radar
work and battlefield command and control systems of 70 years compared to
only 8 years in commercial markets. Corporate reorganisation by the
holding company led it to focus on R and D as opposed to the production of
defence and commercial electronic systems.
In terms of access the operations manager 1 provided the initial contact and
2.1 Interview Programme
In terms of access the operai
itinerary arrangements for the case study data collection. Table 2 shows
TABLE 2: Interview programme at Systemco
Stage in the interview programmeFunction Number of consulted Exploratory Primary Review interviews
data data dataAssistant Chief Buyer * 1
Commercial Director * 1
Commercial Manager * 1
Technical Engineer * 1
Engineering Manager * 1
Major Sub-contracts Manager * * 2
Operations Manager * * 2
Purchasing Director * 1
Quality Assurance Manager * 1
Sales and Marketing * 1 Director
N = 12
1 I am gratefully indebted to the amount of time and effort taken by the operations manager who was able to identify people within the organisation who were willing to participate in the research.
Vol. 2: Work-in-progress papers f|jp 353
that a total of 12 interviews were undertaken at Systemco. Initial
exploratory interviews were undertaken with the operations manager and
the major sub-contracts manager 1 . From this exploratory work, subsequent
interviews were arranged as part of the main period of data collection.
Transcripts were faxed to the operations manager to check and review the
findings and the results for their accuracy.
3. THE SYSTEMATIC ANALYSIS OF QUALITATIVE DATA
The "Qualitative Coding Matrix" enabled a systematic way of achieving
the discrimination between data sources and various interpretations of the
different parties (the researcher, perceived wisdom of the academic
community and the respondents themselves). Essentially this approach
creates theory from data (Glaser and Strauss, 1967). In case study analysis
one useful strategy is to use a coding system. Basically, a code is an
abbreviation or symbol applied to a segment of words - more often a
sentence or paragraph of transcribed field notes in order to classify the
words (Miles and Huberman, 1989: p. 78). They work as retrieval
organising devices that allow the analyst to spot quickly, pull out and then
cluster all the segments relating to the unit of analysis. Initially a 'start list'
of codes was developed prior to the fieldwork (refer to Appendix 1).
The list of codes came from the conceptual framework and unit(s) of
analysis. The first column has a short descriptive label for the general
categories and the individual codes (e.g. EC refers to external context). The
second column shows the codes themselves (e.g. CA to competitors
actions). In this study the coding was by hand rather than computer thus the
use of short coding labels. For example, a coding label of IC-HIST was
attached to observations on the company's purchasing history.
1 As a general guideline procurement contracts over 50,000 pounds were classified as major sub-contracts.
354 14* IMP Annual Conference
The "Qualitative Coding Matrix" includes three levels of codes; 'open',
'axial' and 'selective'. 'Open' codes are low level descriptive labels. They
entail no interpretation, but simply, the attribution of a class of phenomena
to a segment of text. An explanation of the three types of 'open' codes is
given in Figure 1.
Issues raised from the topic guide using the researcher's own interpretation:
Issues raised from the literature as interpreted by the researcher:
Issues raised from field notes used following periods of direct observation:
FIGURE 1: Explanation of 'open' codes
O'Axial' codes are created by the researcher using their own judgement on
those 'open 7 codes which have similar characteristics.
Vol. 2: Work-in-progress papers 355
At the end of the 'axial' coding phase the relationship of one 'axial' code
with another is drawn as a diagram. The final stage is 'selective' coding.
These codes are up-graded 'axial' codes which have been drawn together
from different sources where triangulation can be established.
AEach of the data sources mentioned in i 'e previous section are open to a
range of different interpretations. Lowe identified three:
i) My own pre-understanding.ii) The respondents views as the research unfolds.iii) A priori constructs.
(1991: p. 8).
The "Qualitative Coding Matrix" shows the discrimination between data
and various interpretations which could be made by the different parties
(the researcher, perceived wisdom of the academic community and
thesources respondents themselves). Figure 2 shows that at each stage of
the research process the data sources and interpretations are coded with a
unique symbol and alphanumeric code which allows the reader to penetrate
the researcher's intuition and judgement as they move from analysis to
synthesis and finally interpretation. The process of discrimination between
356 14* IMP Annual Conference
data sources and interpretation occurs at three levels of the research
process:
1. Analysis 'open' coding.
2. Analysis/synthesis 'axial' coding.
3. Synthesis/interpretation 'selective' coding.
(ibid.,: p. 9).
DATA SOURCES INTERPRETATIONS
SELECTIVE CODES
FIGURE 2: The qualitative coding matrix
Vol. 2: Work-in-progress papers ifffe 357
The alphanumeric coding matrix provided in Figure 2 enables data
to be presented in ways which increase the reader's comprehension without
losing the complexity of the phenomena being studied.
3.1 The "Qualitative Coding Matrix" in Operation
The "Qualitative Coding Matrix" will be shown in operation using the three
levels of 'coding'. The section of text in Figure 3 illustrates the footprint
effects of the 'open' coding when they are overlaid on the text . This
extract is taken from an interview with the Major Sub-contracts Manager at
the first case study company. The number inside the circle relates to the
chronological sequence that the issue was raised in the interview. For
instance, 14 refers to issue 14, 15 refers to issue 15) The 'open' codes have
been chosen from the verbatim transcripts of a series of interviews with
procurement personnel at Systemco. The next stage involved the
development of an integrative diagram shown in Figure 4 which illustrates
the footprint effects of the codes overlaid on the text.
The codes are clustered into key themes (e.g. industry structure) and sub-
themes (e.g. new entrants). Job titles are abbreviated and placed inside the
alphanumeric codes and a full list is presented in Table 3 (page 18). From
the analysis of 'open' coding the beginnings of the next stage 'axial'
coding - is indicated by the arrow linkages between separate clusters. Then
at the end of the synthesis of 'axial' coding, the codes were compared with
'axial' coding of other data sources. Final interpretation results in
'selective' coding where the evidence from direct observation, archival
sources and procedural documentation were triangulated and are illustrated
in Figure 5.
358 14th IMP Annual Conference
Job Title Transcript Data Sources/ (Abbreviation) Interpretations
Sub- Buying from the right Contracts supplier is 90 percent Manager 'gut feeling', this job is (SCM) about experience not
qualifications.
Interviewer How might the sales and (GG) marketing personnel of
prospective biddersseek to try and influenceyour decision?
SCM The sales people always paint a beautiful picture about their company and product, I frankly do not believe anything sales people tell me!
GG Is this more true of new suppliers?
SCM In the case of a new supplier, I like to visit the premises. In some cases I have wandered in through the 'back door' rather than reception. This gives me an underlying picture of working conditions and the standards of operations. I'll even chat with the shop floor personnel to help assess the firms capabilities.
14
15
16
FIGURE 3: Interviewee remarks - coding illustration
Vol. 2: Work-in-progress papers 359
INDUSTRY STRUCTURE
Civil Exports
1 Cl
*13
CM
14 1CM 12
S GG • £
* 10*^ ^ CM
v« / V,, l'
OM OM OM
STRATEGY
Business Function
5r 9 ^J^^ CDr^--^7.f.N /-iiGG " ***•
•" if \--QO": CM ^~iOM \ * • *
NOi^~ sc
BUYER/SUPPLIER New Entrants
FIGURE 4: The footprint effect of 'open' codes
360 14* IMP Annual Conference
r
Preferred / \ Suppliers \
/ \
A/ LOW V
EXTERNAL RELATIONS
FIGURE 5: The emergence of 'selective' codes
Vol. 2-. Work-in-progress papers 361
TABLE 3: Abbreviated list of job titles used in the qualitative coding matrix
Function consulted
Assistant Chief BuyerCommercial DirectorCommercial ManagerCompany HistorianEngineering ManagerMajor Sub-contracts ManagerOperations ManagerPurchasing ManagerQuality Assurance ManagerSales and Marketing DirectorAssistant Chief Buyer
Abbreviated job title
ACBCDCMCHEMSCMOMPMQAMSMDACB
3.2 Final Interpretation of the 'Selective' codes
The diagram shows that the respondents identify Ministry of Defence
(MoD) policy to be the major driver of UK industry structural change. At
the strategic level the company has adjusted by undertaking a greater
number of collaborative ventures. At the same time the company is well
aware of the need to develop its commercial business. The 'hands off
approach of the MoD had placed a great deal of risk firmly on the
contractor. This is reflected in the high importance that Systemco attaches
to low risk in its external relations. The contractor sought to avoid risk by
working with preferred suppliers that have established reputations for 'tried
and safe' solutions. Government pressure for freer, competitive tendering
has forced the company to act as 'managing contractors', sharing out any
work that could not be performed cost-effectively on site. While project
complexity had increased the extent of subcontracting,the company had to
balance this against its commitment to maintain in house its high value
adding processes. Nevertheless, a clear finding from this extract of case
362 I|p. 14* IMP Annual Conference
research is that it was primarily MoD pressure which had driven Systemco
to implement a more commercial 1 purchasing strategy.
4. CONCLUSION
This paper attempts to make a contribution to the 'research methods'
category of the IMP conference theme entitled 'theoretical and
methodological developments'. My line of argument is that existing
concepts and knowledge are mainly positivistic and do not adequately
capture the 'true' nature of business purchasing (Konrad and White, 1991;
Lowe, 1991; Hakansson, 1995; Ford and Saren, 1996). Case studies as a
research method are gradually receiving scientific approval within and
beyond the IMP group in both Europe and the US (Wilson, 1995). The
"Qualitative Coding Matrix" is unique in that it triangulates at three coding
levels - between sources of data and interpretations - within a case (Yin,
1985). It provides the IMP group, with a more rigourous, scientific
approach which does not clone quantification, in order to ground business
purchasing theory.
REFERENCES
Berger, P and Luckmann, T (1966) The Social Construction of Reality, London: AlienLane.
Bonoma, T (1985) Case Research in Marketing: Opportunities, Problems and a Process,Journal of Marketing Research, Vol. 22, May, 199-208.
Dess, G and Davies, P (1984) Porter's (1980) Generic Strategies as Determinants ofStrategic Group Membership and Organisational Performance, Academy ofManagement Journal, Vol.27, 467-488.
Easterby-Smith M, Thorpe R and Lowe A (1991) Management Research: AnIntroduction, London: Sage.
Ford, D (1980) The Development of Buyer-Seller Relations in Industrial Markets,European Journal of Marketing, Vol. 14, No's. 5-6, 339-353.
Ford, D (1982) Understanding Business Markets: Interaction, Relationships andNetworks, Ed: Hakan Hakansson, 288-304, London: John Wiley and Sons.
1 The peculiar cultural barriers to commercial strategy in the defence industry are detailed by Smith and Smith (1992).
Vol. 2: Work-in-progress papers t$|ffe>- 363
Ford, D (Ed) (1990) Understanding Business Markets, London: Academic Press.Ford, D and Saren, M (1996) Technology Strategy for Business, London: Thomson
International Press. Ford, D (Ed) (1997) Understanding Business Markets: Interaction, Relationships and
Networks, The IMP Group, 2nd edtn, London: The Dryden Press. Glaser, B and Strauss, A (1967) The Discovery of Grounded Theory, New York: Aldine. Gronroos, C (1997) Value-Driven Relational Marketing: From Products to Resources
and Competencies, Journal of Marketing Management, Vol.13, 407-419. Gummesson, E (1991) Qualitative Methods in Management Research, London: Sage. Gummesson, E (1995) Relationsmarknadsforing. Frdn 4P till 30R__(Relationship
marketing from 4P to 30R), Malmo, Sweden :Liber-Hermods. Habermas, J (1974) Theory and Practice, London: Heinemann. Hakansson, H (1982) International Marketing and Purchasing of Industrial Goods,
London: Wiley. Hakansson, H (Ed) (1995) Developing Relationships in Business Markets, London:
Routledge. Hosking, D (1991) Organisations, Structures, and Processes in Analysing
Organisational Behaviour, Ed: Smith, M , 85-103, London: Macmillan. Hunt, S (1983) Marketing Theory, London: R.D . Irwin.Kenny, B (1982) The Defence Industry Environment and Its Impact on the Military-
Industrial Firm. Unpublished Ph.D thesis, Department of Business andManagement Studies, University of Salford.
Kortha, S and Vadlamani, B.L (1995) Assessing Generic Strategies: a EmpiricalInvestigation of Two Competing Typologies in Discrete ManufacturingIndustries, Strategic Management Journal, Vol. 16, 75-83.
Knox, S and White, H (1991) Retail Buyers and Their Fresh Produce Suppliers: APower or Dependency Scenario in the UK?, European Journal of Marketing,Vot.25, No. 9, 22-37.
Levine, H (1980) Teaching Participant Observation Research Methods: A_SkillsBuilding Approach, New York: Anthropological Education.
Lowe, A (1991) The Qualitative Coding Matrix: a New Approach to Improving theRigour of Phenomenological Research in Services Marketing.. Paper presentedat the Institute d'Administration des Enterprises, University d'Aix Marseilles,France, June 1991.
McCall, G and Simmons, J (1969) Issues in Participant Observation, New York,Addison-Wesley.
Miles, M and Huberman, A (1989) Qualitative Data Analysis, London: Sage. Miller, D and Friesen, P (1986) Porter's (1980) Generic Strategies and Performance: An
External Examination with American Data: Part 1. Testing Porter, OrganisationStudies, Vol.7, No. 1,37-55.
Patton, M (1995) Qualitative Evaluation and Research Methods, London: Sage. Porter, M (1980) Competitive Strategy, New York: The Free Press. Porter, M (1985) Competitive Advantage, New York: The Free Press. Porter, M E (1990) The Competitive Advantage of Nations, London: Macmillan. Reason, P (1988) Human Inquiry in Action, Developments in New Paradigm Research,
London: Sage. Sharp, B (1991) Competitive Marketing Strategy: Porter Revisited, Marketing
Intelligence and Planning, Vol.7, No's. 5/6, 8-11.Silverman, D (1985) Qualitative Methodology and Sociology, Aldershot: Gower. Smith, D and Smith, R (1992) Corporate Strategy, Corporate Culture and Conversion:
Adjustment in the Defence Industry, Business Strategy Review, Summer, 45-58.
364 «Mp, 14* IMP Annual Conference
Speed, R (1989) Oh Mr Porter! An Re-appraisal of Competitive Strategy. MarketingIntelligence and Planning, Vol 7, No's. 5/6, 8-11.
Stockport, G and Kakabadse, A (1991) Using Ethnography in Small Firms Research.Paper presented at the 14th National UK Small Firms Policy and_ResearchConference, Blackpool, November 1991.
Strauss, A and Curbin, J (1995) Basics of Qualitative Research, Grounded TheoryProcedures and Techniques, London: Sage.
Turnbull, P amd Cunningham, M. (1981) International Marketing and_Purchasing: ASurvey Among Marketing and Purchasing Executives in 5 European Countries,London: Macmillan.
Van Maanen, J. (1979) Qualitative Methodology, London: Sage. White, R (1986) Generic Business Strategies, Organisational Context and Performance.
An Empirical Investigation, Strategic Management Journal, Vol. 7, 217-231. Wilson, D (1995) An Integrated Model of Buyer-Seller Relaionships, Journal of the
Academy of Marketing Science, Vol. 23, No. 4, 335-345. Wright, P (1987) A refinement of Porter's strategies: research notes and
communications, Strategic Management Journal, Vol. 8, No. 1, 93-101. Wright, P., Kroll, M., Howard, T. & Helms, M. (1991) Generic Strategies and ^Business
Performance: An Empirical Study of the Screw Machine Product Industry,British Journal of Management, Vol. 2, 57-65.
Yin, R (1985) Case Study Research Design and Methods, London, Sage.
Vol. 2: Work-in-progress papers 365
APPENDIX 1 Start List of Codes
EXTERNAL CONTEXT EC
EC: COMPETITORS ACTIONS
EC: PRODUCT
EC: NEW ENTRANTS
EC: TECHNOLOGY
EC: NEWINNOVATIONS
EC: ADOPTION PROCESS
EC: CUSTOMERS
EC: HOME
EC: EXPORT
EC: SUPPLIERS
EC: PRODUCT
EC: NEW ENTRANTS
INTERNAL CONTEXT 1C
1C: CHARACTERISTICS
1C: NORMS
1C: PURCHASING HISTORY
1C: ORGANISATION PROCEDURES
PROCUREMENT ACTIVITIES PA
PA: EVENT CHRONOLOGY - OFFICIAL VERSION
PA: EVENT CHRONOLOGY- SUBTERRANEAN
PA: PROCUREMENT PROCESS
PA: CONTACT FREQUENCY
PA: ORGANISATIONAL LEVEL
PA: FUNCTIONAL BREADTH
EC-CA
EC-PROD-CA
ECENT-CA
EC-TEC
ECNEW-TEC
ECADOPT-TEC
EC-CUS
ECHOM-CUS
ECEXP-CUS
EC-SUPP
ECPRO-SUPP
ECENT-SUPP
IC-CHAR
IC-NORM
IC-HIST
IC-PROC
PA-CHRON/OFF
PA-CHRON/SUB
PA-PRUC
PA:FREQ/PRUC
PA: LEV/PRUC
PA: FUNC/PRUC
366 14th IMP Annual Conference
BUYER EVALUATION BE
BE: EVENT CHRONOLOGY - OFFICIAL VERSION
BE: EVENT CHRONOLOGY - SUBTERRANEAN
BE: LOCATIO
BE: RULES/NORMS
BE: TYPE
BE: RATING CRITERIA
BE: PERFORMANCE
BE: EFFECTS
BE: ASSESSMENT BY RECIPIENTS
BE: LINKAGE
EMERGING CAUSAL LINKS CL
CL: NETWORKS
CL: RULES
BE-CHRN/PUB
BE-CHRN/PRIV
BE-LOC
BE-RULE
BE-TYPE
BE-RATE
BE-PERF
BE-EFF
BE-ASS
BE-LINK
CL-NET
CL-RULE