Data Collection and Sampling

Primary Data

• There are various methods for collecting primary (original) data– Eg questionnaire, survey, interview,

observation

• Control over investigation much greater• Can more easily avoid “data-driven”

research • Cost can be prohibitive• Pilot studies can be very helpful

Choice of method

• Shipman: choice often between sampling and case study

• Intensive versus extensive research design

• Qualitative versus quantitative data

• Interpretivists favour the former; positivists favour the latter

• All primary research involves selection

• Most methods require sampling

Sampling: general principles

• No a priori superiority of any method• Trade-offs: standardisation versus control,

generalisability versus flexibility• Shipman: sampling method used dependent on

nature of study undertaken• Basis for sample must be transparent• Cost of surveying entire population is prohibitive

(e.g. census)• Constraint of feasibility

Sampling: definitions

• Population: must be defined

• Finite population: e.g. voters

• Sampling unit: single potential member of sample

• Sampling frame: list of sampling units (NB 1936 US Presidential election)

• Sample: drawn from sampling frame

Probability Sampling

• Probability of each sampling unit being chosen is known (often equal probability)

• Simple random sampling: classic method, regarded as most reliable, least biased

• List numbered sampling frame members and select via random number generator

• Other probabilistic methods are available

Systematic sampling

• List members of sampling frame

• Choose first sample member randomly

• Then choose every Kth unit, where K=N/n

• More convenient than SRS for large popn

• Can be a systematic pattern in sample list, leading to bias; e.g. corner shops

Stratified sampling

• Divide population into groups of alike members

• Strata sizes usually proportionate to popn

• Draw randomly from groups

• Cost effective

• Ensure representativeness

• Can lead to excessive number of sub-groups

Cluster Sampling

• Select large groups• Select sampling units from clusters

randomly• Example: take a city, divide into areas,

number areas, select areas randomly, number units within areas, select units randomly

• Very cost-effective• Very good if sampling frame poorly defined

Non-probability Sampling

• Convenience sampling: select whoever is available

• Quota sampling: collect data according to proportions of the population

• Selection of subjects absolutely crucial• Requires great skill of interviewers• Snowball sampling: select next subject from

previous subject

Non-Probability Sampling

• Theoretical sampling: select those most likely to be affected by an issue

• Can ignore things which do not fit

• Can interpret observations according to the theory

• Non-prob sampling cannot claim representativeness as easily but gives much more discretion and control

Response Rates

• Another possible trade-off is on response rates

• R = 1 - (n-r)/n

• Even if initial sample size is appropriate (n’ = n/(1+(n/N)) where n = s2/SE2: see F-N and N: 194-9) response rates can be low

• Postal questionnaires: typically 20-40%

• Non-response bias

Response Rates

• Non-respondents could affect findings

• If reason for non-response is related to issue: e.g. reluctance to interview drunks hampers study on alcoholism

• Response rate can be improved by cover letter, callbacks, skill of researcher, length of questionnaire, types of question

Conclusions

• All types of primary data require selection

• If sampling used: various methods possible

• Sampling method relates to research tool

• Different data collection techniques: questionnaires, interviews, etc. - all to be studied in Research Methods 2 - all have advantages and disadvantages

Secondary Data

Introduction

• Primary quantitative data has several advantages, particularly control; qualitative data too

• Do not equate primary and qualitative

• Today: advantages of secondary data

• Searching on electronic data sources including the Internet

Secondary data

• Primary/secondary is not = qualitative/quantitative

• Qualitative can include secondary data sources such as personal documents, auto/biographies, etc.

• Secondary: collected by someone else, e.g. another academic researcher, business, government agency, etc.

Secondary data

• Used extensively in social science– Durkheim: suicide– Marx: wages, incomes, prices– Weber: church records

• Economists mainly use secondary data

Advantages of Secondary Data

• Might be the only data available• Enables longitudinal /time series work• Cheaper (cost and time) and more convenient than

primary data• Aids generalisation• Arises from natural settings

(nonreactive/unobtrusive data)• Allows replication and checking - validity

Disadvantages of Secondary Data

• May be not exactly the data required

• Differences in underlying sampling, design, questions asked, method of ascertaining information, etc.

• Differences lead to bias

• Method of data generation crucial to econometric studies

Electronic Data Sources

• Through the library system

• Through the internet

• Known versus unknown sources

• Known sources via library catalogue

• Problem of reliability/credibility is common to all electronic sources (more than non-electronic sources)

Electronic Data - Literature

• You can search by author or subject across journals, via several static websites/portals:

• www.econlit.org/

• www.sosig.ac.uk

• www.mimas.ac.uk

• www.economics.ltsn.ac.uk

• www.esds.ac.uk

Electronic Data: Databases

• There are many databases available online

• Most have standardised, national data free to download in various formats

• Common file format is .csv; but .html and even .xls files also common

• OECD: • ONS: • UN: • Penn World Tables: • BEA (US): • Ameristat:• Eurostat: • World Bank: • CIA: • US Statistical Abstract:

• See Dissertation homepage/hb

Conclusions

• Secondary data has many advantages and disadvantages relative to primary

• There is a wide range of secondary data available

• Much data is available on the internet

• Internet sources must be scrutinised more closely than other sources

Qualitative Data

Introduction

• Principals of research design and sampling basically hold for quantitative and qualitative data

• However, they apply most easily to quantitative analysis

• Qualitative analysis has different foci

• Qualitative analysis relatively (to quant; other soc sci) unused in economics

Qualitative techniques: types

• Case study

• Fieldwork (ethnography)

• Observation

• Unstructured interviews

• Analytic induction/grounded theory

• Discourse analysis

• Theoretical sampling

Qualitative techniques: principals

• Qual often = not quantitative

• Can use quant for pattern detection, qual for causal analysis

• Or use qual and quant as equals in inference (triangulation)

• Quantification often inappropriate

Qualitative techniques: principals

• Interpretivism, verstehen

• Used to be associated only with using autobiography, letters, personal documents, diaries

• Ethnography fairly recent:

• Focus on cases rather than generality

Qualitative techniques: principals

• Analysis not really a separate stage of research• Design, data collection and analysis all

simultaneous and continuous• Open-ended approach: Theory and conclusions

formed iteratively• Imagination is crucial• Recognise importance of exceptions• Context is crucial

Fieldwork

• Study of people acting in their daily lives

• Access a group but remain somewhat detached

• Approach with key questions

• Teams get range of perspectives

• Danger of self-perception and bias

Participant Observation

• Adopt perspectives of subject group in order to understand them

• Learning language, customs, behaviours, work, leisure, etc.

• Hanging around and learning the ropes • Being an outsider can changes subjects’ behaviour• Complete participation - researcher wholly

concealed – contamination and artificiality

Participant Observation

• Researchers can go native (internalise group lifestyle)

• Covert researchers can be in danger or create detrimental behaviour

• Researchers can be “piggy in the middle”• Covert: recording observations can be difficult

(e.g. need hidden cameras)• Serious ethical issues with covert observation

Employ analytic induction

• Go in with prejudices and theories

• Revise theory in light of evidence

• Generate new theories until evidence seems to fit

• Flexibility accorded but also required by the researcher

• Need to be open to disconfirming cases

Grounded theory

• Data collected• Develop categories (with inevitable

theoretical priors and language)• Categories checked by data• Once categories seem secure and grounded

in the evidence, formulate interconnection between categories

Evaluation

• Broad range of qualitative techniques • Control over the investigation; less data driven;

flexibility much greater than quantitative studies • Logistically difficult: Huge amounts of data

produced and problems with manipulation (although Nvivo will help with this)

• Must be careful to collect evidence widely to avoid bias

• Can be ethical issues re: data collection and reporting