URBDP 591 I Lecture 3: Research Process

Objectives• What are the major steps in the research process?• What is an operational definition of variables?• What are the relationships between validity and the

research process?• How do we select a research method?

Research is a Dynamic Tension BetweenTheory and Real World

Theory

Real World

Research Question is the point of contact that links theories to the real world.

SubjectsMeasures

ProceduresProblem Analysis

Research Question

Field of InquiryArea of InterestProblem-Idea

TheoryHypothesis

Research Process

Hypothesis development•Conceptualization•Construct•Operational Definition

Design Structure•Experimental•Individual Difference•Descriptive

Specification•Sampling•Instruments•Statistical testsImplementation

•Data Collection•Data Analysis•Evaluation

Research Question

TheoryDevelopment

Interpretation•Draw conclusions•Assess uncertainty •Evaluate process

Research objectives

• What is the broad topic area/academic discipline?

• What are the main originating and specifying questions to be investigated?

• What is the rationale of the research? What debates, controversies,areas of knowledge does it aim to address and/or extend?

Types of Research Projects

1. Fact-finding/filling gaps in knowledge

2. Testing hypotheses

3. Establishing a relationship between variables

4. Examining the adequacy of models or theories

5. Critical analysis of a theoretical position or proposition

6. Uncovering or explicating the meaning of a concept, practice, or situation

Good Research Questions

• Grounded in theoretical and empirical literature

• Testable by empirical methods

• Stated clearly and simply

• Not too abstract

• Not too complex

Hypothesis

A hypothesis is a tentative statement, subject to empirical testing, of the expected relationship between variables; sometimes in practice it is little more than "an educated guess". Essentially what happens is, a scientist will think "if my theory is correct, I should be able to predict that event X will occur with a higher than chance probability”.

Evaluating Theory

• Testable

• Logical soundness (deductive validity)

• Communicable

• General

• Parsimonious: Necessary and Sufficient conditions

• Accurate (inductive validity) 0

OperationalizationA procedure by which one selects observable indicators (variables) to

represent theoretical concepts.

A variable is a theoretically relevant concept which may be

observed to take different values in different cases

1. Independent variable (explanatory variable) - A variable that is taken as a given in our research and used to explain other phenomena.

2. Dependent variable (outcome, response variable) - A variable whose values we try to explain by looking at other variables.

Operationalization of a variableOperationalization of a variable defines how the variables we have

identified and defined will be measured with real (available) data.

Obstacles to operationalizing variables

1. Conceptual obstacles to operationalization

2. Practical obstacles to operationalization

a. Reliability: a measure is reliable if it yields the same values for a particular case in repeated measurements

b. Validity: a measure is valid if it is an appropriate measure of the concept in which you are interested

Types of ValidityMeasurement is a tool of research, and validity is the attempt to determine whether a type of measurement measures what it is presumed to measure.

A. Construct validity. Construct validation is interested in the degree to which the construct itself is actually measured. It makes use of the traits of convergence and discriminability.

B. Internal validity. Internal validity is the freedom from bias in forming conclusions in view of the data. It seeks to ascertain that the changes in the dependent variable are the result of the influence of the independent variable.

C. External validity. This type of validity is concerned with the generalizability of the conclusions reached through observation of a sample to the universe.

D. Statistical validity. This type of validity is concerned with the appropriate choice of statistical test.

ReliabilityReliability: the extent to which a measurement procedure yields the same answer however andwhenever it is carried out.

1. Quixotic reliability: Refers to the circumstances in which a single method of observation continually yields an unvarying answer.

2. Diachronic reliability: Refers to the stability of an observation over time. This type of reliability is only appropriate when the phenomenon observed is not assumed to change over time.

3. Synchronic reliability: Refers to the similarity of results from the use of multiple measures within the same time period.

Relationships between Validity and the Research Process

State QuestionsDefine HypothesesIdentify Variables

DetermineDesignStructure

IdentifyPopulationAnd Sample

Design InstrumentsAnd ClassifyOperationalDefinitions

SelectStatisticalTests

CollectData

AnalyzeData

InternalValidity

Relationships between Validity and the Research Process

State QuestionsDefine HypothesesIdentify Variables

DetermineDesignStructure

IdentifyPopulationAnd Sample

Design InstrumentsAnd ClassifyOperationalDefinitions

SelectStatisticalTests

CollectData

AnalyzeData

ExternalValidity

Relationships between Validity and the Research Process

State QuestionsDefine HypothesesIdentify Variables

DetermineDesignStructure

IdentifyPopulationAnd Sample

Design InstrumentsAnd ClassifyOperationalDefinitions

SelectStatisticalTests

CollectData

AnalyzeData

StatisticalValidity

Relationships between Validity and the Research Process

State QuestionsDefine HypothesesIdentify Variables

DetermineDesignStructure

IdentifyPopulationAnd Sample

Design InstrumentsAnd ClassifyOperationalDefinitions

SelectStatisticalTests

CollectData

AnalyzeData

ConstructValidity

Research Approaches

General Purpose Explore relationships between variables Describe variables

General Approach Experimental Non-Experimental Descriptive

SpecificApproach

RandomizedExperimental

Quasi-Experimental

Comparative Associational Descriptive

Specific Purpose DetermineCause

ExamineCause

Compare Groups Find Associations Summarize Data

Type of HypothesisDifference

Relate variables Descriptive

Statistics Difference Inferential Statisticse.g. ANOVA

AssociationalInferentialStatistics(eg. Correlation,Multiregression

Descriptive StatisticsEg, Histograms,Means etc.

Selecting a research method for data collection

Experiment, Quasi-Experiment, Cross-Section, Longitudinal, Case study

Selecting a methods depends on: • the research question/purpose • the operational definition of the construct

of interest (difference/associational) • the required protocols for reliability and

validity

Qualitative/Quantitative Research Continuum

Inductive Inquiry Deductive InquiryUnderstanding Social Phenomena Establishing RelationshipsGrounded-theory Theory-BasedHolistic Inquiry Focused on ComponentsContext-specific Context-FreeObserver-Participant Detached Role of ResearcherNarrative Description Statistical Analysis

QUALITATIVE QUANTITATIVE

Ethnographic Experimental Historical Quasi-experimental

Phenomenological SurveySource: Newman & Benz 1998

Mixed-Method• Triangulation: tests the consistency of findings obtained through

different instruments.

• Complementarity: clarifies and illustrates results from one method with the use of another method.

• Development: results from one method shape subsequent methods or steps in the research process.

• Initiation: stimulates new research questions or challenges results obtained through one method.

• Expansion: provides richness and detail to the study exploring

specific features of each method.