Du Feng, Ph.D.

Professor

School of Nursing

University of Nevada, Las Vegas

Use of the Quantitative-Methods

Approach in Scientific Inquiry

The Scientific Approach to Knowledge

Two Criteria of the Scientific Approach

Logical

Empirical

Fundamental Basis of Social/Behavioral

Sciences

Theory

Research Method

Statistics

Video: Battling Bad Science by Ben Goldacre

http://www.ted.com/playlists/130/the_dark_side_of_d

ata

Good science? Bad science?

- Research method makes the call

The Social Science Research Process

The research problem - interest, idea

The hypothesis - theory

The research design

research method, population & sampling

Measurement

conceptualization & operationalization

Data collection - observations

Data analysis - data processing, analysis

Generalization & Application

The research journey

authored by Stephanie Fleischer © SAGE publications Ltd 2014

Anatomy of A Research Article

Title

Abstract

Introduction

Methods

Results

Discussion

References

Appendix

Elements of a Research Proposal

Specific Aims - What do you intend to do

Significance - Why is the work important?

Background - Literature Review

Subjects - How are you going to select your sample?

Measurement (of the IV’s and DV’s)

Data-collection methods (research design)

Data analysis plan

Timeline

Budget

Asking Testable Questions

The first step in conducting scientific

research

Not all questions are testable

In order to be testable, research questions

should be empirically grounded, precise,

specific, and value-free

Identify the independent and dependent

variables

https://webcampus.unlv.edu/bbcswebdav/courses/OE_NURS-780-

DEVELOPMENT-WB-MASTER/research-question/

The variable language

Variables: logical groupings of attributes

gender;

age

Attributes: characteristics

male, female;

young, old

Deciding the level of numerical measurement

authored by Stephanie Fleischer © SAGE publications Ltd 2014

Criteria for Evaluating Social and

Behavioral Scientific Research

Internal Validity: To what extent does the

research design permit us to reach causal

conclusions about the effect of the IV on

the DV?

External Validity: To what extent can we

generalize from the research sample and

setting to the population and settings

specified in the research hypothesis?

Internal Validity

Threats to Internal Validity

Attrition

History

Instrumentation

Maturation

Selection

Statistical regression

Testing effects

Interaction effect

Maximize Internal Validity

– The Goal of Experiments

Random assignment of individuals to

treatment conditions means that

confounding variables are equally

distributed across conditions

As such, confounding variables are

unlikely to be responsible for observed

differences between treatment conditions

External Validity

Threats to External Validity

Reactive effect of testing

Interaction of selection and treatment

Reactive effects of experimental

arrangements

Multiple-treatment interference

Biased sample

Maximize External Validity

- Random and Nonrandom Sampling

The extent to which the results of a

study generalize to the population of

interest

To be confident about such a

generalization, the sample must be

representative of the population of

interest

Figure 7.1 Population, sample and individual cases

authored by Stephanie Fleischer © SAGE publications Ltd 2014

Two Ways to Get a Sample

Probability Sampling

Every element of the population has a known nonzero

probability of being selected

Random selection is used at some point in the process

Nonprobability Sampling

Something else.

Bottom Line:

Nonprobability sampling makes it impossible to

estimate sampling errors

With nonprobability sampling, judgments about external

validity are rarely on firm ground.

Measurement

Thinking about Total

Variability -

If X = T + E, then:

var (X) = var (T) + var (E)

Scales (Levels) of Measurement

Nominal

Ordinal

Interval

Ratio

- An example: “Age”

measured at the nominal level: 1=young; 2=old

measured at the ordinal level:

1=young; 2=middle aged; 3=old

1=infant; 2=child; 3=adolescent; 4=adult

measured at the interval level: years of age

Quality of Measurement

Reliability

- the extent to which a measuring technique would yield

consistent results when applied repeatedly

- More technical: To what extent do observed scores

reflect true scores?

- Less technical: How consistent is the assessment?

Validity

- the extent to which a measuring technique adequately

reflects the real meaning of the concept under

consideration

- Am I measuring what I intend to measure?

Reliability Coefficients

Reliability coefficients reflect the proportion of true score variance to observed score variance

Therefore reliabilities range from 0.0 (no true score variance) to

1.0 (all true-score variance)

var( )

var( )xx

Tr

X

Various Types of Reliability

Internal Consistency (Content)

Random error affects responses to items

on an assessment

Test-Retest (Time)

The construct stays the same. However,

random errors vary from one occasion to

the next.

Inter-Rater (Observer Biases)

Various Types of Validity

Content

Criterion-related

Construct

Concurrent

Convergent

Discriminant

construct

(general evidence-gathering)

content

(more theory-based)

criterion-related

(more data-based)

Research Design

Experiments vs Surveys

Cross-sectional vs

longitudinal

The ultimate goal:

- To establish causality

Experimental Approach to

Studying Causal Relations

Intervention and Control

Random assignment of participants to

two or more conditions of an experiment

The Independent variable is the variable

that is manipulated or the experimental

conditions (e.g., treatment or no-

treatment)

The Dependent variable is the outcome

of interest in the particular study

(observed/measured)

Design 1: Randomized Two-Group Design

Pool of

Participants

Treatment

Control

Outcome

(DV)

Outcome

(DV)

Design 2: Pretest-Posttest Two-Group

Design

Treatment

Control

Posttest

(DV: Post - Pre) Pretest

Pretest

Pool of

Participants

Posttest

(DV: Post - Pre)

Design 3: Solomon Four-Group

Treatment

Control

Posttest

Posttest

Pretest

Pretest

Treatment

Control

“Posttest”

“Posttest”

Clinical Trials

Explanatory trials - aim to test whether an

intervention works under highly controlled

and optimal situations

- High internal validity

Pragmatic trials - designed to evaluate the

effectiveness of interventions in real-life

routine practice conditions

- High external validity

In the ideal world

A representative sample obtained by

random sampling

A sufficiently large N

RCT

Valid and reliable measurement

No missing or incomplete data

No attrition in longitudinal studies

High fidelity of intervention studies

Challenges Relating To

Research Design

Random sampling is infeasible -> sample bias -> low

external validity

Difficult to recruit participants

Random assignment of recruited participants is in

feasible - > low internal validity

Hard to measure outcome variables

Poor measurement quality, measurement error

Missing data due to drop-outs

Incomplete data due to refusal to answer certain

questions

Other challenges

Prospective Power Analysis

The purpose is to determine the

minimum sample size required

for achieving a desirable power for a

specific hypothesis test,

for obtaining a confidence interval

with a specified width,

to estimate a parameter with a

maximum error of estimation for a

stated probability

Inputs for Power Analyses

The desired power

Effect size (or, statistics that can be used to calculate effect size) Cohen’s d, Hedges’ g: standardized mean or mean difference ω2, Cohen’s f , or f 2: the variance explained regression slope, odds ratio, etc.

The α level

One-sided or two-sided test

The number of in independent variables (and correlations between them)

The number of groups based on each independent variable

Distributional properties (e.g., standard deviation) of the random variables in the analytical model

Standard deviation of the residuals (e.g., for linear regression)

The number of occasions

Bivariate correlation between repeated measurements (e.g., for repeated measure ANOVA)

etc.

Software for Power Analysis

Stand-alone power/specialized programs for power analysis only G*Power – FREE Pass SPSS/Sample Power

General purpose statistical packages SAS Stata SPSS faculty pack R– FREE

Software for multilevel modeling PINT– FREE Optimal Design Software– FREE MLPowsim– FREE Mplus

Some learning sources

Jacob Cohen (1992), "A power primer", Psychological Bulletin 112 (1): 155–159.

http://www.unt.edu/rss/class/mike/5030/articles/Cohen1992.pdf

http://web.vu.lt/fsf/d.noreika/files/2011/10/Cohen-J-1992-A-power-primer-kokio-reikia-imties-dyd%C5%BEio.pdf

http://classes.deonandan.com/hss4303/2010/cohen%201992%20sample%20size.pdf

UCLA Statistical Computing Seminars: Introduction to Power Analysis:

http://www.ats.ucla.edu/stat/seminars/Intro_power/

Intermediate Power Analysis: http://www.ats.ucla.edu/stat/seminars/power_analysis/Power_analysis_-_intermediate_course_for_UCLA_white.pdf

Advanced Power Analysis: http://www.ats.ucla.edu/stat/seminars/power_analysis/Power_analysis_-_advanced_course_for_UCLA_white.pdf

Statistical Analysis

The goal of statistical analysis is

hypothesis testing

Research strategies (including type of

research design, sampling strategy,

temporal design of measurements,

measurement of expected outcomes,

manipulation or measurement of the

independent variables, etc.) should be

guided by theory

Statistical models should be consistent

with theoretical reasoning and chosen

research methods

Choosing appropriate statistical tests

Clearly stated specific, empirically

grounded, and testable hypothesis

Numbers of IVs and DVs

Type (level of measurement) of each IV/DV

Continuous variables are measured at the

interval or ratio levels

Discrete variables are measured at the

nominal or ordinal levels

A dichotomous variable is discrete, but

can be treated as a continuous variable

statistically

How to communicate with your

statistician?

How to communicate with your

statistician? ~continued

http://pubs.acs.org/doi/abs/10.1021/om4000067

Variables How many? What type? Analysis Online text Movies

Group Differences: Comparing Group Means

DV -> One -> Continuous

IV One Discrete-two independent

groups

Independent t-test Statsoft

StatsLecture

Andy Field

IV One Discrete-three or more

independent groups

One-way ANOVA Statsoft StatsLecture

Andy Field

IV One Two time points (pre- and

post-tests), or two dependent

groups

Paired t-test U of M Paired t-test using SPSS

IV One Three or more time points Repeated measure

ANOVA

PowerPoint StatsLecture

Andy Field

IV Two or more Discrete; between-subject

factors only

Factorial ANOVA PowerPoint StatsLecture

Andy Field

IV Two or more Discrete and continuous;

between-subject factors only

ANCOVA Andy Field Andy Field

IV Two or more Two or more time points & at

least one between-subject

factor ( all discrete)

Mixed model ANOVA Mixed Model ANOVA/

ANCOVA

Andy Field

IV Two or more Two or more time points & at

least one between-subject

factor (discrete and

continuous)

Mixed model ANCOVA Mixed Model ANOVA/

ANCOVA

Same as above

DV-> Two or more -> Continuous

IV Any number All discrete MANOVA MANOVA/ MANCOVA Andy Field

IV Any number Discrete and continuous MANCOVA MANOVA/ MANCOVA Factorial MANCOVA

Variables How many? What type? Analysis Online text Movies

Relationships Among Variables

DV One Continuous

IV One Continuous Pearson’s r Hypothesis testing with

Pearson’s r

StatsLecture

Hypothesis testing

with Pearson’s r

IV Two or more Continuous and/or

dichotomous

Multiple regression Statsoft MR_ SPSS (Brief)

MR_ SPSS (more

depth)

DV Two or more Continuous

IV Two or more Continuous Canonical correlation or

path analysis

Introduction Canonical Correlation

DV Multiple Continuous

IV None Exploratory factor

analysis

Statsoft Andy Field

IV Latent variables Confirmatory factor

analysis

CFA CFA

CFA using Amos

DV One Discrete

IV One Discrete 2 test for contingency

tables

StatsLecture StatsLecture

IV Two or more Continuous Discriminant analysis Statsoft

DFA

DFA using SPSS

IV Two or more Continuous, discrete, and

dichotomous

Logistic regression Overview Logistic regression

Suppose that you are interested in

whether students learn better in the

morning or in the afternoon. You

recruited 20 students and taught

them vocabulary words at 7 am one

day and at 2pm another day.

Which statistical analysis would be

appropriate for the following research

example?

Suppose that you have designed an

intervention program to help people with

addiction to alcohol stop drinking. In order

to evaluate the program, you selected 30

subjects and randomly assigned 15 of them

to receive the intervention, and the other 15

subjects to the control group. After the

intervention, you measure the number of

drinks each person has in a week, and you

want to know if the two groups are different

in terms of drinking.

Which statistical analysis would be

appropriate for the following research

example?

Suppose you are interested in whether a

training program improves nurses’ skills in

identifying sepsis. Two groups of nurses are

randomly assigned to an experimental group

who receive the training (n=50), and a

control group (n=50) who do not receive the

training. Then, both groups are asked to

evaluate a simulation case which involves

sepsis. It is observed whether the nurses

successfully identify sepsis with the

hypothetical patient.

Which statistical analysis would be

appropriate for the following research

example?

Write down a clear statement of your

specific research hypothesis

Identify the independent variable(s)

and dependent variable(s) of the

hypothesis,

Identify the type of each of the IV(s)

and DV(s) as being either “discrete”

(or, categorical) or “continuous.”

More research examples from you?

Can data lie?

http://pss.sagepub.com/content/22/11/1359.abstract

Videos: When is

reproducibility an ethical

issue?

by Keith Baggerly

http://www.birs.ca/events/2013/5-

dayworkshops/13w5083/videos/watch/20130814112

1-Baggerly.mp4

Background of the Case

http://www.nature.com/news/2011/110111/full/469139a/box/1.html

Data Management

Entering Data

Cleaning Data

Scoring

Processing data

Documentation

Codebook

Annotated forms

A notebook of everything: proposal,

description of the project, codebook,

annotated forms, instrument and their

scoring instructions, consent forms, data

error reports and responses, all copies of

communications, etc.

Software Packages

R & Rstudio

SAS

SPSS

STATA

R and RStudio

Free

R (http://cran.r-project.org)

RStudio (https://www.rstudio.com)

From the very basic to very sophisticated

Examples of descriptive statistics and plots

A Shiny Apps example

http://shiny.rstudio.com/gallery/google-charts.html

United States

Health.Expenditure:3747.692121

Life.Expectancy:75.62195122

Region:North America

Population:266278000

United States

Health.Expenditure:8607.875672

Life.Expectancy:78.64146341

Region:North America

Population:311591917

Questions?

Thank you!