Design, Measurement, Information Design, Measurement, Information SourcesSources

ConceptualizationConceptualizationWhat is a concept?What is a concept?Dimensions of a conceptDimensions of a conceptIndicators of a conceptIndicators of a conceptValidity and reliability of indicatorsValidity and reliability of indicators

OperationalizationOperationalizationFactors to considerFactors to considerLevels of measurementLevels of measurement

Concept = mental imageConcept = mental image

Name for a category of thingsName for a category of things General, abstractGeneral, abstract Meaning is agreed upon by a groupMeaning is agreed upon by a group

no $, run-down apt., welfare, mom w/ kids, minority, …“Poverty” =

Examples of sociological conceptsExamples of sociological concepts

SocietySociety CultureCulture Social structureSocial structure Social institutionSocial institution Primary groupPrimary group Formal organizationFormal organization DevianceDeviance Social controlSocial control Social classSocial class Socioeconomic statusSocioeconomic status

PovertyPoverty InequalityInequality PrejudicePrejudice Minority groupMinority group Social distanceSocial distance LiberalismLiberalism Value of childrenValue of children Marital satisfactionMarital satisfaction ReligiosityReligiosity Social movementSocial movement

Nominal definitionNominal definition

Gives researcher’s definition of a conceptGives researcher’s definition of a concept Not exact or specificNot exact or specific Distinguishes it from similar conceptsDistinguishes it from similar concepts

Love is attachment.

Love is a very strong liking.

Love is a romantic and sexual attraction to another person.

Love is putting another’s needs before your own.

Violence is any kind of maltreatment.

Violence is a physical act that causes injury.

Violence is a physical act that is intended to cause injury.

Violence is an act that is intended to cause injury or pain, no matter how minor.

DimensionDimension Part or aspect of a conceptPart or aspect of a concept Component of a conceptComponent of a concept Not the entire concept in itselfNot the entire concept in itself

Love

IntimacyPassion

Commitment

Researchers may disagree about Researchers may disagree about dimensions of a concept:dimensions of a concept:

Love

AttachmentSexual attraction

Caring

Love

Attachment

OR:

4 Conceptualizations of Inequality4 Conceptualizations of Inequality

SES

EducationOccupation

Earnings

Class

Relationship to means of production

Status

Prestige

SES

Wealth

Prestige

Power

Knowledge

Indicators of a ConceptIndicators of a Concept

Measurable, observable variablesMeasurable, observable variables Indicate presence or absence of Indicate presence or absence of

conceptconcept At least one indicator per dimensionAt least one indicator per dimension

indicator = “clue” to a concept

Example: Indicators of LoveExample: Indicators of Love

Attachment: Attachment: “I would not like to live without “I would not like to live without .” .”

“ “When When is gone, I feel like part of me is is gone, I feel like part of me is missing.”missing.”

Intimacy: Intimacy: ““ and I are very close.” and I are very close.”

“ “ I can share my feelings with I can share my feelings with .”.”

“ “I trust I trust .”.”

Caring: Caring: “I put “I put ‘s needs before my own.”‘s needs before my own.”

“ “ I want I want to be happy.” to be happy.”

Example: Indicators of Social Example: Indicators of Social StratificationStratification

Status, prestige: Admiration of others, Status, prestige: Admiration of others, ranking of occupations by public.ranking of occupations by public.

Wealth: Income, assetsWealth: Income, assets Knowledge: Yrs. of education, degreeKnowledge: Yrs. of education, degree Power: Positions in organizationsPower: Positions in organizations Class, relationship to means of production:Class, relationship to means of production:

– Owns businessOwns business– Has employeesHas employees– Controls others’ labor (hire/fire, make rules, Controls others’ labor (hire/fire, make rules,

etc.)etc.)– Sells own laborSells own labor

Operational definition vs. Operational definition vs. Similar TermsSimilar Terms

The specific definition of a concept The specific definition of a concept that is used in a particular studythat is used in a particular study

May involve exact wording of a May involve exact wording of a measuremeasure

Indicators Indicators can have more than one can have more than one operational definitionoperational definition

““Variable”Variable” may refer to an indicator may refer to an indicator or operational definitionor operational definition

Examples: Operational Definitions Examples: Operational Definitions of “Income”of “Income”

Individual’s (or couple’s) gross Individual’s (or couple’s) gross annual income before taxes, as annual income before taxes, as reported on tax return.reported on tax return.

““Would you say your income last Would you say your income last year was between: 0 – 19,999; year was between: 0 – 19,999; 20,000 – 39,999; 40,000 – 49,999….”20,000 – 39,999; 40,000 – 49,999….”

Total family income from all sourcesTotal family income from all sources Top 25%, middle 50%; bottom 25%Top 25%, middle 50%; bottom 25%

Examples: Operational Definitions Examples: Operational Definitions of “Unemployment”of “Unemployment”

% of people who are not employed% of people who are not employed % of people 16-64 who are not % of people 16-64 who are not

employedemployed % of people 16-64 who are looking % of people 16-64 who are looking

for work for work % of people 16-64 looking for work or % of people 16-64 looking for work or

who gave up looking…who gave up looking…

Level of measurementLevel of measurement

Name categories, count frequencies

Rank categories in order

Numerical values; equal distances b/t categories

Meaningful zero point No

No

No No

No No

Yes Yes Yes

YesYesYes

Yes Yes

Yes

Yes

NominalInterval

RatioOrdinal

Examples of nominal variablesExamples of nominal variables

GenderGender (Male/Female) (Male/Female) Race Race (White, Black, Hispanic, Asian, (White, Black, Hispanic, Asian,

Native American, other)Native American, other) Political preferencePolitical preference (Democrat, (Democrat,

Republican, other)Republican, other) OccupationOccupation (job title) (job title) Favorite breakfast cerealFavorite breakfast cereal State of residenceState of residence

Examples of Ordinal VariablesExamples of Ordinal Variables

Opinion statementOpinion statement (strongly agree, (strongly agree, agree, don’t know, disagree, strongly agree, don’t know, disagree, strongly disagree)disagree)

Social classSocial class (lower, working, middle, (lower, working, middle, upper)upper)

Level of educationLevel of education (< high school, high (< high school, high school grad, vocational/technical degree, school grad, vocational/technical degree, bachelor’s degree, graduate degree)bachelor’s degree, graduate degree)

Church attendanceChurch attendance (never, several (never, several x/year, monthly, weekly)x/year, monthly, weekly)

Examples of Interval VariablesExamples of Interval Variables(Numeric, but scale is arbitrary)(Numeric, but scale is arbitrary)

TemperatureTemperature (Celsius, Fahrenheit) (Celsius, Fahrenheit) IQ scoreIQ score (average = 100) (average = 100) Occupational prestigeOccupational prestige (0 – 100) (0 – 100) Rating scalesRating scales (0-5, 1-10, etc.) (0-5, 1-10, etc.) Any numerically scaled variableAny numerically scaled variable

Examples of ratio variablesExamples of ratio variables

Income in Income in dollarsdollars YearsYears of education of education Age in Age in yearsyears NumberNumber of children of children PopulationPopulation size size RatesRates (births, deaths, crimes, etc.) (births, deaths, crimes, etc.)

What is the level of measurement?What is the level of measurement?

Marital status Marital status (never married, married, separated, (never married, married, separated, divorced, widowed)divorced, widowed)

Market value of home Market value of home (dollars)(dollars)

Approval of president’s job performance Approval of president’s job performance (strongly approve, approve, disapprove, strongly (strongly approve, approve, disapprove, strongly disapprove)disapprove)

Grade point average Grade point average (0.0 – 4.0)(0.0 – 4.0)

Letter grade Letter grade (A, A-, B+, B, B-,….)(A, A-, B+, B, B-,….)

Marital satisfaction Marital satisfaction (scale of 0 – 10)(scale of 0 – 10)

Participation in extracurricular activities Participation in extracurricular activities (Yes/No)(Yes/No)

Evaluating Indicators: ValidityEvaluating Indicators: Validity

Validity = “Truth” Validity = “Truth” Are we measuring what we think Are we measuring what we think

we’re measuring?we’re measuring? Does the indicator really reflect the Does the indicator really reflect the

concept?concept?

Ways to evaluate validityWays to evaluate validity

Face validity: does it make sense? Face validity: does it make sense? Agrees best judgment, or opinion of Agrees best judgment, or opinion of experts.experts.

Example: Makes sense to use income as a Example: Makes sense to use income as a measure of social position.measure of social position.

Predictive (criterion) validity: does it Predictive (criterion) validity: does it work? Is it a good predictor?work? Is it a good predictor?Example: ACT score is a predictor of college GPAExample: ACT score is a predictor of college GPA

Ways to evaluate validityWays to evaluate validity

Content validity: Does indicator cover all Content validity: Does indicator cover all the dimensions of the concept?the dimensions of the concept?Example: Measure of “risk taking” must include attitudes Example: Measure of “risk taking” must include attitudes (enjoy taking risks) and behaviors (actually takes risks).(enjoy taking risks) and behaviors (actually takes risks).

Construct validity: Does the indicator Construct validity: Does the indicator behave as expected? How is it related behave as expected? How is it related to other indicators of other concepts?to other indicators of other concepts?Example: Measure of “marital stress” should correlate Example: Measure of “marital stress” should correlate positively with divorce, negatively with measure of marital positively with divorce, negatively with measure of marital satisfaction.satisfaction.

Evaluating Indicators: ReliabilityEvaluating Indicators: Reliability

Reliability = dependabilityReliability = dependability Freedom from measurement errorFreedom from measurement error Measure yields the same results Measure yields the same results

every time.every time.

Reliability Reliability ≠ Validity≠ Validity

We can measure accurately without We can measure accurately without reflecting the underlying concept.reflecting the underlying concept.

Examples of reliable but invalid Examples of reliable but invalid indicators:indicators:Foot size as indicator of intelligenceFoot size as indicator of intelligenceRace as indicator of criminal behaviorRace as indicator of criminal behaviorNumber of children as indicator of masculinityNumber of children as indicator of masculinityAge as indicator of wisdomAge as indicator of wisdomLength of marriage as indicator of satisfactionLength of marriage as indicator of satisfaction

What causes measurement error?What causes measurement error?

Poorly designed measurePoorly designed measure Respondent or subjectRespondent or subject People working with measurePeople working with measure Changes over timeChanges over time

Measuring Reliability: Measuring Reliability: most common methodsmost common methods

Internal Consistency – correlation between Internal Consistency – correlation between several indicators of the same conceptseveral indicators of the same concept

Test-retest – same measure applied twice Test-retest – same measure applied twice should yield same results should yield same results

Reliability coefficient = results of a Reliability coefficient = results of a reliability test. Range from 0-1.0. Should reliability test. Range from 0-1.0. Should be very high (> .9) if measure is reliable.be very high (> .9) if measure is reliable.

Maintaining ReliabilityMaintaining Reliability

Careful wording of survey questionsCareful wording of survey questions Ask questions R’s can answerAsk questions R’s can answer Pretest questions or measuresPretest questions or measures Train people who will work with Train people who will work with

measures (e.g. interviewers, coders)measures (e.g. interviewers, coders) Be cautious of variables that change Be cautious of variables that change

over time.over time.