Week 8 sampling and measurements 2015
Transcript of Week 8 sampling and measurements 2015
Data CollectionPopulationSampling
More reminders ..
Get your variables sorted by now Chapters 1, 2, 3,4 are not yours Strong remark Use of phrases from another article Citing secondary sources Block quotation and Page reference In text citation Colloquial – “In my honest opinion….” Connecting ideas – on the other hand,
moreover, in support of, This is supported by.., in contrast to, in addition, nonetheless….
Instructor assigned task (25%)
TASK : Design a research
Survey
Correlational
Experimental
Causal Comparative
Case study – Mixed method
Ethnography
TOPIC : 21st century learners
Group work
Group of 3 (max)
Study task description on i-Learn
Assignment/Projects
Instructor Assigned Task
Use Blendspace
Due 30th May 2015
Marks given for product and comments/discussions
Where we are now…Observation Background survey
Broad area of research
Literature review
Problem definition
Problem statement
Research questions
Theoretical framework
Variables clearly identified
Hypothesis generation
Research design
Methods
Data collection
Analysis
Interpretation
Deduction
Hypothesis substantiated?
RQ answered?
Lesson Objectives
Describe procedures
Obtaining permissions for data collection
Selecting participants for data collection
Identifying data options
Recording and administering data collection
Let’s move on to data collectionWhat is data?Collection of facts from which conclusions may be drawn
Key ideas
Who will you study (unit, sampling, sample size)?
What permissions will you need? (levels,MOE)
What information will you collect? (types of data, links to questions/variables)
What instrument(s) will you use? (selecting an instrument, scales of measurement, validity, reliability)
How will you administer the data collection? (standardization, ethical issues)
Who will you study?
Unit of analysis is the level (e.g. individual, family, school, school district) the data will be gathered.
There may be different units of analysis
one for the dependent variable
one for the independent variable
Procedures for Collecting Quantitative Data
(1) Obtain permissions
secure permissions obtain informed consent
from participants
Obtaining Permissions
Institutional or organizational (e.g. school district)
Site-specific (e.g. secondary school)
Individual participants or parents
Campus approval (e.g. university or college)
Obtaining Informed Consent from EPRD,MOE
Obtain Approval via State Education Department
Have participants sign an informed consent form
Procedures for Collecting Quantitative Data
(2) Select participants
specify a population and sample
use probability and non-probability sampling
choose a sample size
Procedures for Collecting Quantitative Data
(3) Identify data options
specify variables operationalize variables select scales of
measurement choose types of data
measures
Procedures for Collecting Quantitative Data
(4) Record and administer data collection
locate or develop an instrument
obtain reliable and valid data develop administrative
procedures for data collection
Population and Sampling
POPULATION and SAMPLING
A population is a group of individuals that comprise the same characteristics
A sample is a sub-group of the target population that the researcher plans to study
Select Participants: Specify a Population and Sample
Samples
for the purpose about making generalizations about the target population (quantitative research).
samples are only estimates the difference between the
sample estimate and the true population is the “sampling error.”
Manageable research population
Populations and Samples
Sample
TargetPopulation
Population• All science teachers
in secondary schools in Kuantan
• College students in all community colleges
• Adult educators in all faculties of education
Better?All sec. school biology teachers in KuantanStudents in one community collegeAdult educators in faculties of education in the East Malaysia
Probability and Non-Probability Sampling
Probability sampling is the selection of individuals from the population so that they are representative of the population
Non-probability sampling is the selection of participants because they are available, convenient, or represent some characteristic the investigator wants to study.
Types of Sampling
Sampling Strategies
Probability/Random Sampling
Non-Probability/Purposeful Sampling
Simple Stratified ClusterRandom Sampling Sampling Sampling
Convenience SnowballSampling Sampling
Differences Between Random and Purposeful Sampling
Random “Quantitative” SamplingSelect Representative individuals
To generalize from sample to population
To make claims about the populationTo build/test “theories” that explain the population
Purposeful “Qualitative” SamplingSelect people/sites who can best help us understand our phenomenon
To develop detailed understanding
That might be “useful: informationThat might help people “learn” about the phenomenonThat might give voice to “silenced” people
Types of Probability Sampling
Simple Random: selecting a sample from the population so all in the population have an equal chance of being selected
Systematic: choosing every “nth” individual or site in the population until the desired sample size is achieved
Types of Probability Sampling Stratified sampling: stratifying the population on a characteristic (e.g. gender), then sampling from each stratum.
BoysN=6000
GirlsN=3000
Population(N=9000)
.66 of pop. 200
.33 of pop 100
Sample = 300
Types of Probability Sampling
Cluster Sampling:
Selects groups, not individuals
All members in the groups have similar characteristics
Useful when the population is large or spread over a wide geographical area
Example: Cluster Sampling
Population : All primary school teachers in Klang valley (5000)
The desired sample : 400 Cluster: School No. of primary schools in Klang valley: 150 Average number of teachers per school : 40 Number of cluster : 400/ 40 10 out of 150 schools are randomly selected All teachers in the selected schools make up
the sample
Try one
You want to study the resilience of UiTM students.
Population?
Sampling?
Select Participants: Choose a Sample Size
Select a sample size as large as possible from the individuals available
Select a sufficient number of participants for the statistical tests you will use
Calculate the sample size using a sample size formula
Select Participants: Choose a Sample Size
A rough estimate:
15 participants in each grp in an expt
30 participants for a correlational study
350 individuals for a survey study but depend of several factors
Calculating sample size
Krejcie and Morgan (1970)
EXAMPLE
Gender difference in vocational interest of post matriculation at UiTM
Independent variable: Dependent variable: Research Design: Population: Sample size: Instrument:
EXAMPLE
Gender difference in vocational interest of post matriculation at UiTMIndependent variable: GenderDependent variable: Vocational interestResearch Design: SurveyPopulation: Post matriculation studentsSample size: 350Instrument: Self developed
Calculating sample size
Use web calculator
http://www.raosoft.com/samplesize.html
http://survey.pearsonncs.com/sample-calc.htm
Non Probability Sampling
Types of Purposeful SamplingWhen Does Sampling Occur?
After Data Collection has startedWhat is the intent?
To developmanyperspectives
ExtremeCase
Sampling
To describeparticularlytroublesomeor enlighteningcases
TypicalSampling
To describe whatis “typical” to those unfamiliarwith the case
What is the intent?
To take advantageof whatever caseunfolds
OpportunisticSampling
To locatepeople orsites to study
SnowballSampling
To exploreconfirming ordisconfirmingcases
Confirming/Disconfirming
Sampling
MaximalVariationSampling
To generate a theory or concept
Theory or ConceptSampling
To describe somesub-group in depth
HomogenousSampling
Before Data Collection
One may sample..
Maximal variation
Most hardworking/ Highest achiever
Lowest achiever
Extreme case
??
Examples of Non-Probability Samples
Convenience Sampling: participants are selected because they are willing and available to be studied
Snowball Sampling: the researcher asks participants to identify other participants to become members of the sample.
Non Probability sampling example
Study delinquent behaviour during recess. I selected 1 school out of 4 - Interviewed all 4
principals and toured all 4 schools. I chose school to which I was given most access with fewest restrictions. Also school that reported the widest variations in delinquent behaviour during recess (very high to virtually no display).
Then sampled different locations with camera to find most varied activity and least self-conscious/guarded behavior. Where? Turned out to be behind the surau.
Later used snowball approach in choosing children to interview.
Your sample?
Proceed to collecting quantitative data
Collecting quantitative data
Flow of Activities in Collecting Data
1.Identify the variable2.Operationally define the
variable3.Locate data (measures,
observations, documents with questions and scales)
4.Collect data on instruments yielding numeric scores
Identify Data Options: Specify Variables
Independent Variables
Dependent Variables
Intervening
Moderating
Identify Data Options: Operationalize Variables
Operational Definition: The specification of how the variable will be defined and measured
typically based on the literature
often found in reports under “definition of terms”
Sometimes the researcher must construct it
Flow of Activities in Collecting Data
Identify the variable
Operationally define the variable
Locate data (measures, observations, documents with questions and scales)
Collect data on instruments yielding numeric scores
Self-efficacy for learning from others
Level of confidence that an individual can learn something by being taught by others
13 items on a self-efficacy attitudinal scale from Bergin (1989)
Scores of each item ranged from 0-10 with 10 being “completely confident.”
Flow of Activities Example
Flow of Activities in Collecting Data
Identify the variable
Operationally define the variable
Locate data (measures, observations, documents with questions and scales)
Collect data on instruments yielding numeric scores
Learning motivation of adult learners
Interest and level of engagement of an individual
----- items on a questionnaire developed
Scores of each item ranged from 1-5 with 5 being “most interested”
Flow of Activities Example
Scales of measurement
Identify Data Options: Select Scales of Measurement
Nominal (Categorical): categories that describe traits or characteristics
participants can check
Female Male
Ordinal: participants rank the order of a characteristic, trait or attribute
Identify Data Options: Select Scales of Measurement
Interval: provides “continuous” response possibilities to questions with assumed equal distance ; scale with no true zero
Discrete (SD ---------------------SA)
Metric (oC)
Ratio: a scale with a true zero and equal distances among units
PracticeIdentify the level of measurement
Measurement level? Age Religion Gender Income bracket Test scores CGPA Frequency of asking
questions Time spent on task Level of acceptance
(0 – never, 5 – all the time)
Activity ( 0 –not active, 5 – very
active)
Ratio ?
Nominal?
Ordinal?
Interval?
Measurement level? Age Religion Gender Income bracket Test scores CGPA Frequency of asking
questions Time spent on task Level of acceptance
(0 – never, 5 – all the time)
Activity ( 0 –not active, 5 – very
active)
Ratio Nominal Nominal Ordinal Ratio Ordinal Ratio Ratio Interval/
Ordinal
Interval/Ordinal
Identify Data Options: Choose Types of Data Measures
An instrument is a tool for measuring, observing, or documenting quantitative data
Types of Instruments Performance Measures (e.g. test performance) Attitudinal Measures (measures feelings
toward educational topics) Behavioral Measures (observations of
behavior) Factual Measures (documents, records)
Record and Administer Data Collection: Locate or Develop an Instrument
Develop your own instrument
Locate an existing instrument
Modify an existing instrument
Record and Administer Data Collection: Locate or Develop an Instrument
Strategies to use
Look in published journal articles Run an ERIC search and use the
term “instruments” and the topic of the study
Go to ERIC web site for Evaluation and Assessment
Examine guides to commercially available tests
Developing a questionnaire
Obtain Reliable and Valid Data
Reliability: individual scores from an instrument should be nearly the same or stable on repeated administrations of the instrument Bathroom scale
Reliability
Types of reliability
Test-retest (scores are stable over time)
Internal consistency (consistent scores across the instrument)
Cronbach coefficient alpha if items are scored as continuous variables (SA—SD)
Inter-rater reliability (similarity in observation of a behavior by two or more individuals)
Validity
Validity: the ability to draw meaningful and justifiable inferences from the scores about a sample or a population
Types of validity
Content (representative of all possible questions that could be asked)
Criterion-referenced (scores are a predictor of an outcome or criterion they are expected to predict
Construct (determination of the significance, meaning, purpose and use of the scores)
Let’ s look at a Rosenberg Self Esteem Scale
Collecting Qualitative Data
Key Ideas
Gaining site permission
Purposive sampling
Types of qualitative data
Protocols and Issues regarding administering and recording qualitative
Gaining Permission
Gain permission from Institutional Review Board
Gain permission from “gatekeepers” at the research site
Gatekeepers: individuals at the site who provide site access, helps researcher locate people and identifies places to study
The gatekeeper may require written permission about the project
Information for the gatekeeper Why their site was chosen
What time and resources are required What will be accomplished at the site What potential there is for your
presence to be disruptive What individuals at the site will gain
from the study How will you use and report the results
Types of data to Collect
Observations
Interviews
Documents
Audio-Visual Materials
Sources of Qualitative Data From People:
Interviews Surveys Focus Groups Participant
Observation (field notes)
From Things: Agency case records Miscellaneous
documents Historical Artifacts Media Published materials
ObservationInterviewsDocumentsAudio Visuals
Types of Data to Collect: Observations
An observation is the process if gathering first-hand information by observing people and places at a research site.
Observational roles
Participant observer
Non-Participant observer
Observational roles can be changed
Types of Data to Collect: Observations
Conduct multiple observations Record both descriptive and reflective
field notes during the observation Descriptive field notes describe the
events, activities and people Reflective field notes record
personal reflections that relate to their insights, hunches or broad themes that emerge
Administering and Recording Data: Observational Protocols
The header: essential information about the observation
Left column to record descriptive notes
Right column to record reflective notes
A picture of the site may be sketched
Types of Data to Collect: Interviews
Types of Interviews
Individual
Focus group
Telephone
Structured, Unstructured, and Semi-Structured Interviews
Approach toData
Collection
Type ofResponse Options toQuestions
Types of Interviews
Leading to
Data
Quantitative Close-Ended
Structured/semi-structuredInterviews
Scores toanswers
Qualitative Open-Ended
UnstructuredInterviews
Transcriptionof words
Types of Data to Collect: Interviews
General open-ended questions are asked
allows the participant to create options for responding
participants can voice their experiences and perspectives
Information is recorded then transcribed for analysis
Administering and Recording Data: Interview Protocols
The header: essential information about the interview
Open-ended questions include
“ice-breaker”
ones that address major research questions
probes that clarify and elaborate
Closing comments thanking the participant
EXAMPLE: Semi structured Interview protocols
Name: Date: Time: Venue: Experience – please describe Questions
1. Is the method a valid approach to addressing the learning needs of adult learners? Please describe your thoughts of the method
2. What do you think are the barriers to the utilization of the method in UiTM?
3. How would you rate the quality of learning using the method?
4. What overall suggestions do you have for improving the method?
Types of Data to Collect: Documents
Public and private records Good source for text data You must obtain permission before
using documents Scan documents when possible
Types of Data to Collect: Audio-Visual Materials
Determine the material that can provide evidence to address your research questions
Determine if the material is available and obtain permission to use it
Check the accuracy and authenticity of the material if you do not record it yourself
Collect the data and organize it
Administering and Recording Data: Field Issues Time needed for data collection
Limit initial collection or one or two observations or interviews
Time is needed to establish a substantial data base
Administering and Recording Data: Field Issues Obtaining permission to use materials
Ethical issues Anonymity of participants Convey true purpose of study without
deception
Let’s gather data
Fill in the Qnaire Tabulate results Answer research Qs Qualitative data
??
NEXTQuantitative Data Analysis