Survey exercise follow-up. Survey population Sample? Random Purposive Convenience Entire...
-
date post
21-Dec-2015 -
Category
Documents
-
view
218 -
download
2
Transcript of Survey exercise follow-up. Survey population Sample? Random Purposive Convenience Entire...
Survey exercise follow-up
Survey population Sample?
Random Purposive Convenience
Entire population Small, bounded population
All the people who work here
Non-response bias an issue in each case
Sample design Survey element – unit ABOUT WHICH information is
collected; unit of analysis E.g., Person; household; event
Survey universe – theoretical and hypothetical aggregation of elements E.g., Americans; users; visits
Population – specified aggregation of survey elements: Households in the US, defined as one person, or a
collection of related or unrelated persons…as of March, 2002
Visits to any web page in defined domain during March, 2002
Sample Design, cont. Survey population -- Aggregation of elements from
which survey sample is actually selected Households in 50 states with listed telephone numbers
Sampling unit – elements consider for selection at some stage listed residential phone numbers If multi-stage sample:
State; telephone directory; phone numbers Sampling frame – list of sampling units
Telephone directories for specified places Observation unit – element from which info collected
person over 18 who either answers the phone or is fetched by a person under 18 answering the phone
Sample selection Consider possible biases of sample design Mixed methods may give uninterpretable samples
E.g. mail to a group AND hand out in Sproul Plaza – so what population does your sample represent? And people on the mailing list who walk through Sproul
have *2* chances of being sampled Consider practical issues
Confidentiality often precludes getting a list, e.g. from the university
If you can’t quantify units (people, events), you can’t sample proportionately
Non-users or non-affiliated users (e.g., potential students; web users with no particular connection to the site) are hardest to track down
Introducing survey Purpose of introduction
Justify your asking Induce their cooperation Allay possible fears
“this is not a test…” Help them understand domain of survey, questions
Describing purpose of survey Enough to gain cooperation, not enough to bias
answers E.g. “a study of compensation in this field,” not “a study of
whether women’s salaries lag behind men’s in this field” SHORT but not at the expense of clarity.
Getting cooperation
Worthwhile topic, survey Not threatening Not a waste of time Interesting to them
Good design If you are not credible, not worth their
effort Short and easy to answer
Directions Short and clear People skim! The more you ask of them, the more likely
they are to mess up “please rate from 1, most important, to 7, least
important” Give instructions
For skip patterns “Did you use the library today? If not, go to
question 3. If yes,… For what to do if question does not apply
Choice of questions
Think through: what will you do with the answers?
If you don’t know, don’t ask If you do know, word in a way that
helps you E. g. “Have you ever used a library?”
What difference will it make if they answer “no”?
Choice of questions Respondents must be qualified to answer
“What would be the best way to bring peace to the Middle East?”
Respondents must think the topics of the questions are: Not invasion of privacy Not “dumb questions”; worth their time Not a test
Of them Of some idea or proposal they may not like
(unless you mean it) “if we had to cut the budget, should we lay people
off or reduce salaries?”
Time dimension to questions General, usual, most often:
What do you usually do? Specific
What did you do today? What have done in the last month?
NOT: What do you use this site for? (today? Ever?)
Memory What’s important to you may not be to them
“what did you do the last time you used this site?” Time and frequency are hard to remember
“Approximately how many times in the last month…”
Question wording Resolve any possible ambiguity!
Definitions, illustrations, examples Have you ever used a library? What’s a library? “Did you use the library’s homepage?”
Which page is that? What proportion of your income do you pay for housing?
Individual or household income? Housing: rent or mortgage; taxes? Insurance?
You cannot use “etc.” unless the reader can complete the list NO: “borrow library books, ask ref questions, etc.” YES: “borrow library books, ask reference questions, or
otherwise use the library’s resources or staff, in person or remotely.”
Use parallel construction What did you use this site for today?
Searching for information Placing an order Checking a previous orderNOT Searched for information I wanted to check on an earlier order
What did you use this site for today? If yes… What was the question to which “yes” was the
answer?
Answers Closed-ended are easier (which is usually,
but not always, a good thing) Easier for users (recognition, not recall; less
writing) They are less likely to think of different choices
Easier for you to summarize Be as exhaustive as possible with choices
If you leave out an answer that a user might reasonably give, they may or may not think to add it
They may become confused about the question or the survey
Answers, cont. Allow degrees of answers
Yes/ No, always/never are rarely the case E.g., Usually; Sometimes; Rarely
Allow multiple dimensions of answers How satisfied are you with your visit today?
Very: found what I was looking for. Not very: did not find.. What if I found what I needed but with
difficulty? I may have been successful but am dissatisfied with the experience.
Answers, cont. Give clear instructions
Check one? Check the one that best describes…? Check as many as apply?
Define! Lay out clearly
E.g. Ordinal scales all on one line How satisfied are you?
Very satisfied somewhat satisfied not at all OR
Very somewhat not at all
Satisfied satisfiedsatisfied
NOT: How satisfied are you? Very satisfied Somewhat satisfied Not at all satisfied
Ranges and intervals Want answers to be easy to for your reader
to interpret Generally, use either:
Intervals of equal size OR Intervals reflecting meaningful groupings
Age: under 18 18-25 25-40 40-55… OR intervals get larger when you expect fewer
responses How many years have you worked here?
Less than 1 1-3 4-10 10-20 more than 20
Avoid: Years worked here
1 year 10% 2-5 years 15% 6-7 years 20% 8-15 years 50%
Examples Have you checked out a book from
the UC Berkeley Library system? Yes no
If you use Pathfinder, Gladis, or Melvyl to find books or periodicals, where do you use them? Terminal at main library Terminal at my departmental library I don’t use these services
Examples If you could allocate funds for more salary and had
to pick from the following places, how would you prioritize where you would obtain these funds?(1 – first choice to find money through 7 – last place you
would go) tuition Pension plan Taxes Decreased health coverage State bonds Fund raising Less vacation/sick leave
Examples How long has your library offered live online
reference service? How long have you been a Reference
Librarian? (providing reference services comprises at least 40% of your day) Up to 5 years Between 5 and 10 years Between 10 and 15 years Over 15 years
Examples
Which part(s) of this web site interests you the most?
Which part(s) of this web site do you find the most uninteresting?
What parts of this web site do you like the most?
Which parts of this web site do you dislike the most?
Examples 1. Have you ever used the following online
services: [list]
2. If you checked any box in question 1, how important do you think online help would be in each of these activities? (only respond as applicable) [Item] Absolutely unimportant |--|--|--||--|--|--| absolutely important