REC 375—Leadership and Management of Parks and

Recreation Services

Jim Herstine, Ph.D., CPRP

Assistant Professor

Parks and Recreation Management

UNC Wilmington

Evaluation

REC 375—Leadership and Management of

Parks and Recreation Services

Evaluation Defined

• Evaluation is the systematic process of determining the effectiveness of current practices, procedures, and plans

• Evaluation is a process that judges worth

Why Evaluate?

• Determine that funds are wisely spent• Conduct services efficiently• Reinforce staff efforts or recommend new

directions• Assist policy makers in directing the

organization toward more productive channels

• Evidence the strengths of services

Types of and Approaches to Evaluation

• Outcome Evaluation

• Process Evaluation

Outcome Evaluation

• Also called product evaluation, program evaluation, impact evaluation or summative evaluation

• Documents the effect of a service on a client

Process Evaluation

• Also called formative evaluation• Determines whether the service has been

conducted in an efficient, legal, and ethical manner

• Produces information during and after the delivery of the program or service that judges the worth of its development and operation

Scope of Evaluation• Annual evaluation of goals and objectives• Annual review of policies• Needs assessments• Management information systems• Performance appraisal of personnel• Program effectiveness• Equipment cost-effectiveness• Risk management• Fiscal accountability

Steps in the Evaluation Process

1. Determine the evaluation issues

2. Select an evaluation design

3. Collect the information

4. Analyze the information

5. Prepare a report

6. Implement the findings

Maintaining Research Integrity

• Be objective

• Minimize bias

• Approach the project fairly and honestly

• Understand the principles and procedures of evaluation

• Have exceptional mastery of the tools of research

• Be concerned with honest and careful evaluation

Sampling Techniques

• Probability Sampling

• Non-probability Sampling

Probability Sampling

• Probability sampling methods are based on randomness• Desire is to be able to “generalize” to the entire population

from which the sample is drawn• Most common method is simple random sampling—all

members of the study population have an equal opportunity to be chosen• Fishbowl Technique

• Random sampling with replacement• Random sampling without replacement

• Also can use a table of random numbers or computer based selection

Probability Sampling, continued

• Not always possible to select respondents with complete randomness• Stratified sampling

• Systematic sampling

• Deliberate sampling

Stratified Sampling

• The overall population of respondents is divided into several different subgroups according to common characteristics—age, gender, ethnic or racial identity, socio-economic status, etc.

• 20 basketball teams, 30 soccer teams, 25 volleyball teams, 35 baseball teams• Want a 25% survey sample

• Take 25% of each category as the stratified sample

Systematic Sampling

• This selection method occurs when some “system” is applied to the subjects• Interview every 5th person that comes by

• While systematic selection approximates a random sample, it does not strictly satisfy the definition of random sampling and, hence, is not entirely bias free

• It is quick, efficient and saves time and energy

Deliberate/Purposive Sampling

• With this method, the researcher knows that specific characteristics exist in a certain segment of a population

• Since these traits are extremely critical to the results of the investigation, the researcher deliberately selects those subjects who contain the characteristics

• This selection process is always biased

Non-Probability Sampling

• Samples not selected at random• Intact classes, volunteers, a typical group or

individual

• Intact or available groups impose a serious restriction on the researcher’s ability to generalize the data obtained to the larger population from which the sample was drawn

Information Gathering Methods

• Questionnaires and structured interviews

• Unstructured interviews and focus groups

• Observations

• Document study/secondary data review

• Non-reactive or unobtrusive measures• Physical traces

Analysis of Information

• Information gathered or measured is essentially nothing more than numbers or words—”raw” data

• Need to analyze the “raw” data to determine exactly what it means

Quantitative Data Analysis

• Describing Data• Frequency distributions• Measures of central tendency

• Mean, mode and median

• Measures of variability• Standard deviation, spread of scores

• Measures of relationship• Chi-square test (named variables) and Pearson

correlation (numerical variables)

Quantitative Data Analysis

• Inferring or inferential statistics• Measure differences between variables

• T-test—is there a significant difference between the scores of 2 groups (compares means)

• Analysis of variance—is there a significant difference between the scores of 3 or more groups

Quantitative Data Analysis

• Always have issues regarding “Reliability” and “Validity”

Reliability

• Refers to whether the repetition of a study or measurement will result in the same or a very similar answer• Do I get the same answers again and again?

• If subjects stand on a scale and the subjects are the same weight, does the scale record their weight as equal

Validity

• Refers to whether the evaluation actually measured what was intended to be measured• Is the weight of the subjects who stood on the

scale accurate? Do they really weigh what the scale says?

Qualitative Data Analysis

• Not all data that is collected is quantitative

• Some of the data is qualitative • Text (narrative), photographic or video form

• Look for themes and commonalities

Evaluation

Jim Herstine, Ph.D., CPRP

UNCW

herstinej@uncw.edu

910.962.3283