Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program...
Transcript of Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program...
![Page 1: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/1.jpg)
Data CollectionImproving Program Effectiveness Through the Use of Data
Presented to:What Works! Summit
Sept. 30, 2015
![Page 2: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/2.jpg)
Objectives • Participants will understand the point and purpose of data collection
• Participants will understand the use of data to inform business decisions
• Participants will be able to identify tools and methods of data collection
• Participants will be able to list barriers to accurate data collection
![Page 3: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/3.jpg)
What Is This Data Collection Thing? • Love It?• Answers?• Outcomes? • Statistics?• Information? • Evaluation?• Checks & balances?
• Measures?• Quantitative?• Qualitative?• Math? • A Process?• Necessary?• Variables?
• A Pain?• Nonsense?• A Ding? • Don’t care
about it?
![Page 4: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/4.jpg)
Data Collection Is……• the process of gathering and measuring information on variables of interest, in an established systematicfashion that enables one to answer stated research question hypotheses, and evaluate outcomes.
![Page 5: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/5.jpg)
This Data Collection Thing is NOT • Love It• Answers• Outcomes • Statistics• Information• Evaluation• Checks & balances
• Measures• Quantitative• Qualitative• Math • A Process• Necessary• Variables
A Pain
Nonsense
A Ding
Don’t care about it
Don’t care about it
![Page 6: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/6.jpg)
Why Collect Data?• If no data is collected, several questions will
naturally come to mind when observing how a program operates.– Are the services being provided correctly?– Are enough people being served?– Are the funds for the program being utilized properly?
• Without data being collected it is difficult to make determinations on the various aspects of the services being provided.
![Page 7: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/7.jpg)
Consider….• An agency wants to launch a new treatment/service but remains
concerned about staff expansion and utilization – Q. Should we move forward?
• Community stakeholders want to reduce school drop out and high crime in a particular area in the city intervention addresses only part of the issue – Q. Are funds being used properly?
• An agency director knows that treatment results should yield higher levels of success but something wrong – Q. Why is it not working for the population – Q. What if anything needs to be adjusted or changed?
![Page 8: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/8.jpg)
Why Collect Data!• Data is the life line of any business
– It helps one marketing decisions and fine‐tune new product launch strategy
– Breaks a macro picture into a micro one– Acquires meaningful insights from the dataset– Bases critical decisions on findings– Rules out human bias through proper statistical treatment
• Data is a decision supporting system
![Page 9: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/9.jpg)
Improving Program Effectiveness• A program’s effectiveness can be determined through the collection of data. This allows us to see: – What works? How can we continue it?– What isn’t working? How can we improve it?
![Page 10: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/10.jpg)
How to Collect Data• Develop time frame to collect data
– How often does the data need to be collected?• Repeatable
– The mechanism used to collect the data (both the form and the submission process) must be something that can be used as often as data needs to be collected.
• Clear and Concise– The requirements of the people generating the data cannot be
so demanding that they struggle to pull the data together or cannot dedicate the time to collect it
• Hold people turning data in responsible
![Page 11: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/11.jpg)
Data Collection Methods Include • Face‐to‐face or telephone interviews • Computer‐assisted data sheets• Questionnaires that are either personally, electronically or
administered through the mail • Observation of individuals and events with or without
videotaping or audio recording
![Page 12: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/12.jpg)
Example Of Computer Assisted Data Sheet
• Bob could you insert a DC tracker to show an example of bullet #2
![Page 13: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/13.jpg)
Obstacles to Data Collection• Accuracy • Non – Existent Standards of Protocol• Interpretation • Connected to locally and clearly defined goals • Training • Collection Tools
![Page 14: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/14.jpg)
Consequences from Improperly Collected Data Include
• Inability to answer research/impact questions accurately• Inability to repeat and validate interventions• Distorted findings resulting in wasted resources• Misleading others to pursue fruitless avenues of investigation • Compromising decisions for public policy • Causing harm to human participants
![Page 15: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/15.jpg)
Why Protocols
• Irrelevant or duplicate data collected • Pertinent data omitted • Erroneous or misinterpreted data collected • Too little data acquired from client • Data base format causes disorganized profile • Poor documentation from staff • Conflicting data • Insufficient time • Lack of equipment
![Page 16: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/16.jpg)
Clearly Defined Goals • Stakeholders goal or expected outcome is too general • Goals or expected outcomes does not derive meaningful
results that can inform decisions• Goals or expected outcomes is not measurable or observable• Client, provider, stakeholder not involved – what is measured
is of little to no value
![Page 17: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/17.jpg)
Getting to Clearly Defined Goals Make sure you have • A brief description of the project• The specific data that is needed• The rationale for collecting the data• What insight the data might provide (to a process being
studied) and how it will help • What will be done with the data once it has been collected
![Page 18: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/18.jpg)
Determining WHAT to Measure• Too much data is almost as bad as no data
– Effective measurement is the goal– “Paralysis by analysis” must be avoided
• Identify data points that touch on all aspects of program operation for collection– All facets of program operation should be considered (cases being seen,
cases completed, staffing, finances, etc.)• Make sure that the data requested is data that can be readily obtained
– If a data point requires too much time to collect, the chances of it being consistently collected decrease significantly
![Page 19: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/19.jpg)
Determining HOW to Measure• Once data is obtained, it must be interpreted correctly in
order to understand how successful operations are• Reasonable stratifications of performance are important in
proper evaluation• Periodic reviews of stratifications must be done to make sure
that they are giving the performance evaluations that are desired
![Page 20: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/20.jpg)
Using Data to Make Decisions• Having identified the data points to monitor, it is important to use
the data to help make program management decisions• Perspective needs to be used to understand trends vs. individual
data points– One must not read too much into small sample sizes
• Specific events that have taken place in operations (an employee/therapist leaving the company, etc.) must be considered when interpreting the data
![Page 21: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/21.jpg)
ReportingData reporting is the process of preparing and disseminating findings to the community • Monitor, convert & deploy your data
– Observe for trends– Converted into more understandable and precise presentation formats, such as
charts, files or graphs– Create written data report– Distribute report to those parties that need it
• Determining your audience– Whenever reporting out on data, it is crucial that you understand who is
going to be reviewing the reporting– Understand that different levels of staff maybe interested in different
levels of detail
![Page 22: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/22.jpg)
When Reporting• Beware of External Influences that could skew report
– Professional competition – Job security – Unclear guidelines from funding sources – Publication pressure
• Beware of Internal Influences that could skew report – Individual ego or vanity– Personal financial gain – Incompetence– Sloppy writing/reporting
![Page 23: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/23.jpg)
![Page 24: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/24.jpg)
![Page 25: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/25.jpg)
Be Sure to Report • If your intervention/ program had exactly the effects you expected and hoped it would.
• If your program/ intervention had no effect.• If your program/ intervention had a negative effect.• If your program/ intervention had the effects you hoped for and other effects as well.
• If your program / intervention gives you a clear indication that what you’re doing is accomplishing your purpose
![Page 26: Data Collection Presentation eba slides final 1 [Read-Only] · Data Collection Improving Program Effectiveness Through the Use of Data Presented to: What Works! Summit Sept. 30, 2015](https://reader034.fdocuments.in/reader034/viewer/2022043020/5f3c8d97a555160b773dcbf4/html5/thumbnails/26.jpg)
QUESTIONS?• Leslie Byam, DC Project Director, EBA, [email protected]• Bob Sayles, IT Director, EBA, [email protected]• Leslie Serena, Executive Director, Camelot Community Care,
[email protected]• Jennifer Bonness, Camelot Community Care, Clinical Director,