Data Quality 101 What is Data Quality? · 5/2/2020 · 101 course (basics, beginnings, foundation)...
32
1 Data Quality 101 What is Data Quality? May 5 th , 2020 Meradith Alspaugh & Alissa Parrish
Transcript of Data Quality 101 What is Data Quality? · 5/2/2020 · 101 course (basics, beginnings, foundation)...
1
Data Quality 101What is Data Quality?
May 5th, 2020
Meradith Alspaugh&
Alissa Parrish
• Webinar will last about 60 minutes
• Access to recorded version
• Participants in ‘listen only’ mode
• Submit content related questions in Q&A box on right side of screen
• For technical issues, request assistance through the Chat box
Webinar Instructions
Presenter
Presentation Notes
Alissa Slides 2-9 Before we begin, I need to cover a few “housekeeping” items. We are recording today’s session, which will last approximately 60 minutes. We will post a link to the recording and the presentation slides to the HUDExchange in the near future. During this training, all participants will be in “listen-only” mode. We encourage you to submit content related questions in the Q&A box, located on the right side of your screen. Please feel free to submit questions at any time. Should you experience any technical difficulties, please request assistance by using the Chat box, and we will respond to them. If you have trouble hearing the audio through your computer speakers, please request assistance using the Chat box or connect your audio using your phone.
• Questions?
• Please submit your content related questions via the Q&A box
•Send to Host, Presenter and Panelists
Webinar Instructions
Presenter
Presentation Notes
Before we begin, I need to cover a few “housekeeping” items. We are recording today’s session, which will last approximately 60 minutes. We will post a link to the recording and the presentation slides to the HUDExchange in the near future. During this training, all participants will be in “listen-only” mode. We encourage you to submit content related questions in the Q&A box, located on the right side of your screen. Please feel free to submit questions at any time. Should you experience any technical difficulties, please request assistance by using the Chat box, and we will respond to them. If you have trouble hearing the audio through your computer speakers, please request assistance using the Chat box or connect your audio using your phone.
• Please submit any technical issue related questions via the Chat box
• Send the message directly to the Host
• Host will work directly with you to resolve those issues
Webinar Instructions
Presenter
Presentation Notes
Before we begin, I need to cover a few “housekeeping” items. We are recording today’s session, which will last approximately 60 minutes. We will post a link to the recording and the presentation slides to the HUDExchange in the near future. During this training, all participants will be in “listen-only” mode. We encourage you to submit content related questions in the Q&A box, located on the right side of your screen. Please feel free to submit questions at any time. Should you experience any technical difficulties, please request assistance by using the Chat box, and we will respond to them. If you have trouble hearing the audio through your computer speakers, please request assistance using the Chat box or connect your audio using your phone.
About NHSDC
The National Human Services Data Consortium (NHSDC) is an organization focused on developing effective leadership for the best use of information technology to manage human services. NHSDC provides information, assistance, peer to peer education and lifelong learning to its conference participants, website members and other interested parties in the articulation, planning, implementation and continuous operation of technology initiatives to collect, aggregate, analyze and present information regarding the provision of human services.
NHSDC holds two conferences every year that convene human services administrators primarily working in the homeless services data space together to learn best practices and share knowledge. The past 3 events have been put on with HUD as a co-sponsor. Learn more on our web site www.nhsdc.org.
After this virtual conference is over, NHSDC will be sending out a survey to learn about your experience. Please help us by signing up for emails and participating in the survey!
Learning Objectives
Explain HUD’s vision and strategy for data and understand how data quality fits into that context
Discuss the core elements, definitions, and metrics of data quality
Understand the roles that the CoC, HMIS Lead, HMIS Vendors, and HMIS Participating Organizations/Users play in ensuring high data quality
Session Overview
101 course (basics, beginnings, foundation)
Participant engagement will help guide the discussion (don’t be shy)
Next steps
Presenter
Presentation Notes
Foundational session that discusses the basics of data quality and what it is Even though we’re virtually presenting this, we’d still really like participant engagement through the chat Once there is a solid understanding of the basics of data quality, next steps can be identified that will help your community address data quality
Who’s With Us Today?
Options (select all that apply):• CoC• HMIS Lead/Administrator • HMIS Vendor• HMIS Participating Organization/End User• Person with Lived Experience• Government Entity• Funder• Other
Presenter
Presentation Notes
Adobe connect poll
Why Did You Choose This Session?
? ? ?? ?
Presenter
Presentation Notes
Please use the chat box to let us know why you’re here and what you’d like to get out of this session.
SNAPS Data Strategy and Data Quality
10
Presenter
Presentation Notes
Meradith Slides 10-14
SNAPS Data Strategy and Data Quality
• SNAPS Strategy is intended to be aspirational and not used to monitor projects for compliance
• Focus on ensuring CoCs have data-driven local planning to work towards ending homelessness
• CoCs, HMIS Leads, and Organizations work together to review the strategy and set local goals and performance indicators
SNAPS Data TA Strategy to Improve Data and Performance
11
Presenter
Presentation Notes
The Data TA Strategy has been available since 2018 It serves as the framework by which TA is provided to communities related to data quality and performance While the goals within the strategy are aspirational, the belief is that they are realistic The strategy addresses all pieces of data quality – completeness (including system coverage), timeliness, accuracy, and consistency The strategy also provides benchmarks to assist communities in making data-informed decisions based on accurate information CoCs, HMIS leads, and participating organizations all have a part to play in a community’s data quality efforts All relevant stakeholders should be involved in setting local benchmarks and performance indicators
SNAPS Data Strategy and Data Quality
3 specific strategies and today, we will highlight Strategy #2, as it focuses on data quality
Data Systems collect Accurate, Comprehensive, and Timely Data
12
Presenter
Presentation Notes
The Strategy lays out 3 specific areas of focus and for this session, we will center around Strategy #2 because of its overall theme of data quality Data systems can only collect accurate, comprehensive, and timely data if the community understands these concepts and makes them a priority
SNAPS Data Strategy and Data Quality
13
Presenter
Presentation Notes
Full bed coverage in HMIS ensures a full picture of what is available within the homeless services system, who it is serving, and how it is performing. Which status does your community currently represent related to bed coverage? (Adobe Connect poll?) High-quality data means it is complete, accurate, and timely. We will define these terms more throughout this session. Based on what you understand these terms to mean, which status does your community currently represent related to quality data? (Adobe Connect poll?)
What is Data Quality?
14
Data Quality Defined
Data Quality refers to the reliability and comprehensiveness of your community’s data
Components of data quality include:• Timeliness• Completeness• Accuracy• Consistency
15
Timeliness Completeness
Accuracy Consistency
Presenter
Presentation Notes
We’ll break down each one of these terms more during the rest of this session, and provide examples of how to measure each Alissa Slides 15-18
Requirements for Data Quality
2004 HMIS Data and Technical Standards
4.2.2. Data Quality (Baseline Requirement)
• “PPI collected by a CHO must be relevant to the purpose for which it is to be used. To the extent necessary for those purposes, PPI should be accurate, complete and timely.”
2004 HMIS Data and Technical Standards
16
Presenter
Presentation Notes
HUD’s baseline requirement for data quality is in the 2004 Data and Technical Standards and really only includes one sentence. How each community measures and quantifies accuracy, completeness, and timeliness varies
Data Quality Strengths
On which data quality component is your community doing well?
Options:• Timeliness• Completeness• Accuracy • Consistency
Why are you doing well?
17
Presenter
Presentation Notes
Adobe Poll
Data Quality Limitations
With which data quality component is your community struggling?
Options:• Timeliness• Completeness• Accuracy • Consistency
Why are you struggling?
18
Presenter
Presentation Notes
Adobe Poll
Timeliness • Data Quality Framework report includes a timeliness measure
• Other reports can also be used to report on data timeliness
• Reviewing timeliness of data for all phases of a client’s project activity helpful to understand where a lack of timeliness may be affecting a system’s data quality
• Most communities measure timeliness of project enrollments but just as important to measure timeliness of updates and project exits
• It may also be useful to look at which parts of the system need to be timelier in data entry than others, based on how quickly the system needs to respond to the data once it’s entered
19
Timeliness Completeness
Accuracy Consistency
“The degree to which the data is collected and available
when it is needed.”
Presenter
Presentation Notes
Which project types need to have their data into the system as soon as possible so that the homeless services system can act upon that data? Which project types may be given a little more leeway on data entry timeliness because the system isn’t relying on it to be able to serve clients appropriately? What tools do you use to measure this? What’s working well? What’s not working well? How do you set your baseline for timeliness? Meradith Slides 19-22
Completeness
20
Timeliness Completeness
Accuracy Consistency
“The degree to which all required data is known and documented. Coverage and utilization are both forms of
completeness.”
• Data completeness includes collecting and entering all required data elements into HMIS
• Also includes bed coverage & utilization
• Reporting on whether all required data elements are entered into the system is generally easy to measure
• It may also include setting baselines for an acceptable rate of responses that are “client doesn’t know”, “client refused”, and “data not collected”. At a minimum, a flag or alert for high % of these responses could help decide when to check in with projects to review data quality.
• A lack of bed coverage in HMIS can significantly impact understanding your homeless services system
• Working with non-HMIS providers to understand why they don’t use HMIS can help find ways to increase bed coverage
Presenter
Presentation Notes
There may be many barriers to lower HMIS coverage/utilization – Lack of agency resources – too few staff, lots of volunteers; no HMIS budget; Limited computer skills or even computers; may not have reliable internet Don’t see the value of HMIS; Existing HMIS ease of use for high volume shelters
Accuracy
“The degree to which data reflects the real-
world client or service.”
21
Timeliness Completeness
Accuracy Consistency
Data accuracy can be difficult to measure because the system doesn’t know what it doesn’t know. There are some pieces that you can look at related to data accuracy:
• 1 and only 1 head of household for any given household • Date of Birth = Project Start, especially for clients defined as
head of household • Clients under the age of 18 are not veterans • Prior living situation, length of time, approximate date, # of
times, and # of months (3.917 questions) congruency
Other pieces of data accuracy that are just as important but can be more difficult to report on include:
• All clients served are entered into the system• All clients exited have been exited from the system
• Helps to look at utilization
Consistency
“The degree to which the data is equivalent
in the way it is collected and stored”
22
Timeliness Completeness
Accuracy Consistency
Consistency across the HMIS is not always easy to measure
• Do all organizations understand the data elements in the same way?
• Are all intake workers collecting the information from clients in a consistent manner?
Presenter
Presentation Notes
Who’s Involved?
Who’s involved in the data quality process in your community?
Options (select all that apply):• CoC• HMIS Lead/Administrator • HMIS Vendor• HMIS Participating Organization/End User• Funder• Other
23
Presenter
Presentation Notes
Adobe Poll Alissa Slides 23-29
Stakeholders
24
CoC
HMIS Lead
HMIS VendorParticipating Organizations
Local / State Funder
Presenter
Presentation Notes
For data quality to be fully addressed within a community, each stakeholder must be involved and understand their roles and responsibilities related to HMIS and data quality.
CoC
• Celebrate successes and allow room for growth from all involved
• Make connections between data quality efforts and other CoC efforts
• Empower HMIS Lead to carry out a comprehensive DQMP
• Serve as the enforcement and encouragement of the DQMP
25
CoC
HMIS Lead
HMIS VendorParticipating Organizations
Local / State Funder
Presenter
Presentation Notes
The CoC is the entity that can serve as the enforcer of data quality. They are the entity that will have both the carrot and the stick. Partnership between the CoC and HMIS Lead to provide consistent messaging about the importance of data quality and how it fits into the larger community priorities is essential.
HMIS Lead
• Conducts monitoring of data quality in HMIS
• Works closely with participating organizations and end users to address data quality issues
• Collaborates with CoC to ensure consistent messaging and connections between data quality and other CoCwork
26
CoC
HMIS Lead
HMIS VendorParticipating Organizations
Local / State Funder
Presenter
Presentation Notes
While the HMIS Lead carries out the day-to-day work of monitoring the data quality in HMIS, they must have the support of the CoC to enforce those efforts. The HMIS Lead must be empowered by the CoC to carry out the work and participating organizations should know that any requests to address data quality that come from the HMIS Lead are backed fully by the CoC.
HMIS Vendor
• Ensure HMIS software is compliant with HUD data standards and reporting specifications
• Provide sufficient documentation for HMIS Leads and other partners for software-specific workflows, reports, and other system functionality important to understand
27
CoC
HMIS Lead
HMIS VendorParticipating Organizations
Local / State Funder
Presenter
Presentation Notes
The entity that holds the contract with the HMIS vendor has the direct line of communication with the vendor. However, all stakeholders should provide feedback on how HMIS could be improved to ensure effective and efficient data entry into HMIS. The HMIS Lead should ensure the vendor and the software are compliant with HUD standards. Any feedback about system features or functionality that could improve the user experience and increase data quality should be provided to the vendor and any responses from the vendor should be communicated back to the requesting stakeholder.
Participating Organizations
• Partner with and be responsive to the HMIS Lead and CoC to address data quality issues that arise
• If you don’t understand something, ASK, don’t GUESS
• Utilize resources that are made available (reports, HMIS help desk, visual guides, helper guides, training opportunities, etc.)
28
CoC
HMIS Lead
HMIS VendorParticipating Organizations
Local / State Funder
Presenter
Presentation Notes
For participating organizations to understand the role they play in HMIS data quality, they must understand the CoC’s overall goals and how data quality plays a part in those goals. When participating organizations don’t have a connection to the larger CoC and the CoC’s priorities, data quality suffers. Additionally, if the organizational culture doesn’t include an understanding of the importance of data quality and how the organization fits into the larger whole, users don’t connect how what they do every day and enter into the system fits into the picture. Making data-informed decisions is a culture and it starts at the top.
Local / State Funder
• Consider requiring the use of HMIS for grantees (both entering data into HMIS and reporting data out of HMIS)
• Partner with the CoC to understand community initiatives, goals, and how your funding can support those
29
CoC
HMIS Lead
HMIS VendorParticipating Organizations
Local / State Funder
Presenter
Presentation Notes
While HMIS is required for HUD and federal partner funding streams, state and local funders can play a large part in ensuring a comprehensive system by requiring their grantees to also enter data into HMIS. HMIS has the ability to be a robust and comprehensive data set about those at-risk of and experiencing homelessness in your community, and this often requires partnership with local and state funders that provide grants for homeless service providers in the community. Find ways to partner with funders and speak their language.
Action Plan
30
• When will these action steps occur?
• Why are these your next steps?
• Who will you involve?
• What will you focus on next?
What Who
WhenWhy
Presenter
Presentation Notes
Let’s talk about next steps: Who What When Why Meradith & Alissa together
Resources
CoC Data Quality Brief (April 2017)
Data Quality and Analysis for System Performance Improvement (July 2017)
Introductory Guide to Submitting LSA Data: Appendix LSA Data Quality Table Shells (October 2018)
31
Thank you!
Meradith Alspaugh Alissa Parrish
The Partnership Center ICF
[email protected] [email protected]
32
