ASQ - National Institute on Out-of-School Time · Welcome to ASQ —After‐School Quality: The...
Transcript of ASQ - National Institute on Out-of-School Time · Welcome to ASQ —After‐School Quality: The...
After‐School Quality:The Process of Program Improvement
Learn how to:
• Pair ASQ with program quality assessment tools
• Involve multiple stakeholders
• Select and train an ASQ Team
• Gather and organize data
• Develop an action plan to reach improvement goals
ASQA guide for facilitating a 5‐step, team‐based approach to improving program quality.
Welcome to ASQ — After‐School Quality: The Process of Program Improvement.
With today’s increasing interest in program quality, there are a growing number of tools that programs
can use to assess and improve quality. What’s missing is a process for involving multiple stakeholders in
the effort to document program quality. That’s where ASQ comes in.
ASQ is designed to assist your program to become the best it can be. ASQ is a five‐step self‐assessment
process that helps you examine the quality of your program and then make changes that will lead to
better practices and more positive youth outcomes.
The cornerstone of ASQ is a team approach. ASQ relies on the ideas and energy of all the people who
care about your afterschool program— staff, families, children, youth, program directors, boards of
directors, and hosts. All involved are invested, have a voice, and can help shape the solutions because
ASQ includes everyone in program improvement.
The ASQ process is about change. ASQ is based on the belief that the most successful program
improvements happen when key stakeholders are involved in a slow, gradual change.
ASQ supports the use of any youth program quality assessment tool. Open‐ended and flexible, ASQ
helps you make the most of any program observation tools and questionnaires/surveys. The ASQ process
focuses on how one uses an instrument, who is involved in using it, and, most importantly, what is
learned from the information gathered. This leads to developing strategies for program improvement.
ASQ: Supporting Program Assessment through the Years
ASQ was first developed in 1991 and has changed over time to meet the needs of programs like yours. ASQ
originally stood for “Advancing School‐Age Childcare Quality” and was used as the basis for the National
Afterschool Association program improvement and accreditation system.
Today ASQ (now known as “After‐School Quality: The Process of Program Improvement”) has been
reformatted to support anybody who is interested in improving the quality of their program—for self‐
assessment purposes, state level quality rating system advancement, and as a way to become accredited
from the Council on Accreditation. In each case, ASQ is a flexible, effective tool for ensuring that the
process of improving program quality comes from a team of people who are invested in and care about
your program.
‐ i ‐
Welcome
In gratitude to the original authors of ASQ, the many individuals that have used ASQover the decades in support of youth, and those who contributed to this version.Please turn to page 32 where we acknowledge our authors and supporters.
National Institute on Out‐of‐School TimeA program of the Wellesley Centers for Women at Wellesley College
May 2012
Copyright © 2011 National Institute on Out‐of‐School Time. All rights reserved. Programs that have purchased this material for internal self‐assessment and program improvement are allowed to copy this guide for use by their staff and ASQ Team. For all other organizations and purposes, no part of this work may be reproduced without prior written permission.
Introduction
p. iii Overview & Goals
p. iv Introduction to the ASQ Process
p. vii The ASQ Approach
Step 1: Preparing the Program Community for Improvement
p. 2 Overview & Goals
p. 3 Prepare the Staff
p. 5 Prepare the Program Stakeholders
Step 2: Preparing the ASQ Team
p. 8 Overview & Goals
p. 9 Select the ASQ Team and the Tools
p. 12 Train the ASQ Team
Step 3: Gathering the Information
p. 15 Overview & Goals
p. 16 Observe the Program
p. 19 Complete the Questionnaires
p. 21 Summarize the Information
Step 4: Creating the Action Plan
p. 25 Overview & Goals
p. 26 Make Sense of the Information
p. 28 Share the Information
p. 30 Create the Action Plan
Step 5: Taking Action
p. 33 Overview & Goals
p. 34 Coordinate the Tasks of the Action Plan
p. 35 Check on the Process of Program Improvement
p. 36 Solve Problems
Acknowledgements
p. 37 Our Authors and Contributors
Table of Contents
‐ ii ‐
After School Quality: The Process of Program Improvement (ASQ) is a five‐step process
that helps you and a team of people committed to program improvement examine the
quality of your program and then make changes that will lead to better practices. You
will involve the program community in an open dialogue about your program.
Together, you will set a program vision and assess how well your program is meeting
the needs of the children, youth, and families it serves. Your ASQ Team will develop an
action plan to reach improvement goals. The ASQ process takes into account that
"you"—administration, staff, families, children, youth, and community members—are
the best people to make the changes that will improve your program.
ASQ is the process you will need to make the most of the program observation tools
and questionnaires/surveys you are using. This guide will help you understand the ASQ
five‐step process and how to use it. Each step will be discussed in depth with an
overview and goals, and tips for success. In addition, the ASQ Web Portal at
www.niost.org contains downloadable resources to accompany each step. This guide is
intended to provide a starting point for implementing a process that will support your
program quality assessment efforts. ASQ Team leaders should feel free to adapt the
materials in the way that makes sense for their programs.
OVERVIEW
• Become familiar with the ASQ process
• Understand the ASQ approach
• Prepare to launch the five steps of ASQ
GOALS
Introduction
Visit the ASQ Portal for More ResourcesAt www.niost.org under Our Work/Training, the portal link is http://www.niost.org/asq_login
As a companion to this guide, the ASQ Portal contains materials to support each step of the ASQ process. Resources include customizable forms and letters, step‐by‐step instructions for running meetings, and more.
‐ iii ‐
INTRODUCTION TO THE ASQ PROCESS
What Are the Five Steps in the ASQ Process?
Step 1: Preparing the Program Community
ASQ involves people in defining the problem and discovering the solutions
Introduce staff and the broader program stakeholders to ASQ and program improvement,
and clarify what they want children and youth to gain from the program.
Step 2: Preparing the ASQ Team
Select the ASQ team, determine which tools, questionnaires and surveys
to use, train and prepare team for observation and survey collection.
Step 3: Gathering the Information
Learn how your program is doing through observation and questionnaires surveys.
Review the “big picture” by summarizing your data.
Step 4: Creating the Action Plan
Identify areas of strength and areas that need to be improved, set priorities, and identify
strategies for improvement.
Step 5: Taking Action
Coordinate the tasks set forth in the Action Plan and check on the process of
making program improvements.
Who Uses ASQ?
Any organization that wants to improve the quality of their afterschool program. In the past,
programs found they were most successful implementing ASQ when they were financially stable and
not undergoing significant changes. Ideally programs will have committed leadership that has been in
place for several years, several staff members who have been with the program for some time,
agencies that fully support their improvement efforts, and actively involved families.
Who Makes the ASQ Process Happen?
The ASQ process is set in motion by an ASQ Team leader. The team leader is usually
the program director, but it can be anyone with the time, authority, and enthusiasm
to make changes in the program. The Team works best when it includes
representatives from all the stakeholders in the community, including staff, families,
older children or youth, program administration, and host sites (school, community
center, etc.). If your program is not ready to involve the larger community of
stakeholders, it is fine to begin the ASQ process with internal staff as a first step
towards involving a broader team.
Do People in the Program Who Are Not on the Team Have a Say?
Absolutely. ASQ encourages two meetings that are held for all who are interested in the success of
the program—one at the very beginning of the process, and one toward the end. Families, older
children or youth, staff, administration, the host, and other interested members of the program
community are encouraged to attend so they can learn about ASQ, program quality, and progress
made by the ASQ Team.
‐ iv ‐
ASQ involves peoplein defining the problem and discovering the
solutions.
ASQ is designed to be flexible. Ideally the ASQ Team represents all
program stakeholders. If you are not ready for a
team of staff and stakeholders, you can begin using ASQ with
just staff.
INTRODUCTION TO THE ASQ PROCESS
How Does ASQ Support Your Work with Quality Assessment Tools?
You may already be working with a specific program observation tool or questionnaires/surveys. If
you have been using these tools with your staff only, ASQ recommends expanding involvement to a
new, broader team by holding a community meeting and inviting other stakeholders to get involved.
If you have not yet chosen questionnaires/surveys or an assessment tool or are not required to use
specific tools, look to Step 2 to learn how to select the tools that make sense for your program.
How Long Does the ASQ Process Take?
The time it takes to assess your program and develop an action plan depends on your program, your
program’s needs, and who is involved. It may take several months to a year. Once you identify areas
that need improvement, it is best to choose a few areas to begin working on and then create an
action plan. The time it will take to accomplish the goals in the action plan depends on how ambitious
the action plan is and what resources will be needed to implement it.
This process is a continuous cycle (see Figure 1 on pg. vi). You will continue to reflect on your practice,
identify areas of improvement and establish better practices.
Can You Use ASQ When You Are Pursuing COA Accreditation?
Yes! The ASQ process allows for effective self‐assessment, making it a valuable tool for any program
that is in the process of obtaining accreditation from the Council of Accreditation.
How Will You and Your Program Benefit from the ASQ Process?
• You will have a way of communicating what is working well to those directly connected to your
program, as well as to the larger community.
• You will demonstrate to the families who rely on you for out‐of‐school time programming that
you are truly interested in meeting their needs.
• You will gain increased parental involvement. This may be the first time that families have a say in
the decision‐making process through questionnaires/surveys, speaking up at program‐wide
meetings, or being on the ASQ Team.
• You will provide a way to involve children and youth in planning the program. Have them
complete questionnaires/surveys and involve them in the ASQ Team.
• You will ensure that families and stakeholders gain a deeper understanding of what it takes to run
a quality program.
• You will gain both increased support and valuable feedback by opening up the process to the
whole program community.
• You will build relationships among staff, families, the host and the program community. ASQ is a
team‐building experience.
‐ v ‐
Questionnaires Available at the ASQ Portal• Visit the ASQ Portal to download ready‐to‐use questionnaires for Staff, Family, Child and
Youth, Host, and Director. The ASQ Observation tool that measures quality items linked to the NAA national standards is available.
• In addition, go to www.niost.org for information on useful tools such as the APT and SAYO tools which make up the Afterschool Program Assessment System (APAS) system.
INTRODUCTION TO THE ASQ PROCESS
How Will You Benefit from the ASQ Process?
• You will be energized by the valuable input of people who care about the program.
• You will learn a process that you will be able to use as an ongoing tool for improvement. You will
make time to reflect on the way you do your work.
• You will have the opportunity to compare the way you work now to quality standards.*
• You will become part of a national movement to work on program improvement through self‐
study.
• You will learn better ways to communicate to people within and outside the program what your
program does well.
• You will be able to clarify your goals for the future.
• You will improve your observation skills.
• You will share leadership, which will reduce the burden on the director by sharing the power and
sharing the work.
‐ vi ‐
Set Goals& Select Outcomes
Document& Track. Collect Data
Reflect on Findings & Make
Action Plan
Make Program Changes
Re‐Assess.Collect Data
Figure 1: The Continuous Improvement Cycle
*You will be using observation tools and questionnaires to gather information about your program. In general, these tools are research‐based instruments which focus on standards of quality for out‐of‐school time programs in the areas of relationships, activities, program planning, staff development, administration, etc.
For information on national standards visit the National Afterschool Association at http://www.naaweb.org/default.asp?contentID=612. The NAA Standards for Quality School‐Age Care, are divided into 36 “keys” of quality. NAA offers these standards to programs via two publications.
THE ASQ APPROACH
There is always room for improvement.
Every program can become better by building on its strengths. Children and youth grow, learn, and
change every day. We must be ready to change and improve to meet their needs. Everyone with a
connection to the program can suggest ways to help meet the program’s goals.
Change which involves everyone is the most successful.
ASQ is based on the belief that the most successful program improvements happen when the key
stakeholders are involved in slow, gradual change. This is the same belief found in the Japanese
model for organizational learning known as the Kaizen model. It is human nature to want things to
remain the same. But sometimes resistance comes from not being clear about what needs to change
and why. When you are part of the process, you can see and understand the need for changes. It is
then easier to let go of the reluctance to change.1
Slow, gradual change will last.
Another principle from the Kaizen model is that it takes time to change. Start with a few changes and
work on them over time. See what happens before you take on other changes. Work at a pace that is
comfortable for your program. Small but constant improvement that involves everyone is apt to have
lasting effects. Broad, sweeping change is hard to maintain.
People working together are the best resource.
When people openly share ideas, creative solutions can result. When you respect each other's ideas,
you will find that you can handle problems you thought were impossible to solve. Using the ASQ
process, programs have produced amazing results without spending much money.
Sharing the leadership role will increase participation.
An important concept for the ASQ Team is shared leadership. What do we mean by that? The success
of ASQ really depends on the strength of the ASQ Team. This team is made up of a leader and
members. The contribution of every member of the team is necessary for the process to work. Shared
leadership does not mean no one is in charge. It means that all group members participate by sharing
leadership behaviors. See the “Who Does What?” chart (page viii) for a description of the
responsibilities of the team leader and team members.
‐ vii ‐
1Masaaki Imai, KAIZEN: The Key To Japan's Competitive Success. New York: Random House, 1986, pp. 2‐7.
THE ASQ APPROACH
‐ viii ‐
2Peter Senge, The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday, 1990, p. 6.
Who Does What?
ASQ Team Leader ASQ Team Members Ideally Will
• Organizes meetings • Introduces agendas • Makes sure the process is moving along • Encourages leadership behaviors • Gives the group an opportunity to talk about the process
• Coordinates the work of the action plan
• View each other as valued players • Listen to each other • Contribute to meeting agendas • Summarize what is being said • Identify problems • Suggest solutions • Point out conflicts • Encourage each other • Work toward general agreement • Implement the action plan • Solve problems which arise
General agreement comes with compromise.
Any group of people is bound to have a range of opinions. Sometimes the result is conflict. Opening
up the process to many points of view can present new possibilities. Negotiation and compromise are
the way to reach realistic goals. Work toward general agreement. Search for strategies that everyone
can live with.
One change will lead to another.
Change has a ripple effect. For example, you may decide to create new interest areas in your
program. This simple act will probably spark changes in children’s or youth’s activity choices. You may
notice changes in their behavior. Observing how a change in one area of the program affects other
areas is an important part of the ASQ process.
Make time for reflection.
Build in time to reflect on your findings as well as the process of self‐assessment. Reflection involves
pausing to examine and inquire about beliefs, goals, and practices. Looking at what you do, and how
and why you do it can lead to deeper understanding.
ASQ helps you create a "learning organization"
In his book The Fifth Discipline, Peter Senge defines “learning organizations” as organizations where
people continually expand their capacity to create the results they truly desire, where new and
expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people
are continually learning how to learn together. Learning organizations "continually enhance their
capacity to realize their highest aspirations.”2 Senge has identified five core disciplines that influence
the ability of organizations to "learn": Personal Mastery, Mental Models, Shared Vision, Team
Learning, and Systems Thinking. Many ideas from these disciplines have informed the philosophy and
practice of the five steps of the ASQ.
THE ASQ APPROACH
‐ ix ‐
3Senge, p. 7, 8, 9, 10, 13, 208‐9, 239.
How ASQ Aligns with Senge’s Learning Organization Disciplines3
Learning Organization Discipline What ASQ Does
Personal Mastery• Clarifying and deepening our personal vision
• Focusing our energies • Seeing reality objectively • Developing needed skills
The ASQ process encourages staff, community, and ASQ Team members to state their goals and priorities. During the ASQ process, Team members learn to observe and rate what they see. They also compare what they see to what others see. The questionnaires/surveys help Team members to understand others' points of view. Team members are asked what training they need to master needed skills.
Mental Models• Deeply ingrained assumptionsthat influence how we understand the world and how we take action
• Be open to hearing new ways of looking at some old problems
As ASQ Team members listen to people discuss the program, they may discover that it has been stuck in a pattern that is not best serving the needs of children and youth. For example, the daily routine may be to serve a snack as soon as the kids come in. But other models are possible: a snack can be offered at different times, or it can be served buffet‐style and left out because kids are hungry at different times.
Shared Vision• A pull toward a goal which people truly want to achieve
• Shared “pictures of the future” foster genuine commitment rather than compliance
During Step 1, the program community meets to establish a shared vision. The ASQ Team will revisit this vision as it sets goals and strategies for an action plan. ASQ Team meetings provide time for a continual dialogue about individual and shared visions for the organization.
Team Learning• Starts with “dialogue” so people can enter into a genuine “thinking together”
• Collectively we can be more insightful than we can possibly be individually
The focus of the ASQ process is teamwork. The ASQ Team assesses program quality and discusses what has been learned. The action plan will be developed with analysis from the group as well as input from the whole community. The Team works together to solve new problems that arise as the plan is put into place.
Systems Thinking• You understand the system by contemplating the whole
• Continually discovering how people create their reality and how they can change it
The observations and questionnaires/surveys typically cover the elements of program quality. These elements are interconnected. For example, you might consider offering youth more activity choices. The attempt to solve one problem may inadvertently solve another. You may find that by changing the schedule for outdoor physical activity and homework, that the children may be more focused when they return from outside.
STEPPreparing the Program Communityfor Improvement
Prepare the Staff
Prepare the Stakeholders( Staff | Family | Children & Youth | Host )
Step 1 Step 2 Step 3 Step 4 Step 5
1
STEP 1 Preparing the Program Community for Improvement
From the moment you decide to improve your program, participation is the key to
success. The process begins by introducing staff to the ASQ process. They may want
some time to become familiar with the ASQ approach and the five steps of the process,
as well as local or national quality standards. When you are ready to involve other
program stakeholders (families, children, youth, the host, and others who play a role in
the program), start with a program‐wide meeting to clarify what benefits the
stakeholders want children and youth to gain from their program experiences. In
addition to a community‐wide meeting, reach out to those who cannot attend a
meeting through phone interviews, surveys, and conversations at pick‐up time with
parents or guardians. Once general agreement is reached on your vision and goals,
allow plenty of time for people to play a part in the decisions and changes.
OVERVIEW
• Introduce staff and the broader program stakeholders to ASQ and program improvement
• Clarify what staff and program stakeholders want children and youth to gain from the program
• Involve staff and program stakeholders in deciding to work on program improvement
GOALS
Step 1: Preparing the Program Community for Improvement ‐ 2 ‐
PREPARE THE STAFF
Success depends on community involvement, and the foundation of that success is the "buy‐in" of staff,
the people who work to create great experiences for the children and youth. They are on the front lines
and have important contributions to program improvement. Many of the changes will need to be
implemented by the staff. This means they have to believe in the goals of the program and the strategies
used to reach those goals. Sometimes staff feel overworked and underpaid. They may feel nervous
about program improvements when they feel they are being overworked already, being observed or
worrying that their needs are not understood by the community they serve. It is wise to invest the time
needed to ensure staff are comfortable and invested with what is happening.
Start with yourself.
If you will be leading the drive for improvement, think about what this role will mean. How much
time will it take? What supports will you need?
Introduce the idea of ASQ to your staff.
Take a few minutes during a staff meeting to introduce ASQ and suggest that you schedule time at
the next staff meeting to consider working more formally on program improvement. Remind staff
that meetings often provide time for reflection on program practice. The ASQ process takes reflection
a step further. It takes a comprehensive look at the program through gathering information using
surveys, questionnaires, or observation tools. It involves other members of the program community.
Tell staff that this meeting will be an opportunity for everyone to step back and take a look at the “big
picture”. See page 3 for a sample meeting agenda.
Find ways to compensate staff for time spent on work issues.
Pay staff for the meeting time, allow comp time, offer free training opportunities, or provide some
other special recognition.
Step 1: Preparing the Program Community for Improvement
“ We have always spent time in staff meetings to talk about how things are going, but we have never done a formal assessment, nor invited parents and others in the community to share their opinions about the direction of the program. We are excited and nervous at the same time.” — Program Director
‐ 3 ‐
Step 1: Preparing the Program Community for Improvement ‐ 4 ‐
SAMPLE AGENDA
ASQ Resources available on NIOST Portal• Tips for Preparing Staff • Sample Meeting Agenda – Staff Meeting • Leader’s Guide to the Staff Meeting • What Is Strong and What Could Improve?
Note: The times listed in all sample meeting agendas are suggestions. Feel free to modify the time allotments based on the size of your group
A staff meeting is a great way to introduce the ASQ process and establish buy‐in from staff.
Sample Agenda: Introducing Program Improvement to Staff
Goals• Involve staff in deciding to work on program improvement• Introduce ASQ process
5 minutes WelcomeThank staff for their time and review the agenda.
10 minutes Establish ground rulesSet guidelines to help people feel comfortable participating.
15 minutes Warm‐up exercise: “Looking at what we do”Ask staff to brainstorm a list of rewards and challenges in their work. Look for patterns within each category and notice similarities/differences between categories.
35 minutes What is strong/What could improveConduct an exercise to help identify program strengths and areas for improvement.
15 minutes What is ASQIntroduce ASQ and the five steps. Invite staff to ask questions and offer ideas on implementing ASQ.
5 minutes Next stepsSet next steps for planning a program‐wide meeting to introduce other stakeholders to ASQ.
PREPARE THE PROGRAM STAKEHOLDERS
Once you and your staff are ready to involve more stakeholders in the ASQ self‐assessment process, you
may wonder if it is necessary to convene a program‐wide meeting. Why not simply ask stakeholders to
be part of the ASQ Team? Let staff know you want the broader program community to "buy into" the
idea of improvement. Allowing staff, families, and other program stakeholders to talk about the
commitment that ASQ and program improvement takes will ensure there will be more support for
changes. It is also important because you begin to create a dialogue between staff, family members, and
stakeholders about goals and visions for the program. Programs that have taken the time to hold a
program‐wide meeting have found it improved communication, brought new energy to the program,
and provided important insights about the goals and value of their work.
Use the program‐wide meeting to build widespread support.
This meeting is your first and best opportunity to energize stakeholders and generate enthusiasm for
the ASQ process. It gives stakeholders the opportunity to “buy into” the ASQ process. See Page 5 for a
sample meeting agenda.
Think carefully about who should be “at the table.”
Invite everyone who cares about your program's success. This could include the custodian of your
school space, the swimming instructor who teaches the youth every week, or the woman across the
street who is patient about the children’s outdoor noise. Be sure to invite the people who control the
money, the space, or the rules about how your program runs, as well as youth attending the program.
Choose a convenient time, date, and place for the meeting for a successful turnout.
Ask your staff, several parents, members of the host community, and even children and youth to tell
you what strategies they think will get the broadest attendance at the meeting.
Publicize to ensure the best possible attendance.
Announce the upcoming meeting through your newsletter, “kid mail,” and other venues.
Prepare the meeting space ahead of time.
Gather all necessary materials in advance, arrange the furniture so people can talk to each other, and
assign someone to take notes so you have a good record of the meeting.
Continued communication is important.
Once the meeting is over, be sure to communicate important information about program
improvement to the stakeholders.
Keep in MindASQ is designed to be flexible. If you are not ready for a team of staff and stakeholders, begin using ASQ with internal staff only as a first step toward involving the broader community.
“ ASQ gave me a concrete way to work together with our school. By having the principal as part of the ASQ Team, she was able to understand some of the challenges we face. It really helped to demystify what happens in afterschool.” — Program Director
Step 1: Preparing the Program Community for Improvement ‐ 5 ‐
Step 1: Preparing the Program Community for Improvement ‐ 6 ‐
SAMPLE AGENDA
ASQ Resources available on NIOST Portal• Sample Announcement for Program Community Meeting • Sample Meeting Agenda – Program Community Meeting • Leader’s Guide to the Program Community Meeting• Memory Exercise Form• What Do We Want for Our Children – Exercise
A program‐wide meeting allows stakeholders to become involved in decision making and clarifies their
vision for the program. Invite everyone involved in the program to this meeting.
Sample Agenda: Program Community Meeting
Goals• Clarify what stakeholders want children and youth to get out of the program• Introduce the ASQ process• Involve stakeholders in deciding to work on program improvement
10 minutes Welcome and Introductions• Introductions• Goals of the meeting• Review agenda• Setting ground rules
20 minutes Memory ExerciseHave participants think about how they spent their out‐of‐school time and compare how things are different for kids today.
45 minutes What do we want for our children and youthConduct an exercise to explore what participants want children and youth to accomplish during program hours. Identify top priorities.
5 minutes Break
20 minutes What is ASQIntroduce the ASQ process. Invite participants to ask questions and offer ideas on implementing ASQ.
5 minutes Next stepsBuild the ASQ team. Broad‐based support
and participation will be necessary to help you achieve the best results from your efforts. To help you accomplish this, make sure you reach out to those who cannot attend a community meeting to include them in the process and keep them informed.
STEP
• Orient the ASQ Team• Train the ASQ Team• Complete a Work Plan
Step 1 Step 2 Step 3 Step 4 Step 5
2
Select the ASQ Team( Staff | Families | Children & Youth | Host )
Preparing the ASQ Team
STEP 2 Preparing the ASQ Team
The work of ASQ is done by an ASQ Team that represents the key "players" in an out‐
of‐school time program. The team should include staff, family members, youth and
older children in your program, representatives of organizations that have a say in your
space or budget, and interested community members. ASQ has been designed so
everyone in the program can participate. People do not need to be experts about out‐
of‐school‐time programs to be on the team. In fact, a successful ASQ Team contains
people of many backgrounds who are enthusiastic about being involved and can make
the commitment the ASQ process requires.
During Step 2, you will select and train team members so that they are well‐prepared
for the work ahead. For many members, it may be the first time they have gathered
together to talk about the program. People who have volunteered to be on ASQ Teams
around the country have found it a rich learning experience. Carefully choosing and
training the ASQ Team sets a solid foundation for the work ahead.
OVERVIEW
• Select the ASQ Team that represents the key players in your program.
• Select appropriate tools, surveys, or questionnaires.
• Orient the ASQ Team.
• Train the ASQ Team and complete a work plan for the months ahead.
GOALS
Step 2: Preparing the ASQ Team ‐ 8 ‐
SELECT THE ASQ TEAM AND THE TOOLS
Determining the ASQ team
The best ASQ Team will include people who share the many different perspectives of various groups
of people in the program. Team members might include the program director (most likely as the ASQ
Team leader), staff, people who do the daily work that makes the program run, family members, and
even children and youth. Consider including a board member, a member of the host agency, and key
community people (child‐care advocates, school principals, government officials).
Include the Host Site on the ASQ team
The host is any place where an afterschool program is located such as a school, church, or
community‐based organization. A program has a lot to gain from a good working partnership with the
host. The host may make crucial decisions about program resources. But the "host" is rarely a single
person. If it makes sense, include more than one member of the host community on your team.
ASQ team size
The size of the team depends of the size of the program and the various groups that want to be
included. A medium‐sized program with about 60 children might have a team of eight: two family
members, three staff (including the director), two older children or youth, and a host representative.
A larger program may want to include more people, but it is always important to remember it is hard
to get groups of more than 10 people to meet and make decisions.
The use of a Coach
For many afterschool programs, successful program improvement happens with the support of
coaches—individuals with the technical expertise to guide programs through the process. Who are
coaches and what do they offer? Coaches are typically local experts in afterschool, quality
improvement, and mentoring. They provide ongoing training and technical assistance as the program
assesses quality and develops an action plan. Coaches can:
• answer questions
• keep programs on task by periodically checking in
• help in tool selection, set up databases, and input data
• assist in explaining data and setting goals for program improvement based on the reports.
Coaches can provide key assistance in helping you summarize and make sense of your data. In
addition, coaches help staff establish a climate of dialogue and build capacity for reflective practice.
Coaches build positive, trusting relationships and use open‐ended questions to facilitate dialogue and
inquiry. NIOST is happy to provide coaching or facilitation services.
‐ 9 ‐
“ Participating in this process, as a staff person, brought me to a deeper level of practice. Even though I was really nervous about participating, it was one of the best things I’ve done.” — Assistant Group Leader
Step 2: Preparing the ASQ Team
SELECT THE ASQ TEAM AND THE TOOLS
For multi‐site agencies
Each site is unique, even when they are all run by the same agency. Consider having a separate ASQ
Team for each site. You may also choose to coordinate the work by having an ASQ Team with
representatives from each site. In this case, each site would have a smaller "action team," which
conducts its own program observation, collects its own questionnaires/surveys, and develops its own
action plan. The coordinating ASQ Team develops the action plan for administrative issues that apply
to all sites.
Qualities of an ASQ team member
• Look for people who like to work with others toward a common goal
• Look for individuals who are sensitive and respectful toward other people
• Look for energy and enthusiasm
Tool selection
Tool selection should be based on the outcomes desired for your program. It could be helpful to have
an internal data specialist or coach to assist with tool selection. You can review the Forum for Youth
Investment’s report “Measuring Youth Program Quality: A Guide to Assessment Tools, Second
Edition,” available for free at: http://www.forumforyouthinvestment.org/node/297
This guide compares 10 youth program quality assessment tools and offers guidance on issues to
consider when selecting and using these tools. Information on the Assessment of Afterschool
Program System (APAS) tools are accessible at www.niost.org.
Sample ASQ Team:
Program director, site coordinator, line staff, board/advisory council member(s), leadership from
a large organizations (i.e. where fiscal and policy decisions are made), host agency (school principal,
school liason, community center representative, etc.), familes, older youth.
How much of a time commitment do ASQ members need to make?
Once your team is in place, the members will need to set aside approximately 15 hours for their core
functions. These include:
• Orientation
• Training
• Program observation
• Making sense of the information
• Developing an action plan
• Checking on progress
• Attending the second community‐wide meeting
‐ 10 ‐
“ I feel that I now really understand how much skill it takes to provide my child and large groups of children with consistently good experiences.” — Parent
Step 2: Preparing the ASQ Team
SELECT THE ASQ TEAM AND THE TOOLS
What if some of the most valuable team members cannot spare 15 hours?
Sometimes a school principal or a family member who has much to contribute to the team cannot
attend every meeting that has been listed. Programs that have used ASQ say that there are three
important times to have the involvement of key people with busy schedules. They feel everyone
should be involved in:
• the program‐wide meeting, where everyone shares their opinions about the vision for the
program;
• program observation/observation‐meeting, which allows team members to rate the program, or
hear the results of observations
• developing an action plan
‐ 11 ‐
“ The program has been operating in my school for several years. Now I see what an important part of the day this is for the children and what an essential service it provides to families.” — School Principal
“ ASQ is the foundation that openedthe minds of my staff and broughtus together as a team.” — Site Leader
Step 2: Preparing the ASQ Team
“ This was the first time that staff and family members had talked about the program. It was great to have time to hear another point of view. One night, we got into a long discussion about homework. There had been some tensions about it, but we had never talked it over in a group. In addition to coming to a compromise, I think that each of us learned something important about communication that night.” — ASQ Group Leader
TRAIN THE ASQ TEAM
Now that you’ve selected members of the ASQ team, it’s time to prepare them by running an orientation
and a training meeting. The sample agendas (see Pages 10 and 11) outline suggested content for each of
these meetings.
An orientation meeting helps team members become comfortable working together.
‐ 12 ‐
ASQ Resources available on NIOST Portal• Leader’s Guide to the ASQ Team Orientation• Sample Agenda ASQ Team Orientation Meeting
Step 2: Preparing the ASQ Team
Sample Agenda: ASQ Team Orientation
Goals• To set a foundation for smooth teamwork• To set up a schedule of team meetings
5 minutes Statement of GoalsUpdate the group on what has happened to introduce the ASQ process up to this point to staff or to the program‐wide community. Review the goals from the list of “What do we want to accomplish for our children” generated at the program‐wide meeting and the “What is strong/what could improve” list identified in the staff introduction meeting from Step 1.
10 minutes Introductions IcebreakerConduct an icebreaker activity to help team members get to know each other.
15 minutes Establishing Ground RulesSet guidelines to help people feel comfortable participating.
25 minutes What’s in a TeamConduct an exercise to come to agreement on ways the group can best work together.
35 minutes What Do We Do When We DisagreeConduct an exercise to learn how to work together and to handle conflict.
10 minutes Next stepsSchedule the next meeting. Review what team members should prepare (e.g., become familiar with the observation tool and questionnaires/surveys, review local or national standards, etc.). Part of the next meeting will be devoted to training ASQ Team members to use the assessment tools.
TRAIN THE ASQ TEAM
‐ 13 ‐
ASQ Resources available on NIOST Portal• Sample Agenda ASQ Team Training Meeting• Leaders Guide to the ASQ Team Training Meeting• Developing a Work Plan handout • ASQ Team Work Plan Chart • Keeping Meetings on Track• Checking In on Group Process• Exercise: Understanding the Developmental Needs of Children & Youth• Handout: The Developmental Needs of Children & Youth
Note: The content of your agenda will depend on the tools being used.
Step 2: Preparing the ASQ Team
The training ensures all team members share a basic understanding of the standards and tools they will
be using.
Sample Agenda: ASQ Team Training
Goals• Become familiar with local or national quality standards• Learn how to use the observation tool and questionnaires/surveys
5 minutes Welcome
30 minutes Standards of QualityConduct an activity to become familiar with the observation tool and questionnaires/surveys. Examine how these tools relate to local or national quality standards.
60 minutes Observing the ProgramConduct an activity to practice using the observation tool.
25 minutes Developing a Work PlanMake a plan about how work will be done, when, and by whom.
10 minutes ASQ Team Meeting Reporting FormIntroduce a process to keep the entire staff informed about what is happening in the ASQ process (e.g., updates through email, staff meetings, or forms placed in staff boxes). Assign someone to be the staff representative. It is best to have staff on the ASQ Team take responsibility for communication between the ASQ Team and the whole staff. The staff can communicate issues directly with the staff representative to the Team.
5 minutes Next stepsLet all team members know where they can get copies of tools. You will not be meeting again until all the observations have been completed and the questionnaire/survey results have been gathered.
STEP
Step 1 Step 2 Step 3 Step 4 Step 5
3
ASQ Team Observes:
• Before school• After school• Kindergarten• Summer• Full day
Program participants fill out questionnaires / surveys:
• Family• Child/youth• Staff• Host• Director
Program observationsare tallied
Questionnaires / Surveysare tallied
Self‐Assessment Summary
Gathering the Information
STEP 3 Gathering the Information
You have arrived at the heart of the self‐assessment process. This is an exciting time of
learning. Depending on the assessment tools you are using, team members will observe
the program and everyone involved in the program will be encouraged to complete a
questionnaire/survey. The program observation process and questionnaires/surveys
are rich sources of information. People experience the program very differently. You
will learn how many people share the same feelings and how strong those feelings are.
As you analyze the information, you will begin to see patterns to guide your work in
developing an action plan.
OVERVIEW
• Learn how the program is doing through the observation process.
• Learn how the program community feels about the program throughfeedback from questionnaires/surveys.
• Review the "big picture" by summarizing the information.
GOALS
Step 3: Gathering the Information ‐ 15 ‐
OBSERVE THE PROGRAM
ASQ Team members often report that observing the program is the most valuable experience in the self‐
assessment process. Staff enjoy the chance to step back from everyday tasks to reflect on how the
program works. Hosts often find new ideas about how to support the program. Families are usually
amazed at how much goes into each program day. The team frequently finds that this observation
experience can bring a deeper commitment to the program improvement work that lies ahead. Here are
some tips to make the observation a positive and productive experience:
Each team member brings their own special perspective to the observation.
Remember, there are no right or wrong answers when it comes to what you see in the observations.
The strength of the team process is that everyone will contribute something. You may see things that
other people will not; others may see things you do not.
Come prepared.
Review the observation tool that you have chosen to use the night before. Bring it on a clipboard,
along with a pencil for easy rating. Dress comfortably and be prepared to go outside when the
children and youth do. Arrive early enough to get a short tour from the director before the children
and youth arrive and confirm that you are observing a typical day.
See the the full range of program activities.
All members of the team should visit the afterschool program as an “observer” rather than as the
role they usually play (i.e. parent, director, etc.). It is recommended to see the program in its entirety,
from arrivals and departures, transition times, and indoor and outdoor activities. Three hours should
be enough time to observe. At least two members of the team should visit other program
components: before‐school, kindergarten, full‐day, and summer. You will find that each part of the
program may have different strengths and challenges.
‐ 16 ‐
Consider this
• Watching staff and children struggle with a space that is too small and has few supplies means much more to an observer than hearing about it at a meeting.
• Feeling either the stuffiness or the chill in a program's space motivates people to search immediately for solutions.
• Seeing how hard the staff work to care for and challenge the youth inspires the team to increase its support for the staff.
“ By watching the program in action, I realized how difficult it is to have so many program activities in the gym. I want to explore other space possibilities, so the gym can just be used for sports and games.”— Host
“ Having the afternoon to observe the program was one of the best things I've done all year. I have a renewed sense of what the day feels like for the kids, and I have new ideas for the space and for our schedule.” — Staff Member
Step 3: Gathering the Information
OBSERVE THE PROGRAM
Be respectful of program staff.
Even the most experienced staff can feel "on the spot" when being observed. Remember that they
are working hard every day to create a good program for the children and youth. Before you begin
observing, make sure you are introduced to each staff member and refer to them by name. While you
observe, be aware of your facial expressions and body language. Maintain a neutral expression—you
do not want staff to worry that you look displeased or uninterested. Speak to other observers only
when necessary. Most of all, maintain confidentiality about what you observe. For example, if you
hear a staff person mishandle a situation with a child, do not discuss this with your family and friends.
When you describe it to the ASQ Team, describe the situation without naming the staff member. Only
the director needs to be told the name of the person in order to address the situation during
supervision. Also, do not discuss the observation with other team members in public places (e.g.
grocery store, restaurant, public area of the program).
Make careful ratings
Move through the whole program so you can see as much as possible. Write down what you see,
then decide on your ratings while the visit is fresh in your memory (some tools may require that you
rate all before you leave the observation, some may say rate and revisit). Rate fairly. Do not rate high
because you feel sympathy for problems you see or low because something annoyed you. If you are
unable to decide on a rating because you do not have enough information, ask the director questions
to learn what you need to know. Ask open‐ended questions that start with "What do you do about
...?" or "Tell me about ..." You will get better information than if you ask "Do you ...?" which can be
answered with a yes or no. If it was raining when you visited the program, you will need to ask: "What
do the children and youth do on a typical day for outdoor play?“
‐ 17 ‐
ASQ Resources available on NIOST Portal• Checklist for Program Observation • Sample Program Observation Confirmation Letter
“ As a parent, I had never really taken the time to watch the program in action. I was always in a hurry to pick up my child. It was a wonderful experience to spend the afternoon at the program, using the program observation. It made me realize how much goes into making a program that works well for my 5-year-old AND for my 10-year-old. I am so impressed with the staff.” — Parent
Step 3: Gathering the Information
OBSERVE THE PROGRAM
‐ 18 ‐Step 3: Gathering the Information
Questions & Answers about the Observation
How can we make the time for staff to observe?
One of the first hurdles the ASQ Team leader faces is how to free up the staff for an observation. You
may feel tempted to skip this step in the self‐assessment. But staff members often feel this is the most
valuable step in the process. Here are some ideas:
• Most programs start by spreading the observations over a period of days so only one staff person
is missing at a time.
• Some programs hire a substitute or ask a volunteer or agency staff member to step in, making
sure they meet regulations in filling the empty spot. Substitutes should not be observed.
• Other programs have the director, site coordinator, or specialist fill in.
Plan to leave money in the budget to cover this cost, or add it to a request for funds to support self‐
assessment from a local foundation or civic organization.
Should the observers visit every room of the program?
Visit every room and as many activities as possible. Staying for longer periods per activity is preferable.
Observers can comfortably view four to five rooms and outdoor areas in the course of an afternoon
observation. If your program is spread over more than four areas and serves more than 80 children,
observers will find it difficult to see everything in one afternoon. In this case, you can divide up the
program so each group of observers works in only one‐half of the program. You will then analyze the
observations of each area separately.
How does the observer make the best use of his or her time in the program?
Timing is often a problem both for observers and programs. We suggest seeing an activity from start
to completion during your observation. Several visits may be needed to see the full range of a
program.
Do we need to observe the summer program?
Some summer programs are quite similar to the school‐year program, but more often, summer
programs are quite different. At the least, the pace of the day and the activities offered are usually
different. It is important to observe summer programs as an important component of your offerings,
but address it separately with its own action plan.
How do we rate a standard when staff and their actions differ so much?
You will be observing a number of staff members at once and observing hundreds of interactions they
have with the children. Usually, the patterns you observe make choosing a rating easy. But what if
there is a staff member or situation that breaks the pattern?
Imagine you are observing how staff interacts with youth. There are 3 staff members, two are
completely involved with the youth but one staff person is disengaged. Not only is this person distant,
but is so busy straightening the cupboards that he or she refuses to help a child who is crying. How do
you average this into the rating? Ask yourself how much this one staff person and his or her behavior
affects the individual child and the tone of the group. If the individual child quickly gets help from
another member of the staff and the children tend to avoid the problem staff person, you may only
bring the rating down slightly. But if it has a major impact on the child and the group, you’d drop your
rating more. Be sure to describe the situation in your comments and discuss it with the ASQ Team
leader. This is the type of situation that needs attention through staff supervision.
COMPLETE QUESTIONNAIRES/SURVEYS
Questionnaires/surveys are a wonderful way to involve everyone in the self‐assessment process and give
them a say in how the program will change. The team may learn that families love the program, but find
the 5:30 closing time difficult. Children and youth may wish for a nicer outdoor play space and larger
snacks. The staff may want more training. This information will help you create an action plan that takes
the range of experiences into account.
Get lots of volunteers to help.
The ASQ Team leader should help identify parents, staff, and even children and youth who can help
distribute questionnaires/surveys.
Offer convenient times and methods for people to fill out the questionnaires/surveys.
You get the best results when people have time to give thoughtful answers. Staff may prefer to fill out
the questionnaires/surveys at home or at staff meetings. Children may want to have a chance to eat
and play in the gym first. Parents and guardians usually do best if they have a quiet place to fill out
questionnaires/surveys when dropping off/picking up or at home. You can provide
questionnaires/surveys in hardcopy and electronically through email. You might post a PDF of the
questionnaire/survey on your web site for respondents to print. Also consider using a web‐based
survey tool, like Survey Monkey. Note that programs that have mailed family questionnaires home or
sent them by “kid mail” found they had low return rates.
Provide support for family or staff members with limited English proficiency.
For people who speak other languages, translate the questionnaires/surveys or have a translator read
the questions and then write the answers in English so the team can include responses in the
summary. For some people, the telephone interview may be a good option.
Ensure Confidentiality of Responses.
Everyone who fills out a questionnaire wants to feel certain that their responses will be confidential
and carefully reported to the ASQ Team.
Aim for a high return.
The more questionnaires/surveys returned, the more accurate your information will be. Offer
incentives for questionnaire/survey returns. Keep a record of who has filled out
questionnaires/surveys so reminders can be given to those who still need to complete them.
‐ 19 ‐Step 3: Gathering the Information
Ideas for Incentives
To encourage participants to complete questionnaires/surveys, try:
• Offering free pizza at pick up time for youth and parents. Stay, enjoy a bite to eat, and complete the survey.
• Giving families a coupon for a small credit of their fee.
• Having the children make or bake something that can be given to their family member who completes the survey.
• Asking local businesses to donate stickers or pens and other small gifts for youth. Try tickets to the movies or bowling for families.
COMPLETE QUESTIONNAIRES/SURVEYS
‐ 20 ‐Step 3: Gathering the Information
ASQ Resources available on NIOST Portal• Checklist for Program Observation• Questionnaires for Staff, Family, Child and Youth, Host, and Director• Checklists for the Family, Child/Youth, Staff, and Host Questionnaires • Sample Letters for Family, Staff, and Host Questionnaires • Family Focus Groups • Tips for Running Successful Focus Groups • Tips for Talking with Family Members
Questions & Answers about the Questionnaires/Surveys
If our families do not come in for pick‐up, when can they complete the surveys?
Interview family members by telephone. You will often get fuller responses when you talk with family
members. They may only spend five to 10 minutes filling out the survey, but they will probably speak
with you on the telephone for 10 to 20 minutes. Telephone calls to parents can also be a great learning
experience for the caller.
Provide a brief training for ASQ Team members. Role‐play responses to various issues which might
come up in the conversation, for example, a parent talking too long; a complaining parent. (See the
ASQ Portal for Tips for Talking with Family Members.)
Conduct a focus group meeting with family members.Many parents enjoy the opportunity to meet
with each other to discuss the program. Some family members will feel more comfortable answering
questions in a group than answering by themselves on paper. Some programs use the focus group to
get the surveys completed by every parent. Other programs select just a sample to attend the focus
group.
Here are some tips on conducting a focus group:
• Select a date for the focus group.
• Limit the number of participants to eight to 10 people. This size makes it possible for everyone to
speak.
• Hand out surveys to each person attending if they have not already filled one out.
• Analyze the responses by the date agreed in the work plan.
• See the ASQ Portal for the following resources: Family Focus Groups, Tips for Running Successful
Focus Groups, and Keeping Meetings on Track.
Families enjoy special events. The youth in one program created a play called "Future Program
Improvements." Children expressed through a play their hopes for new and improved snacks, fabulous
field trips, and a new play area. Your program could do the same. Family members could come to
watch this or some other type of performance. Potluck dinners or family pizza nights could include 20
minutes for children and families to complete surveys.
SUMMARIZE THE INFORMATION
You have collected a lot of information from multiple tools (observation tool and questionnaire/surveys)
and sources (families, children and youth, staff, the host, and other stakeholders). Now is the time to
summarize the feedback. This is the first step in examining the big picture without overlooking important
details and feelings.
Summarize your data by following the directions provided by your assessment tool. Here are some
additional tips. It is recommended that at a minimum, an electronic database such as Microsoft Excel be
used for organizing, exploring and summarizing data.
To summarize the information, we suggest that you organize data by program standards or quality
elements. For example, you may look at “Staff relate to all children and youth in positive ways” or
“Children and youth can choose from a wide variety of activities.” For each standard or element, pull
together the data collected from each source (observation tool, child & youth survey, family survey,
director survey, staff survey, and host survey). This will give you a collective impression about how things
are going.
1. Summarize the Observations
Create a summary sheet, or use one provided by your assessment tool.
This sheet should list each person who observed the program, the group they observed, and the
date of the observation. List the ratings for each observer. You may want to use a separate tally
sheet for each separate group observed so you can clearly see the ratings for each group.
Determine an average rating for each standard observed.
When there is substantial disagreement among observers, you can still do a simple average, but
CIRCLE that standard so you can note the disagreement in the self‐assessment summary. You can
note the average and the range. This will alert the ASQ Team to discuss their disagreement when
they are reviewing the summary.
You may also want to circle any standards that receive low ratings and high ratings so they can
also be discussed during Step 4.
Categorize the comments.
Categorize the frequency of comments and summarize (see next page).
‐ 21 ‐Step 3: Gathering the Information
Consider working with a partner or coach to analyze responses
It can be useful to work with a coach who can assist with setting up the database. It is also often easier to work in a team especially when you have more than 25 questionnaires/surveys.
SUMMARIZE THE INFORMATION
2. Summarize the Questionnaires/Surveys
Create a summary sheet or use one provided by your assessment tool.
This sheet should include individual responses from each person surveyed, the number of
questionnaires/surveys completed, total number of possible respondents in the programs, and
the date tallied.
Determine an average rating for each questionnaire/survey item or categorize individual
responses.
Summarize the comments for open‐ended questions.
Summarize comments and write number of times they were mentioned. You do not need to
write out each whole comment. Use short phrases and capture the general flavor. You will not be
able to write down every comment, but be sure to summarize them when:
- the comment explains why ratings are particularly high
- a number of people have a similar comment
- the comment explains why ratings might have been particularly low
For example:
“Five parents want more communication about their child.”
“One parent suggests a sliding fee scale.”
“Three staff members want training on behavior management.”
It is common for there to be disagreements in the comments section. For example, some observers
may feel there are enough materials for the program while others don’t. Bring the information to
the ASQ Team when you meet again in Step 4. Remember that involving many people in the process
means dealing with different opinions. The team will discuss these differences and work toward
general agreement.
Important: Questionnaires and surveys should be anonymous. Do not use any names in your
summary. Delete names or change them to pronouns in the quotes you use.
Address negative information.
You are bound to find some criticism of the program. Criticism not directed at specific individuals
can be discussed by the ASQ Team. The team will need to decide if it is an isolated complaint or
shared by a number of people. If many people mention the issue, you may want to have an
immediate response.
If the negative comments are about specific staff members, it is best to share them with the
director. The issue can then be handled through ongoing supervision. If the issue is about the
director, the team should let the director know and, if necessary, pass the information on to the
Board of Directors or the Advisory Board.
‐ 22 ‐Step 3: Gathering the Information
SUMMARIZE THE INFORMATION
3. Create a Self‐Assessment Summary
Merge all the information to create a rich portrait of your program. You have collected data from
multiple sources informing how things are going at your program. By pulling together all the
information, you will see the "big picture" without losing sight of details.
Each summary should include:
• The average ratings from the program observations
• Comments from anything the observers want to bring to the group’s attention
• The tallied responses from all questionnaires/surveys
• Summarized comments from the open‐ended questions in the questionnaires/surveys
Summarize the most frequently cited comments from the program observation process and
questionnaires/surveys. Note questionnaire/survey items and program observations that received
very low ratings. These are areas that will most likely have priority in the Action Plan. In addition,
include items that have very high ratings and share this information.
‐ 23 ‐Step 3: Gathering the Information
ASQ Resources available on NIOST Portal• Tally Sheets for Staff, Family, Child/Youth,
and Host Questionnaires/Surveys• Directions for Summarizing Comments from Staff,
Family, Child/Youth, and Host Questionnaires/Surveys
The tools are a rich source of information. People experience the program very differently which will be reflected in the data but you will begin to see patterns, frequencies, and themes. This will guide your work in developing an action plan.
Step 1 Step 2 Step 3 Step 4 Step 5
4
Make sense of the information
• Identify areas of strength • Identify areas that need to improve• Identify priorities
Create an action plan
Share informationwith the program
community
STEP Creating the Action Plan
Self‐Assessment Summary
STEP 4 Creating the Action Plan
Now is the time to put together all the pieces of information to create a portrait of your
program. This will give you a clear idea of the program's strengths and areas that need
improvement. You will have an opportunity to share your work with the community
again and hear their ideas about how to approach your priority areas. Most important,
you will develop an Action Plan that will guide your work for the months to come.
OVERVIEW
• Review the information from the observations and questionnaires/surveys.
• Identify areas of strength and areas that need to be improved.
• Develop priorities for program improvement.
• Share this information with the program community and solicit feedback.
• Create an Action Plan.
GOALS
Step 4: Creating the Action Plan ‐ 25 ‐
MAKE SENSE OF THE INFORMATION
By the time you meet again as a team, the hardest part of organizing the information is complete. Now is
the time to gather the team together to make sense of the information (see page 23 for sample meeting
agenda) and reflect on your findings.
When you look at the information, some information will jump out at you. It is natural to start thinking of
strategies as soon as you identify a problem. Some members of your team will be eager to roll up their
sleeves and get to work. It will be very tempting to let them. However, if you take action at this stage,
you will be doing it without broad agreement from the community. What you think is a high priority now
may turn out to be less important later when you see the whole picture. Finally, you may be using
precious resources that you will later realize should have been used in a different way.
The assessment tools you are using may already provide guidance on how to make sense of the data
collected. Here are some additional tips:
Start by sorting out clear areas of strength.
Depending on the instruments you are using, these may include observation items that receive high
ratings, questionnaire/survey items with positive responses, and specific notes and comments that
are also positive.
Then sort out areas that clearly need improvement.
An area that needs improvement may include observation items with low ratings and
questionnaire/survey responses at the low end of the scale.
Note areas where more information is needed.
Sometimes the information is in conflict. The program observation may indicate that there are a
variety of activities, for example, but over one‐third of the children say they do not have enough
activity choices. Or, perhaps a comment was made that raises questions for the team. These items
will need further discussion and or further information gathered.
Set priorities for working on improvements.
Once you have identified areas of strength and areas that needs improvement, you are ready to
decide whether an improvement item will receive high priority attention or low priority attention.
“ Creating the action plan was a wonderful time of creativity within our program. Everyone shared their best ideas and immediately began bringing in resources to make things happen.” — ASQ Team Member
‐ 26 ‐Step 4: Creating the Action Plan
MAKE SENSE OF THE INFORMATION
‐ 27 ‐
ASQ Resources available on NIOST Portal• Sample Meeting Agendas‐Make Sense of the Information Meetings• Leader’s Guide to Make Sense of the Information Meetings
Step 4: Creating the Action Plan
Working together to make sense of the information will help set the stage for creating the action plan.
Note: This sample agenda below combines two meetings into one to show the steps that will take place
at this stage. Most programs will find that these steps take time and therefore are best conducted over
two meetings. On our portal, there is an agenda and leaders guide for these steps as two meetings.
Choose how you would like to present the information.
Sample Agenda: Make Sense of the Information
Goals• Create a list of strengths and areas for improvement• Prioritize areas for improvement• Build the foundation for an action plan
5 minutes WelcomeThank everyone for their efforts so far and congratulate yourselves on having come this far.
10 minutes Warm‐up ExerciseConduct an activity to help team members share their experience of working on ASQ so far.
90 minutes Strengths and areas for improvement• Small group work • Reporting backAsk small groups to use the summary sheets created in Step 3 to develop a list of strengths and areas for improvement. Have each group share their lists. Then review the results of the entire team. If you discovered an issue that needed further discussion, explore that issue before you go further into the agenda.
15 minutes Checking in on group progressConduct an exercise to check that the team is working well.
90 minutes Prioritize areas for improvementCome to an agreement on the level of priority for each area for improvement.
5 minutes Next stepsPlan who will present your results at the program‐wide meeting.
SHARE THE INFORMATION
During Step 1, you held a program‐wide meeting at which the ASQ process was discussed and approved.
Now is your opportunity to share your findings with the community and be sure the community is
comfortable with them. Remember, improvements will be easier to make if everyone is on board with
you.
The best way to share the information is to hold a second program‐wide meeting.
When people work together and build on each other’s ideas, often the soundest strategies will
emerge. Programs that chose this route reported that the meeting was the kick‐off to a huge
volunteer effort for putting the chosen strategies into place. See page 25 for a sample meeting
agenda.
If you think a meeting will have low attendance, find other ways to involve the community.
Create a brief written report and share it with the Board of Directors, at a staff meeting, and in the
program newsletter. You might also call families directly, create a video presentation that families can
watch at home, or tap expertise outside of the program for new ideas. Go over the list of areas that
are strengths and those that need to be improved. Ask for feedback and ideas for solutions.
‐ 28 ‐Step 4: Creating the Action Plan
SHARE THE INFORMATION
‐ 29 ‐
ASQ Resources available on NIOST Portal• Sample Meeting Announcement • Sample Meeting Agenda Program Community Meeting• Leader’s Guide to the Program Community Meeting
Step 4: Creating the Action Plan
By sharing findings with the program community, you can establish support and invite the community to
identify strategies to address areas for improvement. Note: the purpose of this meeting is to pre
Sample Agenda: Program Community Meeting
Goals• To present the findings of the ASQ Team which identify program strengths and areas for improvement.
• To celebrate program strengths. • To brainstorm specific strategies which can help address the areas for improvement.
10 minutes Welcome and Introduction• Goals of the meeting• Review of the agenda• Setting of the toneWelcome everyone, introduce the ASQ Team members to the group, and review the agenda.
30 minutes The work of the ASQ Team• Presentation• DiscussionBriefly summarize the steps in the process that the ASQ Team completed and present the findings. Discuss the areas for improvement and prioritize the ones most important to children and families.
40 minutes Generating strategies for improvement• Small‐group brainstorming• Reporting back• Whole‐group feedbackAssign a priority to each small group and have them brainstorm strategies to address the priority. Have small groups share their ideas and gather additional ideas from the whole group.
10 minutes Introduce the ASQ Team Action PlanExplain that the ASQ Team will be using ideas generated at the meeting to develop an Action Plan. Let people know the plan will be made available to all.
5 minutes Next stepsAsk for a show of hands from people who would be willing to work on specific improvements.
As you share the information, be sure to highlight program strengths and acknowledge people who work hard to keep the program running.
CREATE THE ACTION PLAN
You have learned a great deal about your program. You are about to create a plan which will improve
the program for the children and families who rely on it. Take a moment, as a team, to feel good about
the work you have done. Here is a 10‐step guide for developing the Action Plan. This guide will steer you
through the brainstorming process, evaluating strategies most likely to succeed, and allocating
resources. Remember that small but constant improvements that involve everyone will have lasting
effects on your program. Start by making a few small changes and build off these successes as your team
continues to work on program improvement.
ASQ Resources available on NIOST Portal• Action Plan worksheet • Problem Solving Model
Remember that participation is the key to successful change. Assume that people want to help. Spread the work around. If everyone is involved in making the improvements, there will be less resistance to change. There will also be a greater sense of community spirit.
Constant, gradual change is change that will last. Start with the goals you can realistically handle.
“ There were moments when I felt our plan was too ambitious, but as everyone pitched in, I knew we would reach our goals. And it wss a relief to have everyone involved, and to feel it wasn’t my job alone to tackle these issues.” — ASQ Team Leader
Step 4: Creating the Action Plan ‐ 30 ‐
CREATE THE ACTION PLAN
Guidelines for Developing an Action Plan
1. Review your list of areas of strength and areas that need improvement. Then review the
priorities and strategies that have been generated from the ASQ Team and the program
community.
2. Combine the issues that can be solved by addressing one goal.
As you look at the information you have, you will see that several ideas and areas of concern
really fit into one category. Be careful not to make a goal too broad.
3. Write clear goals that specifically name the actions you will take.
This will help when you have to come up with strategies for each goal.
4. Generate possible strategies.
Start by reviewing the strategies generated by the ASQ Team and the community. Then add
other possible strategies.
5. Choose the strategies most likely to succeed.
For each strategy, consider:
• How much will it cost?
• Who would undertake the effort (families, staff, children and youth, other stakeholders)?
• How much work is involved?
• What are the advantages and disadvantages?
• Can it be done? Is it realistic?
6. Assign Tasks. Assign tasks to the people who know the most or care the most about a particular goal.
7. Decide what resources will be needed to meet the goal. Decide how much the plan will cost and how many people will have to be involved. You may need to accomplish your goals in stages. Work on one to three of the top priority goals in the first year. Save the low‐priority goals for the second year or later.
8. Create a timetable. Be realistic about when you can complete the task. Check to see how the timetable might be affected by other goals and tasks.
9. Review the completed plan.
Look over your work and ask these questions:
• Is it realistic?
• Do we have enough resources to complete it?
• Have we picked the best strategies?
• Is it clear who is working on each task?
• Will people in the program help to put the plan into action?
10. Send an outline of the plan to everyone in the program.
This Action Plan is a map for their future and you will need their help to implement it.
‐ 31 ‐Step 4: Creating the Action Plan
Step 1 Step 2 Step 3 Step 4 Step 5
STEP Taking Action5
The ASQ Team and Program Community
Checking in on Tasks
Checking in on Progress
SolvingProblems
CelebratingSuccess
STEP 5 Taking Action
During Step 5 you will finally begin to see some of the results of your work. You will feel
the excitement that comes with making changes to improve your program. However,
things don't always go as you may expect. You will be listening to all members of the
community and watching for signals that the plan may need to be adjusted. Remember,
when you are guiding change you have two jobs that are equally important. The first is
to keep track of the progress toward the goals you have established. The second is to
keep track of the process and the feelings people have about making changes.
OVERVIEW
• Coordinate the tasks set forth in the Action Plan.
• Check on the process of making program improvements—and celebrate the successes.
• Solve problems in areas which are not progressing as well as expected.
GOALS
Step 5: Taking Action ‐ 33 ‐
COORDINATE THE TASKS OF THE ACTION PLAN
Program improvement means getting involved in the process of change. Some changes will require
simple, decisive action. Other changes are complex and require more thought and effort. To successfully
accomplish your goals, your team must have effective leadership and work well together. It is important
to check on progress every four to six weeks to get the most from your efforts. Here are some tips to
keep you moving in the right direction to ensure all tasks laid out in the Action Plan are accomplished.
Give each person involved in the Action Plan a full copy of the plan.
It is important that each person involved in implementing the Action Plan sees how his/her task fits
into the larger picture.
Make it clear how each person reports back on progress or problems.
Check on progress regularly so you can catch problems before time is lost or people become
discouraged.
Continue to build "buy‐in" and commitment to the Action Plan.
It is essential to let people in the program community know about your progress and problems.
Decide how you will update the full staff, families, the host, and key stakeholders.
Check back to make sure goals are met.
Spot checks with staff, families, children, and youth will help you learn if the strategy developed by
the ASQ team is working the way you planned. When all goals in the Action Plan have been met, the
ASQ Team can conduct a program observation to check on overall progress.
‐ 34 ‐
Questions & Answers about Tasks
What if the ASQ Team is not able to carry out the Action Plan?
Many programs hope the ASQ Team will be able to continue the self‐assessment work until the Action
Plan is complete, but sometimes this is not possible. There may have been a lot of turnover on the
team, or team members might no longer have the necessary time. Programs that have experienced
this have shifted the work over to others. The staff team or a committee from the Board of Directors
might take over the work. No matter who takes on the responsibility, it is important that
representatives from each group of stakeholders still have a chance to comment on the progress of
the Action Plan.
What do we do if a change leads to unanticipated results?
Change can have a ripple effect—one change often leads to another. Some of these changes may be
unanticipated, but positive. Other changes may not be so fortunate. When you check in to see how a
change is working, be sure to ask if it has had any unanticipated results. Take these results into
account as you fine‐tune your Action Plan.
Step 5: Taking Action
CHECK ON THE PROCESS OF PROGRAM IMPROVEMENT
Sometimes we welcome change, but more often, change can feel unsettling. Some people may want to
race ahead with changes. Other people may feel overwhelmed or resist changes. To be successful in
program improvement, it is important that you pay as much attention to the process of change as to
tasks themselves. Here are some tips on ways to check on how people are feeling.
Take time to celebrate your efforts and successes.
Find ways to let people know about progress being made and publicly recognize the people who are
making things happen.
Check in frequently with the people who are doing the work.
Even those who are 100% behind the Action Plan may feel discouraged or overwhelmed at times. Talk
individually with people to see how things are going. Include time during ASQ Team meetings to
check in formally.
Check in frequently with the people affected by the changes.
Staff, families, children, and youth often give informal feedback about changes being made. Find ways
to check in with larger groups so you avoid assuming that one small group speaks for everyone.
Take the time you need to make successful, lasting changes.
Focus on a few goals at a time. If you take on too much, people may become overwhelmed or the
results might not be as successful as possible.
Note when things are not going well.
As with any developmental process, growth will be uneven. Programs often report it is easy to make
concrete changes. You might sail through improvements in materials, equipment, and indoor space. It
is more difficult to make changes that affect long‐standing habits or behavior. Problems can appear in
how people react to the goals and strategies of the Action Plan, when work is not getting done, or
when there is low energy and much resistance.
‐ 35 ‐Step 5: Taking Action
SOLVE PROBLEMS
It is human nature to resist change. Think of the last time you wanted to change something in your life.
Perhaps you decided to sign up for an exercise class. Even when you feel determined to succeed, things
can get in the way. You may find that money, time, unexpected crises, or your psychological readiness to
change throw roadblocks in your path. Making changes in a program requires the cooperation of many
people. Expect that there will be some problems. Success will depend on your ability to solve some of
the typical problems that arise.
An unexpected crisis occurs.
When a crisis will require the energy of only a small group of people, you can assign the problem to a
small committee. Seeing new successes in the midst of a crisis is often especially important for
morale. For situations where funding is unexpectedly cut, space is terminated, a director leaves, or a
staff member or family experiences a crisis, it might be wise to stop the action on ASQ. The staff
leadership, board and ASQ Team should decide jointly what to do. It may help to ask yourselves how
much effort it will take to resolve the crisis:
• Does the program’s survival depend on resolving the crisis?
• Is the crisis affecting the morale of staff, children and youth, or families?
• Will the resolution take the full energy and attention of the leadership core?
• Is it impossible to offer a quality experience to children and families until the crisis is solved?
If you answer “yes” to any of these questions, it might be best to temporarily set aside the Action Plan.
Maintaining quality becomes the most important issue. A temporary slow down in the process will not
alter your long range goals.
The NIOST portal has several resources to help guide the director and the team through the change
process and to help find ways to work through issues that may arise.
Congratulations! You have embarked on the journey of program improvement, to make your program
the best it can be for the youth and families you serve as well as those who work at the program and are
affiliated with your organization.
‐ 36 ‐Step 5: Taking Action
ASQ Resources available on NIOST Portal• Questions and Answers about How to Solve Typical Problems • How to Keep Turnover from Derailing the ASQ Process • The Role of the ASQ Team Leader During Stage 5• Progress Report Form
We hope that you find the ASQ process useful. We welcome your comments, suggestions, and feedback. Please contact us at [email protected] and let us know how the ASQ process is being used at your program and what new ideas you have to share.
ACKNOWLEDGEMENTS
NIOST Director Ellen Gannett and all those associated with NIOST who assisted with this project
including Karen Lachance, Jenny Lisle, Rebecca Krumholz, Kristel Dupaya, Melissa Magner, Julie
Dennehy, Wendy Surr, Georgia Hall, Genevieve Solomon , Donna Tambascio, Rebecca Mongeon, Sue
Sours, Gwen Holbrow and Kathy Schleyer, project manager.
Original Authors
Susan O'Connor, Ellen Gannett, Cathy Heenan and Kathryn A. Wheeler.
NAA formerly NSACA
ASQ, the program improvement process, was gifted by NIOST to the National School Age Care
Association, now NAA, almost twenty years ago. At that time, ASQ was the program improvement
process that NSACA required for afterschool programs seeking national accreditation. Embedded
within the ASQ process were the NAA National Quality Standards that had also been developed in
collaboration with NIOST and with support from other leaders and organizations in the afterschool
field. Those standards are reflected in the ASQ Observation Tool and the surveys and questionnaires
that made up the original ASQ. Two years ago, ASQ came back to NIOST. We are grateful to NAA for
returning ASQ to NIOST.
Proponents of the original ASQ that encouraged and supported the development of this ASQ Guide
Susan O’Connor, Leslie Roesler, Susan Brasel, Sara Espinosa, Judy Nee, Lynn Siebert, Betsy Saatman,
Carmen Gatti, Thomasine Washington, Pat Hess, Virginia Brown, and others who helped this project.
And our deepest gratitude to Liz Joye and Juanita Svendsen for their
commitment to ASQ over the years as ASQ instructors, coaches and proponents.
1Masaaki Imai, KAIZEN: The Key To Japan's Competitive Success. New York: Random House, 1986, pp. 2‐7.
‐ 37 ‐
Why ASQ?
ASQ was originally developed as the afterschool program improvement process to guide programs seeking national accreditation. This process was tied to the ASQ Observation Tool and NAA’s national afterschool quality standards. Today, a wide range of afterschool programs use a variety of observation and survey tools.
What hasn’t changed is the need for the process to enact the benefits that comes from using measurement tools. This is why ASQ is still vital. Today’s ASQ, is available on the internet via the NIOST website portal. We now offer the ingredients of the process including all the agendas, forms and other support materials electronically so programs and sites can download them and adapt them to their own needs. And it works with any assessment tools or surveys that a program chooses to use. Join those who are actively engaged in continuous program improvement.