Welcome to the Allegany Franciscan Ministries 2014 Major Grant Evaluation & Success Measures Webinar...
-
Upload
valentine-pierce -
Category
Documents
-
view
219 -
download
1
Transcript of Welcome to the Allegany Franciscan Ministries 2014 Major Grant Evaluation & Success Measures Webinar...
Welcome to the Allegany Franciscan Ministries2014 Major Grant Evaluation & Success Measures Webinar
We will begin PROMPTLY at 11:00 a.m.
~~ Please MUTE your phone ~~
Please do not use a headset as they cause feedback that is disruptive to others on the call.
April 29th, 2014
ReflectionBlessed are they who give without expecting even thanks in return, for they shall be
abundantly rewarded.Blessed are they who translate every good thing they know into action, for ever higher
truths shall be revealed unto them.Blessed are they who do God's will without asking to see results, for great shall be their
recompense.Blessed are they who love and trust their fellow beings, for they shall reach the good in
people and receive a loving response.Blessed are they who after dedicating their lives and thereby receiving a blessing, have
the courage and faith to surmount the difficulties of the path ahead, for they shall receive a second blessing.
Adapted fromfrom the book "Peace Pilgrim"
Agenda Welcome and Reflection The Application Evaluation – our philosophy Success Measures & Outcomes
Definitions Examples Resources
Q & APresentersJessica Gonzalez, Program Officer Erin Baird, Director, Grants and TechnologyShelley Robertson, Evaluation Consultant
Our goals for this webinar To help you and your staff team…
Understand our evaluation philosophy, process & expectations.
Better understand the difference between program outcomes & indicators; and the difference between those outcomes that are measureable & meaningful and those that are not.
Begin to conceptualize program success measures before your initial meetings with the Program Officer.
Have the tools and knowledge to submit a successful application.
2014 Major Grants
In each region, 4 to 6 organizations are likely to be funded.
Complete Major Grant Applications are due by May 29, 2014 at 12 pm (noon).
Site Visits will occur in late July and early August.Board will finalize decisions in September. Grant agreements will then be developed and signed;
initial payments will be made between. November and January, depending on the project.
Major Grant Timeline
Hillsborough, Pinellas, Palm Beach, Martin & St. Lucie
CountiesNotes
LOI Due March 20, 2014 Online application must be submitted by 12 pm (noon).
Notification of our decision; decline or invite to submit full application By April 25 We will e-mail and send a letter to you.
Major Grant Full Application Technical Assistance Webinar April 30 For those selected to submit a full application; please hold date
(date subject to change).
Deadline to submit full application May 29, 2014 Online application must be submitted by 12 pm (noon).
Site Visits & Meetings July & August
Decisions Announced late September We will work with applicants between September & December to finalize budgets & success measures.
Grant Period Begins November 1, 2014 or later
2014 Allegany Franciscan Ministries Strategies Community education and outreach designed to
increase knowledge and support behavioral changes.
Healthcare services and basic needs. Increased capacity and coordination of the
healthcare delivery system. Advocacy and community efforts that educate,
engage, raise awareness, or mobilize communities.
The Application – How to ApplyRead the entire 2014 Major Grant Application
Instructions document.Go to our web site, www.afmfl.org.Click on the “Current Partner” button.Log on to the online application system.Confirm or edit all contact information.Under “LOI” you will see “Application”.Click on “Edit Application”.
The Online Application – Some Basics Complete your responses in Word© and then
paste into the appropriate field. Pay attention to character limits. Document will
not save if you are over the character count. Don’t use acronyms or jargon. Save your work frequently. When uploading documents, click “save as draft”
to finalize each upload. Before submitting, “Print Packet” and read/have
someone else read.
The Application
The Application
Budget
Evaluation Philosophy
As a good steward, Allegany Franciscan Ministries is dedicated to assuring that our investments move us toward our mission. Our evaluation process is clear, simple, and focused on outcomes. The evaluation process is respectful of organizations with different size, sophistication, values, beliefs, and traditions. It balances qualitative and quantitative methods to inform and guide our decisions while improving services provided by our grant partners.
Getting to measures An organization should articulate how the
program leads to long-term change for people.
Success measures document progress being made towards that long-term change.
Allegany Franciscan MinistriesSUCCESS MEASURES FORM
Organization Name: Project Title: Year: From (month/year): To (month/year):
Outcomes Please state one outcome per row.
Indicators What is the specific statistic you will track to determine your success on this outcome? What data tool will you use to collect this data?
ActivitiesWhat are the key activities that will lead to each outcome?
Measurement planWhat is your measurement plan? For example, how often will you administer your data tool? How will you assess whether you were successful?
Participants increase knowledge about how to cook healthy meals.
85 percent of participants are able to identify four healthy cooking methods and up to 10 healthy food choices.
Provide 10 hours of training to 20 participants in 5 two-hour sessions.
We will administer a paper pre and post test to participants.
Coalition members will increase their advocacy activities
90 percent of members will take at least two advocacy actions during the six months after training.
Recruit and train a coalition of up to 20 advocates.
Members will report actions at monthly coalition meetings; staff will document efforts.
complete one page – and one page only - per year of grant
Definitions
Outcomes: improvement and/or change in knowledge, skills, attitudes, or behavior. Outcomes may also refer to desired changes in a community in response to community-level and system-wide activities.
Indicators: the specific items used to track a program’s success. Indicators describe observable, measureable characteristics or changes. An indicator should include the specific characteristic measured, what data tool is used, when it is measured, and who collects the data.
Definitions
Activities: what a program does with its resources to fulfill its mission.
Measurement plan: states any sampling strategies, how the data is entered, how you will analyze the characteristics to assess whether you were successful, and how data reported. For example, what counts as an increase?
Program outcomes
Program outcomes state the changesin clients’ knowledge, skill,
attitude, or behavior.
Example: Program outcomes Clients increase their knowledge about how to reduce risk
behaviors. An indicator could be that 80% of clients increase their
knowledge about what are risk behaviors, consequences of risk behaviors, and how to reduce risk behaviors as measured by comparing post test to pretest scores on the curriculum-embedded tool.
Clients increase their skills in preparing healthy food. Clients demonstrate improved confidence toward changing
behavior. Clients experience decreased health risk behaviors.
Organizational capacity outcomes
Organizational outcomes state changes in capacity areas such as revenue, fiscal
management, board development, operations, etc.
The organization should be able to explain – although probably not measure – how the change in capacity will impact those they serve, such as improved service quality or more clients served.
Example: Organizational capacity outcomes
The organization increases the capacity of its board of directors. An indicator would be that the organization meets current
needs on 10 benchmarks on the board development section of the Capacity Benchmarking tool (see resources).
The organization increases its fundraising capacity. An indicator could be that the organization achieves
benchmarks on a capacity tool or that the organization (a) develops at least two new revenue streams, (b) reduces expenses by 5 percent in comparing 2010 to 2009, and (c) that 90% of staff report that the new system saves them time.
Choosing outcomes: meaningful and measurable What we want to see:
Fewer youth exhibit at risk behaviors due to undiagnosed mental health problems.
What we don’t want to see:Fewer youth
abuse substances and show increased stability and life skill.
Choosing outcomes: meaningful and measurable What we want to
see:75% of recipients
increase their intake of healthy food daily.
What we don’t want to see: 75% of recipients
report eating more fruits, vegetables and protein after starting our food delivery program.
Choosing outcomes: meaningful and measurable What we want to see:
“Organization A” will provide primary medical services to individuals and families with no health insurance.
What we don’t want to see: Provide safety net
medical services for individual and families without health insurance or who cannot afford or are not eligible for conventional health care insurance.
25
Creating Outcome Statements:
• Step one: describe your client (i.e., age, gender, status)
• Examples: at-risk middle school youth, adults with mental illness, frail seniors, Hispanic teens, adults with disabilities, low-income parents
The Outcome Formula
26
• Step two: identify the issue or topic you are addressing.
• Examples: healthy lifestyle choices, employment, substance abuse, grades, sports, self-esteem, coping skills, health status.
Creating Outcome Statements:
The Outcome Formula
27
Client Verb Adjective
Type Example/topic
Will Increase Knowledge about
Decrease
Skill(s) in
Maintain Attitude about
Behaviors such as
Examples:
1. At -risk middle school youth will decrease behaviors such as participating in gangs, getting in fights, and using drugs. 2. Hispanic teens will increase skills in job seeking.
Creating Outcome Statements:
The Outcome Formula
Demonstrate
Outputs vs. Outcomes Outputs answer the
question “how much” .
Outputs include number of clients, service units, hours, and appointments.
Outcomes are client-focused.
Outcomes relate to knowledge, skills, attitudes, or behaviors.
29
Characteristics:Observable and measurable.Client-focused.Unambiguous.State a time frame.Match.Understandable.
Indicators
30
The Indicator FormulaTells what you will measure (see table below), how, when, and who is measuring.
For this type of outcome
Tell us…. Examples:
Knowledge
What three main things the clients learn
Skill What, specifically, they can demonstrate
Attitude
What they will or will not feel
Behavior
What they will or will not do
31
Indicator ExamplesTool:
• Pick one: survey, pretest and posttest, posttest only, interview, case records, observation form, or official statistics. Tell us the name of the tool and who created it.
• Examples: curriculum pretest and posttest, online survey created by USF, case records kept by the agency, report cards from the school, ADL checklist provided by the State Department of Elder Affairs, survey provided by DJJ.
Target: • Put the percentage of your clients that you think will achieve the
outcome. The target is NOT the client population. • Examples: 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90% .
32
The Indicator FormulaHow examples:
• how the tool is administered.
When examples: • at the end of the program, beginning, every 30
days, 90 days after completing the program, etc.
Who examples: • program staff, non-program staff, outside agency
personnel, other agencies.
33
Indicator Examples
Knowledge: [Clients] will learn three elements of sportsmanship as measured by an end of
the program posttest administered by staff. Skill:
[Clients] will demonstrate that they can prepare a resume, find an open job through the paper or online, and answer interview questions as measured by non-program staff observation during the job fair at the end of the program.
Attitude: [Clients] will state that they feel hopeful about a positive future as
demonstrated by exit interviews conducted by program staff prior to leaving the shelter.
Behavior: [Clients] will not be adjudicated for gang activity, fights, or drugs while they are
in the program as measured by DJJ statistics collected by the School Resource Officer.
Making It Measurable
Direction of change.
Area of change.Target population.Degree of change. Time frame.As measured byBaseline.
Example: To decrease the percentage of 18 to 20 year old college students reporting binge drinking from 13 percent to 12 percent by 6/30/2015 as measured by the CORE survey.
Hints Outcomes: State one outcome per row.
Indicators and measurement: What is the specific statistic will you track to determine your success on this outcome? What data tool will you use to collect this data?
Activities: What are the key activities that will lead to each outcome?
Measurement plan: What is your measurement plan? For example, how often will you administer your data tool? How will you assess whether you were successful?
Evaluation Resources
In addition, Dr. Shelley Robertson is available at no charge to you for up to two hours.
Dr. Robertson can assist you in identifying which of these resources might be most helpful for your specific project.
She can also help as you develop or refine the outcomes to be tracked and reported, and determine appropriate indicators and measurement tools.
Next steps
Log in to the grant administration system and begin the application.
Contact Jessica with application questions earlier rather than later.
Complete the Full Application by May 29, 2014, 12 pm (noon).
Communicate with your team and discuss possible outcomes with your team.
Contact InformationFor all questions and technical support: contact
Jessica Gonzalez at 727-507-9668 or [email protected]
For help with outcomes and success measures, first contact Jessica Gonzalez and then our evaluation consultant, Shelley Robertson, 239-633-3241, [email protected]
Questions?