Independent

External

Evaluation

April, 2012

Transformational

Development Results (TDR)

International

AccountabilityLearning &

Improvement

Knowledge development:

Oversight and compliance

Relevance

Efficiency

EffectivenessImpact

Sustainability

Purposes of evaluationsGlobal criteria for programme evaluations

Background

AIMS -SA

Academic

Postgraduate DiplomaCollaborative Honours

Degree

Research AIMSSEC

Mathematical Thinking, Problem Sovling and

Technology in Teaching and Learning

Advanced Certificate in Education

AIMS has produced graduates, researchers and improved the skills of Math teachers…

SO WHAT??? What has changed?

What is the correlation between the AIMS experience and what the alumni are doing today or

what they are planning to do?

Is the AIMS-SA model a relevant and critipoint on the change continuum towards cal addressing the

development challenges of Africa?

Key Evaluation Questions

Evaluation Methodology

108 documents reviewed

86 Semi-structured Interviews

04 focus group

discussion04 focus group discussion

204 Alumni interviewed

Evaluation Methodology

Evaluation Framework

Impact of any education program is long term, complex and best achieved through a longitudinal study of graduates

High response rate from recent graduates, where ability to make a difference may still be limited while evidence of impact may lie with the earlier graduates

Stratified FGD with alumni as a result of primary responses provided was not undertaken

All data sets utilized highly subjective in nature and thus triangulation process required significant organization and time

Data analysis

Limitations of the Evaluation

RELEVANCE EFFICIENCY

IMPACT SUSTAINABILITY

KEY FINDINGS

AIMS-SA contributes in building a strong base of African researchers who will address Africa’s development challenges

89% of Alumni reported AIMS-SA increased their awareness on the role mathematical sciences can play in addressing Africa’s challenges

Recognition and value of AIMS-SA in helping to meet the mandates of NEPAD and UNESCO .

01020304050607080

Pe

rce

nta

ge

Before arriving When you graduated

FIGURE 13. ROLE AND PURPOSE OF AIMS AS

IDENTIFIED BY THE ALUMNI, BEFORE

ARRIVING, IMMEDIATELY AFTER GRADUATION

AND CURRENTLY

RELEVANCE

AIMS-SA, through its partnership with Stellenbosch University, demonstrates efficiency through:

Shared services and resources with local and international partners

Use of volunteer international lecturers contributed significantly towards efficiency

PGD, the Research Centre and the AIMSSEC program, despite being a part of AIMS-SA, function independently in areas where increases in efficiencies could be achieved e.g. resource mobilization

EFFICIENCY

EFFICIENCY – COST/PGD STUDENT

AIMS-SA facilitated the creation of the South African Centre for Epidemiological Modelling (SACEMA)

Significant contribution to Alumni’s current careers or educational paths

Active participation in several national, continental and global networks

AIMS-SA contributes to instilling an increased understanding of Africa’s challenges among the alumni

Research outputs (publications) and the growth of the Research Centre

EFFECTIVNESS

Government of South Africa +++

80% of respondents in semi structured interviews are concerned about sustainability at the institutional level

Strong evidence of sustainability for alumni at the individual level for knowledge and skills acquired at AIMS-SA

Strong evidence of the contribution towards impact at the individual level for AIMS-SA Alumni.

Alumni survey reported strong contribution, and in some cases attribution of current success, of Alumni to AIMS-SA.

With the absence of a counterfactual scenario or control group as part of this evaluation, the results from the analysis of the findings have been classified as contributions towards impact.

Evidence continues to be built that will further demonstrate AIMS-SA’s contribution towards impact within the Mathematical Sciences sector and in addressing Africa’s development challenges.

IMPACTSUSTAINABILITY

Evaluating the effectiveness of an AIMS-SA Education

AIMS-SA

University

In Sweden

Private

Sector

University

of Rochester

Church

Civil Society

Developing

Country

Gov’ts

Family

support

Elevating the value and need for mathematical sciences and positioning mathematical sciences

AIMS-SA is achieving its goal of building the capacity of Africans in mathematical sciences and doing so in manner that is unique and empowering for the young Africans who have gone through AIMS-SA.

The relevance of AIMS-SA is clear and strong; it is a highly efficient institution and has produced outputs within the last seven years that speak strongly to its effectiveness.

There remains, however, important challenges AIMS-SA needs to address related to:

How it constitutes and reports on its results and contribution towards impact

Sustaining the AIMS-SA model in the absence of external (International) support in the long term.

Recognizing and working with in addressing the brain exchange vs brain drain challenge.

CONCLUSIONS

The design of the AIMS-SA model

Reporting of results and achievement of AIMS-SA

The evaluation

Lessons learnedKey Recommendations

Integration

Theory of Change

Academic-research

centre and outreach

Future career paths

Engage alumni

beyond current effort

Strategic and operational

plan

The End

Thank YOU!!!!

“What we see depends mainly on what we look for ”

−Sir John Lubbock

“The important thing is not to stop questioning.”

−Albert Einstein