2

Skills Development through Infrastructure for Educati on

Skills Development through Infrastructure for Education

FEBRUARY 2017

IMPACT EVALUATION REPORTIMPACT EVALUATION REPORTIMPACT EVALUATION REPORTIMPACT EVALUATION REPORTIMPACT EVALUATION REPORT

CONTENTSCONTENTSCONTENTS

ABBREVIATIONS, ACRONYMS & GLOSSARY ....................................................... i

ILLUSTRATIONS ............................................................................................. ii

Figures ............................................................................................................ ii

Tables ............................................................................................................. ii

EXECUTIVE SUMMARY .............................................................................. 1-2

1. OVERVIEW .............................................................................................. 3

1.1 Background ........................................................................................ 3

1.2 Purpose and Scope of Evaluation ........................................................... 3

1.3 Key Evaluation Questions .................................................................... 4

1.4 Approach ........................................................................................... 4

1.4.2 Data Collection & Analysis .......................................................... 5

1.4.3 Limitations ............................................................................... 5

1.4.1 Evaluation Criteria .................................................................... 4

CONTENTS

2. EVALUATION FINDINGS .................................................................... 7

2.1 Demographics .............................................................................. 7

2.2 Relevance of the Training to Participants ........................................ 8

2.3 Impact of Training to Participants .................................................. 9 -14

APPENDICES ................................................................................. 15

1. Verbatim Responses ................................................................... 15-16

2. Evaluation Questionnaire ............................................................ 16 - 18

1.4.4 Evaluation Logic Model ............................................................... 6

i

DAC Development Assistance Committ eeECD Early Childhood Development CentreOECD Organisati on for Economic Co-operati on and Development RCT Randomized Control Trial SETA Sector Educati on & Training Authority

Program: Refers to the interventi on that adds value, brings new informati on, teaches a skill, expands knowledge, or otherwise infl uences a situati on either directly or indirectly. A program consti tutes the products and services an organisati on apply to address a problem. Program examples may include: classes, training workshops, lobbying, public awareness, campaigns etc.

Outputs: Refers to the products and services produced by an organisati on. Examples may include graduates of a training program, meals served to the disabled and needy and beds fi lled in a homeless shelter are all outputs.

Outcome: An outcome is the direct, intended benefi cial eff ect on the stakeholders or interests an organisati on and programs exist to serve. For example, the outcome of a smoking cessati on program is the number or percent of those completi ng the program that actually stop smoking. The outcome of a job training program is the number of program graduates who get a desirable positi on. The outcome of a remedial reading program could be either an improvement in grades among those who completed the program or the number who report that, of their own voliti on, they read a certain number of books, newspapers, or magazines over the course of the summer school vacati on.

Impact Refers to the long-term or indirect eff ects of a program’s outcomes on the stakeholders (benefi ciaries) it serves.

Aggregate: A whole formed by combining several separate elements (e.g., if you have a scale with Strongly Agree, Agree, Neutral, Disagree, and Strongly Disagree, we will aggregate the Strongly Agree and Agree together to form a new variable representi ng any agreement on that specifi c scale).

Composite variables: Measurements based on multi ple data items. Multi ple data items from a scale are combined (computed) to give a single score (e.g., overall Confi dence in bricklaying knowledge and skills). Measures are combined looking at the median average score we measure the “middle” rather than the “average mean” score as we do not assume equi-distance (mean) between degrees of answers because our scales have not stati sti cally validated/proven reliability.

Frequencies: Tools used via SPSS or excel to show the total for each category or group of data (e.g., the number of responses for a certain variable).

Likert Scales: A scale used to represent people’s atti tudes on a topic, normally with 5 potenti al choices (e.g., rate where you fall on a statement between Strongly Agree, Agree, Neutral, Disagree, and Strongly Disagree).

Pre & Post Evaluati on: A Pre-Post Evaluati on Design method measures parti cipants before and aft er an interventi on (or training), where the same indicators are measured at both ti mes and then compared. The beginning evaluati on forms the ‘Baseline’ from which the parti cipants’ responses are compared to at the end of the interventi on, or the “Post-Evaluati on.”

Post-Test Evaluati on: A type of non-experimental evaluati on design conducted with one group (no comparison group) or two groups (with a comparison group) of parti cipants. In post-test, parti cipants receive an interventi on and are tested aft erwards.

Proxy Indicators: Indicators that are used in order to provide some way to measure the Strategic Indicators.

SPSS: Soft ware used for stati sti cal analysis, “Stati sti cal Package for the Social Science”.

Strategic Indicators: In an interventi on, one develops a programme design to measure exactly what that interventi on should seek to change. What is being sought to change are the Strategic Indicators.

ABBREVIATIONS, ACRONYMS & GLOSSARYCONTENTS

2.1 Demographics .............................................................................. 7

2.2 Relevance of the Training to Participants ........................................ 8

2.3 Impact of Training to Participants .................................................. 9 -14

1. Verbatim Responses ................................................................... 15-16

2. Evaluation Questionnaire ............................................................ 16 - 18

ii

Figure 1: Evaluati on logic 6Figure 2: Distributi on of respondent’s by race & gender 7Figure 3: Respondents’ age 7Figure 4: Respondents’ current occupati on 7Figure 5: Extent to which training met the needs of respondents 8Figure 6: Relevance of training to parti cipants’ needs 8Figure 7: Besides the money you earned, what was most rewarding about the project to you? 9Figure 8: To what extent did the training enhance your life skills? 9Figure 9: Post training engagements (knock-on eff ects) 10Figure 10: Looking back over the last two years, what positi ve changes in your life can say were made possible by the SETA training and constructi on experience that you acquired? 11Figure 11: Confi dence in ability to apply bricklaying skills 12Figure 12: Confi dence in ability to apply carpentry skills 12Figure 13: Confi dence in ability to apply plumbing skills 13

Table 1: Analysis of occupati on by gender 8Table 2: Parti cipants verbati m responses to questi ons regarding post training involvements 15-16

Evaluati on Specialist: Adam Andani (MA, MA)Ruhr University Bochum, Germany, University of the Western Cape, Cape Town South Africa E: dunkurugu@gmail.com T: 078 268 4761

ILLUSTRATIONS

FIGURES

TABLES

1

ILLUSTRATIONS EXECUTIVE SUMMARY

This report details the fi ndings of an evaluati on undertaken by an independent evaluator to assess the relevance, eff ecti veness and impact of AMAMA Social Enterprise’s SETA accredited building and constructi on training on 15 trainees, two years following completi on of the training. Specifi cally, the evaluati on was conducted to generate answers to the following key questi ons:

1) How relevant was the training to the needs of trainees?2) To what extent did the trainees apply the skills and knowledge acquired from the training?3) Did the training produce any benefi ts for the trainees?

The evaluati on methodology entailed a non-experimental (post-test) design incorporati ng the DAC evaluati on criteria (relevance, eff ecti veness and impact). The approach was largely qualitati ve, and included the design and administrati on of a semi-structured questi onnaire to elicit data from 15 trainees. Parti cipants were purposively selected to parti cipate in the study based on their availability and convenience. The questi onnaire was informed by the DAC evaluati on criteria, and contained close-and-open-ended questi ons aligned to the key evaluati on questi ons outlined above. A descripti ve stati sti cal approach was applied to analyse the raw data from the survey using excel to generate stati sti cal outputs (frequencies, proporti ons and graphs). The survey yielded an 80% (n=12) response rate upon completi on, which gives an adequate representati on of the sample, thereby ensuring reliability of the results. The main fi ndings of the evaluati on are summarised below.

Demographics

All (100%) the parti cipants resided in Gauteng, and were predominantly Black African. There were more parti cipants above 30 years (75%), than those under 30 (25%), and most were female (66.7%), whilst male consti tuted 33.3% of the cohort. With respect to current occupati on, 41.7% of the trainees were engaged in part-ti me work and 25% in full-ti me. Unemployed trainees were 33.3%. Proporti onally, female trainees were more likely to work part-ti me (25%) and full-ti me (16.7%) than males. They are also more likely to be unemployed (25%) than their male counterparts (8.3%).

Relevance of training to participant’s needs

The training was found to be “extremely relevant’ to the needs of all (100%) the trainees surveyed. Overall, the training met most of the expectati ons of a majority (66.7%) of trainees.

To more than half (58.3%) of the parti cipants, the skills and knowledge acquired from the training was considered as the most rewarding experience. In additi on, 25% of the trainees reported to have used the money earned from the constructi on of the ECD to support their families.

“I was able to support my family

but above all gained skills because

I knew nothing about construction,

so I am confi dent that even though the remuneration was not enough

but it really assisted”

Eff ectiveness & Impact of training on participants

The analysis revealed that two-thirds (66.7%) of the trainees have secured constructi on related jobs in the last two years because of their training with AMAMA. Most of them were involved in full-ti me work with constructi on companies, while others worked part-ti me. Some were involved with RSP housing constructi on aft er their training.

2

Whilst the rammed earth technology has not been usedanywhere else by trainees to build, half (50%) of the trainees have coached their peers on various aspects of constructi on and building such as plumbing, bricklaying, erecti ng a roof, glazing and how to set up a building.

Overall, parti cipants were relati vely confi dent in their ability to perform bricklaying related tasks. The analysis illustrates that, the majority (75%) of the parti cipants were “very confi dent” in their ability to ‘identi fy tools and their functi ons’, and ‘mix and cast foundati on concrete’.

Most parti cipants (83.3%) rated themselves as being “very confi dent” in their ability to ‘identi fy carpentry tools and their functi ons’. With respect to plumbing, many of the trainees (75%) said they were “very confi dent” in their ability to ‘identi fy plumbing tools and their functi ons’, and ‘excavate trenches for pipe laying’.

Above all, parti cipants have acquired the ability to do their own basic home maintenance and to eke out a living by earning income through piece jobs.

However, what is most striking from the fi ndings is the realizati on that, the training has shift ed the mindsets of trainees – by insti lling in them an imminent sense of possibility of a bett er future.

Although the training was generally eff ecti ve, it is recommended that specifi c aspects of plumbing such as: ‘testi ng below ground drainage systems (25%)’, ‘constructi ng a scepti c tank (18%)’, and ‘installing a sink (11%)’ should be given more att enti on during training as some parti cipants reported a lack of confi dence in their ability to carry out tasks in these areas.

“Jobs are scarce but I am

confi dent with knowledge”

“I have a certifi cate to prove my knowledge.”

“I used my certifi cate to acquire a job”

“Yes, though I am not doing

some community work but I

learned tolerance”

“It assisted a lot and I am

looking forward to getting

jobs in construction in order

to take the skills forward”

EXECUTIVE SUMMARY continued

3

EXECUTIVE SUMMARY continued OVERVIEW

AMAMA Social Enterprise is a Non-Profi t Organisati on (NPO) that seeks to contribute to the alleviati on of some of the developmental and educati onal challenges in South Africa through training unskilled and unemployed members in a community to build educati on infrastructure in disadvantaged areas using the indigenous rammed earth building constructi on method.

AMAMA strives to work together with others to fi nd ways to improve the prospects for a bett er skilled and bett er educated citi zenry in South Africa. This is important since the quality and extent of our individual and collecti ve futures in South Africa hinge greatly upon achieving this.

Through a community mobilizati on process, 15 members from the Sicelo community were recruited to parti cipate in the constructi on of the Sicelo Village Early Childhood Development (ECD) centre in Midvaal – Gauteng. These community members parti cipated in AMAMA’s SETA accredited building and constructi on training, and were subsequently directed to build the ECD centre, uti lizing the age-old rammed earth green building technology as part of their practi cal to qualify for their certi fi cates.

There was no formal schooling requirement other than the ability to read and write in English. There were no gender requirements; however, eff ort was made to ensure a higher representati on of females by recruiti ng at least more than 50% women primarily in the “youth” category. The trainees worked under supervision and were paid a market related daily wage during the building process.

1.1 Background

This report presents the fi ndings of an evaluati on commissioned by AMAMA Social Enterprise to ascertain the impact of its Sector Educati on & Training Authority (SETA) accredited “building and constructi on” training on 15 parti cipants. The assessment was conducted by an independent evaluator, two years following the training, to assess the eff ects of the training on parti cipants.

Essenti ally, the evaluati on sought to empirically demonstrate the relevance, eff ecti veness and impact of the training on parti cipants , by investi gati ng whether the training met the needs and expectati ons of parti cipants, what they have done with their skills, and if the training has produced any positi ve results such as employment, changes in life-skills and confi dence, amongst others.

The purpose of the evaluati on was to assess the impact of the training on 15 parti cipants who completed the training program two years ago. The ensuing fi ndings therefore att empt to shed light on some of the positi ve changes (eff ects) in the lives of the parti cipants, which can be att ributed to the infl uence of AMAMA’s training.

In terms of scope, the evaluati on was limited to the 15 parti cipants and AMAMA’s project staff who provided valuable insight to address the evaluati on questi ons. The users of the evaluati on results include the organisati on and its stakeholders (board, donors and partners).

1.2 Purpose and scope of evaluation

1 DAC evaluati on criteria. htt p://www.oecd.org/development/evaluati on/qualitystandards.pdf 2 Parti cipant is used synonymously with trainee in this evaluati on. 3 See htt p://www.amama.co.za/ for details about the organisati on and its services. 4 A post-test only design is a straightf orward evaluati on conducted with one group (no comparison group) or two groups

4

In order to fulfi l the purpose of the evaluati on outlined above, the following broad evaluati on questi ons were formulated to inform the assessment. These questi ons formed the choice of evaluati on design and the measurement tool used for data collecti on.

1) How relevant was the training to the needs of trainees? 2) To what extent did the trainees apply the skills and knowledge acquired from the training? 3) Did the training produce any benefi ts for the trainees?

OVERVIEW

1.3 Key evaluation questions

A non-experimental evaluati on design, specifi cally a post-test design was employed to address the evaluati on questi ons. The process involved the design and administrati on of a semi-structured survey questi onnaire to collect data from 15 parti cipants.

The questi onnaire was informed by the above key evaluati on questi ons, and was aligned to the Development Assistance Committ ees’ (DAC) (of the OECD) evaluati on criteria described below. The key evaluati on questi ons were further developed into sub-questi ons which were used to measure specifi c att ributes (proxy indicators) with regards to relevance, eff ecti veness and impact (short-medium-long term eff ects) the training might have produced for the parti cipants. The various elements of the DAC evaluati on criteria are described below.

1.4 Approach

The evaluati on adopted three of the fi ve DAC evaluati on standards/principles, namely: relevance, eff ecti veness andimpact as its main criteria. These principles were used from the onset of this evaluati on to inform the design of the evaluati on tool, and analysis of results. The DAC criteria, defi ned below, are premised on the view that evaluati on should seek to “determine the relevance, and fulfi lment of objecti ves, developmental effi ciency, eff ecti veness, impact and sustainability of eff orts supported by aid agencies (OECD, 1992).

The post-test design has been considered for this evaluati on because the evaluati on sought to track changes or eff ects with regards to relevance, eff ecti veness and impact for only the interventi on group (benefi ciaries of the SETA training), due to limited resources to select a control/comparison group, and the absence of pre-interventi on indicators (baseline indicators).

1.4.1 Evaluation Criteria

1. Relevance: This measures the extent to which the objecti ves of an interventi on are consistent with recipients’ requirements. The current evaluati on assessed relevance of training to the needs of trainees, and the extent to which their expectati ons were met.

2. Eff ecti veness: Measures the extent to which an interventi on’s objecti ves were achieved, or are expected to be achieved, taking into account their relati ve importance. The current study looked at the extent to which trainees applied their knowledge and skills to earn a living, or transferred their skills to others.

The OECD-DAC evaluation criteria6

5

OVERVIEW 3. Effi ciency: A measure of how economically resources/inputs (funds, experti se, ti me, equipment, etc.) are converted into results. This did not form part of the ex-post evaluati on.

4. Impact: Positi ve and negati ve primary and secondary long-term eff ects produced by the interventi on, whether directly or indirectly, intended or unintended. This evaluati on investi gated the positi ve changes produced by the training for trainees, coaching others, changes in parti cipants’ life-skills, value of training during and post constructi on of the ECD centre, employability etc.

5. Sustainability: The conti nuati on of benefi ts from the interventi on aft er major development assistance has ceased. Interventi ons must be both environmentally and fi nancially sustainable. Where the emphasis is not on external assistance, sustainability can be defi ned as the ability of key stakeholders to sustain interventi on benefi ts – aft er the cessati on of donor funding – with eff orts that use locally available resources. In this evaluati on, sustainability was not included.

Overall, 12 (80% response rate) parti cipants out of 15 were successfully surveyed to generate data for the evaluati on. Three parti cipants were not available to respond to the survey. The achieved response rate is stati sti cally representati ve of the target group, thus making it possible to draw generalized conclusions on the results.

With regard to data analysis, the raw data was processed and analysed using a descripti ve stati sti cal approach with the aid of Microsoft excel. The data was fi rst captured in excel and thereaft er cleaned, and analysed to generate descripti veoutputs, which was later interpreted for the report.

Numeric data (generated from closed-ended questi ons) was analysed and presented with frequency tables, percentages and graphs, whilst non-numeric (generated from open-ended questi ons) was analysed using a content analysis approach – the process involved coding and streamlining data into themes and categories aligned to key questi ons. The qualitati ve data has been presented using verbal quotati ons and themes.

1.4.2 Data collection & analysis

The chosen evaluati on design has a number of limitati ons, which must be considered. To begin with, because there is no pre-test or comparison group, it is diffi cult to determine the magnitude of the outcome (e.g., how much did knowledge increase compared to what parti cipants knew before), and att ributi on (whether the outcomes are due to the interventi on or to some other causes) with a post-test only design.

In short, non-experimental designs are less rigorous in establishing a cause-and-eff ect relati on between an interventi on and its impact, compared to the “gold standard”, i.e. Randomized Control Trial (RCT) and other experimental designs, which are considered much more rigorous for impact objecti ves.

Furthermore, the non-experimental design employed for this evaluati on does not control for any external factors (e.g., other trainings, exposures), as well as being non-random (trainees were chosen based on a selecti on criteria to parti cipate in the training). Therefore, this report does not seek to provide any causal relati ons, but to empirically highlight changes in the lives of trainees aft er completi on of their training.

1.4.3 Limitations

6 See Organisati on for Economic Co-operati on and Development – Development Assistance Committ ee, ‘Evaluati on of development programmes, DAC Criteria for Evaluati ng Development Assistance’. htt p://www.oecd.org/dac/evaluati on/daccriteriaforevaluati ngdevelopmentassistance.htm

6

Figure 1: Evaluati on logic The logic model provides a schemati c overview of the linkages between the required resources, acti viti es, outputs and expected outcomes of the training for its parti cipants.

1.4.4 Evaluation Logic Model

INPUTS ACTIVITIES OUTPUTS OUTCOMES & IMPACT

Project Staff

Taining materials

Funding

Building equipment

Offi ce space

Partnerships

Recruit trainees

Facilitate training

Engage trainees in aconstructi on project

Issue certi fi cate post training to successfultrainees

SETA accredited building and constructi on training curriculum

The rammed earth technology training materials

Training certi fi cate

Trainees acquire knowledge and skills in constructi on

Bett er chance of securing employment in the building and constructi on sector

Improved self-confi dence and abilty to apply skills to solve constructi on related problems

Trainees are able to start their own business ventures for self sustenance

Improved skills andlivelihoods

Increased opti mism about the future - or sense of imminent possibility for a brighter future

7

2. EVALUATION FINDINGS

The key demographic indicators assessed were locati on, gender, age, race and occupati on. In terms of locati on and populati on group, the analysis revealed that all (100%) parti cipants resided in the Gauteng province, and were predominantly Black African. Most of the trainees were female (66.7%), whilst male trainees consti tuted 33.3% (Figure 2) of the cohort.

Moreover, there were more parti cipants above 30 years (75%), than those under 30 (25%) as shown in Figure 3. With respect to current occupati on, it was found that most (41.7%) of the trainees were engaged in part-ti me work. However, 25% worked full-ti me and 33.3% (Figure 4) were Unemployed. Proporti onally, female trainees were more likely to work part-ti me (25%) and full-ti me (16.7%) than males. They were also more likely to be unemployed (25%) than males (8.3%) (Table 1).

2.1 Demographics

Overall, feedback was gleaned from 12 (n) parti cipants upon completi on of the data collecti on phase of the evaluati on. The emerging fi ndings are presented in relati on to key evaluati on questi ons, with graphical illustrati ons.

Indian

White

Coloured

Black African

Race & Gender (n=12)Female

Male

33.3%66.7%

0 20 40 60 80

Figure 2: Distributi on of respondents by race & gender

50

White

0

Age (n=12)

White33.3%

18 - 20 21 - 25 26 - 30 31 - 35 36 - 40

41.7%16.7%8.3%0%

Figure 3: Respondents’ age

Student, part-ti me

Student, full-ti me

Unemployed

Self-employed

Working part-ti me

Working full-ti me

Current occupati on (n=12)

33.3%

41.7%

0 5 10 15 20 25 30 35 40 45

41.7%

33.3%

0%0%

25%

0%

Figure 4: Respondents’ current occupati on

0%0%0%

8

Table 1: Occupati on by Gender

Occupati on Male Female # Type

Studentet - - - -

Unemployed, looking 8.3% (1) 25% (3) 4 33.3%

Self-employed - - - -

Work, full-ti me 8.3% (1) 16.7% (2) 3 25%

Work, part-ti me 16.7% (2) 25% (3) 5 41.7%

Table 1: Occupati on by gender

In measuring the relevance of the training to parti cipants, parti cipants were asked to rate the extent to which they found the training relevant to their needs on a scale – from being “extremely valuable” to “not valuable”. In additi on, they were asked to indicate the extent to which the training met their expectati ons on a scale – from meeti ng “most expectati ons” to “not meeti ng most expectati ons”.

The results which have been presented graphically in fi gures 5 & 6 show that with respect to relevance, all (100%) of the parti cipants found the training to be “extremely valuable” to their needs (Figure 6).

In terms of expectati ons, the training met most of the expectati ons of a majority (66.7%) of the trainees, whilst 33.3% indicated that ‘some of their expectati ons were met’ (Figure 5).

2.2 Relevance of the training to participants

Most expectati ons were not met

Some of my expectati ons were met

Most of my expectati ons were met

To what extent did the SETA training meet your needs? (n=12)

33.3%

66.7%

Figure 5: Extent to which training met the needs of respondents

To what extent did the SETA training meet your needs? (n=12)

66.7%

To what extent did the SETA training meet your needs? (n=12)

0 10 20 30 40 50 60 70

0%

200

White

0

How would you rate the relevance of the SETA training to your needs? (n=12)

White

Extremely Valuable Somewhat Valuable Not Valuable

0%

Figure 6: Relevance of training to parti cipants’ needs

0%

100%

9

0 10 20 30 40 50 60 70

Parti cipants were asked to indicate their most rewarding experiences besides the daily wages that they earned from the constructi on of the ECD. As presented in fi gure 7, it appears that “acquiring the knowledge and skills in building and constructi on” was the most rewarding experience to more than half (58.3%) of the group. Moreover, the money earned from the ECD constructi on was deemed most rewarding and 25% of the trainees also claimed they used the money to support their families. Lastly, the training certi fi cate was also seen as most rewarding experience to a few of the trainees (16.7%).

2.3 Impact of training on participants

• Most rewarding training experiences

100

White

0

Most rewarding training experience (n=12)

White

The Certi fi cate Acquiring the skills I used all the money received and knowledge to support my family

16.7%

Figure 7: Besides the money you earned the ECD constructi on, what was most rewarding about the project to you?

58.3% 25%

• Impact of training on participants’ life skills

35%

30%

25%

20%

15%

10%

5%

0%

Impact of training on trainees’ life skills (n=12)

25%

I can build and do Enhanced my Improved my I know what is concrete mix employability confi dence to take right and wrong in easily in the building on projects constructi on & constructi on sector

25%

33%

8.3%

Figure 8: To what extent did the training enhance your life skills?

I am able to do my own constructi onrelated maintenance

at home

8.3%

10

The analysis of the results presented in fi gure 9 demonstrates that two-thirds (66.7%) of the trainees have secured constructi on related jobs in the last two years because of their training with AMAMA. Most of them were involved in full-ti me work with constructi on companies, while others worked part-ti me. Some were involved with RSP housing constructi on aft er their training.

Interesti ngly, since completi on of the training, half (50%) of the parti cipants have coached their peers on various aspects of constructi on and building such as plumbing, bricklaying, erecti ng a roof, glazing and how to set up a building. However, parti cipants have not used the rammed earth technology anywhere else to build.

In the last two years, have you had any jobbecause of your training in building and

constructi on?

Have you coached anyone on any of the aspects ofbuilding and constructi on such as plumbing,

bricklaying, erecti ng roof, glazing and setti ng up?

Besides the constructi on of the ECD centre in yourcommunity, have you used the rammed earth

Post training involvements

33.3%66.7%

Figure 9: Post training involvements (knock-on eff ects)

66.7%33.3%

0% 20% 40% 60% 80% 100% 120%

0%

50% 50%

100%

No Yes

While it may not be possible to establish a cause-eff ect relati onship between the training and its eff ects due to the weakness of the chosen evaluati on design, the insights gleaned from the parti cipants, nonetheless, suggest that the training has contributed signifi cantly to insti lling positi ve att ributes such as confi dence and opti mism in parti cipants (Figure 10).

To ascertain any knock-on eff ects, parti cipants were asked to refl ect over the last two years and identi fy positi ve changes in their lives, which they can say were made possible by AMAMA’s SETA building and constructi on training experience that they acquired.

The results revealed that, a third (33.3%) of the parti cipants secured employment because of the certi fi cate in building and training they received aft er their training with AMAMA. More importantly, 25% of the respondents stated that they have grown in confi dence since completi ng their training. When looking for work, respondents are confi dent to show their certi fi cate and prove to be experienced when employed.

A few (8.3%) of the respondents said they have “learned to be more tolerant of others”, whilst others highlighted that “the knowledge and skills from the training have assisted them tremendously, and thus certain about getti ng employment”. Furthermore, 16.7% of the parti cipants said they are now able to do any constructi on related work in their communiti es because of the training experience.

• Knock-on eff ects

The evaluati on delved deeper to assess parti cipants post-training acti viti es in order to understand if the training has had any possible knock-on eff ects on trainees. Three proxy indicators were used: securing employment in the last two years as a result of the training, coaching or mentoring others on any of the various aspects of plumbing, carpentry and bricklaying, and if parti cipants have used the rammed earth technology anywhere to build.

• Post-training engagements (knock-on eff ects) When looking for work I am confi dent to provide my certi fi cate and when taken I prove to be experienced

The knowledge and skills from the training have assissted me greatly and I am positi ve that I will get a job done soon

I have been able to secure a job with my certi fi cate

I am able to do any constructi on related project

I have learned to be more tolerant of others

11

0% 20% 40% 60% 80% 100% 120%

When looking for work I am confi dent to provide my certi fi cate and when taken I prove to be experienced

The knowledge and skills from the training have assissted me greatly and I am positi ve that I will get a job done soon

I have been able to secure a job with my certi fi cate

I am able to do any constructi on related project

I have learned to be more tolerant of others

Knock-on eff ects (n=12)

8.3%

25%

Figure 10: Looking back over the last two years, what positi ve changes in your life can say were made possible by the SETA training and constructi on experience that you acquired?

0% 5% 10% 15% 20% 25% 30% 35%

33.3%

16.7%

8.3%

The evaluati on explored the extent to which parti cipants were confi dent in their ability to translate the skills acquired to acti on. They were asked to indicate how confi dent they were in their ability to do carpentry, plumbing and bricklaying tasks on a scale of – “very confi dent” to “not confi dent”.

• Confi dence in ability to perform ‘bricklaying’, ‘carpentry’, and ‘plumbing’ related tasks

Overall, parti cipants were relati vely confi dent in their ability to perform bricklaying related tasks. The analysis depicted in fi gure 11 illustrates that, the majority (75%) of the parti cipants, said they were “very confi dent” in their ability to ‘identi fy tools and their functi ons’, and ‘mix and cast foundati on concrete’, concerning this component of the training.

Furthermore, half of the parti cipants (50%) were “confi dent” in their ability to ‘build foundati on wall in stretcher bond’, whereas 42% of the parti cipants maintained they were “very confi dent” in performing this task.

Similarly, nearly half (45%) of parti cipants rated themselves as being “very confi dent” with respect to their ability to ‘set out and excavate foundati on’, whilst 46% were “confi dent” in their ability to ‘set out and transfer levels’.

Bricklaying

In terms of confi dence in carpentry skills, the parti cipants responded more confi dently with respect to their ability to carry out carpentry related tasks amongst the three major skills areas examined.

The analysis revealed that, most parti cipants (83.3%) rated themselves as being “very confi dent” in being able to ‘identi fy carpentry tools and their functi ons’, as shown in fi gure 12.

Likewise, more than half (58.3%), respecti vely reported they were “very confi dent” in their ability to: ‘set out and construct basic joints’, ‘install roof truss’ and ‘prepare steel frame to hang a door’.

In additi on, 50% of the cohort indicated they were “very confi dent” with respect to setti ng out and preparing to erect ceiling, whereas another 50% said they were “confi dent” in their ability to erect a ceiling ti mber brandering.

Carpentry

1

Confi dence in ability to perform bricklaying related tasks (n=12)

Very Confi dent

Confi dent

Somewhat Confi dent

Not Confi dent

Identi fy tools & their functi on

Set out &transfer levels

Set out & excavate foundati on

Build foundati onwall in stretcherbond

Mix & castfoundati onconcrete

75%

16.7%

8.3%

0%

27.2%

45.5%

27.3%

0%

45.4%

27.3%

18.2%

9.1%

41.7%

50%

8.3%

0%

75%

25%

0%

0%

80%

70%

60%

50%

40%

30%

20%

10%

0%

Figure 11: Confi dence in ability to apply bricklaying skills

Confi dence in ability to peform carpentry related tasks (n=12)

Very Confi dent

Confi dent

Somewhat Confi dent

Not Confi dent

Identi fy carpentry tools& theirfuncti ons

Set out &construct basicjoints

Install rooftruss

Prepare steelframe to hanga door

ISet out andprepare to erectceiling

Erect a ceilingto ti mberbrandering

83%

8.3%

8.3%

0%

58.3%

25%

8.3%

8.3%

58.3%

8.3%

25%

8.3%

58.3%

33%

0%

8.3%

50%

42%

8.3%

0%

41.7%

50%

8.3%

0%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

Figure 12: Confi dence in ability to apply carpentry skills

13

Confi dence in ability to perform plumbing related tasks (n=12)

Very Confi dent

Confi dent

Somewhat Confi dent

Not Confi dent

Identi fy plumbing tools & their functi on

Drainage Excavate trenches for pipelaying

Plan & prepare materials according to spec

Install below ground waterborne drainage systems

Testbelow ground drainage systems

Construct a septi c tank

Install a sink

Install a water closet (toilet)

Install a wash basin

75%

16.7%

0%

8.3%

41.7%

16.7%

33.3%

8.3%

75%

8.3%

16.7%

0%

56.3%

8%

25%

8.3%

58%

8%

25%

8.3%

50%

17%

8.3%

25%

27%

46%

9%

18%

22%

56%

11%

11%

33.3%

50%

8.3%

8.3%

50%

33.3%

16.7%

0%

80%

70%

60%

50%

40%

30%

20%

10%

0%

Figure 13: Confi dence in ability to apply plumbing skills

14

This evaluati on was undertaken to establish the relevance, eff ecti veness and impact of AMAMA’s SETA accredited building and constructi on training on 15 trainees who underwent the training two years ago. Despite the limitati ons of the evaluati on design applied, the results nonetheless have shown that the training indeed produced signifi cant shift s in knowledge and skills pertaining to constructi on. The training met most of the expectati ons of a majority (66.7%) of trainees, and was rated by all (100%) trainees as being “extremely valuable” to their needs.

The impact of the training manifested in the many (66.7%) trainees who secured employment in the last two years upon completi ng their training. Parti cipants reported high confi dence in their ability to apply their skills and knowledge acquired from the training. They were relati vely confi dent in their ability to undertake various tasks across carpentry, plumbing and bricklaying. Many of the parti cipants have increased their confi dence to take on constructi on related projects.

To more than half (58.3%) of the parti cipants, the skills and knowledge acquired from the training was considered as the most rewarding experience. In additi on, 25% of the trainees reported to have used the money earned from the constructi on of the ECD to support their families.

As a knock-on eff ect, half (50%) of the parti cipants have transferred their building skills and knowledge to their peers through coaching and mentoring engagements. Above all, parti cipants have acquired the ability to do their own basic home maintenance and to eke out a living by earning income through piece jobs.

However, what is most striking from the fi ndings is the realizati on that, the training has shift ed the mindsets of trainees – by insti lling in them an imminent sense of possibility of a bett er future.

Although the training was generally eff ecti ve, it is recommended that specifi c aspects of plumbing such as: ‘testi ng below ground drainage systems (25%)’, ‘constructi ng a scepti c tank (18%)’, and ‘installing a sink (11%)’ should be given more att enti on during training as some parti cipants showed lack of confi dence in their ability to carry out tasks in these areas.

3. CONCLUSION & RECOMMENDATIONS

o AMAMA Social Enterprise – SETA Accredited Training. htt p://www.amama.co.za/o Chianca, T. (2008). The OECD/DAC Criteria for Internati onal Development Evaluati ons: An Assessment and Ideas for Improvement, Journal of Multi -Disciplinary Evaluati on, Volume 5, Number 9.o Organisati on for Economic Co-operati on and Development – Development Assistance Committ ee, ‘Evaluati on of development programmes, DAC Criteria for Evaluati ng Development Assistance’, web page, OECD. htt p://www. oecd.org/dac/evaluati on/daccriteriaforevaluati ngdevelopmentassistance.htm o Penna, R.M (PhD) (2011). The nonprofi t outcomes toolbox. A complete guide to program eff ecti veness, performance measurement and results. John Wiley & Sons, Inc.

BIBLIOGRAPHY

15

1. Verbatim Responses

A) Besides the constructi on of the ECD centre in your community, have you used the rammed earth technology anywhere to build?

B) Since completi ng your training, have you parti cipated in any constructi on project?

C) Have you coached anyone on any of the aspects of contracti on such as plumbing, bricklaying, erecti ng roof, glazing, and setti ng up building, since you completed your training?

D) In the last two years, have you had any job because of your training/skills in building and constructi on that you acquired?

POST TRAINING INVOLVEMENTS YES NOPOST TRAINING INVOLVEMENTS YES NOPOST TRAINING INVOLVEMENTS YES NOPOST TRAINING INVOLVEMENTS YES NO

0 12 (100%)

6 (50%) 6 (50%)

B1) If yes, please elaborate?

• By assisti ng at a constructi on of RSP houses in Sicelo• I am working on a farm assisti ng with laying of irrigati on pipes and general constructi on maintenance.• Acquired a full ti me job on a company that does concrete units.• Building of RSP houses in Sicelo Township• I am working with constructi on company laying out pipes• I have done a lot of piece jobs in constructi on and sti ll prefer that skill

6 (50%) 6 (50%)

C1) If yes, please elaborate?

• When faced with a problem on the farm I use my skills to solve it by sharing with my peers what needs to be done.• I have a family in Sasolburg, they were building RSP houses, and I coached a group on bricklaying.• My peers during the constructi on of RSP houses• We live in an area where constructi on is happening a lot must come for advises, especially in plumbing.• Some of peers working at RSP houses needed a coach and I assisted especially in plumbing• There were people who were working on plumbing and when it did not work I assisted them and shown then how they should lay the pipes and it worked.

8 (66.7%) 4 (33.3%)

D1 If yes, please elaborate?

• When working at the RSP site in 2016• As menti oned above I am working part ti me on a farm.• Acquired a job in a concrete units constructi on.• Aft er building this crèche I was involved with RSP constructi on as menti oned above and training was provided.• As explained above I got involved with the constructi on of RSP houses• I am laying pipes with a constructi on company.• Piece jobs in constructi on and I fi nd it easy to work• I got temporary job on constructi on in building houses.

Apart from the money that you earned during the constructi on of the ECD center, what was most rewarding about the project to you?

• The certi fi cate and knowledge

3. CONCLUSION & RECOMMENDATIONS APPENDICES

16

• The certi fi cate and training• Acquiring the skills• I am now working in that fi eld• Acquired knowledge• I used all the money to support my family, could not buy opportunity as it was not enough.• Acquired skills on constructi on• I was able to support my family but above all gained skills because I knew nothing about constructi on, so I am confi dent that even though the remunerati on was not enough but it really assisted.• Acquiring of building skills• The skills• Acquiring the skills• Yes it assisted to support my family

How has the training improved your life skills?

• I can build and do concrete mix easy.• By providing knowledge and power by off ering me a certi fi cate, I will start looking for a job as I had a baby.• Confi dently getti ng the job.• A lot• Know what is right and wrong in constructi on.• Can do this and that at home.• By knowing what is required in case of constructi on work• Very much because as menti oned I can get a job in the constructi on• I am confi dent to take on any building project• By getti ng the job• With knowledge• A lot because I am confi dent in what I was trained on

Looking back over the last two years, aft er completi ng your training, can you identi fy any positi ve changes in your life that was made possible by the SETA training and constructi on experience you acquired?

• Jobs are scares but I am confi dent with knowledge.• Not much but confi dence is boosted.• I have a certi fi cate to prove my knowledge.• Used my certi fi cate to acquire a job.• Not really• Yes, though I am not doing some community work but I learned tolerance• It assisted a lot and I am looking forward to getti ng jobs in constructi on in order to take the skills forward• Yes when looking for work, I confi dently produce my certi fi cate and when taken I prove to be experienced.• by getti ng the job I need• Yes because I know what of skills I have and only working for constructi on• Yes because I can do any building project

2. Evaluation questionnaire

Please note that all your responses will remain confi denti al and will be treated with utmost respect for anonymity

1. Gender: a) Male [ ] b ) Female [ ] c) Other (please specify): _____________________________

2. Which one of the following populati on groups are you? a) Black African [ ] b) Coloured [ ] c) White [ ] d) Indian [ ]

3. Age: a) 18-20 [ ] b) 21-25 [ ] c) 26-30 [ ] d) 31-35 [ ] c) 36-40 [ ]

17

4. In which province do you reside? _______________________________________

5. Work status (which one of the following most describes your status?) a) Working full-ti me [ ] b) Working part-ti me [ ] c) Self-employed [ ] d) Unemployed [ ] e) Student, full-ti me [ ] f) Student, part-ti me [ ]

6. If self-employed, what is the nature of your business? __________________________________________________

8. Using the following scale, please indicate the extent to which the training on building and constructi on met your expectati ons (Please ti ck answer) a) Most of my expectati ons were met [ ] b) Some of my expectati ons were met [ ] c) Most expectati ons were not met [ ]

9. Overall, how would you rate the relevance of the training to your needs? (Please ti ck answer) a) Extremely valuable [ ] b) Somewhat valuable [ ] c) Not valuable [ ]

10. Since completi ng your training, have you parti cipated in any constructi on project? a) Yes [ ] b) No [ ]

11. If yes, please elaborate: _____________________________________________________________________________________

12. Besides the constructi on of the ECD center in your community, have you used the rammed earth technology anywhere to build? a) Yes [ ] b) No [ ]

13. If yes, please elaborate: _____________________________________________________________________________________ 14. How confi dent are you in your ability (skills) to carry out tasks in the following areas? (Please ti ck)

Bricklaying

• Identi fy bricklaying tools & their functi ons• Set out and transfer levels• Set out and excavate foundati on• Build foundati on wall in stretcher bond • Mix and cast foundati on concrete

Carpentry

• Identi fy carpentry tools and their functi ons• Set out and construct basic joints• Install roof truss• Prepare steel frame to hand a door• Set out and prepare to erect ceiling• Erect a ceiling to ti mber brandering

FOCUS AREAS VeryConfi dent

Confi dent Somewhat Confi dent

NotConfi dent

15. Have you coached anyone (Family or Peers) on any of the aspects of constructi on such as plumbing, bricklaying, erecti ng roof, glazing, and setti ng up building, since you completed your training? a) Yes [ ] b) No [ ]

16. If yes, please elaborate: __________________________________________________________________

17. In the last two years, have you had any job because of the training on building & constructi on that you acquired? a) Yes [ ] b) No [ ]

If yes, please elaborate: ________________________________________________________________

18. Apart from the money (daily wage) that you earned during the constructi on of the ECD center, what was most rewarding (valuable) about the project to you? _____________________________________________________________________________________

19. How has the training improved your life skills? ______________________________________________

20. Looking back over the last two years, aft er completi ng your training, can you identi fy any positi ve change(s) in your life that was made possible by the SETA training and constructi on experience you acquired?

Thank You for contributi ng to this study!

Plumbing

• Identi fy plumbing tools and their functi ons• Drainage • Excavate trenches for pipe laying• Plan and prepare materials according to specifi cati on• Install below ground waterborne drainage systems• Test below ground drainage systems• Construct a scepti c tank• Install a sink• Install a water closet (toilet)• Install a wash basin