Interim Evaluation Report 01 - NHS Employers/media/Employers...This report summarises the results...

SupportiveLeadership &Management BehaviourTraining

01

PSYCHOLOGY WITH BUSINESS IN MIND

Pilot Training Programme MAY 2016Report Number: NHSE008Based on date received up to November 2015

Interim Evaluation Report

CONTENTSCONTENTS

SUPPORTIVE LEADERSHIP AND MANAGEMENT BEHAVIOUR - EVALUATION REPORT 2016

02

BACKGROUND 3

An integrated model of leadership 3Programme evaluation 4

MANAGER/LEADER BEHAVIOUR 6

MANAGER/LEADER CAPABILITY 8

TRAINING REACTIONS 10

QUALITATIVE COMMENTS 12

CLOSE 13

1

6

2

3

4

5

BACKGROUND

www.zeal-sbt.co.uk Email: [email protected] Telephone: 01159 932 324

03

1 - BACKGROUND

There were three broad objectives of the Supportive Leadership &Management Behaviour SBT training programme. These objectiveswere met by trainers delivering a series of activities and exercisesusing scenario based videos and other types of exercises as part ofa one day training programme.

1. Increase awareness and importance of everyday manager/leader behaviour2. Reflect on your own potential leadership/ manager behaviour3. Develop a plan to action supportive behaviour in your own work situation

AN INTEGRATED MODEL OF LEADERSHIP

Although the programme itself was designed to increase supportive leadership behaviour, helpingleaders to become more aware of their own performance and the factors that can influence theirperformance was integral to the success of this training intervention. Leaders were thereforeintroduced to the model of human performance that is shown below in Figure 1.

The approach used in this training starts out from the basis of understanding human health andperformance and then extends this understanding towards leadership and management andthen situates this behaviour within an organisational context. The supportive leader behaviourmodule is built upon the principle that human performance in any given situation is the culminationof three components that interact with and influence each other. These components are whatwe call 1) Personal Factors; 2) Behavioural Factors; and 3) Environmental Factors.

PERSONAL

ENVIRONMENTAL BEHAVIOUR/SKILL

MANAGEMENT/LEADERSHIPEFFECTIVENESS

Figure 1The interactive nature ofpersonal, environmentaland behavioural factors

BACKGROUND


04

The use of this model helps leaders to understand that no one component can be isolated fromthe other, and knowledge of all three is crucial to understanding human behaviour, learning andperformance. As shown above, these three components/factors act in a constant cyclical pattern,interacting with each other and dictating human behaviour. It also helps leaders to realise thatthe performance of their staff is similarly influenced by these factors. By developing a greaterunderstanding and appreciation of these factors, leaders can begin to target any specific areasfor personal development and also apply the model to develop those they lead using a commonframework and language. In addition, and to enable leaders to develop a greater understandingof their own personal, behavioural and environmental factors, the training programme was alsooffered with a 360 assessment called the Supportive Manager Behaviour Index (SMBI). TheSMBI enables leaders to seek feedback on their own approach from their staff, their peers andtheir managers. This process results in a personal feedback report that can be used to plan theirpersonal development.

PROGRAMME EVALUATION

Evaluating the impact and effectiveness of the programme was an important part of this project.As shown below, the SBT is being evaluated in multiple ways:

1. Reviewing the responses that have been recorded on the evaluation questionnairescompleted before and after training and in a transfer survey that was issued to asample of managers in October.

2. Reviewing any change in the responses recorded through the SMBI (360 behaviourassessment).

3. Reviewing any impact of the SBT management development training on measuredorganisational outcomes.

This report summarises the results from reviewing and – where possible - comparing pre andpost evaluation questionnaires (Figure 2)

Figure 2Illustrating the evaluation process for the SBT management development programme

Pre-courseevaluation

TRAININGPost-courseevaluation

BACKGROUND


05

Various measures where used to assess the impact of the training, and these are summarisedin Table 1 below. Some items (e.g. personality and mindset) were included as control factors toallow for the fact that individual differences may play a role in determining how effective thetraining is.

Measure Pre Post Follow-up

Management behaviour (awareness) a a aManager capability perceptions a a aPsychological symptoms of stress (PSS) a aJob satisfaction a aIntentions to quit a aOrganisational commitment a aReactions to training aTransfer of training aPersonality aMindset aPerceived organisational support a

Table 1Evaluation Measures


06

BEHAVIOUR A key aspect of this training is to also help managers/leaders

become more aware of supportive and unsupportive behaviours aswell as to develop a more informed understanding about the impactthat certain behaviours can have on employee health, well-being,motivation and performance.

To assess any change in awareness and understanding, delegates were asked to rate 17 behavioursin terms of how supportive or unsupportive they considered each behaviour to be. Assessments ofbehaviour were provided both before and immediately after the training. The 17 behaviours can begrouped into three main categories as shown below in Figure 3. An example behaviour is alsoprovided for information.

2 - MANAGER/LEADERBEHAVIOUR

Figure 3Behavioural categories assessed before and after training.

The before and after training scores (immediately post training) on the three behavioural categoriesare shown below in Table 2.

Positive orsupportivebehaviour

Encourages staff to sharetheir views

Performanceand targetmanagement

Sets demandingexpectations for their staff

Negative orunsupportivebehaviour

Is quick to blame whenthings go wrong

Behavioural Category

Positive or Supportive Behaviour

Performance and Target Management

Negative or Unsupportive Behaviour

AfterTraining

6.20

4.30

2.63

BeforeTraining

5.90

4.27

2.95

SignificantDifference

YES P<.001NOYES P<.001

Information:

N=485

Scale range: Minimum =1 (Very Unsupportive) and Maximum = 7 (Very Supportive)

P<.001 - there is less than a 1 in 1000 probability of such a difference in awareness ofthe supportiveness of manager behaviour happening ‘by chance’

Table 2Scores on the behaviour categories before and after training

BACKGROUND


07

These results suggest that immediately post course, delegates had a shift in their awarenessof supportive and unsupportive behaviours so that;

1. Delegates perceive the positive behaviours as being more supportive than theyperceived them to be at the start of the training. This change in awareness andunderstanding of the impact of the supportive behaviours was found to bestatistically significant.

2. At the end of the training, delegates perceive the negative behaviours to be moreunsupportive than they perceived them to be at the start of the training. This changein awareness and understanding of the impact of the negative behaviours was alsofound to be statistically significant.

3. Scores on the behaviours that were associated with performance managementand target setting did not change as a result of the training and remained neutral(i.e. at the scale mid-point).

CAPABILITY


08

Leadership capability is assessed using a five item scale. Each itemassesses the confidence with which leaders believe they can dealwith various scenarios and the scale range is from 0 (no confidence)to 10 (complete confidence). Leaders are asked to assess theirconfidence to deal with specific workplace situations/scenarios.Each scenario is described in detail in the evaluation survey, for thepurpose of this report a summary of the scenario domain is providedbelow in Table 3.

Analysis of the data showed that manager’s perceived capability to deal with different scenariossignificantly increased following attendance on the Supportive Leadership & Management Trainingprogramme. This difference is evident when comparing pre-course training scores with theimmediate post-course training scores on all of the measured scenarios (Table 3). This findingsuggests that managers left the training feeling more confident in their ability to deal with a rangeof management scenarios.

3 - MANAGER/LEADERCAPABILITY

Scenario Domain

Dealing with team conflict

Dealing with staff sickness absence

Dealing with organisational change

Dealing with violence and aggression

Dealing with poor communications

AfterTraining

7.57

7.71

7.35

7.87

7.69

BeforeTraining

6.72

6.61

6.20

7.08

6.64

SignificantDifference

YES P<.005

YES P<.005

YES P<.005

YES P<.005

YES P<.005

Information:

N = 498

Scale range: Minimum = 0 (not at all confident) and Maximum = 10 (completely confident)

P<.005 - there is less than a 5 in 1000 probability of such an increase in perceived capability happening ‘by chance’

Table 3Manager Capability scores

In addition to looking at the scores on the individual capability items, the five leader capability surveyitems can be aggregated to enable an overall leader capability score to be calculated. Using thedata from across the participating organisations (N= 498), Figure 4 below depicts the trend line inleadership capability from before to immediately after training.

CAPABILITY


09

Impact of training on: Management capability perceptions

40

38.15

33.20

39

38

37

36

35

34

33

32Before Training

Management Capability P

erceptions

Significant p<0.005

Time of measurementAfter Training

Figure 4Management capability perceptions pre-course and immediately after training

REACTIO

NS


10

As part of the post course evaluation questionnaire, delegate’sexperience of the training was measured. Although a number ofitems were asked within the questionnaire, further analysis of theserevealed that an overall measure of training reactions could bereduced to 7 items – example items are shown in Table 4 below.

Items are measured on a scale of 1- 5 (1=strongly disagree, 5 = strongly agree), with a higherscore indicating a more positive reaction/ experience of the training programme.

Table 5 below provides an overview of the delegate’s scores on the training reaction indicators.Although, and on their own, trainee reactions do not provide a rigorous assessment of any trainingprogramme, they are useful indicators of training quality. The response option with the highestfrequency for each training reaction indicator is highlighted in Table 5.

4 - TRAINING REACTIONS

StronglyAgree

163

116

158

138

133

171

255

Agree

218

268

261

272

267

215

203

Neutral

90

89

69

79

81

71

39

Disagree

23

29

14

22

20

38

5

StronglyDisagree

7

4

4

4

6

12

2

ReactionIndicator

The training met myexpectationsThe training context waswell suited to the trainingprocessIn my opinion, the plannedobjectives were metThe training was realisticThe issues were dealt within as much depth as thecourse allowedThe training enabled meto share professionalexperiences with colleaguesThe trainer/s wereconfident in their delivery

Training Reactions sample items

In my opinion, the planned objectives were met

The trainer(s) were confident in their delivery

The training met my expectations

Table 4Trainee Reactions sample items

Table 5Training reaction scores

REACTIO

NS


11

Based on a sample of approximately 507 (note: not all rows will add up to 507 due tomissing cases)

75% of delegates agreed or strongly agreed that the training had met theirexpectations

76% of delegates agreed or strongly agreed that the training context was wellsuited to the training process

83% of delegates agreed or strongly agreed that the planned objectives were met

81% of delegates agreed or strongly agreed that the training was realistic

79% of delegates agreed or strongly agreed that the issues were dealt with inas much depth as the course allowed

76% of delegates agreed or strongly agreed that the training enabled them to shareprofessional experiences with colleagues

91% of delegates agreed or strongly agreed that the trainer(s) were confident intheir delivery

COMMENTS


12

5 - QUALITATIVE COMMENTS

To capture any additional reactions that delegates have to thetraining, the post training questionnaire provides delegates with theopportunity to add further comments. A sample of these can be seenbelow and further support the positive reactions to the training.

Very inspiring at present time. Having the opportunity to reflect upon personalstrengths and weaknesses has been very refreshing.

Useful and practical course. Techniques can be applied easily and seamlesslyinto operational practice. Good opportunities to share experience, reflect and learnfrom others.

Very reflective – open discussions help reflect on own practice. Good looking atdepth – the positives/ negatives of managers behaviours. Liked coming up with goals.

I found the training very relevant to my development and also what I need to knowand change in being a better manager and leader. This has helped me to recognisequalities that I have and areas which I need to develop. There was a good mix of staffon the training course and the experience shared has really helped me and I feel betterequipped to manage and hopefully become a good leader and merge both together inachieving organisational goals.

Group discussion initiated by real life scenario was very helpful in looking atdifferent ways of tackling issues… the experience of others gave me a differentperspective of looking at situations.

I have greatly enjoyed this training and I would suggest that it becomes mandatoryin my trust both for new but also existing managers and especially senior managers.

“ ”“

”

“

”“

”“ ”

“ ”

CLOSE


13

6 - CLOSE

On the basis of the evaluation results, it can be seen that the trainingitself is both valuable and valued by trainees. Of particular note is thatmanagers are leaving the training with a significantly increased senseof their own ability to deal with various potential managementsituations that they may encounter in their job roles. In addition tothis, managers left the session with a greater awareness ofsupportive and unsupportive leadership behaviours and reportedbeing keen to apply this learning. In addition, 91% felt that the trainerswere confident in their delivery, which suggests that the delivery oftraining across the different Trusts has been to a high standard.

Following the distribution of the follow-up (time 3) survey in October/ November 2015, it isanticipated that the long term impacts of the training will be explored within the next full report.

NOTES


14

ZEAL SO

LUTIONS


15

At Zeal, we are all passionate about the role of psychology in theworkplace. We bring together leading edge thinkers, doers andcommunicators who know what it means to solve customerproblems in practical and useful ways. Our core aim is to enhanceindividual and organisational health and effectiveness. What doesthis mean, you might ask?

Do you ever ask questions like… How can we improve motivation at work?

Do our leaders and managers have all the skills and competencies they need?

How can we better manage workplace violence and aggression?

How can staff retention be improved?

What can we do to reduce absence?

Are we doing all we can to manage workplace bullying/harassment?

Do we have a 'stress problem'? What can we do about it?

What do people really think about their job and the organisation?

Do we have the 'right' organisational culture?

Does the design and layout of our workplace supportwhat we ask people to do?

Is our training effective? Can it be improved?

Our aim is to answer questions suchas these by:

Bringing useable science to bear on allaspects of human psychology at work

Making sure that there is evidence forany solutions advocated

Translating fundamental knowledge intoeveryday ‘jargon free’ tactics, strategiesand actions

Evaluating actions and interventions

Encouraging individual and organisationalproblem solving

Bringing energy and enthusiasm toeverything we do

Management& Leadership

Health &Well-Being

Recruitment& Selection

Training &Development

Consultancy Advice

Research Evaluation

We offer consultancy, advice, researchand evaluation services. And we workacross key organisational functions.

Call us now for a discussion.www.zeal-sbt.co.uk

Email: [email protected]: 01159 932 324

ABOUT ZEAL SOLUTIONS

PSYCHOLOGY WITH BUSINESS IN MIND

www.zeal-sbt.co.ukEmail: [email protected]: 01159 932 324

Zeal Solutions Ltd15 Beck StreetNottinghamNG1 1EQ.

Zeal

Sol

utio

ns L

td ©

201

6. R

ef.S

LM

BM

IER

.

“”

Evaluation isa tool thatcan supportlearning.

Interim Evaluation Report 01 - NHS Employers/media/Employers...This report summarises the results...

Documents

Transcript of Interim Evaluation Report 01 - NHS Employers/media/Employers...This report summarises the results...