Success Profiles

Scoring Case Studies

November 2018

1 | Success Profiles - Vacancy Toolkit

Main Menu

2 | Success Profiles - Scoring Case Studies

NavigationIntroduction

Using and navigating this toolkit

These case studies have been designed to provide detailed examples of how to score throughout different recruitment campaigns, using Success Profiles, and make appointment decisions.

Before you read this tool you should understand what Success Profiles are and be familiar with the vacancy holder toolkit and the interview methodology. You should also be familiar with your departmental recruitment processes and know where you need to record information.

Behaviours, experience and technical skills are scored on a 1-7 rating scale as we are seeking to evaluate the balance of evidence (i.e. how much or how little) a candidate presents in their response, in relation to a single dimension.

Strengths are scored on a 1-4 rating scale as we are seeking to evaluate the product of three dimensions (level of engagement, capability and frequency of use) as indicated by a candidate’s response.

Throughout the recruitment process the vacancy holder has the discretion to set benchmarks when assessing candidates. Benchmarks however exclude candidates and so careful consideration should be given about whether they are required and what they are set at. The vacancy holder must weigh up how selective to be, influencing factors may include the level of risk associated with the job, the number of roles, the availability of eligible applicants and the extent to which training and development is provided. All benchmarks must be agreed in advance and applied consistently to all candidates in the campaign.

When selecting ability or aptitude tests, it is important to follow guidance on the pass/fail benchmarks. This will minimise the risk of any adverse impact for candidates with protected characteristics and ensure fair selection.

You can navigate to the section that interests you using the menu, or you can browse the full guidance using the previous and next page buttons. Where sections have multiple tabs, you simply need to click on the tab to view the information it contains.

Throughout this guidance you will see the following buttons:

Previous - return to the last page viewed

Next page - proceed to the next page in the topic

Section menu - return to the sub menu

Home - return to the main menu

�� External link

3 | Success Profiles - Scoring Case Studies

Entry-level role

Assessment flow

Application form

Civil Service Judgement Test Interview

Start

Selection of successful candidate

Entry-level roles (example only)

The vacancy holder has chosen the below assessment flow for their entry level vacancy.

4 | Success Profiles - Scoring Case Studies

Entry-level role

The initial sift uses the Civil Service Situational Judgement Test (CSJT).

To proceed to the next stage of selection candidates must meet the default pass score set for the test.

Unsuccessful candidates are notified, feedback is not usually provided following tests.

Initial sift

5 | Success Profiles - Scoring Case Studies

Sift

Entry-level role

Candidates are sifted using a behaviour based application form, with 250 word behaviour examples.

A high number of applications were expected, so the vacancy holder decided to only score the lead behaviour ‘Working Together’ at sift. This was explained to candidates in the job advert.

There are limited places for interview and therefore only the top 50 candidates, when placed in merit order, are invited to interview.

Note: if all of the behaviours on the application form had been assessed, those scores would have been combined to reach a total behaviours score.

6 | Success Profiles - Scoring Case Studies

Blended interview

Entry-level role

Candidates will be assessed using a blended behaviour and strength-based interview.

The vacancy holder has selected four relevant behaviours and set a minimum benchmark that candidates must score at least a 4 for each behaviour.

The vacancy holder has selected four relevant strengths and set a minimum benchmark that candidates must achieve a total score of at least 12. Although this is the equivalent of an average score of 3 on each strength, candidates could score lower or higher on some questions if their total score is at least 12.

Following the interviews, the vacancy holder created a merit list based on the overall interview score (the sum total of all strengths and behaviours scores).

Candidate A

• Scored 16 for behaviours (there were no scores less than 4)

• Scored 16 for strengths

• So an overall score of 32.

Candidate A was top of the merit list and appointable to role.

Candidate B

• Scored 20 for behaviours (they had a score lower than than 4 for one behaviour)

• Scored 12 for strengths

• So an overall total interview score of 32.

Although Candidate B achieved the same overall score as Candidate A, they were not appointable because they did not meet the standard set for behaviours.

7 | Success Profiles - Scoring Case Studies

Specialist role

Assessment flow

Specialist roles (example only)

Start

Interview

Technical Simulation Exercise

CVSelection of successful candidate

The vacancy holder has chosen the below assessment flow for their specialist vacancy.

8 | Success Profiles - Scoring Case Studies

Sift

Specialist role

At sift, candidates submit a CV and personal statement.

To proceed to the next stage, candidates must demonstrate they have the essential experience and behaviours.

Evidence of the candidates’ behaviours and experience are scored separately using a 1-7 scale.

The vacancy holder has set a minimum benchmark of at least a score of 4 for both elements to progress to the next stage of assessment.

 

9 | Success Profiles - Scoring Case Studies

Technical simulation exercise

Specialist role

The technical simulation exercise used assesses two technical skills and two behaviours.

Exercise elements are scored individually, using the 1-7 rating scale, and totalled to give an overall technical simulation exercise score.

The vacancy holder commissioned the Government Recruitment Service to work with Subject Matter Experts to design an exercise that replicates and assess the tasks undertaken in the role. The technical simulation exercise involves solving a coding problem and explaining the results to a stakeholder. This exercise was trialled and piloted before being used in live recruitment.

The vacancy holder set a minimum benchmark of at least 16 for the overall simulation exercise score. This is because they decided an equivalent of an average score of less than 4 on each behaviour or technical skill would be unsatisfactory.

10 | Success Profiles - Scoring Case Studies

Blended interview

Specialist role

Candidates will be assessed using a blended behaviour, strength and experience-based interview.

The questions will cover:

• Two behaviours with a minimum benchmark overall score of 8 (the equivalent of an average of 4 on each behaviour)

• Two areas of experience with a minimum benchmark overall score of 8 (the equivalent of an average of 4 on each area of experience)

• Four strengths with a minimum benchmark overall score of 8 (the equivalent of an average of 2 on each area of strength).

11 | Success Profiles - Scoring Case Studies

Combing assessment scores

Specialist role

Since the candidates were assessed using two selection methods at this stage (technical simulation exercise and blended interview) the scores must be combined.

The vacancy holder will then create a merit list of the sum total score, of all behaviours, strengths, technical skills and experience scores, across both the interview and simulation exercise. Appointment is made from the top down, only those who achieved the minimum benchmarks will be eligible for appointment.

Candidate A

• Technical simulation exercise total score: behaviours + technical skills = 17 (above the benchmark score of 16)

• Interview scores: behaviours = 9, experience = 8, strengths = 9. (No scores below the minimum benchmark)

• Sum total score for this stage = 43.

Candidate A is considered appointable to role.

Candidate B

• Technical simulation exercise total score: behaviours + technical skills = 25 (above the benchmark score of 16)

• Interview scores: behaviours = 4 (below the minimum benchmark), experience = 8, strengths = 6 (below the minimum benchmark)

• Sum total score for this stage = 43. 

Candidate B is not considered appointable to role, as they did not meet the standards set for behaviours and strengths in the interview.

Candidate C

• Technical simulation exercise total score: behaviours + technical skills = 10 (below the benchmark score of 16)

• Interview scores: behaviours = 6 (below the minimum), experience = 8, strengths = 9

• Sum total for this stage = 33.

Candidate C would not be considered appointable to role, as they did not meet the standard set for the technical simulation exercise, nor the standard set for behaviours in the interview.

12 | Success Profiles - Scoring Case Studies

Senior leadership role

Assessment flow

Senior leadership roles (example only)

CV Staff Engagement Exercise

Individual Leadership

Assessment (ILA)

Start

Interview

Selection of successful candidate

Presentation

The vacancy holder has chosen the below assessment flow for their senior leadership vacancy.

13 | Success Profiles - Scoring Case Studies

Sift

Senior leadership role

At sift candidates submit a CV and personal statement.

Evidence of the candidates’ behaviours and experience are considered together and a single score given, using a three point scale of:

• does not meet the criteria

• borderline/meets some of the criteria

• clearly meets the criteria.

The vacancy holder has decided that those who clearly meet the criteria will progress to the next stage. Those who meet some of the criteria may progress to the next stage of assessment depending on the places available.

14 | Success Profiles - Scoring Case Studies

Pre-interview assessment

Senior leadership role

The pre-interview assessment consists of an Individual Leadership Assessment (ILA) and a Staff Engagement Exercise (SEE). The assessments are not scored.

The findings will be shared with the panel for the next stage of assessment, along with suggested probing questions. The panel will use this information to decide what probing questions to ask to explore whether candidates have demonstrated the required behaviours and experience for the role.

Individual Leadership Assessment (ILA)  

An occupational psychologist uses psychometric tests of personality and verbal/numerical reasoning, along with evidence from a psychological interview, to identify areas for the panel to target further probing at interview.

For example, a candidate might be assessed as having a strong tendency for complex strategic thinking and generating ideas, but with a risk of over-playing this at times and coming up with impracticable solutions for staff/stakeholders.

Staff Engagement Exercise (SEE)

An occupational psychologist observes candidates interacting and engaging with a real group of staff, representative of the organisation associated with the role.

For example, a candidate might be assessed as potentially being perceived as uninspiring to their staff.

15 | Success Profiles - Scoring Case Studies

Blended interview with presentation

Senior leadership role

The blended interview assess behaviours and strengths. It also includes a 5-minute presentation covering a set topic related to the candidate’s leadership experience (with probing questions from the panel).

To increase the reliability of the scores, two questions were asked per strength. To provide an overall score per strength, the scores for the two questions in each case are arithmetically averaged and the score rounded to the nearest whole number.

For example: A candidate was given the scores 2 (question 1) and 3 (question 2) for the same strength, the average score was 3 (i.e. 2 + 3 = 5, 5 ÷ 2 = 2.5 and rounded to 3).

Benchmarking

The vacancy holder has set the following minimum benchmarks:

• A total score of at least 12 for behaviours (as this is the equivalent of an average of 4 on each behaviour), with no score in any individual behaviour less than a 3.

• Candidates with an overall score of 1 (‘weakness’) in two or more strengths will be unsuccessful.

• A total score of at least 8 for experience (as this is the equivalent of an average of 4 on each area of leadership experience).

16 | Success Profiles - Scoring Case Studies

Combing assessment scores

Senior leadership role

The vacancy holder will create a merit list of the sum total score, i.e. of all behaviours, strengths and experience scores across both the interview and presentation. Appointment is made from the top down, only those who achieved the minimum benchmarks will be eligible for appointment.

Candidate A

• Scored 15 for behaviours, 9 for strengths (scored above 1 on all three strengths) and 12 for experience

• So an overall total of 36.

Candidate A met the standards for all the elements and were the highest scoring candidate and appointed to the role.

Candidate B

• Scored 13 for behaviours, 7 for strengths (scored 1 on one of the three strengths) and 8 for experience

• So an overall total of 28.

Candidate B met the standards for all the elements but were not the highest scoring candidate so was placed on the reserve list for a period of 6 months.

Candidate C

• Scored 11 for behaviours, 6 for strengths (scored 1 on two of the three strengths), and 9 for experience

• So an overall total of 26.

Candidate C did not meet the standard set for behaviours and strengths so was not appointable.

17 | Success Profiles - Scoring Case Studies

High volume roles

Assessment flow

High volume roles (example only)

Start

Selection of successful candidate

Civil Service Judgement test

Assessment centre

Role Play

In-Tray ExerciseAptitude Test

Interview

Basic Application Form

The vacancy holder has chosen the below assessment flow for their high volume vacancies.

18 | Success Profiles - Scoring Case Studies

High volume roles

The sift uses the Verbal Reasoning Test (VRT) and Civil Service Judgement Test (CSJT).

These tests will act as the gateway to the next stage of selection.  

To proceed to the assessment centre, candidates must meet the set pass score for each test. As this demonstrates whether they have the baseline ability required for the role. Ability will not be tested any further.

Unsuccessful candidates are notified, feedback is not usually provided following tests.

Sift

19 | Success Profiles - Scoring Case Studies

High volume roles

The assessment centre comprises a blended interview, an e-tray exercise and a role-play.

Blended Interview

The vacancy holder has chosen three behaviours, four strengths and two experience questions, all relevant to the role.

In this instance the behaviours being assessed are: working together, managing a quality service and delivering at pace.

The vacancy holder has set the following minimum benchmarks:

• A strengths total score of 8 or more (the equivalent of an average of 2 on each strength).

• An experience total score or 8 or more (the equivalent of an average of 4 on each experience question).

Note that behaviours scores are not benchmarked within the interview, but at the overall assessment centre level.

 

In-tray Exercise

The following behaviours are assessed as part of the in-tray: delivering at pace, making effective decisions, managing a quality service.

 

Role Play

The following behaviours are assessed as part of the role play: working together and making effective decisions.

Assessment centre

20 | Success Profiles - Scoring Case Studies

As demonstrated in the table below, each behaviour was assessed twice to improve reliability.

The individual scores for each activity are then combined and the arithmetic average is calculated to get one overall score for each behaviour.

For example:

A candidate scores a 5 on working together in the role play and a 4 for the same behaviour in the interview. So their averaged total score for this behaviour is 4.5. For this campaign, the vacancy holder has decided to round all decimal scores to the nearest whole number so the final total score is 5.

The vacancy holder has set a minimum benchmark of 4 in each overall behaviour score.

High volume roles

Assessing the same behaviours multiple times

In-Tray Interview Role Play

Working Together

Managing a Quality Service

Delivering at Pace

Making Effective Decisions

21 | Success Profiles - Scoring Case Studies

High volume roles

The vacancy holder will create a merit list of the sum total score, i.e. of all the strengths, experience and overall behaviour scores to get an assessment centre total score.

There are 20 roles available. Appointment is made from the top down, only those who achieved the minimum benchmarks will be eligible for appointment.

The vacancy holder decided that where candidates achieve the same assessment centre total score and meet the minimum benchmarks, they should be considered for appointment in the following priority of scores: behaviours, strengths, experience.

Combing assessment centre scores

Candidate A

Scored 23 for behaviours (with no scores less than 4)

Scored 11 for strengths (above the benchmark total of 8)

Scored 11 on the experience questions (above the benchmark total of 8).

So their total score was 45.

Candidate A was top of the merit list and appointable to the role.

Candidate B

Scored 21 for behaviours (with no scores less than 4)

Scored 9 for strengths

Scored 11 on the experience questions.

So their total score was 41.

Candidate B was appointable.

Candidate C

Scored 19 for behaviours (with no scores less than 4)

Scored 10 for strengths

Scored 12 on the experience questions

So their total score was 41.

Candidate C was appointable but behind Candidate B in the merit list as they had a lower score for behaviours.

Candidate D

Scored 22 for behaviours (with one score of 3)

Scored 13 for strengths

Scored 7 for experience

So their total score was 42.

Candidate D was not appointable because they did not meet the minimum benchmarks for behaviours and experience.

Candidate E

Scored 17 for behaviours (with no scores less than 4)

Scored 8 for strengths

Scored 9 for experience

So their total score was 34.

Although candidate E met the required standards, they were placed on a reserve list because the vacancy holder had already appointed the top 20 candidates from the merit list.

22 | Success Profiles - Scoring Case Studies

Further materials

Additional resources

�� Success Profiles Overview One page summary of what the Success Profile Framework is

�� Civil Service Behaviours Core Guidance which is replacing the CS Competency Framework

�� Civil Service Strengths Dictionary Core Guidance providing an overview of using strengths

�� Civil Service Ability Core Guidance providing an overview of using ability

�� Technical Core Guidance providing an overview of using technical

�� Experience Core Guidance providing an overview of using experience

�� Recruitment process flowchart Overview of the recruitment process timeline from start to finish

�� Vacancy Holders Overview Summary of what Success Profiles means for a Vacancy Holder

�� Vacancy Holder Toolkit Detailed guide of end to end recruitment including how to recruit using the Success Profile Framework

�� Defining the job role and selection requirements Guidance on the beginning of the recruitment process

�� Civil Service Interview Methodology Guidance on interviewing the elements of the Success Profile

�� Civil Service Interview Scoring Tool Calculates candidates’ total score for blended interviews and enables creation of a merit order list 

�� Designing strength-based questions Guidance on how to create good strengths questions for interview