Evaluating Bids & Tenders

©ALL RIGHTS RESERVED - Confidential

Spend Matters UK/Europe

Evaluating Bids and Tenders

Peter Smith May 20th 2014




Agenda

1. Introduction

2. The purpose of evaluation

3. Key steps in the process

4. Evaluation criteria and weightings

5. Evaluating and scoring

6. Wrap up




Part 1 - INTRODUCTION

The Real World Sourcing Series 2014 is a series of sessions covering hot topics for procurement professionals.

The Real World Sourcing Series is promoted and supported by Tejari, and developed by Peter Smith (Spend Matters) and Guy Allen (Real World Sourcing Ltd.)

Peter Smith started his procurement career with Mars Confectionery, then was CPO for Dun & Bradstreet Europe, the Department of Social Security and the NatWest Group. He is now a consultant, author, non-executive director and editor of the Spend Matters website.

He was President of CIPS in 2002/3, is an senior adviser to the UK’s National Audit Office and co-wrote “Buying Professional Services”, published by the Economist Books in 2010.




Part 2 – THE PURPOSE OF EVALUATION

• Simply, choosing the suppliers that our organisation works with (or assisting colleagues to do so) must be a fundamental part of procurement’s role.

• Given that suppliers contribute so much to the organisation's achievement of corporate goals, choosing the right ones (the best ones) must be vital.





• Yet many organisations see tender evaluation as a routine, mundane, unimportant exercise

• It is not – it is complex, challenging and one of the fields where procurement should be able to show some real professional expertise





Why is it worth doing this properly?

• We want to make sure we choose the ‘best’ supplier or suppliers to meet our needs.

• Providing evidence to the market that it’s worth time and effort trying to be our suppliers.

• Guarding against fraud and corruption – internal and external.





• A bid or tender is a proposal for the supplier to deliver goods or provide services to the buying organization

• So we want the ‘best’ proposal in terms of value for money (measured ultimately by contribution to the buying organisation's goals)

We’ll come back to that later...





• BUT... within that value for money argument, there are two further elements to consider over and above the usual suspects.

• Particularly true for the more complex contracts (generally but not always services).

• What are they?





1. The CONFIDENCE we have in the supplier's ability to provide what they say they will – that it isn’t just a well written proposal.

2. The ability to handle CHANGE that we may require through the contractual period. A great proposal is useless if it can’t be delivered – and lack of change capability can lead to major issues through the contract period.




Part 3 – KEY PROCESS STEPS

Decide how many stages in the evaluation process

1. Single stage – one tender

2. Two stages – a pre-qualification and tender

3. Multiple stage – competitive dialogue in the public sector, iterative negotiations in the private sector





Single stage process – Pros and Cons

• Simplicity and speed of execution

• No pre-qualification - so may look friendlier to small suppliers etc.

BUT

• Look at all aspects at the same time

• Can end up with high numbers of bidders (if ‘open’ approach)

• Complex process – or we end up in effect running it as a two stage process (evaluating part of the proposal first, like a PQQ)





Two stage – Pros and Cons

• Enables a down- select and rules out ‘stupid’ bids

• Tender from just a manageable number of serious bidders

• Focus on capability before get into detailed solutions

BUT

• Can be lengthy

• Easy to rule out good suppliers before they get a chance to propose solutions





Multiple or Iterative – Pros and Cons

• Greater flexibility

• Can refine requirements as you proceed

• Explore supplier ideas and inputs

BUT

• Potentially complex, and lengthy process

• Expensive for bidders and buyers

• Hard to maintain a level playing field – potential for corruption





The PQQ usually has 3 purposes, although it may sometimes just address the first and second:

1. To collect important information about the supplier: registration.

2. To establish whether the supplier has the basic capability, capacity and financial strength to carry out the contract: qualification.

3. To compare bidding organisations so a sensible number can be taken forward to tender or final selection: ranking.





PQQ good practice:

• Be open and transparent on evaluation processes; we recommend

publication of full process detail

• Use thresholds sensibly and in a non-discriminatory manner e.g:

o only reject firms on size of turnover where truly significant (and

publish the thresholds),

o ask for reasonable insurance levels for the risk involved

• Accept references and examples of previous work that are relevant but

not exactly the same as your work

• Only ask questions where you are going to use the responses!





PQQ good practice:

• How should we evaluate the PQQ responses?

• What are the three main ways we can treat each response

within the PQQ?

• When should we use each?




Part 4 –EVALUATION CRITERIA AND WEIGHTINGS

• Evaluation criteria help us decide which tender / supplier should win the contract

• Weightings indicate the relative importance of those criteria

• Everything should flow from the strategic objectives of the purchase and contract




Part 4 – EVALUATION CRITERIA AND WEIGHTINGS

Strategic objectives of

organisation

Objectives of procurement /

what is supplied

What we want from contract

/ supplier

Evaluation criteria





Typical examples:

• Experience, capacity, capability (often PQQ stage in government sector)

• Technical solution proposed / how product meets needs

• Quality systems and processes

• Flexibility and approach to change

• Collaborative working / account management

• Management issues – invoicing, security, etc.





• Start with high-level criteria and then work down through sub-criteria

• Look for ‘independent’ (in a mathematical sense) criteria to avoid double counting

• Avoid overly small criteria, group where appropriate . Nothing worth less than 2% is worth marking....

• Some factors are critical but not suitable for high weighting (or any weighting at all) – consider threshold approach and fail supplier if they don’t reach it

• E.g. ‘data security’ – essential but may be binary




Part 5 – EVALUATING AND SCORING

• There is a whole set of ‘project management’ issues here that we don’t have time to go into fully

• Who is going to do the evaluation?

• Ensuring propriety – no one person should be able to determine the outcome

• How will they be organised?

• How will the results be gathered and audit trail maintained?

• Timing? Moderation? Sign-off?

• Use of clarification, interviews etc?





• In terms of marking, we can’t distinguish between more than seven different levels in marking scales

• We need to get enough discrimination whilst being able to explain / justify marks

• There is a philosophical debate to be had around marks for providing a more than fully acceptable option

• Some organizations use non-linear scale – e.g. 10, 7, 4, 3, 1

• Others going for more of an all or nothing approach

• Acceptable to have a threshold for critical criteria – score too low and bid is rejected





Typical scoring:

5 Meets all requirements and offers some added value

4 Meets all requirements

3 Generally meets critical requirements with some minor issues

2 Meets some requirements, but a few major gaps or issues

1 Fails to meet overall requirements; serious concerns

0 Does not meet requirements / fail to answer / irrelevant





Mark Weighting Score

Technical solution 4/5 60% 48

Quality systems 5/5 15% 15

Account management 3/5 10% 6

Innovation and change 4/5 5% 4

Management processes 2/5 10% 4

Total 77%





• We need to bring together (in most methodologies) cost (price / whole life cost) and non-cost factors.

• How do we do that?

• In the olden days, and today in many private sector firms, we draw a chart like the next slide. Then the evaluators just talk!

• However, that is not enough in many cases – e.g. if you want to have a proper audit trail and a basis for considered judgement and decision making.




NOT

ACCEPTABLE B

D

A

C

Non-price score

100

Price / cost

(£M)

HIGH

10

12

40

HIGH

Quality Price

A 54 10

B 84 12

C 98 13

D 66 13

Part 5 - EVALUATING AND SCORING

WHO WINS?

13M





• We have to devise a method to convert cost into a score so

we can bring that together with the non-price scores, and

decide on the weightings of cost and non-cost factors

• We are in effect using the concept of ‘utility’ – critical to

decision theory

• For example, a lower cost of AED 1 million in one bid is

compared to the ‘utility’ we receive from slightly higher

quality, service or other factors in another bid

• But there are many ways to do this...





• Our maximum budget is AED 100,000

• Our market research tells us that this work is probably impossible to do for less than AED 50,000

• Costs from tenders are:

Supplier A AED 100,000

Supplier B AED 80,000

Supplier C AED 70,000

• How would you score those bids?





Alternative scoring methods

• Variation i.e. Lowest Price / Bidder’s price (x100)

• Average price as a baseline (scores 50 perhaps) then %

difference from baseline

• Highest or lowest price as a baseline and percentage difference

• Ranking (lowest cost 100, highest 0)

• Pre-determined scoring system (e.g. We decide in advance

that AED 50000 will score 100 points, and AED 150000 zero)





Alternative scoring methods - results

C

Variation Average benchmark

Lowest benchmark

Ranking Pre-determined

A 70 33.33 57 0 50

B 87.5 53.33 86 50 70

C 100 63.33 100 100 80

Demonstrates how different methodologies drive different scores





Mark Weighting Score

Non-cost factors 77/100 60% 46.2

Cost factors 80/100 40% 32

Overall Total 78.2




Part 5 – EVALUATING AND SCORING - Issues

• Some methods can be skewed by unnaturally low bids

• Is it fair if one supplier’s score depends on others? That is not how we mark non-price factors which are usually absolute, not relative?

• How does the cost scale look compared to the non-cost scale? Should it be comparable?

• Some scoring methods aren't linear and therefore arguably illogical including the Lowest Price / Bidder’s price method which is probably the most widely used method in (UK) public sector

• Most logical is probably the pre-determined scoring system




Part 5 - EVALUATING AND SCORING

Recommendations

• Model what you’re doing before you decide on which option to use. Compare the effect of cost differences with the non-cost factors

• Look at scale not just weighting e.g. Price may have a high weighting, but if your scheme means all the marks come out between say, 80 and 90, it hasn’t really got a high weighting!





Bringing CHANGE and CONFIDENCE into the picture

• CHANGE – not about what the supplier has done in the past, or their specific ‘solution’ today

• It is about how they will respond to change in the future and pro-actively look to offer change and innovation in the future

• Tough to assess – but can be important. We can ask questions in the tender about how the supplier has handled change in the past, where they have formal processes, etc.





CONFIDENCE

• However good the proposal looks, how do we know the supplier will actually deliver what they offer?

• HOW CAN WE DO THIS?

• There is a world of difference between a well-written and attractive proposal and deliver

• Have they done this before? How can they give us confidence that they have the experience, skills, resources, inclination to do what they say they will?




THREE KEY TAKE-AWAYS

1. Think carefully about the evaluation process – it plays a major

part in determining which supplier will win your contract

2. Follow a logical flow through strategic objectives, key success

factors, to evaluation criteria and weightings to ensure the

results meet your fundamental aims and objectives

3. Model your specific scoring system, particularly scoring cost,

to make sure it does not have unintended consequences




Real World Sourcing Scholarship

• AED 10,000 towards any procurement based training

• You will receive an email with your Tejari Education Network login details

• Complete the online certification programme

• Answer as many questions right as possible

• The top 10 scorers will be invited to the Awards dinner

• The winner will be announced at the dinner!




QUESTIONS

1. Have you seen any examples where an evaluation process

went wrong or came up with an unexpected / unacceptable /

unintended answer?

2. Why did that happen?

3. What did you learn from that?




THANK YOU!

Please feel free to contact me on

[email protected]

mailto:[email protected]

Evaluating Bids & Tenders

Education

Transcript of Evaluating Bids & Tenders