Daily Scratch Rating (DSR) PLAYING CONDITIONS

DailyScratchRating(DSR)

PLAYING CONDITIONSObjective: The system should reflect

variations in playing conditions (Principle 6).

Why is it important to assess variations in playing conditions?

- It is axiomatic amongst the designers of all handicap systems that a gross score must first be standardised before it can be used for handicapping.

- Standardisation enables us to meaningfully assess the value of a score, and to meaningfully compare it with all other scores. For example, is 78 a good score? In order to answer the question we need to know the difficulty of the course.

Why is it important to assess variations in playing conditions? Contd.

- The objective of a course rating system is to enable us to standardise scores.

- Course ratings are intended to precisely measure the difficulty a course presents to a golfer in the playing of their round.


- If the rating of a course is not a true reflection of the difficulty it presented to a golfer in the playing of a round, the player’s standardised score for that round will be inaccurate.

- If the standardised score is inaccurate, the player’s handicap will be distorted (ie if inputs are inaccurate, so must the output also be inaccurate).


- Does the difficulty of a course vary from day to day?

- We know this can happen.- Daily fluctuation can be caused by changed

hole placements, varying green speeds & green firmness, and changed weather.


- So how often will a rating be accurate if it is static and unable to vary in-line with changes in course difficulty from day to day?

- GA’s statisticians have conducted extensive analysis of the comprehensive repository of competition data in the GOLF Link database.


- The statisticians have looked for competitions where the weighted average net score varies by 1 stroke or more from the average net score that would be expected from the given composition of players.

- The weighting is for field size – for small fields the variance from the expected average net score will need to be greater than 1 stroke. [More detail on weighting for field size is contained later in the presentation.]


- Could it be that whole fields are just having a good or bad day?

- Our statisticians have factored into their analysis the standard deviation of player scores.

- They have assessed that the likelihood of large fields simply playing well, or playing poorly, is very low (and the weighting for field size accommodates the potential for this to happen with small fields).


- Our statisticians have assessed that actual course difficulty aligns with the static Scratch Rating on only 33% of occasions.

- Are we suggesting that a daily rating system can achieve 100% accuracy?

- No, but 90-95% is a substantial improvement on 33%.

- But if a rating is sometimes a little high and sometimes a little low, does it all not just ‘even out’ under a static Scratch Rating System?

- IMPACT ON NATIONAL TRENDS & AVERAGES. DSR will make handicaps in general less volatile.

- Why? What is currently happening is that good scores on easy days are resulting in larger downward movements than will happen under DSR (because the course rating stays artificially high under the static Scratch Rating system). This causes a player’s handicap to dip temporarily, it becomes lower than it should be and the player struggles to play to the new value, before drifting out again. This is accentuated by scores on hard days being evaluated against ratings that are artificially low. A further impact of decreased volatility is that across Australia, the average handicap will increase very slightly.


- INDIVIDUAL IMPACT. Whilst the impact on national averages is important, in this case it only paints a small part of the picture. Examples of the impact that not having a DSR has on individual golfers are as follows:• Bill always plays in the morning when it’s calm, and Tony always plays in the afternoon when it’s

windy. If there’s no DSR, both players will be having their scores evaluated against incorrect course ratings (one will be too high and the other too low). The impact of the two cases net off against each other to give an average that looks perfect. However, this doesn’t help Bill whose handicap will always be too low, and Tony won’t be happy either because he’s trying to improve his handicap and yet it’s always kept artificially high!


• Jenny’s underlying ability doesn’t change from summer to winter but because of the heavier winter conditions, she doesn’t score as well. Without DSR, Jenny’s handicap increases in winter and then decreases again in summer. Jenny doesn’t play much at the start of summer, so when she does start to play regularly at the height of summer her handicap is artificially high compared to those players whose handicaps have already adjusted to the easier conditions.

• John’s handicap is only based on an average of 8 scores. As a result, it doesn’t take too many inaccurate ratings to distort his handicap at any given point in time. Distortion can be caused either by ‘top 8’ scores that were actually returned against artificially high ratings, or by scores in the worst 12 that would have been in the top 8 if they had been assessed against ratings more reflective of the true course difficulty than the Scratch Rating. Whilst the ups and downs may even out over the course of a substantial number of rounds, it seems hopeful to expect they will even out at any given point in time.

• Fiona has been playing in the Melbourne winter. Without DSR, her handicap will increase because of the harder conditions. She visits her friend in tropical Cairns and plays golf. Coming out of the Melbourne winter with her artificially high handicap, Fiona wins the Cairns competition.


- But if a golfer plays on a very difficult day, won’t a poor score fall into the worst 12 of their most recent 20 scores? And if so, isn’t the need to adjust the course rating negated? If this is correct, why do we need DSR?• Firstly, handicap golfers are just as capable of playing well on difficult days as they are on easy

days. And whilst the score may be worse on a hard day, it is not necessarily because the quality of the round is worse. In a proportionate number of cases it is instead because the difficulty of the course was increased. As a result, a round should not be dismissed just because the conditions were difficult and the score itself is commensurately worse.


• Secondly, a handicap system should be flexible enough to accurately assess the standard of rounds played by players under varying conditions.

• Thirdly, a score may be assessed as poor when compared against the static Scratch Rating, but it may be in a player’s top 8 if it is compared against the true rating.

• Fourthly, if a static Scratch Rating is being used, a score from a difficult day that is in a player’s top 8 will be assessed as being worse than it should be as it is being compared against an artificially low rating. This will make the handicap higher than it should be.

• Fifthly, if a static Scratch Rating is being used, a score from an easier day that is in a player’s top 8 will be assessed as being better than it should be as it is being compared against an artificially high rating. This will make the handicap lower than it should be.

• Sixthly, it is possible that the swings and roundabouts of the above may cancel each other out. However, a sample of 8 is not large and whilst the ups and downs may even out over the course of a substantial number of rounds, it seems hopeful to expect they will even out at any given point in time.


• But how much does course difficulty really vary from day to day and with the seasons? Below is a graph of DSRs that have been calculated from scores at a typical Australian golf club.


• The graph shows clear fluctuations in course difficulty from day to day.

• The graph also shows clear seasonal fluctuations in course difficulty.

• But how much of an impact does all of this have on the calculation of handicaps?

• One measure is to compare the proportion of Anchored players before and after the introduction of DSR by using the data from our typical Australian club.


- So we can see that providing ratings that align with the actual difficulty presented to the golfer in the playing of their round does make a material difference to handicaps.

- (Note: Approx 70% of the reduced Anchorage impact is caused by DSR (the remainder is caused by a mix of Slope and the Stableford Handicapping Adjustment).)


- DSR inputs are:•Gender•Type of competition (Stroke, S’ford, Par)•Field size•Average handicap of field•Average net score of field

- The suite of DSR algorithms is provided at Appendix A.

Exactly what is being taken into consideration when making the DSR calculation?

The use of the Mean Net Score for a given Competition• The Mean Net Score expected from a

given competition (under normal conditions) is able to be well determined by establishing the average handicap of the field, and type of competition (Par, Stableford, or Stroke), and whether the competition was for Men or Women.

Exactly what is being taken into consideration when making the DSR calculation? Contd.

• This expected Mean Net Score will be less than the net score players would need to achieve in order to play to their handicaps. Let’s call this difference the Normal Deduction.

• It is related to handicaps as follows: [see next slide]


• The difference between the Actual Mean Net Score and the Expected Mean Net Score is a first approximation to an adjustment that should be made for handicap purposes to account for Course Conditions.

• It will be positive on an easy day, and negative on a difficult day.


• However, it is necessary to give the value above a weight, which accounts for field size, and which is also dependent on the average handicap of the players in the field. This weighting factor is close to 100% for large field sizes, but as low as 25% for a field size of one.

• The following table gives some examples: [see next slide]

The Weighting Factor and how the system can work even for very small fields.

Weight Men’s Stableford

Women’s Stroke

Average Handicap of

the field15 25

Field Size20% 2 4

50% 10 16

80% 40 60

90% 90 140

• Only Modest Field Sizes are needed to get a good result.

• A DSR can be calculated from a dataset of 1 player.

The Weighting Factor and how the system can work even for very small fields. Contd.

• So we ascribe a greater weight to the average net score of a large field than we do to the average net score of a small field.

• But how have we related field size to confidence? Our statisticians have used Bayes’ Theorem.


• From Bayes’ Theorem is derived the formula for weighting the Daily Estimate compared to the measured Scratch Rating.

• Bayes proved that it is optimal. There can be no better estimate of the Daily Condition than the Bayes estimate.

• More information on Bayes’ Theorem is provided at Appendix B.


• There is no hard and fast rule.• GA’s direction to its clubs is that scores for

a given day are to be processed through GOLF Link as soon as is practicable (irrespective of whether a daily rating system is in operation).

• For some clubs this will be on the day of play (preferable).

• For other clubs this will be within a week of the day of play (acceptable).

When is the cut-off for scores to be submitted in order for the DSR to be calculated for a given day?

- GOLF Link will calculate all Differentials against the DSR, NOT the Scratch Rating.

- DSR will not provide any administrative impost on clubs.

- The club will enter the scores of players into GOLF Link (in the same way it would if DSR was not in operation), press the button, and GOLF Link will perform all the necessary calculations.

How will the system work in practice in terms of updating players’ handicaps on GOLF Link?

- DSRs will be calculated for competition scores AND extra day scores.

- All extra day scores returned at a course on a given day will be processed through GOLF Link as a single Batch (in the same way that competition scores will be processed).

- If an extra day score is not processed in the appropriate Batch, it will still be eligible to be used for handicapping.

What is the implementation protocol for competitive play and for extra day scores?

- EGA and CONGU both operate a daily rating component.

- EGA and CONGU both treat as indicative the proportion of players in each handicap grade to return a good score.

- DSR uses all scores (ie good AND poor).- The analysis performed by GA’s statisticians

led them to the conclusion that there is material value in using good scores AND poor scores.

Comparison with other methods used worldwide.

- The CONGU component operates by adjusting the scratch rating against which a player’s score is compared (as will DSR).

- The EGA component operates ostensibly by expanding or contracting the buffer zones (however it effectively operates by adding or subtracting a value from a player’s net score so as to account for a variation in conditions).

Comparison with other methods used worldwide. Contd.

- CONGU and EGA prefer a greater degree of statistical certainty than GA in order for the daily rating to vary from the Scratch Rating.

- After consideration, GA took the view that a more dynamic approach is more likely to yield outputs that will align with the golfer’s view of the difficulty of the course on the day.

Comparison with other methods used worldwide. Contd.

- The concepts underpinning the EGA & CONGU daily rating components are all readily understood by golfers.

- GA firmly expects that golfers will readily understand the concepts underpinning DSR.

How easy is DSR to understand?

- The mathematics underpinning these methods are more esoteric.

- GA believes there are two key determinants of whether a daily rating system is accepted by golfers. Firstly, the degree to which it is conceptually understood. Secondly, its ability to produce intuitive outputs.

- Golfers want ratings that align with their view of the difficulty of the course on the day.

How easy is DSR to understand? Contd.

Is there a simpler daily rating solution?

• CCR • DSR

- In Australia we previously operated a simple mathematical model (CCR) where the 12½% net score became the daily rating.

- CCR was necessarily simplistic (as it operated prior to the age of universal computer use) and it was statistically inefficient.

Is there a simpler daily rating solution? Contd.

- The mathematics of CCR were readily understood by golfers and administrators.

- However the commonly-held view of golfers (and administrators) was that CCRs were often more a reflection of the quality of the field than they were of the difficulty of the course (eg veterans’ fields would produce higher CCRs than regular handicap fields).


- CCR did not work so well for clubs in regional areas or for women’s fields, ie in cases where fields were frequently small.

- A further problem with women’s fields was that they typically exhibited materially higher average handicaps than do men’s fields.

- Our statisticians are supremely confident that DSR will deliver materially better outcomes for our golfers than did CCR.


- DSR was developed by the Daily Rating Statistical Review Group – comprised of experienced administrators and highly accomplished statisticians.

- It was laboratory tested against millions of rounds.

- It has also been trialled in a live environment in a diverse selection of Australian clubs.

Results of pilot study.

- There have been three phases to the live trial.

- The first two phases were held across different seasons.

- We are currently engaged in the 3rd and final phase.

- The live trial has involved a diverse selection of 20 clubs (regional and metropolitan, large and small).

Results of pilot study. Contd.

- DSR values have been calculated each day and emailed to officials at each trial club.

- These officials were asked to comment on the degree to which the calculated DSRs have aligned with their intuitive view of the difficulty of the course on the day.

- The feedback from the live trial clubs has been very positive and extremely encouraging.

Results of pilot study. Contd.

Normal Deduction: ND = mH+bWhere H is the average handicap of the field, m and b are taken from the table below, representing the slope and intercept of the straight line of best fit. R is the correlation of this fit.

Weighting Factor = n/((m'H+b')2/CSD2+n)Where n is the field size, H is the average handicap of the field, CSD (the Course Standard Deviation) is estimated at 1.5, and m’ and b’ are taken from the table below representing the slope and intercept of the straight line of best fit for the empirically derived standard deviation of the Normal Deduction.

CPA (Course Parameter Adjustment) = Prior CPA + WCA x 0.02 x (0.7 for Men, 0.5 for Women)Where WCA (Weighted Condition Adjustment) is SR (Scratch Rating) minus DSR. Putting DSR into a single formula:DSR = SR - (S- (36+Par-SR+CPA-mH-b))n/((m'H+b')2/CSD 2+n)Where S is the actual average Stableford points scored in the competition, and the other symbols have their meaning as above.

APPENDIX A – The DSR Algorithms

Men Women m b R m b RPar P (0.052) (2.777) 0.943 (0.062) (2.514) 0.959 Stblfd S (0.111) (3.498) 0.972 (0.117) (3.338) 0.953 Stroke K (0.124) (4.372) 0.973 (0.146) (3.939) 0.984

Men Women m' b' R m' b' RPar P 0.023 3.194 0.856 0.027 3.040 0.760Stblfd S 0.057 3.841 0.829 0.057 3.775 0.917Stroke K 0.083 3.993 0.956 0.081 3.889 0.938

BAYES’ THEOREMBayes’ Theorem starts with an estimate of some value you’d like to ascertain; then it assumes that you can take a sample which gives a further but limited estimate of the value; from these pieces of data the formula gives a final estimate which combines the initial estimate with the sample result. It effectively gives the amount of weight that can be given to the sample compared to the initial estimate, this in turn being based on the sample size and the inherent variability of the estimate and the sample.

EXPLANATORY NOTES ON BAYES’ THEOREMSuppose there is a true answer to a question.

Let’s take two examples to demonstrate:1. What is the proportion of voters that will vote for the Republicans in the next election? 2. What is the true scratch rating of a golf course on which I will play tomorrow? Due to conditions,

the course may play differently to the static scratch rating which may be considered as an average or normal rating.

In the case of the voters, every four years the question is answered. But along the way, there are many polls of voters’ intentions. Eventually we will know the truth.

In the case of the course, we will never know the true answer. We could only know if all the golfers in the land actually played on it tomorrow, clearly impossible.

APPENDIX B – Bayes’ Theorem

EXPLANATORY NOTES ON BAYES’ THEOREM (contd.)When we take a poll of voters, the smaller the sample, the less accurate the poll result. The greater the sample, the greater the accuracy, until on voting day, we get complete accuracy.

When we take a sample of golfers’ scores tomorrow, we will get a “poll”, and by comparing their average score to what would be expected for a field of that composition, we will get an estimate of whether the course played easier, harder, or no different to normal. The more golfers in the sample, the better the poll result. If the average net score is higher than expected, then this is a poll result suggesting that the course played harder.

Bayes however, was able to build in the concept of having an idea of what might be expected as an answer before you take the sample. In the case of voters, it might be voting patterns at the last election. And you can measure how variable this has been over the years. In the case of a daily scratch rating it might be the static scratch rating ascribed to the course.

He then said we will only depart from the first idea if there is enough evidence to prove that there is indeed a better answer. So, in the case of voters in an area which in the past has voted 55% Republican, finding that only 30% of a sample of 100 voters intends to vote Republican at the next election, may not be convincing. The same result with a sample of 10,000 voters may be very significant. Bayes’ theorem allows the optimal weight to be ascribed to the poll result, and to determine if this result is significant.


EXPLANATORY NOTES ON BAYES’ THEOREM (contd.)In the case of the daily scratch rating, application of the theorem allows us to come up with an estimate of the daily rating, but only lets it deviate from the static rating if, firstly, the average scores indicate a deviation and, secondly, the number of players making up the sample is sufficient to make a different result the most likely on the balance of probabilities. Of course the poll of golfer scores may indicate that there is no case for deviation at all, either because the sample is too small, or because the average of their scores was quite close to what would be expected. But if there is a deviation, after the Bayes weighting is applied, we can be certain that this estimate of the daily scratch rating is the optimal value available based on the evidence. It is more likely to be a reflection of the true daily rating on that day than the static rating.

One might say that Bayes’ theorem allows us to anchor the new answer (the daily scratch rating) firmly to the original idea (the static scratch rating), and, conservatively, only allows deviation when the evidence is overwhelming.


QUESTIONS

Daily Scratch Rating (DSR) PLAYING CONDITIONS

Documents

Transcript of Daily Scratch Rating (DSR) PLAYING CONDITIONS