PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary
description
Transcript of PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary
![Page 1: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/1.jpg)
PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics
Conference(s) Summary
Bob Cousins, UCLA and CMS
29 June 2007
![Page 2: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/2.jpg)
TOPICAL TALKS
![Page 3: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/3.jpg)
174 pages!Now that you have p values, do you want to combine more than one? See RC annotated bibliography, http://arxiv.org/abs/0705.2209
![Page 4: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/4.jpg)
Historically, HEP seems to have under-utilized this “direct” calculational way of well-approximating a M.L. estimate. Is our computing now so advanced that we have less use for it?
Jim Linnemann and Andrew Smith, also citing Roger Barlow (1987!) and F. Tkachov.
![Page 5: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/5.jpg)
(I was not able to study.)
Time will tell.
![Page 6: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/6.jpg)
![Page 7: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/7.jpg)
![Page 8: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/8.jpg)
Physicist’s View: Jim Linnemann
![Page 9: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/9.jpg)
The “Other” PDF’s
• Illuminating talk by– Robert Thorne: Uncertainties in parton
related quantities.
• Tough business: uncertainties on functions!
• PDF’s matter: CDF compositeness, high-pT search reach.
![Page 10: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/10.jpg)
However…[see talk]
![Page 11: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/11.jpg)
![Page 12: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/12.jpg)
REV. THOMAS
![Page 13: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/13.jpg)
Durham Phystat 2002:
![Page 14: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/14.jpg)
Bayes' rule is satisfying, convincing, and fun touse. But using Baye's rule does not make one aBayesian; always using it does, and that's where dif-culties begin.
![Page 15: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/15.jpg)
Sir David Cox
![Page 16: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/16.jpg)
Sir David’s list, as augmented by HEP practice
Typically we are saved because likelihood overwhelms prior, or because Poisson upper limit (but not lower) has excellent frequentist properties. But in high dimensions, one is almost certainly adding undesired “information”.
• Priors flat in arbitrary variables.
Six
![Page 17: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/17.jpg)
![Page 18: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/18.jpg)
Perhaps the most important general lesson is that the facile use of what appear to be uninformative priors is a dangerous practice in high dimensions.
![Page 19: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/19.jpg)
![Page 20: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/20.jpg)
Good Practice
• First thing to do with the likelihood is to obtain the profile likelihood answers (MINUIT MINOS)
• First thing to do with multidimensional priors is to get all your answers using constant likelihood function (no information from data). Jacobians may become evident.
Now ready to combine L and Priors!
• With the various priors, get Bayesian answer. Beware if it is different from profile likelihood. (One case where it might be better is if L has spike which is very narrow for any reasonable prior.)
![Page 21: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/21.jpg)
Eilam Gross
![Page 22: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/22.jpg)
Eilam Gross
![Page 23: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/23.jpg)
![Page 24: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/24.jpg)
Jim Berger:
![Page 25: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/25.jpg)
![Page 26: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/26.jpg)
Again, different individuals may react differently, and the sensitivity analysis for the effect of the prior on the posterior is the analysis of the scientific community, so that the answer should now be an interval of posterior values which may be reasonably held by individual scientists.
![Page 27: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/27.jpg)
• “Sensitivity Analysis is at the heart of scientific Bayesianism”– How skeptical would the community as a whole
have to be in order not to be convinced.– What prior gives P(hypothesis) > 0.5– What prior gives P(hypothesis) > 0.99, etc
From Michael Goldstein’s hand-written slides at Durham:
![Page 28: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/28.jpg)
LHC
![Page 29: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/29.jpg)
At LHC, we will benefit tremendously from work at LEP, Tevatron, B factories...
Wade Fisher, Experiences fromTevatron Searches
...but the speakers agree there is room for improvement.And there is more at Tevatron: 31 hits for Spires search FIND C PHRVA,D57,3873 AND COLLABORATION CDF
(See also excellent talk on CLS by Alex Read at Durham 2002 Workshop)
![Page 30: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/30.jpg)
Beautiful talk: Required reading for PhyStat 2009 (and in the meantime for all CMS and ATLAS physicists)!
![Page 31: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/31.jpg)
![Page 32: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/32.jpg)
![Page 33: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/33.jpg)
Eilam Gross, on l.h.s. quoting Fred at 2000 CLW
![Page 34: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/34.jpg)
![Page 35: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/35.jpg)
OUTLOOK
![Page 36: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/36.jpg)
ParkesA warning from Chris Parkes at Durham re LEP:
![Page 37: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/37.jpg)
We seem to be off to a much better start with LHC statistics!
• We benefit from work at LEP, Tevatron, B factories, and many other experiments.
• We understand necessity to compare and combining results in order to maximize return on society’s huge investment in us.
• We seem to be getting over the hump of foundational wars and becoming pragmatists (and are getting some residual skirmishes out of the way before we have data).
• We are excited about potential synergies among LHC experiments, ROOT team, other developers.
![Page 38: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/38.jpg)
![Page 39: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/39.jpg)
A Realistic Goal!• Have in place tools to allow computation of results using a
variety of recipes, for problems up to intermediate complexity:– Profile likelihood (Minuit MINOS)– Bayesian with analysis of sensitivity to prior– Frequentist construction with approximate treatment of nuisance
parameters
– Other “favorites” such as CLS
• The community can then demand that a result shown with one’s preferred method also be shown with the other methods, and sampling properties studied.
• When the methods agree, all we are in asymptopic nirvana.• When the methods disagree, we learn something! E.g.:
– Bayesian methods can have poor frequentist properties– Frequentist methods can badly violate likelihood principle
![Page 40: PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics Conference(s) Summary](https://reader036.fdocuments.in/reader036/viewer/2022062500/568150f7550346895dbf1500/html5/thumbnails/40.jpg)
THANKS!
• On behalf of all of us, to the organizers.
• To all of you, for making this a stimulating workshop.
• To the statisticians, for your help and good-natured tolerance as we sometimes abused your discipline.