Resurrecting Laplace's Demon: The Case for Deterministic ... · God was playing dice, in...
Transcript of Resurrecting Laplace's Demon: The Case for Deterministic ... · God was playing dice, in...
Resurrecting Laplace's Demon: The Case for Deterministic Models
Edward A. Lee Robert S. Pepper Distinguished Professor UC Berkeley Invited Talk: Synchron December 8, 2016 Bamberg, Germany
Notjustinforma.ontechnology:• Cyber+Physical• Computa/on+Dynamics• Security+SafetyProper.es:• Highlydynamic• Safetycri/cal• Uncertainenvironment• Physicallydistributed• Sporadicconnec/vity• Resourceconstrained
Doesitmakesensetotalkaboutdeterminis7cmodelsforsuchsystems?
Automotive
Context:Cyber-PhysicalSystemsApar/cularlychallengingcasefordeterminism
Biomedical
Military
Energy
Manufacturing
Avionics
Buildings
2 Lee, Berkeley
Modelsvs.Reality
Inthisexample,themodelingframeworkiscalculusandNewton’slaws.Fidelityishowwellthemodelanditstargetmatch
Lee, Berkeley 3
The model
The target (the thing being modeled).
Solomon Wolf Golomb
You will never strike oil by drilling through the map!
Lee, Berkeley 4
Engineers often confuse the model with its target
But this does not in any way diminish the value of a map!
Determinacy
Someofthemostvaluablemodelsaredeterminis7c.
Amodelisdeterminis7cif,giventheini/alstateandtheinputs,themodeldefinesexactlyonebehavior.
Determinis/cmodelshaveprovenextremelyvaluableinthepast.
Lee, Berkeley 5
Laplace’sDemon
“Wemayregardthepresentstateoftheuniverseastheeffectofitspastandthecauseofitsfuture.Anintellectwhichatacertainmomentwouldknowallforcesthatsetnatureinmo/on,andallposi/onsofallitemsofwhichnatureiscomposed,ifthisintellectwerealsovastenoughtosubmitthesedatatoanalysis,itwouldembraceinasingleformulathemovementsofthegreatestbodiesoftheuniverseandthoseofthe/niestatom;forsuchanintellectnothingwouldbeuncertainandthefuturejustlikethepastwouldbepresentbeforeitseyes.”— PierreSimonLaplace
Lee, Berkeley 6
Pierre-Simon Laplace (1749–1827). Portrait by Joan-Baptiste Paulin Guérin, 1838
Didquantummechanicsdashthishope?
“Atfirst,itseemedthatthesehopesforacompletedeterminismwouldbedashedbythediscoveryearlyinthe20thcenturythateventslikethedecayofradioac/veatomsseemedtotakeplaceatrandom.ItwasasifGodwasplayingdice,inEinstein’sphrase.Butsciencesnatchedvictoryfromthejawsofdefeatbymovingthegoalpostsandredefiningwhatismeantbyacompleteknowledgeoftheuniverse.”(StephenHawking,2002)
Lee, Berkeley 7
Nevertheless,Laplace’sDemoncannotexist.
In2008,DavidWolpert,thenatNASA,nowattheSantaFeResearchIns/tute,usedCantor’sdiagonaliza/ontechniquetoprovethatLaplace’sdemoncannotexist.Hisproofreliesontheobserva/onthatsuchademon,wereittoexist,wouldhavetoexistintheveryphysicalworldthatitpredicts.
Lee, Berkeley 8
David Wolpert
The Koptez Principle
Many properties that we assert about systems (determinism, timeliness, reliability) are in fact not properties of the system, but rather properties of a model of the system. If we accept this, then it makes no sense to talk about whether the physical world is deterministic. It only makes sense to talk about whether models of the physical world are deterministic.
Hermann Kopetz Professor (Emeritus) TU Vienna
Theques/onswitchesfromwhetheramodelisTruetowhetheritisUseful
“Essen/ally,allmodelsarewrong,butsomeareuseful.”
Box,G.E.P.andN.R.Draper,1987:EmpiricalModel-BuildingandResponseSurfaces.WileySeriesinProbabilityandSta/s/cs,Wiley.
Lee, Berkeley 10
Physicistscon/nuetodebatewhethertheworldisdeterminis/c
Determinismisapropertyofmodels,notapropertyofthesystemstheymodel.
Lee, Berkeley 11
Deterministic model
Deterministic system?
Schema/cofasimpleCyber-PhysicalSystem
Lee, Berkeley 12
What kinds of models should we use? Let’s look at the most successful kinds of models from the cyber and the physical worlds.
SocwareisaModel
PhysicalSystem Model
Single-threadedimpera7veprogramsaredeterminis7cmodels
Lee, Berkeley 13
Considersingle-threadedimpera/veprograms
Thetargetofthemodeliselectronssloshingaroundinsilicon.Ittakes/me,consumesenergy,andfailsifdroppedintheocean,noneofwhichareproper/esofthemodel.
This program defines exactly one behavior, given the input x. Note that the modeling framework (the C language, in this case) defines “behavior” and “input.”
Lee, Berkeley 14
Socwarereliesonanotherdeterminis/cmodelthatabstractsthehardware
PhysicalSystem Model
Instruction Set Architectures (ISAs) are deterministic models
Lee, Berkeley 15
Image: Wikimedia Commons Waterman, et al., The RISC-V Instruction Set Manual, UCB/EECS-2011-62, 2011
…whichreliesonyetanotherdeterminis/cmodel
PhysicalSystem Model
Synchronousdigitallogicisadeterminis7cmodel
Lee, Berkeley 16
Determinis/cModelsforthePhysicalSideofCPS
PhysicalSystem Model
Signal Signal
Differen7alEqua7onsaredeterminis7cmodels
Lee, Berkeley 17
Image: Wikimedia Commons
Signal Signal
18 Image: Wikimedia Commons Lee, Berkeley
AmajorproblemforCPS:combina/onsofdeterminis/cmodelsarenondeterminis/c
Correct execution of a program in all widely used programming languages, and correct delivery of a network message in all general-purpose networks has nothing to do with how long it takes to do anything.
Programmers have to step outside the programming abstractions to specify timing behavior. CPS designers have no map!
Lee, Berkeley 19
Timing is not part of software and network semantics
AStory
In“flybywire”aircrac,computerscontroltheplane,media/ngpilotcommands.
Abstrac/onLayersAllofwhicharemodelsexceptthebofom
Thepurposeofanabstrac/onistohidedetailsoftheimplementa/onbelowandprovideaplagormfordesignfromabove.
Abstrac/onLayersAllofwhicharemodelsexceptthebofom
Everyabstrac/onlayerhasfailedfortheaircracdesigner.Thedesignistheimplementa/on.
Determinism?Really?
CPSapplica/onsoperateinanintrinsicallynondeterminis/cworld.Doesitreallymakesensetoinsistondeterminis7cmodels?
23 Lee, Berkeley
• Inscience,thevalueofamodelliesinhowwellitsbehaviormatchesthatofthephysicalsystem.
• Inengineering,thevalueofthephysicalsystemliesinhowwellitsbehaviormatchesthatofthemodel.
Lee, Berkeley 24
In engineering, model fidelity is a two-way street!
For a model to be useful, it is necessary (but not sufficient) to be able to be able to
construct a faithful physical realization.
TheValueofModels
AModel
Lee, Berkeley 25
APhysicalRealiza/on
Lee, Berkeley 26
ModelFidelity
• Toascien7st,themodelisflawed.
• Toanengineer,therealiza/onisflawed.
I’manengineer…
Lee, Berkeley 27
ForCPS,weneedtochangetheques/on
Theques/onisnotwhetherdeterminis/cmodelscandescribethebehaviorofcyber-physicalsystems(withhighfidelity).Theques/oniswhetherwecanbuildcyber-physicalsystemswhosebehaviormatchesthatofadeterminis/cmodel(withhighprobability).
Lee, Berkeley 28
Determinism?Whataboutresilience?Adaptability?
Determinis/cmodelsdonoteliminatetheneedforrobust,fault-tolerantdesigns.
Infact,theyenablesuchdesigns,becausetheymakeitmuchclearerwhatitmeanstohaveafault!
Lee, Berkeley 29
Enter:SynchronousLanguages
• Determinis/cconcurrency
But:• Timebetween/cks?• WCEToverallreac/ons?• Distributedsystems?
Lee, Berkeley 30
Usefuldeterminis/cmodelsforCPS
Togetdeterminis/cmodelsforCPSwithfaithfulimplementa/ons,wecan:1. Useprocessorswithcontrollable/ming
(PRETmachines).– hfp://chess.eecs.berkeley.edu/pret
2. Extendsynchronouslanguageswitha(superdense)modelof/me– LeeandZheng,EMSOFT2007
3. Synchronizeclocksandcreatedistributedreal-/meexecu/on(PTIDES)– hfp://chess.eecs.berkeley.edu/p/des
Lee, Berkeley 31
Together,thesetechnologiesgiveaprogrammingmodelfordistributedandconcurrentreal-7mesystemsthatisdeterminis7cinthesenseofsingle-threadedimpera7veprograms,andalsodeterminis7cw.r.t.to7mingofexternalinterac7ons.
ExtendingSRtogetDE
• Timetothenext/ckisdeterminedby/me-stampeddiscreteevents.
• Ateach/ck,usealeastfixed-pointseman/cs,asusualwithsynchronouslanguages.
Lee, Berkeley 32
EMSOFT 2007
Abstract:Discrete-event(DE)modelsareformalsystemspecifica/onsthathaveanalyzabledeterminis/cbehaviors.Usingaglobal,consistentno/onof/me,DEcomponentscommunicatevia/me-stampedevents.DEmodelshaveprimarilybeenusedinperformancemodelingandsimula/on,where/mestampsareamodelingpropertybearingnorela/onshiptoreal/meduringexecu/onofthemodel.Inthispaper,weextendDEmodelswiththecapabilityofrela/ngcertaineventstophysical/me…
33 Lee, Berkeley
Ptides – A Robust Distributed Deterministic DE MoC
UsingSynchronizedClocksinDistributedSystems:RootsoftheIdea
ACMTransac/onsonProgrammingLanguagesandSystems,1984.Lee, Berkeley 34
GoogleSpanner–AReinven/on
Googleindependentlydevelopedaverysimilartechniqueandappliedittodistributeddatabases.
Lee, Berkeley 35
Proceedings of OSDI 2012
Bound C1 on computation
time Time stamp sensor data
Lee, Berkeley 36
Ptides: Time stamps bind to real time at sensors and actuators
Bound L on network latency
Bound E on clock synchronization
error
An event here with time stamp T can be processed when the local clock exceeds
T+C1+L+E
Bound C2 on computation
time
Logical delay D
Event is delivered to the actuator on time if D ≥ C1+C2+L+E
Determinis/cDistributedReal-Time
Assumeboundson:• execu7on7me• clocksynchroniza7onerror• networklatencytheneventsareprocessedin.me-stamporderateverycomponentandeventsaredeliveredtoactuatorson.me.
Lee, Berkeley 37
See http://chess.eecs.berkeley.edu/ptides
All of the assumptions are achievable with today’s technology, and are requirements anyway for hard-real-time systems. The Ptides model makes the requirements explicit.
Lee, Berkeley 38
So Many Assumptions?
You will never strike oil by drilling through the map!
Violations of the requirements are detectable as out-of-order events and can be treated as faults.
Non-Synchronized Clocks
A fault manifests as out-of-order events.
… after an event here with a later time stamp has been processed, then fault!
If an event arrives here with an earlier time
stamp… Occurrence of a fault implies one or more of the assumptions was violated.
Lee, Berkeley 39
Handling Faults
But…
Determinismhasitslimits.
Lee, Berkeley 40
• Complexity • Uncertainty • Chaos • Incompleteness
Complexity
• Somesystemsaretoocomplexfordeterminis/cmodels.
• Nondeterminis/cabstrac/onsbecomeuseful.
Lee, Berkeley 41
“Iron wing” model of an Airbus A350.
Complexity
• Somesystemsaretoocomplexfordeterminis/cmodels.
• Nondeterminis/cabstrac/onsbecomeuseful.
Lee, Berkeley 42
Deep Learning, draft book in preparation, by Yoshua Bengio, Ian Goodfellow, and Aaron Courville. http://www.deeplearningbook.org/
But…
Determinismhasitslimits.
Lee, Berkeley 43
• Complexity • Uncertainty • Chaos • Incompleteness
Uncertainty
• Wecan’tconstructdeterminis/cmodelsofwhatwedon’tknow.
• Forthis,nondeterminismisuseful.
• Bayesianprobability(whichismostlyduetoLaplace)quan/fiesuncertainty.
Lee, Berkeley 44
Portrait of Reverend Thomas Bayes (1701 - 1761) that is probably not actually him.
But…
Determinismhasitslimits.
Lee, Berkeley 45
• Complexity • Uncertainty • Chaos • Incompleteness
Determinismdoesnotimplypredictability
Lee, Berkeley 46
Edward Lorenz
Determinismdoesnotimplypredictability
Lee, Berkeley 47
Edward Lorenz
The position of a point is not meaningfully predictable even though the system is deterministic.
Determinismdoesnotimplypredictability
[ThieleandKumar,EMSOFT2015]
Lee, Berkeley 48
But…
Determinismhasitslimits.
Lee, Berkeley 49
• Complexity • Uncertainty • Chaos • Incompleteness
IncompletenessofDeterminism
Anysetofdeterminis/cmodelsrichenoughtoencompassNewton’slawsplusdiscretetransi/onsisincomplete.Lee,FundamentalLimitsofCyber-PhysicalSystemsModeling,ACMTr.onCPS,Vol.1,No.1,November2016
Lee, Berkeley 50
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 51
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 52
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 53
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 54
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 55
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 56
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 57
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 58
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 59
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 60
Illustra/onoftheIncompletenessofDeterminism
Lee, Berkeley 61
ArbitraryInterleavingYieldsNondeterminism
Lee, Berkeley 62
RecalltheHeisenbergUncertaintyPrinciple
Lee, Berkeley 63
IsDeterminismIncomplete?
• InLee(2017),IshowthatthissequenceofmodelsisCauchy,sothespaceofdeterminis/cmodelsisincomplete(itdoesnotcontainitsownlimitpoints).
• InLee(2014),Ishowthatadirectdescrip/onofthisscenarioresultsinanon-construc/vemodel.Thenondeterminismarisesinmakingthismodelconstruc/ve.
Lee, Berkeley 64
Rejec/ngdiscretenessleadstodeterminis/cchaos
Acon/nuousdeterminis/cmodelthatmodelstheballsasspringsischao/c.
Lee, Berkeley 65
Discretebehaviorscannotbeexcludedunlesswealsorejectcausality
Lee, Berkeley 66
Example from Lee, “Constructive Models of Discrete and Continuous Physical Phenomena,” IEEE Access, 2014
Summary
• Determinis/cmodelsareextremelyuseful.
• Combiningofourbestdeterminis/ccybermodelsandphysicalmodelstodayyieldsnondeterminis/cmodels.
• Butdeterminis/cmodelswithfaithfulimplementa/onsexist(inresearch)forcyber-physicalsystems.
• Determinis/cmodelsaren’talwayspossibleorprac/calduetocomplexity,unknowns,chaos,andincompleteness.
• Determinismisapowerfulmodelingtool.Useitifyoucan.Backoffonlywhenyoucan’t.
Lee, Berkeley 67
Conclusion
Modelsplayacentralroleinreasoningaboutanddesigningengineeredsystems.Determinismisavaluableandsubtlepropertyofmodels.
Lee, Berkeley 68
Plato and the Nerd On Technology and Creativity
Edward Ashford Lee
MIT Press, 2017
Forthcoming book My first for a general audience