Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys...

SoftwareComplexityStevenM.Bellovin

https://www.cs.columbia.edu/~smb

WhatHappened?

• Whyisatrainarrivingin-2minutes?• Isthe10:26runningaheadofthe9:38?• (We’llignorethefactthatthey’rebothquitelate.)

ATrainStatusDisplay

AndinWashington…

Optimism

“Theprogrammer,likethepoet,worksonlyslightlyremovedfrompurethought-stuff.Hebuildshiscastlesintheair,fromair,creatingbyexertionoftheimagination.Fewmediaofcreationaresoflexible,soeasytopolishandrework,soreadilycapableofrealizinggrandconceptualstructures.”

FredBrooks,TheMythicalMan-Month

RealityCheck

“[O]nemustperformperfectly.Thecomputerresemblesthemagicoflegendinthisrespect,too.Ifonecharacter,onepause,oftheincantationisnotstrictlyinproperform,themagicdoesn’twork.Humanbeingsarenotaccustomedtobeingperfect,andfewareasofhumanactivitydemandit.”

FredBrooks,TheMythicalMan-Month

RealSoftware

• Softwareisbuggy• Softwareisalwaysbuggy• Thebiggertheprogram,thebuggierthesoftware—always

Why?Interactions

ComplexityKills…

Withncomponents,thereareroughlyn2interactions

• Therearelimitstohowgoodoursoftwarecanbe• Therearethereforethingswecan’tdo• Moreprecisely,whenweincreasecomplexitywe

a) Increasethebugratedramaticallyb) Increasethedevelopmentcostdramaticallyc) Both!

ComplexityandCurrentEvents

• Webankonline• Webuythingsonline• Wecommunicateonline• Whycan’twevoteonline?

ProbablyJustaBug

(PhotobyEdFelten) 12

Avotingmachinetapefromthe2008presidentialprimaryinaNewJerseyprecinct.

EntertheAdversary

13NYTimes

GoingDark

“Asaresult,althoughthegovernmentmayobtainacourtorderauthorizingthecollectionofcertaincommunications,itoftenservesthatorderonaproviderwhodoesnothaveanobligationunderCALEAtobepreparedtoexecuteit.”

ValerieCaproni,GeneralCounseloftheFBI

TheFBI’sSolution

• Allcommunicationsystemsneedsomeformofaccessforlawenforcement• Allencryptionsystemsneeda“backdoor”(whichtheycalla“goldenkey”)• Canwedoit?

WiretapInterfacesareHard

• Someyearsago,theNSAevaluatedthestandardizedwiretapinterfaceon26differentphoneswitches• Allhadsecurityflaws

• Someone(probablyanintelligenceagency)hackedacellphoneswitchinAthensandabusedthewiretapinterface• About100phoneswereillegallytapped,includingthePrimeMinister’s

CryptographyisHard

“Finally,protocolssuchasthosedevelopedherearepronetoextremelysubtleerrorsthatareunlikelytobedetectedinnormaloperation.”

RogerNeedhamandMichaelSchroeder,“UsingEncryptionforAuthenticationinLargeNetworksofComputers”

From“KeysUnderDoormats”

“Wehavefoundthatthedamagethatcouldbecausedbylawenforcementexceptionalaccessrequirementswouldbeevengreatertodaythanitwouldhavebeen20yearsago.Inthewakeofthegrowingeconomicandsocialcostofthefundamentalinsecurityoftoday’sInternetenvironment,anyproposalsthatalterthesecuritydynamicsonlineshouldbeapproachedwithcaution...Thecomplexityoftoday’sInternetenvironment,withmillionsofappsandgloballyconnectedservices,meansthatnewlawenforcementrequirementsarelikelytointroduceunanticipated,hardtodetectsecurityflaws.”

Abelsonetal.

WhyTechnologistsOpposeGoldenKeys

• IthasnothingtodowithdislikeoftheFBIortheNSA• Technologistscanbevictimsofcriminalsandterrorists,too• Rather,it’saquestionofcrimeprevention—thesoftwarenecessarytopermitlawenforcementaccesshasahighprobabilityofopeningupnewsecurityholes• Therootcauseisthecomplexityofsoftware

TheInternetofThings

• We’reconnectingmoreandmore“things”totheInternet• Theserunonsoftware;thissoftwareisoftenpoorlywrittenandneverpatched

Self-DrivingCars

• Almostcertainly,wewillseesomecrashesduetobuggycode• Possibly(thoughnotcertainly),therewillbecrashesduetohacking

• Eventoday’s“dumb”carscontain50-75networkedcomputers• Amoderncarisactuallyamobiledatacenter!

• But—self-drivingcars,flawsandall,willalmostcertainlybesaferthanhuman-drivencars• Carsdon’tgetdrunk,sleepy,distracted,etc.

UsersDon’tSeeMostoftheComplexity

• Goodsoftwareoftenhideshowcomplexitis• But—thecomplexityisstillthere• Often,it’sthepartsyoudon’tknowaboutthatcancausethemosttrouble

SoWhatDoWeDo?

• Giveup?

SoWhatDoWeDo?

• Giveup?• No;thatsacrificesthebenefitsofcomputers.Therearereasons(andgenerallygoodones)whywerelyonsoftware

SoWhatDoWeDo?

• Giveup?• No;thatsacrificesthebenefitsofcomputers.Therearereasons(andgenerallygoodones)whywerelyonsoftware• Often,somesmallrateoffailureisquiteacceptable—nothingelseisperfect,either

SoWhatDoWeDo?

• Giveup?• No;thatsacrificesthebenefitsofcomputers.Therearereasons(andgenerallygoodones)whywerelyonsoftware• Often,somesmallrateoffailureisquiteacceptable—nothingelseisperfect,either• Thetrickisknowinghowtodecide.Wewantmajorbenefits,comparativelylowrisks,andacceptableconsequencesifthereisafailure

“Thecompetentprogrammerisfullyawareofthestrictlylimitedsizeofhisownskull;thereforeheapproachestheprogrammingtaskinfullhumility…”

EdsgerDijkstra,“TheHumbleProgrammer”

SomeSuggestions

• Self-drivingcars• Communicationsapps

• Thesmartgrid?

• (Residential)lightbulbs• Bikelocks• Anti-missilesystems• Votingmachines

• Networkedsextoys?

Questions?

Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys...

Documents

Transcript of Software Complexitysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Why Technologists Oppose Golden Keys...

Cancionero Tuna Filosof%80%a0%a0%Eda y Letras m%80%a0%a0%e1laga

Identity, Security, and Privacysmb%c2%a0%c2%a0%c2%a0%c2%a0%c2%a0/tal… · Reputation should adhere to the real identity A bad guy should not be able to discard a bad reputation by

Secure System Design: E-Commerce Web Sitesmb%C2%A0%C2%A0%C2%A0... · Even if they don’t, compromise of a user’s e-commerce password could permit orders billed to that person’s

Google versus Chinasmb%c2%a0%c2%a0%c2%a0%c2%… · including from Chinese companies like Huawei. ... • Some of the attacks were directed at dissidents and activists ... they could

Contemporary Moral Problemsitethic.pbworks.com/f/Contemporary%C2%A0Moral%C2%A0...Page | 2 Table of Contents Egoism and Moral Scepticism .……….. 3 - 5 Religion, Morality, and Conscience

Introduction to Cryptography, Part IIsmb%c2%a0%c2%a0%c2%a0%c2%… · Homomorphic Vs. Non-malleable •Homomorphic encryption ... •How many people need to be in a room for the probability

New Public Management Is Dead—Long Live Digital-Era …jerrybrito.pbworks.com/f/Digital%C2%A0Era%C2%A0... · Helen Margetts Oxford Internet ... Simon Bastow Jane Tinkler London

ControlCenter2smb%c2%a0%c2%a0%c2%a0...MC3764 CAP (Section 4.2). The CAP will provide the same multiple code population and limited-try fea- tures as the MCCS, but the W82 will not

Viruses, Trojan Horses, and Wormssmb%c2%a0%c2%a0%c2%a0%c2%a0… · Worms in Science Fiction “Let me put it another way. You have a computer with an auto-dial phone link. You put

Frank Miller’s Codebooksmb%c2%a0%c2%a0%c2%a0%c2%a0… · Lots of Google and Google books searches; ... memory of it in 1912 when he formulated his maxim? Does the evidence exist

Authentication - cs.columbia.edusmb%c2%a0%c2%a0%c2%a0%c2%a0… · password as the key + This is where the 8-character limit comes from Any decent cryptosystem can resist ﬁnding

Employment and Computers - cs.columbia.edusmb%c2%a0%c2%a0%c2%a0... · Alcatel “merged” with Lucent; Nokia is buying the new ... remained a monopoly—and didn’t drop in price

Doing History in the Internet Agesmb%c2%a0%c2%a0%c2%a0... · the Freemasons (1829), the Templars (1867), and the Ku Klux Klan (1872) Neither of these two meanings is listed in the

STAINES AP-PLANS PHDD C2-A0 HVAC MECH

The Cybersecurity Challengesmb%c2%a0%c2%a0%c2%a0%c2%a0...The Problem “Our first observation is that we are hard pressed to say that cyberspace is more secure than it was 35 years

Web Server-Side Securitysmb%c2%a0%c2%a0%c2%a0%c2%a0… · Most interesting web sites use server-side scripts: CGI, ASP, PHP, server-side include, etc. Each such script is a separate

Block Diagram Pin Description (Top-side view)C1+ C1-B2+ B2-C2+ C2-B3+ B3-C3+ C3-V DD A0+ A0-D0+ D0-A1+ A1-D1+ D1-A2+ A2-D2+ D2-A3+ A3-D3+ D3-SEL1 SEL2 PI3PCIE3413A 3.3V, PCI Express®

Protecting the Clientsmb%c2%a0%c2%a0%c2%a0... · If someone using the application connects to your server—or if you can trick someone into connecting to your server via a watering

Voluntary Disclosuressmb%c2%a0%c2%a0%c2%a0%c2%a0...update letting friends know that his unit was preparing to go to a West Bank village near Ramallah: On Wednesday we clean up Qatanah,

Security: The Human Elementsmb%c2%a0%c2%a0%c2%a0%c2%a0… · The Human Element “Humans are incapable of securely storing high-quality cryptographic keys, and they have unacceptable