Principles of Software Construction: Objects, Design, and...

115-214

SchoolofComputerScience

PrinciplesofSoftwareConstruction:Objects,Design,andConcurrency

Concurrency:MotivationandPrimitives

ChristianKästner BogdanVasilescu

215-214

Administrivia (1)• SignupprocedureanddeadlineforHW5a– Teamsof2or3.FormyourteamandsignupforapresentationtimebyThursday,Mar30,11:59pm.• Youmayutilizethe"SearchforTeammates"thread @5 tohelpyoufindteammates.

• Stickaroundafterclasstodayifyoudon’thavepartnersyet.• Twoplacestosignup:GoogleSheet&GitHubrepo.See@652

– Shortpresentation(max10min,6slidesorfewer)inrecitation onWednesday,April5 infrontofyourclassmates.• Goal:illustratehowyouachievereuseinadomain• Describedomain,examplesofplugins,decisionsregardinggeneralityvsspecificity,overallprojectstructure(e.g.,howarepluginsloaded),plugininterfaces

• Similartodesignreviewsessions

315-214

Administrivia (2)

• Secondmidterm,ThursdayMar30inclass.• Midtermreviewsessiontoday 7:30pmin GHC4401.• Concurrencynottestedonthemidterm– Buteverythinginthecourseincludingreadingsisfairgame– Wewillfocusonthemiddlepartofthecourseandthethingsthatyouhadmorechancestopractice• e.g.moreUML/designthanAPIdesign

415-214

Administrivia (3)

• GooddiscussiononPiazzaabouttheReadingQuizquestion2(Chapter6of“BeautifulCode”):

• Q:“Whataresomeofthementionedmechanismsthatcanhelpensurebackward-compatibility?”– A:Usingdesignpatterns– A:Usinginterfaces– A:Controllingvisibility

• A:Usingdesignpatterns:– Book:“Providewell-defined‘hookpoints’thatpermitextensibilityin

theplaceswhereyouintendittooccur.”– Exampleofhowthe Observerpatterncanbeusedtoprovidesuch

hookpoints

515-214

Administrivia (4)

• Commitmessagesare(oneof)yourprimarymeansofcommunicationwiththerestoftheteam.– ThiswillbecomemoreobviousinHW5.

615-214

715-214

Part1:DesignataClassLevel

DesignforChange:InformationHiding,

Contracts,DesignPatterns,UnitTesting

DesignforReuse:Inheritance,Delegation,

Immutability,LSP,DesignPatterns

Part2:Designing(Sub)systems

UnderstandingtheProblem

ResponsibilityAssignment,DesignPatterns,GUIvsCore,

DesignCaseStudies

TestingSubsystems

DesignforReuseatScale:FrameworksandAPIs

Part3:DesigningConcurrent

Systems

ConcurrencyPrimitives,Synchronization

DesigningAbstractionsforConcurrency

DistributedSystemsinaNutshell

IntrotoJava

Git,CIStaticAnalysis

GUIsUML MoreGit

GUIsPerformance

Design

815-214

LearningGoals

• Understandthemotivationanddifferentusecasesforconcurrencyandparallelism

• Understandconcurrencyrisks:safety,liveness,performance

• UnderstandanduseJavaprimitivesforconcurrency:threads,synchronization,volatile,wait/notify

915-214

WHYCONCURRENCY

1015-214

Whatisathread?(review)

• Shortforthreadofexecution• Multiplethreadsruninsameprogramconcurrently

• Threadssharethesameaddressspace– Changesmadebyonethreadmaybereadbyothers

• Multithreadedprogramming– Alsoknownasshared-memorymultiprocessing

1115-214

Processorcharacteristicsovertime

1215-214

PowerrequirementsofaCPU

• power=capacitance× voltage2 × frequency• Toincreaseperformance– Moretransistors,thinnerwires

• Morepowerleakage:increasevoltage– Increaseclockfrequency

• Changeelectricalstatefaster:increasevoltage

• Dennardscaling– astransistorsgetsmaller,powerdensityisapproximatelyconstant…– …untilearly2000s

• Now:Powerissuper-linearinCPUperformance

1315-214

FailureofDennard Scalingforcedourhand

• Mustreduceheatbylimitingpower• Limitpowerbyreducingfrequencyand/orvoltage• Inotherwords,buildslowercores…– …butbuildmoreofthem

• Addingcoresupspowerlinearlywithperformance

• Butconcurrencyisrequiredtoutilizemultiplecores

1415-214

Concurrencythenandnow

• Inpastmulti-threadingjustaconvenientabstraction– GUIdesign:eventdispatchthread– Serverdesign:isolateeachclient’swork–Workflowdesign:isolateproducersandconsumers

• Now:required forscalabilityandperformance

1515-214

BenefitsofThreads(1)• ExploitingMultipleProcessors– AllCPUstodayaremulti-core– Butbasicunitofschedulingisathread

• Asingle-threadedprogramrunningona100-processorsystemisgivingupaccessto99%oftheavailableCPUresources

– Also,betterthroughputonsingle-processorsystems:• Single-threadedprogramneedstowaitforsynchronousI/Ooperationtocomplete

• Multi-threadedprogramcandosomethingelseduringtheblockingI/O

• ResponsiveUserInterfaces– AWT,Springhaveseparateeventdispatchthread– Long-runningtasks(e.g.,spellchecking)canbeexecutedinseparatethread

1615-214

BenefitsofThreads(2)

• SimplicityofModeling– Separatingtasks&assigningaseparatethreadtoeach

– Abstractingcommoninfrastructure,asrequestmanagement,loadbalancing,...inconcurrencyframeworks

• SimplifiedHandlingofAsynchronousEvents– Async vssyncI/O:serverthatacceptssocketconnectionsfrommultipleclients;clientreadblocksuntildataisavailable

– Avoiding“callbackhell”(JavaScript)

1715-214

Aside:JavaScript“CallbackHell”• Youdon’twanttheprogramtopause(block)whilewaitingfordownloadtofinish

• Storethecodethatshouldrunafterthedownloadiscompleteina“callback”function

var photo = downloadPhoto('http://coolcats.com/cat.gif')// photo is 'undefined'!

downloadPhoto('http://coolcats.com/cat.gif', handlePhoto)

function handlePhoto (error, photo) {if (error) console.error('Download error!', error)else console.log('Download finished', photo)

}

console.log('Download started')

From: http://callbackhell.com

1

2

3

4

1815-214

Aside:JavaScript“CallbackHell”• Callbackscangetoutofhand

getData(function(a){ getMoreData(a, function(b){

getMoreData(b, function(c){ getMoreData(c, function(d){

getMoreData(d, function(e){ ...

});});

});});

});

From: http://stackabuse.com/avoiding-callback-hell-in-node-js/

1915-214

Weareallconcurrentprogrammers

• Javaisinherentlymultithreaded• Inordertoutilizeourmulticoreprocessors,wemustwritemultithreadedcode

• Goodnews:alotofitiswrittenforyou– Excellentlibrariesexist(java.util.concurrent)

• Badnews:youstillmustunderstandfundamentals– touselibrarieseffectively– todebugprogramsthatmakeuseofthem

2015-214

ConcurrencyvsParallelism

2115-214

CONCURRENCYHAZARDSSafety,Liveness,Performance

2215-214

SafetyHazard• Theorderingofoperationsinmultiplethreadsisunpredictable.

• UnluckyexecutionofUnsafeSequence.getNext

@NotThreadSafepublic class UnsafeSequence {

private int value;

public int getNext() {return value++;

}}

valueà9 9+1à10 valueà10


A

B

Not atomic

2315-214

ThreadSafety

Aclassisthreadsafeifitbehavescorrectlywhenaccessedfrommultiplethreads,regardlessoftheschedulingorinterleavingoftheexecutionofthosethreadsbytheruntimeenvironment,andwithnoadditionalsynchronizationorothercoordinationonthepartofthecallingcode.

2415-214

LivenessHazard• Safety:“nothingbadeverhappens”• Liveness:“somethinggoodeventuallyhappens”

• Deadlock– Infiniteloopinsequentialprograms– ThreadAwaitsforaresourcethatthreadBholdsexclusively,andBneverreleasesità Awillwaitforever• E.g.,Diningphilosophers

• Elusive:dependonrelativetimingofeventsindifferentthreads

2515-214

Deadlockexample

class Account {double balance;

void withdraw(double amount){ balance -= amount; }

void deposit(double amount){ balance += amount; }

void transfer(Account from, Account to, double amount){synchronized(from) {

from.withdraw(amount);synchronized(to) {

to.deposit(amount);}

}}

}

• Twothreads:Adoestransfer(a,b,10); Bdoestransfer(b,a,10)

Execution trace:A: lock a (v)B: lock b (v)A: lock b (x)B: lock a (x)A: waitB: wait

Deadlock!

2615-214

PerformanceHazard• Liveness:“somethinggoodeventuallyhappens”• Performance:wewantsomethinggoodtohappenquickly

• Multi-threadinginvolvesruntimeoverhead:– Coordinatingbetweenthreads(locking,signaling,memorysync)– Contextswitches– Threadcreation&teardown– Scheduling

• Notallproblemscanbesolvedfasterwithmoreresources– Onemotherdeliversababyin9months

2715-214

Amdahl’slaw• Thespeedupis

limitedbytheserialpartoftheprogram.

2815-214

Howfastcanthisrun?

public class WorkerThread extends Thread {...

public void run() {while (true) {

try {Runnable task = queue.take();task.run();

} catch (InterruptedException e) {break; /* Allow thread to exit */

}}

}}

• Nthreadsfetchindependenttasksfromasharedworkqueue

2915-214

JAVAPRIMITIVES:ENSURINGVISIBILITYANDATOMICITY

3015-214

SynchronizationforSafety

• Ifmultiplethreadsaccessthesamemutablestatevariablewithoutappropriatesynchronization,theprogramisbroken.

• Therearethreewaystofixit:– Don'tsharethestatevariableacrossthreads;–Makethestatevariableimmutable;or– Usesynchronizationwheneveraccessingthestatevariable.

3115-214

Exclusion

Synchronizationallowsparallelismwhileensuringthatcertainsegmentsareexecutedinisolation.Threadswaittoacquirelock,mayreduceperformance.

3215-214

Statelessobjectsarealwaysthreadsafe

• Example:statelessfactorizer– Nofields– Noreferencestofieldsfromotherclasses– Threadssharingitcannotinfluenceeachother

@ThreadSafepublic class StatelessFactorizer implements Servlet {

public void service(ServletRequest req, ServletResponse resp) {BigInteger i = extractFromRequest(req);BigInteger[] factors = factor(i);encodeIntoResponse(resp, factors);

}}

3315-214

Isthisthreadsafe?

public class CountingFactorizer implements Servlet {private long count = 0;

public long getCount() { return count; }

public void service(ServletRequest req, ServletResponse resp) {BigInteger i = extractFromRequest(req);BigInteger[] factors = factor(i);++count;encodeIntoResponse(resp, factors);

}}

3415-214

Nonatomicityandthread(un)safety

@NotThreadSafepublic class UnsafeCountingFactorizer implements Servlet {

private long count = 0;



}}



A

B

3515-214

Nonatomicityandthread(un)safety

• Stateful factorizer– Susceptibletolostupdates– The++count operationisnotatomic(read-modify-write)

@NotThreadSafepublic class UnsafeCountingFactorizer implements Servlet {

private long count = 0;



}}

3615-214

Enforcingatomicity:Intrinsiclocks

• synchronized(lock) { … } synchronizesentirecodeblockonobjectlock;cannotforgettounlock

• Thesynchronizedmodifieronamethodisequivalenttosynchronized(this) { … }aroundtheentiremethodbody

• EveryJavaobjectcanserveasalock• Atmostonethreadmayownthelock(mutualexclusion)– synchronized blocksguardedbythesamelockexecuteatomicallyw.r.t.oneanother

3715-214

Fixingthestateful factorizer@ThreadSafepublic class UnsafeCountingFactorizer

implements Servlet {@GuardedBy(“this”)private long count = 0;

public long getCount() { synchronized(this){

return count; }

}

public void service(ServletRequest req, ServletResponse resp) {

BigInteger i = extractFromRequest(req);BigInteger[] factors = factor(i);synchronized(this) {

++count;}encodeIntoResponse(resp, factors);

}}

Foreachmutablestatevariablethatmaybeaccessedbymorethanonethread,all accessestothatvariablemustbeperformedwiththesame lockheld.Inthiscase,wesaythatthevariableisguardedbythatlock.

3815-214



public synchronized long getCount() { return count;

}




}}


3915-214



public synchronized long getCount() { return count;

}

public synchronized void service(ServletRequest req, ServletResponse resp) {

BigInteger i = extractFromRequest(req);BigInteger[] factors = factor(i);

++count;encodeIntoResponse(resp, factors);

}}


4015-214

What’sthedifference?

public synchronized void service(ServletRequest req, ServletResponse resp) {

BigInteger i = extractFromRequest(req);BigInteger[] factors = factor(i);

++count;encodeIntoResponse(resp, factors);

}




}

4115-214

Privatelocks@ThreadSafepublic class UnsafeCountingFactorizer

implements Servlet {private final Object lock = new Object();@GuardedBy(“lock”)private long count = 0;

public long getCount() { synchronized(lock){

return count; }

}


BigInteger i = extractFromRequest(req);BigInteger[] factors = factor(i);synchronized(lock) {


}}


4215-214

Doesthisdeadlock?

public class Widget {public synchronized void doSomething() {...}

}

public class LoggingWidget extends Widget {public synchronized void doSomething() {

System.out.println(toString() + ": calling doSomething");super.doSomething();

}}

4315-214

No:Intrinsiclocksarereentrant• Athreadcanlockthesameobjectagainwhilealreadyholdingalock,i.e.,asynchronizedmethodcancallanothersynchronizedmethodinthesameobject

public class Widget {public synchronized void doSomething() {...}

}

public class LoggingWidget extends Widget {public synchronized void doSomething() {

System.out.println(toString() + ": calling doSomething");super.doSomething();

}}

4415-214

CooperativethreadterminationHowlongwouldyouexpectthistorun?

public class StopThread {private static boolean stopRequested;

public static void main(String[] args) throws Exception {Thread backgroundThread = new Thread(() -> {

while (!stopRequested)/* Do something */ ;

});backgroundThread.start();

TimeUnit.SECONDS.sleep(5);stopRequested = true;

}}

4515-214

Whatcouldhavegonewrong?

• Intheabsenceofsynchronization,thereisnoguaranteeastowhen,ifever,onethreadwillseechangesmadebyanother!

• VMscananddoperformthisoptimization:while (!done)

/* do something */ ;

becomes:if (!done)

while (true)/* do something */ ;

4615-214

Howdoyoufixit?public class StopThread {

@GuardedBy(“StopThread.class”)private static boolean stopRequested;

private static synchronized void requestStop() {stopRequested = true;

}

private static synchronized boolean stopRequested() {return stopRequested;

}


while (!stopRequested())/* Do something */ ;


TimeUnit.SECONDS.sleep(5);requestStop();

}}

4715-214

Youcandobetter(?)volatile issynchronizationsansmutualexclusionpublic class StopThread {

private static volatile boolean stopRequested;


while (!stopRequested)/* Do something */ ;


TimeUnit.SECONDS.sleep(1);stopRequested = true;

}}

4815-214

Volatilekeyword

• Tellscompilerandruntimethatvariableissharedandoperationsonitshouldnotbereorderedwithothermemoryops– Areadofavolatilevariablealwaysreturnsthemostrecentwritebyanythread

• Volatileisnotasubstituteforsynchronization– Volatilevariablescanonlyguaranteevisibility– Lockingcanguaranteebothvisibilityandatomicity

4915-214

Summary:Synchronization

• Ideally,avoidsharedmutablestate

• Ifyoucan’tavoidit,synchronizeproperly– Failuretodosocausessafetyandliveness failures– Ifyoudon’tsyncproperly,yourprogramwon’twork

• Evenatomicoperationsrequiresynchronization– e.g.,stopRequested = true

– Andsomethingsthatlookatomicaren’t(e.g.,val++)

5015-214

JAVAPRIMITIVES:WAIT,NOTIFY,ANDTERMINATION

5115-214

Guardedmethods

• Whattodoonamethodifthepreconditionisnotfulfilled(e.g.,transfermoneyfrombankaccountwithinsufficientfunds)• throwexception(balking)• waituntilpreconditionisfulfilled(guarded suspension)• waitandtimeout(combinationofbalkingandguarded

suspension)

5215-214

Guardedsuspension

• Blockexecutionuntilagivenconditionistrue• Forexample,– pullelementfromqueue,butwaitonanemptyqueue

– transfermoneyfrombankaccountassoonsufficientfundsarethere

• Blockingas(oftensimpler)alternativetocallback

5315-214

MonitorMechanicsinJava

• Object.wait()– suspendsthecurrentthread’sexecution,releasinglocks

• Object.wait(timeout)– suspendsthecurrentthread’sexecutionforuptotimeoutmilliseconds

• Object.notify()– resumesoneofthewaitingthreads

• Seedocumentationforexactsemantics

5415-214

MonitorExampleclass SimpleBoundedCounter {

protected long count = MIN;public synchronized long count() { return count; }public synchronized void inc() throws InterruptedException {

awaitUnderMax(); setCount(count + 1);}public synchronized void dec() throws InterruptedException {

awaitOverMin(); setCount(count - 1);}protected void setCount(long newValue) { // PRE: lock held

count = newValue;notifyAll(); // wake up any thread depending on new value

}protected void awaitUnderMax() throws InterruptedException {

while (count == MAX) wait();}protected void awaitOverMin() throws InterruptedException {

while (count == MIN) wait();}

}

5515-214

Never invokewaitoutsidealoop!

• Looptestsconditionbeforeandafterwaiting• Testbeforeskipswait ifconditionalreadyholds– Necessarytoensureliveness–Withoutit,threadcanwaitforever!

• Testingafterwait ensuresafety– Conditionmaynotbetruewhenthreadwakens– Ifthreadproceedswithaction,itcandestroyinvariants!

5615-214

All ofyourwaitsshouldlooklikethissynchronized (obj) {

while (<condition does not hold>) {obj.wait();

}

... // Perform action appropriate to condition}

5715-214

Whycanathreadwakefromawaitwhenconditiondoesnothold?

• Anotherthreadcanslipinbetweennotify&wake• Anotherthreadcaninvokenotify accidentallyormaliciouslywhenconditiondoesnothold– Thisisaflawinjavalockingdesign!– Canworkaroundflawbyusingprivatelockobject

• Notifier canbeliberalinwakingthreads– UsingnotifyAll isgoodpractice,butcausesthis

• Waitingthreadcanwakeupwithoutanotify(!)– Knownasaspuriouswakeup

5815-214

Interruption

• Difficulttokillthreadsoncestarted,butmaypolitelyasktostop(thread.interrupt())

• Long-runningthreadsshouldregularlycheckwhethertheyhavebeeninterrupted

• Threadswaitingwithwait()throwexceptionsifinterrupted

• Readdocumentation

public class Thread {public void interrupt() { ... }public boolean isInterrupted() { ... }...

}

5915-214

InterruptionExample

For details, see Java Concurrency In Practice, Chapter 7

class PrimeProducer extends Thread {private final BlockingQueue<BigInteger> queue;PrimeProducer(BlockingQueue<BigInteger> queue) {

this.queue = queue;}public void run() {

try {BigInteger p = BigInteger.ONE;while (!Thread.currentThread().isInterrupted())

queue.put(p = p.nextProbablePrime());} catch (InterruptedException consumed) {

/* Allow thread to exit */}

}public void cancel() { interrupt(); }

}

6015-214

BUILDINGHIGHERLEVELCONCURRENCYMECHANISMS

6115-214

BeyondJavaPrimitives

• JavaPrimitives(synchronized,wait,notify)arelowlevelmechanisms

• Formosttasksbetterhigher-levelabstractionsexist

• Writingownabstractionsispossible,butpotentiallydangerous– uselibrarieswrittenbyexperts

6215-214

Example:read-writelocks(API)Alsoknownasshared/exclusivemodelocksprivate final RwLock lock = new RwLock();

lock.readLock();try {

// Do stuff that requires read (shared) lock} finally {

lock.unlock();}

lock.writeLock();try {

// Do stuff that requires write (exclusive) lock} finally {

lock.unlock();}

6315-214

Example:read-writelocks(Impl.1/2)

public class RwLock {// State fields are protected by RwLock's intrinsic lock

/** Num threads holding lock for read. */private int numReaders = 0;

/** Whether lock is held for write. */private boolean writeLocked = false;

public synchronized void readLock() throws InterruptedException { while (writeLocked) {

wait();}numReaders++;

}

6415-214

Example:read-writelocks(Impl.2/2)

public synchronized void writeLock() throws InterruptedException {while (numReaders != 0 || writeLocked) {

wait();}writeLocked = true;

}

public synchronized void unlock() {if (numReaders > 0) {

numReaders--;} else if (writeLocked) {

writeLocked = false;} else {

throw new IllegalStateException("Lock not held");}notifyAll(); // Wake any waiters

}}

6515-214

Caveat:RwLock isjustatoy!

• Ithaspoorfairnessproperties– Readerscanstarvewriters!

• java.util.concurrent providesanindustrialstrengthReadWriteLock

• Moregenerally,avoidwait/notify– Intheearlydaysitwasallyouhad– Nowadays,higherlevelconcurrencyutils arebetter

6615-214

Summary

• Concurrencyforexploitingmultipleprocessors,simplifyingmodeling,simplifyingasynchronousevents

• Safety,livenessandperformancehazardsmatter• SynchronizationonanyJavaobject;volatileensuresvisibility

• Wait/notifyforguards,interruptionforcancelation– buildingblocksforhigherlevelabstractions

6715-214

RecommendedReadings

• Goetzetal.JavaConcurrencyInPractice.PearsonEducation,2006,Chapters1-2

• Lea,Douglas.ConcurrentprogramminginJava:designprinciplesandpatterns.Addison-WesleyProfessional,2000.

Principles of Software Construction: Objects, Design, and...

Documents

Transcript of Principles of Software Construction: Objects, Design, and...