© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 1

Concurrency in Programming Languages

Matthew J. SottileMatthew J. Sottile

Timothy G. MattsonTimothy G. Mattson

Craig E RasmussenCraig E Rasmussen

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 2

Chapter 1 Objectives

• Introduce the topic of concurrency and the current relevance of the topic.

Outline

• Motivation• Where does concurrency appear?• Real-world concurrency• Timeliness of the topic

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 3

Motivation

• Concurrency and parallelism have traditionally existed in niche applications.

• Multicore CPUs and novel architectures (such as GPUs) are bringing the topic to a wider audience.

• Learning how to deal with widespread and pervasive concurrency is critical for being able to program modern computers.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 4

Motivation

• Traditional languages and methods for concurrent programming are:– Awkward: explicit thread management– Error prone: easy to create correctness problems– Difficult for compilation: common language features

prohibit automatic parallelization (e.g.: pointers)

• Languages that are concurrency-aware address these issues for programmers.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 5

Traditional areas

• Operating systems– The most common topic where concurrency is taught

and addressed is in the context of operating systems design and implementation.

• Scientific and high-performance computing– For many decades, the HPC and scientific computing

areas were the primary source of work on parallelism.

• Databases and servers– Server-side applications that needed to handle multiple

concurrent users.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 6

Modern areas

• Any application that is run on a laptop or desktop system with multiple cores can use concurrency for performance gains.– Games– Desktop applications– Development tools– Image and data processing

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 7

Outline

• Motivation• Where does concurrency appear?• Real-world concurrency• Timeliness of the topic

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 8

Operating systems

• Managing concurrently operating hardware devices.

• Multitasking support.• Sharing of resources between multiple programs.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 9

Distributed systems

• Network-based systems.• Servers providing services to multiple clients

simultaneously.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 10

User interfaces

• Multiple programs running concurrently in a graphical environment.

• Applications must support both application logic at the same time as handling user interactions.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 11

Databases

• Supporting multiple simultaneous accesses to a shared data store.

• Transactional systems for providing atomic operations.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 12

Scientific computing

• Parallel computing: implementing algorithms that use many processors in concert to solve a single problem.

• Commonly focus on very large data sets that cannot fit in a single computer.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 13

Outline

• Motivation• Where does concurrency appear?• Real-world concurrency• Timeliness of the topic

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 14

Is concurrency really “hard”?

• Concurrent and parallel computing has a reputation of being difficult or confusing.

• This is a misconception.– In our daily lives outside computers, we are very used to

concurrency. We live in a world where multiple activities are always happening at the same time.

– The reputation likely comes from the difficulty of traditional tools for dealing with concurrency in a programming context.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 15

Is concurrency really “hard”?

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 16

“There is therefore nothing new in the basic idea of parallel programming, but only in its application to computers.”

S. Gill, 1958

Real world example

• Everyone has likely cooked a meal at some point.• Very often the act of cooking even the simplest of

meals involves some degree of concurrency.

• Let’s consider a simple Italian dish – pasta with sauce.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 17

Steps in cooking a pasta meal

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 18

Cooking a meal

• Concurrency is present in a simple cooking activity.

• Parts of the dish are prepared at the same time as others.– Boiling water and cooking pasta can take place while

preparing a tomato sauce.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 19

Real-world concurrency

• We perform many day-to-day activities like cooking, where we “multitask” instead of doing everything in a single sequence of activities.

• It isn’t intimidating to think about multitasking real-world tasks.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 20

Real-world concurrency

• The challenge is how to translate this natural way of thinking concurrently into the context of building programs.

• Can we easily design programs that exploit the same degrees of concurrency as the real world activities that we are comfortable with?

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 21

Outline

• Motivation• Where does concurrency appear?• Real-world concurrency• Timeliness of the topic

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 22

Concurrency is here

• Concurrency is unavoidable with modern computers.

• Nearly all laptops, desktops, and even handheld systems today are based on parallel computing hardware.– Programming them requires addressing issues of

concurrency.

• Concurrency is no longer a niche topic – it touches nearly every area of computing today.

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 23

Approach

• This book approaches the problem of concurrency at the language level.

• How does concurrency impact the languages we use to build programs?

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 24

Approach

• Start with fundamentals: what are the core issues that arise when dealing with concurrency.– Concurrency control– Correctness issues

• Examine the state of the art• Look at history and see how the topic of

concurrency has been addressed– Hardware level– Software level

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 25

Approach

• Look deeper at how concurrency has impacted language development.

• Consider performance implications of concurrency.

• Examine design patterns for parallel programming in the context of concurrent programming languages.– In this text, we choose three specific languages to

motivate the discussion.• OpenMP extensions to C/C++/Fortran• The Cilk derivative of C• The declarative language Erlang

© 2009 Mathew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen 26