Multithreading

Yuriy GutsR&D Engineer

yuriy.guts@eleks.com

Summer School 2012

Summer School 2012

Although threads seem to be a small step from sequential

computation, in fact, they represent a huge step.

They discard the most essential and appealing properties

of sequential computation: understandability,

predictability, and determinism.

Threads, as a model of computation, are wildly

nondeterministic, and the job of the programmer

becomes one of pruning that nondeterminism.

Edward A. Lee, UC Berkeley, 2006

“

”

History: so why did we need threads?

Summer School 2012

• CPU virtualization• Increased robustness• Quazi-multitasking

Process as a resource isolation unit

Summer School 2012

Concurrent executiondoes not exist

on a single logical CPU!

Summer School 2012

Context Switching

Summer School 2012

Scheduling• Processes are not scheduled. Threads are.• 32 different thread priority levels!

Summer School 2012

1. Idle2. Below

Normal3. Normal4. Above

Normal5. High6. Realtime

1. Idle2. Lowest3. Below

Normal4. Normal5. Above

Normal6. Highest7. Time-Critical

ProcessThread

Thread Overhead• Thread Kernel Object• Thread Environment Block (TEB)• User-Mode Stack [1 MB]

• Kernel-Mode Stack [12 / 24 KB]

• DLL: Attach/Detach Notifications

Summer School 2012

Moore’s Law

Summer School 2012

Threads for parallelization?

Summer School 2012

Multiple Logical CPUs

Summer School 2012

• Multi-socket motherboards• Single-core CPUs with Hyper Threading

(HT)• Multi-core CPUs• Multi-core CPUs with per-core HT

Thread Operations

Summer School 2012

I/O-boundCompute-bound

Compute-bound Operations

Summer School 2012

        static void Main(string[] args)        {            Thread workerThread = new Thread(Factorial);            workerThread.Start(10);            workerThread.Join();            Console.ReadKey(true);        }         static void Factorial(object arg)        {            int n = (int)arg;            int result = 1;            for (int i = 2; i <= n; i++) result *= i;            Console.WriteLine(result);        }

Thread Lifecycle

Summer School 2012

Too many threads = too bad

Summer School 2012

• Kernel object overhead• Memory overhead• Context switch overhead• GC overhead

Use threads wisely!

Thread Pool

Summer School 2012

• Designed for background processing• Self-balancing• Avoids much overhead

Thread Pool: Compute-bound

Summer School 2012

        static void Main(string[] args)        {            ThreadPool.QueueUserWorkItem(Factorial, 10);            Console.ReadKey(true);        }

        static void Factorial(object n)        {            int x = (int)n;            int result = 1;            for (int i = 2; i <= x; i++) result *= i;            Console.WriteLine(result);        }

Self-Balancing

Summer School 2012

Too few threads:Allocate more

Too many threads:Remove excessive ones

Task-Based Threading

Summer School 2012

“Work Stealing” Principle!

Tasks: Compute-bound

Summer School 2012

       static void Main(string[] args)       {           Task<Int32> task = new Task<Int32>(Factorial, 10);           task.Start();           task.Wait();           Console.WriteLine("The Sum is: " + task.Result);           Console.ReadKey(true);       }

       static int Factorial(object n)       {           int x = (int)n;           int result = 1;           for (int i = 2; i <= x; i++) result *= i;           return result;       }

Tasks 101

Summer School 2012

• Each task can have child tasks…• …and, therefore, throw multiple exceptions• “Task” does not mean “separate thread”!• Task templating via TaskFactory • Custom scheduling via TaskScheduler • Cooperative cancellation via

CancellationToken• Continuations, WaitAny, WaitAll, …

Tasks: Continuations

Summer School 2012

        static void Main(string[] args)

        {

            Task<Int32> task = new Task<Int32>(Factorial, 10);

            task.ContinueWith(t => Console.WriteLine("Completed"),

TaskContinuationOptions.OnlyOnRanToCompletion);

            task.ContinueWith(t => Console.WriteLine("Canceled"),

TaskContinuationOptions.OnlyOnCanceled);

            task.ContinueWith(t => Console.WriteLine("Error"),

TaskContinuationOptions.OnlyOnFaulted);

task.Start();

        }

Parallel LINQ (PLINQ)

Summer School 2012

enumerable.AsParallel()

Parallel Computations via PLINQ

Summer School 2012

  static void Main(string[] args)  {      Parallel.For(1, 10, n => Console.WriteLine("{0}: {1}", n, Factorial(n))); Enumerable.Range(1, 9).AsParallel().ForAll(n => /* Console... */); new int [] { 1, 2, 3 }.AsParallel().ForAll(n => /* Console... */);   }

Summer School 2012

.AsParallel()

does not mean faster execution

• Possible GC pressure ( i => new { ... } )

• Item acquisition and delegate execution are implemented in virtual methods

• You have to benchmark every particular situation

Synchronous I/O

Summer School 2012

Async I/O

Summer School 2012

Async I/O Benefits

Summer School 2012

• Less threads• Less GC• Faster debugging• Faster I/O if made parallel• Responsive UI (a must for WinPhone &

Silverlight)

Asynchronous Programming Model (APM)

Summer School 2012

Implemented by:

• Streams• Sockets• Serial Ports• SQL Commands• DNS Requests• Web Services …etc.

The Async Model Soup

Summer School 2012

Which one to follow? Stop the madness!

New APM in .NET 4.5 / C# vNext

Summer School 2012

         private byte[] GetURLContents(string url)         {             var content = new MemoryStream();             var webReq = (HttpWebRequest)WebRequest.Create(url);             using (var response = webReq.GetResponse())                 using (Stream responseStream = response.GetResponseStream())                     responseStream.CopyTo(content);             return content.ToArray();         }         private async Task<byte[]> GetURLContentsAsync(string url)         {             var content = new MemoryStream();             var webReq = (HttpWebRequest)WebRequest.Create(url);             using (WebResponse response = await webReq.GetResponseAsync())                 using (Stream responseStream = response.GetResponseStream())                     await responseStream.CopyToAsync(content);             return content.ToArray();         }

Threads and Shared State

Summer School 2012

Threads and Race Conditions

Summer School 2012

Synchronization Constructs

Summer School 2012

User-Mode• Volatile Read/Write• Interlocked

Kernel-Mode• Events• Semaphores• …and everything derived from them

Hybrid• Double-checked locking• Many others…

To make things even worse

Summer School 2012

• Language Compiler optimizations• JIT Compiler optimizations• CPU optimizations – Out-of-order execution – Branch prediction – Memory barriers

Atomic (Interlocked) Operations

Summer School 2012

• Atomic Swap• Test-and-Set• Compare-and-Swap• Fetch-and-Add• Load-Link / Store-Conditional

Kernel-mode Constructs

Summer School 2012

• WaitHandle (base class)• AutoResetEvent• ManualResetEvent• CountdownEvent• Semaphore• Mutex

Hybrid constructs: double-checked locking

Summer School 2012

internal sealed class Singleton {    private static readonly Object s_lock = new Object();    private static Singleton s_value = null;

    private Singleton() { }

    public static Singleton GetSingleton() {         if (s_value != null) return s_value;         Monitor.Enter(s_lock);         if (s_value == null) {             Singleton temp = new Singleton();             Interlocked.Exchange(ref s_value, temp);         }         Monitor.Exit(s_lock);         return s_value;     }}

Concurrent Collections

Summer School 2012

• BlockingCollection<T>

• ConcurrentBag<T>

• ConcurrentDictionary<K, V>

• ConcurrentQueue<T>

• ConcurrentStack<T>

Building Scalable Applications

Summer School 2012

• Avoid thread blocking• Avoid shared state• Avoid statics• Avoid mutable structures

Q & A

Summer School 2012

???yuriy.guts@eleks.com

Thank you!

Summer School 2012