Practical Erlang Programming Simon Thompson University of Kent.

95
Practical Erlang Programming Simon Thompson University of Kent

Transcript of Practical Erlang Programming Simon Thompson University of Kent.

Page 1: Practical Erlang Programming Simon Thompson University of Kent.

PracticalErlang

Programming

Simon ThompsonUniversity of Kent

Page 2: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 2Practical Erlang Programming

Contents

• Erlang History• Erlang Highlights• Open Telecom Platform• Frequency Server Example• Products• The Community• Events & Questions

Page 3: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 3Practical Erlang Programming

Erlang History: The Telecom Industry

• Routers, Switches• Base Stations• Network Infrastructure• Cell Phones

Page 4: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 4Practical Erlang Programming

Telecom Applications: Issues

• Complex

• No down time

• Scalable

• Maintainable

• Distributed

vs

• Time to Market

Access transport and switching networks

CellularPLMN

PSTN/ISDN

Data/ IPNetworks

CATV

Services

Past Single-service networks

Clients/applications

PresentMultiservice networks/client server

BackboneNetwork

Access Access Access

Content Content

Control

Communicationapplications

MediaGateways

Page 5: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 5Practical Erlang Programming

Erlang History

1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998

Prototypes of Telecom

applications

First projects launched

First Erlang Book Published

First products launched

Major projects started

OTP R1 released

Releasedas Open Source

Experiments started at the Computer Science Lab

“Find the right methods - design by prototyping.”

Mike Willliams, CS LAB

Make mistakes on a small scale, not in a production project.

Mike Willliams

It’s not good enough to have ideas – you must also be able to implement

them to know that they work

Mike Willliams

And the rest is history...

Page 6: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 6Practical Erlang Programming

“Erlang is going to be a very important language.It could be the next Java.”

Ralph JohnsonCo-author, “Design Patterns” (the “Gang-of-Four book”)

…if we had to start again we would probably use Erlang…

Mike ShaverMozilla Foundation

Next generation high-performance,highly reliable web sites are going to be implemented using Erlang and Berkeley DB.

Dr. Margo SeltzerOriginal author, Berkeley DB

And this is just the beginning...

Erlang History

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 The Future

January, 36,000 hits on erlang.org

Bluetail AB acquired for 152M USD

August, 1 million hits on erlang.org

Erlang R11 goes

multicore

Major new projects

started within Ericsson

ETC is founded

The first Erlang FactorySan Francisco

December2 million hits on erlang.org

Page 7: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 7Practical Erlang Programming

Erlang History: The Ancestors

Functional languages like ML or Miranda

Concurrent languages like Ada, Modula or

Chill

Logic languages like Prolog

Page 8: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 8Practical Erlang Programming

Contents

• Erlang History• Erlang Highlights• Open Telecom Platform• Frequency Server Example• Products• The Community• Events & Questions

Page 9: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 9Practical Erlang Programming

Erlang: Properties

• DeclarativeFunctional programming language, high

abstraction level, pattern matching and concise readable programs

Page 10: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 10Practical Erlang Programming

Erlang Highlights: Factorial

n! =1

n*(n-1)!

n = 0

n 1

Definition

-module(ex1).-export([factorial/1]).

factorial(0) -> 1;factorial(N) when N >= 1 -> N * factorial(N-1).

Implementation

Eshell V5.0.1 (abort with ^G)1> c(ex1).{ok,ex1}2> ex1:factorial(6).720

Factorial using Recursion

Page 11: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 11Practical Erlang Programming

-module(ex2).-export([qsort/1]).

qsort([Head|Tail]) -> First = qsort([X || X <- Tail, X =< Head]), Last = qsort([Y || Y <- Tail, Y > Head]), First ++ [Head|Last];qsort([]) -> [].

Erlang Highlights: High-level Constructs

QuickSort using List Comprehensions

Eshell V5.0.1 (abort with ^G)1> c(ex2). {ok,ex2}2> ex2:qsort([7,5,3,8,1]).[1,3,5,7,8]

"all objects Y taken from the list

Tail, where Y > Head"

Page 12: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 12Practical Erlang Programming

Erlang Highlights: High-level Constructs

-define(IP_VERSION, 4).-define(IP_MIN_HDR_LEN, 5).

……DgramSize = size(Dgram), <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16, ID:16, Flgs:3, FragOff:13, TTL:8, Proto:8, HdrChkSum:16, SrcIP:32, DestIP:32, Body/binary>> = Dgram, if (HLen >= 5) and (4*HLen =< DgramSize) -> OptsLen = 4*(HLen - ?IP_MIN_HDR_LEN), <<Opts:OptsLen/binary, Data/binary>> = Body, ….. end.

Parsing an IP Datagram using the Bit Syntax

Page 13: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 13Practical Erlang Programming

Erlang Highlights: High-level Constructs

Serialising tree data

[8,6,2,cat,2,dog,emu,fish]

Page 14: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 14Practical Erlang Programming

Erlang Highlights: High-level Constructs

Serialising tree data

deserialize([_|Ls]) -> listToTree(Ls).

listToTree([2,N]) -> {leaf,N};listToTree([N]) -> {leaf,N};listToTree([M|Rest] = Code) ->

{Code1,Code2} = lists:split(M-1,Rest),

{node, listToTree(Code1), listToTree(Code2) }.

treeToList({leaf,N}) -> [2,N];treeToList({node,T1,T2}) -> TTL1 = treeToList(T1), [Size1|_] = TTL1, TTL2 = treeToList(T2), [Size2|List2] = TTL2, [Size1+Size2|TTL1++List2].

Page 15: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 15Practical Erlang Programming

Erlang Highlights: High-level Constructs

Functions as arguments, expressions, results

evens([]) -> [];

evens([X|Xs]) -> case X rem 2 == 0 of

true -> [X| evens(Xs)]; _ -> evens(Xs)

end.

evens([2,3,2,1,5,4]) = [2,2,4]

evens(Xs) -> filter (fun(X) -> X rem 2 == 0

end, Xs).

filter(P,[]) -> [];

filter(P,[X|Xs]) -> case P(X) of

true ->

[X| filter(P,Xs)]; _ -> filter(P,Xs)

end.

Page 16: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 16Practical Erlang Programming

Erlang: Properties

• Declarative

• Concurrency

Functional programming language, high abstraction level, pattern matching and concise

readable programsEither transparent or explicit concurrency, light-weight processes and highly scalable

Page 17: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 17Practical Erlang Programming

Erlang Highlights: Concurrency

Creating a new process using spawn

-module(ex3).-export([activity/3]).

activity(Name,Pos,Size) -> …………

Pid = spawn(ex3,activity,[Joe,75,1024])

activity(Joe,75,1024)

Page 18: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 18Practical Erlang Programming

Erlang Highlights: Concurrency

10 100 1,000 10,000 100,000Number of processes1

10

100

1,000

Mic

rose

con

ds/

pro

cess

erlang

java

C#

Source:Joe ArmstrongSICS

Processcreationtime

Page 19: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 19Practical Erlang Programming

Erlang Highlights: Concurrency

Processes communicate by asynchronous message passing

Pid ! {data,12,13} receive {start} -> ……… {stop} -> ……… {data,X,Y} -> ………end

receive {start} -> ……… {stop} -> ……… {data,X,Y} -> ………end

Page 20: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 20Practical Erlang Programming

Erlang Highlights: Concurrency

Processes communicate by asynchronous message passing

Selective receive:

receive {first,Payload1} -> … process Payload1 … receive {second,Payload2} -> … process Payload2 … end;end;

Whatever the arrival order of the messages

{first,Payload1} {second,Payload2}

the first is processed first.

It won't deadlock if the second arrives first.

Page 21: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 21Practical Erlang Programming

Erlang Highlights: Concurrency

10 100 1,000 10,000 100,000Number of processes1

10

1,000

100,000

Mic

rose

con

ds/

mes

sage

erlang

java

C#

10,000

100

1Source:Joe ArmstrongSICS

Messagepassingtimes

Page 22: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 22Practical Erlang Programming

Erlang: Properties

• Declarative

• Concurrency

• Soft real-time

Functional programming language, high abstraction level, pattern matching and concise

readable programsEither transparent or explicit concurrency, light-weight processes and highly scalable

Response times in the order of milliseconds per-process garbage collection

Page 23: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 23Practical Erlang Programming

Erlang: Properties

• Declarative

• Concurrency

• Soft real-time

• Robustness

Functional programming language, high abstraction level, pattern matching and concise

readable programsEither transparent or explicit concurrency, light-weight processes and highly scalable

Response times in the order of milliseconds per-process garbage collection

Simple and consistent error recovery, supervision hierarchies and "Program for the

correct case"

Page 24: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 24Practical Erlang Programming

Erlang Highlights: Robustness

Cooperating processes may be linked together

usingspawn_link(…,…,…)orlink(Pid)

Page 25: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 25Practical Erlang Programming

Erlang Highlights: Robustness

When a process terminates, an exit signal is sent to all linked processes

… and the termination is propagated

Page 26: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 26Practical Erlang Programming

Erlang Highlights: Robustness

Exit signals can be trapped and received as messages

receive {‘EXIT’,Pid,...} -> ...end

Page 27: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 27Practical Erlang Programming

Erlang Highlights: Robustness

Robust systems can be built by layering

“Supervisors”

“Workers”

Page 28: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 28Practical Erlang Programming

Erlang Highlights: Robustness

Supervision hierarchies are policy-based.

Supervision policies can govern• how many restarts in a given time• when to delegate restarts to your supervisor• do you restart your siblings?• …

Page 29: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 29Practical Erlang Programming

Erlang: Properties

• Declarative

• Concurrency

• Soft real-time

• Robustness

• Distribution

Functional programming language, high abstraction level, pattern matching and concise

readable programsEither transparent or explicit concurrency, light-weight processes and highly scalable

Response times in the order of milliseconds per-process garbage collection

Simple and consistent error recovery, supervision hierarchies and "Program for the

correct case"Explicit or transparent distribution

Network-aware runtime system

Page 30: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 30Practical Erlang Programming

Erlang Highlights: Distribution

Erlang Run-Time SystemErlang Run-Time System Erlang Run-Time SystemErlang Run-Time System

B ! Msg

network

C ! Msg

Page 31: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and ConsultingPractical Erlang Programming

Erlang Highlights: Distribution

loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end.

{rex, Node} ! {self(), {apply, M, F, A}},receive {rex, Node, What} -> Whatend

{rex, Node} ! {self(), {apply, M, F, A}},receive {rex, Node, What} -> Whatend

{rex, Node} ! {self(), {apply, M, F, A}},receive {rex, Node, What} -> Whatend

loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end.

loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end.

loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end.

Simple Remote Procedure Call

Page 32: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 32Practical Erlang Programming

Erlang Highlights: Distribution

Built in support for distribution through socket-based libraries for SNMP, TCP, HTTP, SSL, …

… allowing the systems to have more security than the default distribution mechanism.

Page 33: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 33Practical Erlang Programming

Erlang: Properties

• Declarative

• Concurrency

• Soft real-time

• Robustness

• Distribution

• Hot code loading

Functional programming language, high abstraction level, pattern matching and concise

readable programsEither transparent or explicit concurrency, light-weight processes and highly scalable

Response times in the order of milliseconds per-process garbage collection

Simple and consistent error recovery, supervision hierarchies and "Program for the

correct case"Explicit or transparent distribution

Network-aware runtime system

Easily change code in a running system. Enables non-stop operation Simplifies testing

Page 34: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 34Practical Erlang Programming

Erlang Highlights: Hot Code Swap

Version 1 Version 2

Page 35: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 35Practical Erlang Programming

Erlang: Properties

• Declarative

• Concurrency

• Soft real-time

• Robustness

• Distribution

• Hot code loading

• External interfaces

Functional programming language, high abstraction level, pattern matching and concise

readable programsEither transparent or explicit concurrency, light-weight processes and highly scalable

Response times in the order of milliseconds per-process garbage collection

Simple and consistent error recovery, supervision hierarchies and "Program for the

correct case"Explicit or transparent distribution

Network-aware runtime system

Easily change code in a running system. Enables non-stop operation Simplifies testing

"Ports" to the outside world behave as Erlang processes

Page 36: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 36Practical Erlang Programming

Erlang Highlights: External Interfaces

PortPortExternalprocessExternalprocess

Port ! {self(), {command, [1,2,3]}}

Page 37: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 37Practical Erlang Programming

Erlang Highlights: External Interfaces

PortPortExternalprocessExternalprocess

receive {Port, {data, Info}} ->end

Page 38: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 38Practical Erlang Programming

Erlang Highlights: Jinterface: Erlang+Java

public class ServerNode { public static void main (String[] _args) throws Exception{ OtpNode bar = new OtpNode("bar"); OtpMbox mbox = bar.createMbox("facserver"); OtpErlangObject o; OtpErlangTuple msg; OtpErlangPid from; BigInteger n; OtpErlangAtom ok = new OtpErlangAtom("ok"); while(true) try {

o = mbox.receive(); msg = (OtpErlangTuple)o; from = (OtpErlangPid)(msg.elementAt(0)); n = ((OtpErlangLong)(msg.elementAt(1))).bigIntegerValue(); OtpErlangObject[] reply = new OtpErlangObject[2]; reply[0] = ok; reply[1] = new OtpErlangLong(Factorial.factorial(n)); OtpErlangTuple tuple = new OtpErlangTuple(reply); mbox.send(from,tuple);

} catch(OtpErlangExit e) { break; } }}

-module(myrpc).f(N) -> {facserver, 'bar@STC'} ! {self(), N}, receive {ok, Res} -> io:format("~p! is ~p.~n", [N,Res])end.

-module(myrpc).f(N) -> {facserver, 'bar@STC'} ! {self(), N}, receive {ok, Res} -> io:format("~p! is ~p.~n", [N,Res])end.

Page 39: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 39Practical Erlang Programming

Erlang Highlights: Other interfaces

• C• Ruby• Python• Emacs Lisp• Perl• Scheme• Haskell• …

Page 40: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 40Practical Erlang Programming

Erlang: Properties

• Declarative

• Concurrency

• Soft real-time

• Robustness

• Distribution

• Hot code loading

• External interfaces

• Portability

Functional programming language, high abstraction level, pattern matching and concise

readable programsEither transparent or explicit concurrency, light-weight processes and highly scalable

Response times in the order of milliseconds per-process garbage collection

Simple and consistent error recovery, supervision hierarchies and "Program for the

correct case"Explicit or transparent distribution

Network-aware runtime system

Easily change code in a running system. Enables non-stop operation Simplifies testing

"Ports" to the outside world behave as Erlang processes

Erlang runs on any UNIX, Windows, Vx Works, ...Supports heterogeneous networks

Page 41: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 41Practical Erlang Programming

Erlang: Properties

• Declarative

• Concurrency

• Soft real-time

• Robustness

• Distribution

• Hot code loading

• External interfaces

• Portability

• SMP Support

Functional programming language, high abstraction level, pattern matching and concise

readable programsEither transparent or explicit concurrency, light-weight processes and highly scalable

Response times in the order of milliseconds per-process garbage collection

Simple and consistent error recovery, supervision hierarchies and "Program for the

correct case"Explicit or transparent distribution

Network-aware runtime system

Easily change code in a running system. Enables non-stop operation Simplifies testing

"Ports" to the outside world behave as Erlang processes

Erlang runs on any UNIX, Windows, Vx Works, ...Supports heterogeneous networks

Symmetric multiprocessing support. Takes full advantage of multiple CPU architectures.

Page 42: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 42Practical Erlang Programming

Erlang Highlights: SMP Support

The Industry’s Dilema

• The shift to multicore is inevitable.• Parallelizing legacy C (and Java) code is very hard.• Debugging parallelized C (and Java) is even harder.• ”...but what choice do we have?”

Page 43: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 43Practical Erlang Programming

Erlang Highlights: SMP Support

Erlang SMP ”Credo”

SMP should be transparent to the programmer inSMP should be transparent to the programmer inmuch the same way as Erlang Distributionmuch the same way as Erlang Distribution

• You shouldn’t have to think about it ...but sometimes you must

• Use SMP mainly for stuff that you’d make concurrent anyway

• Erlang uses concurrency as a structuring principle• Model for the natural concurrency in your problem

Page 44: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 44Practical Erlang Programming

Erlang Highlights: SMP Support

• Erlang on multicore

• SMP prototype ‘97, First OTP release May ‘06.

• Mid -06 we ran a benchmark mimicking call handling (axdmark) on the (experimental) SMP emulator. Observed speedup/core: 0.95

• First Ericsson product (TGC) released on SMP Erlang in Q207.

”Big bang” benchmark on Sunfire T2000

Simultaneous processes

http://www.franklinmint.fm/blog/archives/000792.html

16 schedulers

1 scheduler

Page 45: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 45Practical Erlang Programming

Erlang Highlights: SMP Support

Case Study: Telephony Gateway Controller

• Mediates between legacy telephony and multimedia networks.

• Hugely complex state machines• + massive concurrency.• Developed in Erlang.• Multicore version shipped to customer Q207.• Porting from 1-core PPC to 2-core Inteltook < 1 man-year (including testing).

AXE TGC

GWGW GW

Page 46: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 46Practical Erlang Programming

Erlang Highlights: SMP Support

3.17X call/sec

1.55X call/sec

0.4X call/sec

AXDCPB5

14X call/sec

7.6X call/sec

2.1X call/sec

AXDCPB6

ISUP-ISUP /Intra MGW

ISUP-ISUP /Inter MGW

POTS-POTS /AGW

Trafficscenario

5.5X call/sec

3.6X call/sec

X call/sec

IS/GCP1slot/board

7.7X call/sec

One core used

2.3X call/sec

One core used

IS/GEPDual coreOne core running

2slots/board

26X call/sec

13X call/secOTP R11_3

beta+patches

4.3X call/secOTP R11_3

beta+patches

IS/GEPDual coreTwo cores

running2slots/board

Case Study: Telephony Gateway Controller

Page 47: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 47Practical Erlang Programming

Erlang Highlights: SMP supportSpeedup on 4 Hyper Threaded Pentium4

1

1.92 2.05

2.733.11

3.633.79

3.96

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

1 2 3 4 5 6 7 8

# Schedulers

Speddup

• Chatty• 1000 processes created• Each process randomly sends req/recieves ack from all other processes

Page 48: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 48Practical Erlang Programming

Erlang Highlights: SMP Support

Erlang VM

Scheduler

run queuenon-SMP VM

Page 49: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 49Practical Erlang Programming

Erlang Highlights: SMP Support

Erlang VM

Scheduler #1

Scheduler #2

Scheduler #N

run queueCurrent SMP VMOTP R11/R12

Page 50: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 50Practical Erlang Programming

Erlang Highlights: SMP Support

Erlang VM

Scheduler #1

Scheduler #2

run queue

Scheduler #2

Scheduler #N

run queue

run queue

migrationlogic

migrationlogic

Latest SMP VMOTP R13

Page 51: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 51Practical Erlang Programming

• Speedup of ”Big Bang” on a Tilera Tile64 chip (R13α)• 1000 processes, all talking to each other

Memory allocation locks dominate...

Erlang Highlights: SMP Support

Multiplerun queues

Singlerun queue

Speedup: Ca 0.43 * N @ 32 cores

Page 52: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 52Practical Erlang Programming

Contents

• Erlang History• Erlang Highlights• Open Telecom Platform• Frequency Server Example• Products• The Community• Events & Questions

Page 53: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 53Practical Erlang Programming

Open Telecom Platform

O PT

Applications &Libraries

System DesignPrinciples

Page 54: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 54Practical Erlang Programming

OTP: System Design Principles

• A set of abstract principles and design rules. They describe the software architecture of an Erlang System

Needed so existing tools will be compatible with them

Facilitate understanding of the system among teams• A set of generic behaviours.

Each behaviour is a formalisation of a design pattern

Contains frameworks with generic code Solve a common problem Have built in support for debugging and software upgrade

Facilitate understanding of the sub blocks in the system

Page 55: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 55Practical Erlang Programming

Callback module

generic behaviour module

OTP: Design Patterns

• The idea is to split the code in two parts.• The generic part is called the generic behaviour.

They are provided by OTP as library modules.• The specific part is called the callback module.

They are implemented by the programmer.

Server

Page 56: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 56Practical Erlang Programming

OTP: Design Patterns

• Generic Servers Used to model client server behaviour

• Generic Finite State Machines Used for finite state machine programming

• Generic Event Server / Manager Used for writing event handlers

• Supervisor Used for fault-tolerant supervision trees

• Application Used to encapsulate resources and functionality

Page 57: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 57Practical Erlang Programming

OTP: Design Patterns

• An application, its supervision tree and its workers.

• They can be implemented using generic behaviours.

ApplicationSupervisor

Workers

Workers

Page 58: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 58Practical Erlang Programming

OTP:Applications and Libraries

• Basic Applications Erlang Runtime System, Kernel, Compiler, Standard Lib

System Architecture Support Library

• Database Applications Mnesia

• Distributed relational database with query language ODBC

• Interface to accessing SQL databases

• Operations and Maintenance Applications Operating System Monitor Simple Network Management Protocol

• OTP MIBs, generic SNMP Erlang related MIBs

• Corba Object Request Broker Applications

Page 59: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 59Practical Erlang Programming

OTP:Applications and Libraries

• Interface and Communications Applications ASN1 compiler Crypto Graphics System Inets

• Ip related services including TCP, UDP, HTTP and FTP

Java Interface Megaco Stack

• Megaco/H.248 Stack

SSH, SSL XML Parsing Erlang to C Interface

Page 60: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 60Practical Erlang Programming

OTP:Applications and Libraries

• Tool Applications Appmon, Pman, Debugger, Event Trace, Table Visualiser• Graphical debugging tools

Dialyzer• Type checking tool

Docbuilder, Edoc• Documentation tools

Invisio, Observer Percept

• Erlang Concurrency Profiler Tool Parse & Syntax Tools

• YECC Erlang Implementation• Handling abstract Erlang syntax trees

Runtime and Profiling Tools• Tools for debugging production systems• Tools for profiling, coverage analysis, and development support

Page 61: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 61Practical Erlang Programming

OTP ComponentsOTP Components

Standard LibrariesStandard Libraries

Erlang Run-Time SystemErlang Run-Time System

Hardware and Operating SystemHardware and Operating System

OTP Applications written in Erlang

Applicationswritten in C,C++ or Java

OTP: Systems Overview

Page 62: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 62Practical Erlang Programming

Contents

• Erlang History• Erlang Highlights• Open Telecom Platform• Frequency Server Example• Products• The Community• Events & Questions

Page 63: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 63Practical Erlang Programming

Mobile Frequency Server

• Scenario: allocation of frequencies for mobile communication. A client can request that a frequency be allocated.

A client can deallocate a frequency.• Model

The frequency server is a separate process … … communicating with the clients.

• Alternatives Communication mechanism? Communication protocol? Code the server yourself, or use OTP?

Page 64: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 64Practical Erlang Programming

Frequency Allocation and Deallocation

allocate({[], Allocated}, _Pid) -> {{[], Allocated}, {error, no_frequency}};

allocate({[Freq|Free], Allocated}, Pid) -> {{Free, [{Freq, Pid}|Allocated]}, {ok, Freq}}.

deallocate({Free, Allocated}, Freq) -> NewAllocated =lists:keydelete(Freq, 1, Allocated), {[Freq|Free], NewAllocated}.

• A pair of lists models the free and allocated frequencies.

• Return the new pair plus a result message indicating failure or success.

• Deallocation always succeeds, assuming the Freq was already allocated.

• Nice example of pattern matching!

Page 65: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 65Practical Erlang Programming

Server 1: everything explicit

loop(Frequencies) -> receive {request, Pid, allocate} -> {NewFrequencies, Reply} = allocate(Frequencies, Pid), Pid ! {reply, Reply}, loop(NewFrequencies);

{request, Pid , {deallocate, Freq}} -> NewFrequencies = deallocate(Frequencies, Freq), Pid ! {reply, ok}, loop(NewFrequencies);

{request, Pid, stop} -> Pid ! {reply, ok} end.

• Messages are triples of request, process ID of the

sender and service required.

• Replies are a pair reply, result (if any).

• Then loop again, with (possibly) updated frequency data.

• Pattern matching again!

Page 66: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 66Practical Erlang Programming

Setting up the server

start() -> register(frequency, spawn(frequency0, init, [])).

init() -> Frequencies = {get_frequencies(), []}, loop(Frequencies).

% Hard Coded

get_frequencies() -> [10,11,12,13,14,15].

%% The Main Loop

loop(Frequencies) -> receive {request, Pid, allocate} -> etc.

• The system is started by spawning a process to run the init function …

• … and then registering this process with the name frequency.

• init itself will set up the loop data and call the loop for the first time.

• DIY control.

Page 67: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 67Practical Erlang Programming

Client 1: everything explicit

1> c(frequency).{ok,frequency}2> frequency:start().true3> frequency ! {request, self(), allocate}.{request,<0.40.0>,allocate}4> receive {reply,Reply} -> Reply end.{ok,10}5> …

• Messages are triples of request, process ID of

the sender and service

required.

• Replies are a pair reply, result (if any).

• Process structure and message protocol exposed.

Page 68: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 68Practical Erlang Programming

Client 2: a functional interface

allocate() -> frequency ! {request, self(), allocate}, receive {reply, Reply} -> Reply end.

deallocate(Freq) -> frequency ! {request, self(), {deallocate, Freq}}, receive {reply, Reply} -> Reply end.

1> frequency:start().true2> frequency:allocate(). {ok,10}3> …

• A functional API for the operations hides the process information and message protocol

• The function sends the message and handles the reply.

• Compare the example interaction.

• Higher-level API but concurrent behaviour is still hand coded.

Page 69: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 69Practical Erlang Programming

Building the server in OTP: gen_server

• Generic server behaviour. Concurrency, error recovery, supervision protocols, timeouts, … are all handled generically.

In contrast to our spawn / register / init / loop implementation which was all hand coded, and did nothing beyond simple looping.

• Specific behaviour is confined to Message exchanges with the server. Server setup and termination options.

• The system has the same client API as before … another reason for the functional interface.

Page 70: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 70Practical Erlang Programming

Setup and client API functions

-behavior(gen_server).

start() -> gen_server:start_link({local,?MODULE}, ?MODULE,get_frequencies(),[]).

stop() -> gen_server:cast(?MODULE,stop).

%% The client API

allocate() -> gen_server:call(?MODULE,allocate).

deallocate(Freq) -> gen_server:call(?MODULE,{deallocate,Freq}).

• The behavior implies the implementation of various callback functions.

• The server is started using one of the gen_server API functions.

• Communication too, through a call to the gen_server.

• ?MODULE is the name used for the server.

Page 71: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 71Practical Erlang Programming

The callback functions

init(FreqList) -> Freqs = {FreqList, []}, {ok, Freqs}.

terminate(_,_) -> ok.

handle_cast(stop, Freqs) -> {stop, normal, Freqs}.

handle_call(allocate, From, Freqs) -> {NewFreqs, Reply} = allocate(Frequencies, From), {reply, Reply, NewFreqs};

handle_call({deallocate,Freq}, _From, Freqs) -> NewFreqs = deallocate(Freqs, Freq), {reply,ok, NewFreqs}.

• These functions give the particular pieces of behaviour which specify setup, tear down and comms.

• There's nothing here about the communication medium, but the protocol is limited to simple request / response.

• Can be synchronous or asynchronous.

Page 72: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 72Practical Erlang Programming

Contents

• Erlang History• Erlang Highlights• Open Telecom Platform• Frequency Server Example• Products• The Community• Events & Questions

Page 73: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 73Practical Erlang Programming

Products: Ericsson

• Mobility Server (roaming over DECT networks)• Teletrain (Call center solution)• ANx (Broadband access system)• Net Sim (AXE simulator)• AXD301 (ATM Switch)• GPRS (Packet switching over GSM)• Integrated Site • Telephony Gateway Controller

Page 74: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 74Practical Erlang Programming

Products: AXD301 Switch

•A Telephony-Class, scalable (10 –160 GBps) ATM switch

•Designed from scratch in less than 3 years

•AXD 301 Success factors: Competent organisation and people Efficient process Excellent technology (e.g. Erlang/OTP)

Page 75: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 75Practical Erlang Programming

Products: AXD301 Switch

•Erlang: ca 1.5 million lines of code Nearly all the complex control logic

Operation & Maintenance Web server and runtime HTML/JavaScript generation

•C/C++: ca 500k lines of code Third party software Low-level protocol drivers Device drivers

•Java: ca 13k lines of code Operator GUI applets

Page 76: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 76Practical Erlang Programming

Products: AXD301 Switch

•Using Erlang in Complex Systems Fits very well with the incremental design method High programmer satisfaction Outstanding support for robustness and concurrency Very few side-effects easier to add/change single components

Small directed teams can achieve impressive results•Productivity estimates

Similar line/hour programmer productivity 4-10 fewer lines of source code (compared to C/C++, Java, PLEX)

Similar number of faults per 1000 lines of source code

Page 77: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 77Practical Erlang Programming

"We are extremely pleased with the outcome of the initial phase of this project. This is a major step in the phased development of what we believe is a world-leading Next Generation Network," said Richard Newman, General Manager of Planning and Delivery of Network Transport at BT Wholesale.

Ericsson Press Release 5 July, 2002

“As a matter of fact, the network performance has been so reliable that there is almost a risk that our field engineers do not learn maintenance skills”

Bert Nilsson, Director NGS-Programs Ericsson

Ericsson Contact, Issue 19 2002

Products: AXD301 Switch

“Since cut-over of the first nodes in BT’s network in January 2002, only one minor fault has occurred, resulting in 99.9999999% availability.”

Page 78: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 78Practical Erlang Programming

Products: Outside Ericsson

• Telia (Sweden)Call Center Applications

• Tail-F (Sweden)NETCONF, WEB, CLI, SNMP

• Kreditor (Sweden), e-commerce

• T-Mobile (UK)WAP, SMS, IN services

• Teba Bank (South Africa)Commercial Banking Systems

• Facebook (USA)Chat channel servers

Page 79: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 79Practical Erlang Programming

Contents

• Erlang History• Erlang Highlights• Open Telecom Platform• Frequency Server Example• Products• The Community• Events & Questions

Page 80: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 80Practical Erlang Programming

Community: Downloads from erlang.org

0

2,000

4,000

6,000

8,000

10,000

Sep-06Oct-06Nov-06Dec-06Jan-07Feb-07Mar-07Apr-07May-07Jun-07Jul-07Aug-07Sep-07Oct-07Nov-07Dec-07Jan-08Feb-08Mar-08Apr-08May-08Jun-08Jul-08Aug-08Sep-08Oct-08Nov-08Dec-08Jan-09

Windows

Unix

Total

Page 81: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 81Practical Erlang Programming

Community: Page hits on erlang.org

0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1,400,000

1,600,000

1,800,000

2,000,000

2,200,000

2,400,000

2,600,000

2,800,000

3,000,000

De

c-

98

Ju

n-

99

De

c-

99

Ju

n-

00

De

c-

00

Ju

n-

01

De

c-

01

Ju

n-

02

De

c-

02

Ju

n-

03

De

c-

03

Ju

n-

04

De

c-

04

Ju

n-

05

De

c-

05

Ju

n-

06

De

c-

06

Ju

n-

07

De

c-

07

Ju

n-

08

De

c-

08

Page 82: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 82Practical Erlang Programming

• HIPE (Uppsala University, Sweden)

• Telecom Programming (Heriot-Watt University, Scotland)

• Major Projects (University of Corũna, Corũna, Spain)

• Code Refactoring (University of Kent, Canterbury, UK)

• Code Refactoring (Eötvös Loránd University, Hungary)

• Robots & Agents (University of Catania, Italy)

Community: Research

Page 83: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 83Practical Erlang Programming

• CouchDB, A Distributed Robust document database server

• Wings 3D, a 3D modeller based on Nendo

• YAWS, Yet Another Web Server

• RabbitMQ, high performance enterprise messaging

• Ejabberd, instant messaging server

• Erlyweb, component-based web development framework

Community: Open Source

Page 84: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 84Practical Erlang Programming

CouchDB• Schema-less database,

document oriented.• Distributed, fault-tolerant, • Robust, incremental replication.• Bi-directional conflict detection

and resolution.• Queryable and indexable using

a table-oriented view engine.• JavaScript acting as the default

view definition language.

• Apache project … • … supported by IBM.

QuickTime™ and a decompressor

are needed to see this picture.

Page 85: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 85Practical Erlang Programming

Community: Wings 3D

Examples fromwww.wings3d.com

Page 86: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 86Practical Erlang Programming

Community: YAWS

• YAWS thoughput (KBytes/second) vs. load.

Page 87: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 87Practical Erlang Programming

Community: Web Sites

www.planeterlang.org

www.trapexit.org

www.erlang-consulting.com

www.erlang.org

www.erlang-factory.com

Page 88: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 88Practical Erlang Programming

Community: Blogs

http://armstrongonsoftware.blogspot.com/

http://dukesoferl.blogspot.com

http://yarivsblog.com/

http://blog.tornkvist.org/

http://planet.trapexit.org/

Page 89: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 89Practical Erlang Programming

Community: New Books

• Programming Erlang, Software for a Concurrent World by Joe Armstrong

• First English book in 10 years! The previous one published in 2002 was in

French!

• Concurrent Programming with Erlang/OTP by Martin Logan, Eric Merritt, Richard

Carlsson and Robert Calco

• Erlang Web Applications: Problem-Design-Solution by Nick Gerakines

QuickTime™ and a decompressor

are needed to see this picture.

Page 90: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 90Practical Erlang Programming

Shameless Advertisement ….

Erlang ProgrammingErlang ProgrammingA Concurrent Approach to Software Development

Francesco Cesarini Simon

Thompson

www.erlangprogramming.org

Page 91: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 91Practical Erlang Programming

Contents

• Erlang History• Erlang Highlights• Open Telecom Platform• Frequency Server Example• Products• The Community• Events & Questions

Page 92: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 92Practical Erlang Programming

Events

• ACM SIGPLAN Erlang Workshop Held every year as a satellite event of the International Conference of Functional Programming

Alternates between the US and Europe A forum to submit academic papers

• International Erlang User Conference Held every year in Stockholm, Sweden Informal mixture of industrial applications and research

Page 93: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 93Practical Erlang Programming

Events

• Erlang Factory (www.erlang-factory.com) Held twice a year in US and Europe A mix of talks, tutorials, close industrial connections

London June 24London June 24thth and 25 and 25thth, 2009, 2009

• ErlLounge (Noun) A gathering of Erlang developers who meet to drink beer, have a bite and discuss Erlang (In that order). The were first held in Sweden in 2001 and are now regularly held worldwide.

Page 94: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 94Practical Erlang Programming

Questions

Page 95: Practical Erlang Programming Simon Thompson University of Kent.

© 2001-2009, Simon Thompson and Erlang Training and Consulting 95Practical Erlang Programming

Contents

• Erlang History• Erlang Highlights• Open Telecom Platform• Frequency Server Example• Products• The Community• Events & Questions