Webinar #10 - 19 April 2017 Isolates in the Cloud · Webinar #10 - 19 April 2017 Isolates in the...

1

Webinar #10 - 19 April 2017Isolates in the Cloud

2

Agenda

3

Agenda Quick introduction to Futures and

Isolates

4


Isolates Building and launching a Linux VM

with Dyalog APL on the Amazon Elastic Compute Cloud (EC2)

5


Isolates Building and launching a Linux VM


Starting 20 VM's and using them as isolate servers

6

Agenda Quick introduction to Futures and Isolates Building and launching a Linux VM



A quick demo of parallel each (ll.EachXin the isolate workspace since 16.0).

7

Quick Introduction to Isolates

8


iss←isolate.New¨3⍴⊂''

9



10



An isolate looks, tastes, and feelslike a namespace, except thatEach isolate appears as an extension of the workspace,but exists in a separate process.iss is a 3-element vector ofreferences to three isolates.

11


iss←isolate.New¨3⍴⊂''iss.X←(1 2 3) (4 5) 6


12


X←1 2 3

X←4 5

X←6

iss←isolate.New¨3⍴⊂''iss.X←(1 2 3) (4 5) 6


13


X←1 2 3

X←4 5

X←6


iss.((+⌿ ÷ ≢) X)iss.X←(1 2 3) (4 5) 6


14


X←1 2 3

X←4 5

X←6


iss.((+⌿ ÷ ≢) X)iss.X←(1 2 3) (4 5) 6

2 4.5 6


15

Isolate Demo

16

A 2-Core Computer

How isolates work

A DyalogApplication

Dyalog ProcessesComputers

17

A 2-Core Computer

How isolates work

A DyalogApplication


iss←isolate.New¨4⍴⊂⍬

18

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2



19

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2



TCP Sockets

20

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Dyalog Processes Namespaces

1

Computers


TCP Sockets

21

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Isolate 2


1 2

Computers


TCP Sockets

22

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Isolate 2


1 2 3

Computers


TCP Sockets

Isolate 3

23

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Isolate 2


1 2 3 4

Computers


TCP Sockets

Isolate 3

Isolate 4

24

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Isolate 2


1 2 3 4

Computers

Another Computer


TCP Sockets

Isolate 3

Isolate 4

25

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Isolate 2


1 2 3 4

Computers

Another Computer


TCP Sockets

Isolate 3

Isolate 4

StartServer 'ip=10.0.0.1'

26

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Isolate 2


1 2 3 4

Computers

Another Computer

IsolateProcess 1

IsolateProcess 2 iss←isolate.New¨4⍴⊂⍬

TCP Sockets

Isolate 3

Isolate 4


27

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Isolate 2


1 2 3 4

Computers

Another Computer

IsolateProcess 1


AddServer '10.0.0.2'

TCP Sockets

Isolate 3

Isolate 4


28

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Isolate 2


1 2 3 4

Computers

Another Computer

IsolateProcess 1



TCP Sockets

Isolate 3

Isolate 4


29

A 2-Core Computer

How isolates work

A DyalogApplication

IsolateProcess 1

IsolateProcess 2

Isolate 1

Isolate 2


1 2 3 4

Computers

Another Computer

IsolateProcess 1



TCP Sockets

Isolate 3

Isolate 4


30

Isolate DemoFor more on isolates, see for example

"Parallel Programming in Dyalog APL"https://dyalog.tv/Dyalog14/?v=JvLWvyG7JEs

https://dyalog.tv/Dyalog14/?v=JvLWvyG7JEs

31

Agenda Quick introduction to isolates Building and launching a Linux VM



A quick demo of ll.EachX

32

Create an Amazon Machine Image (AMI)

33

Create an Amazon Machine Image (AMI) Select a base AMI with Ubuntu Linux installed

34

Create an Amazon Machine Image (AMI) Select a base AMI with Ubuntu Linux installed Configure Firewall

35

Create an Amazon Machine Image (AMI) Select a base AMI with Ubuntu Linux installed Configure Firewall Launch the VM

36

Create an Amazon Machine Image (AMI) Select a base AMI with Ubuntu Linux installed Configure Firewall Launch the VM Use WinSCP to copy Dyalog installer

37

Create an Amazon Machine Image (AMI) Select a base AMI with Ubuntu Linux installed Configure Firewall Launch the VM Use WinSCP to copy Dyalog installer Install Dyalog APL

38

Create an Amazon Machine Image (AMI) Select a base AMI with Ubuntu Linux installed Configure Firewall Launch the VM Use WinSCP to copy Dyalog installer Install Dyalog APL Use SSH to start Dyalog APL

39

Create an Amazon Machine Image (AMI) Select a base AMI with Ubuntu Linux installed Configure Firewall Launch the VM Use WinSCP to copy Dyalog installer Install Dyalog APL Use SSH to start Dyalog APL Connect to it with RIDE

40

Create an Amazon Machine Image (AMI) Select a base AMI with Ubuntu Linux installed Configure Firewall Launch the VM Use WinSCP to copy Dyalog installer Install Dyalog APL Use SSH to start Dyalog APL Connect to it with RIDE Save the new AMI

41

Create an Amazon Machine Image (AMI) Select a base AMI with Ubuntu Linux installed Configure Firewall Launch the VM Use WinSCP to copy Dyalog installer Install Dyalog APL Use SSH to start Dyalog APL Connect to it with RIDE Save the new AMI

50

+ Max 20 simultaneous instances

66





67

Isolates in the CloudDyalog Processes NamespacesComputers

SSH SessionsTCP Sockets

68

A Windows Computer

Isolates in the CloudDyalog Processes NamespacesComputers


69

A Windows Computer

Isolates in the CloudA Dyalog

Application

Dyalog Processes NamespacesComputers


70

A Windows Computer


Application


SSH Sessions

IsolateProcess

TCP Sockets

71

A Windows Computer


Application


SSH Sessions

IsolateProcess

IsolateProcess

TCP Sockets

72

A Windows Computer


Application


SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

TCP Sockets

73

A Windows Computer


Application


SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

TCP Sockets

74

A Windows Computer


Application


1

Computers

SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

TCP Sockets

75

A Windows Computer


Application


1 2

Computers

SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

TCP Sockets

76

A Windows Computer


Application


1 2

Computers

SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

3

TCP Sockets

77

A Windows Computer


Application


1 2

Computers

SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

3 4

TCP Sockets

78

A Windows Computer


Application


1 2

Computers

SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

3 4

TCP Sockets

79

A Windows Computer


Application


1 2

Computers

SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

3 4

Two Per VM

TCP Sockets

80

A Windows Computer


Application


1 2

Computers

An AWS EC2 Linux VMStartServer 'ip=81.187.219.129'

SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

3 4

Two Per VM

TCP Sockets

81

A Windows Computer


Application


1 2

Computers

An AWS EC2 Linux VM

IsolateProcess


SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

3 4

Two Per VM

TCP Sockets

82

Another VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VMAnother VM A Windows Computer


Application


1 2

Computers

An AWS EC2 Linux VM

IsolateProcess


SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

3 4

Two Per VM

TCP Sockets

83



Application


1 2

Computers

An AWS EC2 Linux VM

IsolateProcess


SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

3 4

21 22 23 24

17 18 19 20

13 14 15 16

9 10 11 12

5 6 7 8

Two Per VM

TCP Sockets

84



Application


1 2

Computers

An AWS EC2 Linux VM

IsolateProcess


SSH Sessions

IsolateProcess

IsolateProcess

IsolateProcess

IsolateProcess

3 4

21 22 23 24

17 18 19 20

13 14 15 16

9 10 11 12

5 6 7 8

Two Per VM

TCP Sockets

85





86

Summary Dyalog is creating tools, frameworks and samples

for distributed / cloud computing, including• APLSSH client to securely start processes on remote machines• Running Isolates in the Cloud• Microservice Frameworks (e.g. JSONServer)• Docker / Container tools

DAAAS (Dyalog APL As A Service) coming soon• We will publish "premium" AMI's with Dyalog APL,

JSONServer and MiServer pre-installed• Pre-build Docker containers with Dyalog installed• Hardest problem is to decide on pricing

We aim to publish a reference architecture for scalable services.

87

Webinar 11 Thursday, May 17th at 15:00 UTC:

Creating and managing your own User Commands

Webinar #10 - 19 April 2017 Isolates in the Cloud · Webinar #10 - 19 April 2017 Isolates in the...

Documents

Transcript of Webinar #10 - 19 April 2017 Isolates in the Cloud · Webinar #10 - 19 April 2017 Isolates in the...