Matt Greenwood & Scott Draves - The Polyglot Beaker Notebook

Post on 08-Jul-2015

311 views 4 download

Preview:

Click to see full reader

Report this document

description

The Beaker Notebook is a new open source tool for collaborative data science. Like IPython, Beaker uses a notebook-based metaphor for idea flow. However, Beaker was designed to be polyglot from the ground up. That is, a single notebook may contain cells from multiple different languages that communicate with one another through a unique feature called autotranslation. You can set a variable in a Python cell and then read that variable in a subsequent R cell, and everything just works – magically. Beaker comes with built-in support for Python, R, Groovy, Julia, and Javascript. In addition, Beaker also supports multiple kinds of cells for text, like HTML, LaTeX, Markdown, and our own visualization library that allows for the plotting of large data sets. This talk will motivate the design, review the architecture, and include a live demo of Beaker in action.

Transcript of Matt Greenwood & Scott Draves - The Polyglot Beaker Notebook

Matt Greenwood

Scott Draves

Beaker is a notebook-style development environment for

working interactively with complex datasets.

Its polyglot architecture allows you to switch between

languages or add new ones with ease.

Beaker is Polyglot

Autotranslation

# set in Python

beaker.x = 10

// use in Javascript

beaker.x + 1

JSON

server

client

Web Browser, HTML5, Angular, MVC

JSON

Doc

Model

nginx

Core Server

Jetty

Jersey

Jackson

Guice

Groovy

GroovyR Python Javascript…

Evaluator Plugins

IPython/ZMQ

Python

Output Plugins

HTML Plot Table …

R

Java/Rserve

…

server

client

Web Browser, HTML5, Angular, MVC

JSON

Doc

Model

nginx

Core Server

Jetty

Jersey

Jackson

Guice

Groovy

GroovyR Python Javascript…

Evaluator Plugins

IPython/ZMQ

Python

Output Plugins

HTML Plot Table …

R

Java/Rserve

…

server

client

Web Browser, HTML5, Angular, MVC

JSON

Doc

Model

nginx

Core Server

Jetty

Jersey

Jackson

Guice

Groovy

GroovyR Python Javascript…

Evaluator Plugins

IPython/ZMQ

Python

Output Plugins

HTML Plot Table …

R

Java/Rserve

…

server

client

Web Browser, HTML5, Angular, MVC

JSON

Doc

Model

nginx

Core Server

Jetty

Jersey

Jackson

Guice

Groovy

GroovyR Python Javascript…

Evaluator Plugins

IPython/ZMQ

Python

Output Plugins

HTML Plot Table …

R

Java/Rserve

…

server

client

Web Browser, HTML5, Angular, MVC

JSON

Doc

Model

nginx

Core Server

Jetty

Jersey

Jackson

Guice

Groovy

GroovyR Python Javascript…

Evaluator Plugins

IPython/ZMQ

Python

Output Plugins

HTML Plot Table …

R

Java/Rserve

…

server

client

Web Browser, HTML5, Angular, MVC

JSON

Doc

Model

nginx

Core Server

Jetty

Jersey

Jackson

Guice

Groovy

GroovyR Python Javascript…

Evaluator Plugins

IPython/ZMQ

Python

Output Plugins

HTML Plot Table …

R

Java/Rserve

…

server

client

Web Browser, HTML5, Angular, MVC

JSON

Doc

Model

nginx

Core Server

Jetty

Jersey

Jackson

Guice

Groovy

GroovyR Python Javascript…

Evaluator Plugins

IPython/ZMQ

Python

Output Plugins

HTML Plot Table …

R

Java/Rserve

…

open source

polyglot

notebook

autotranslation

easy to extend

easy to use

www.BeakerNotebook.com

Autotranslation

# set in Python

beaker.x = 10

// use in Javascript

beaker.x + 1

Core Server // Groovy

beaker: { }# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

// Groovy

// GroovyCore Server

beaker: { }# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

Core Server // Groovy

beaker.x = 5+5

beaker: { }# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

Core Server // Groovy

beaker.x = 5+5

beaker: { }# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

set(“x”, 10)

Core Server // Groovy

beaker.x = 5+5

beaker: { }# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

se

t(“x

”, 1

0)

Core Server // Groovy

beaker: { x: 10 }# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

“be

ake

r.x +

1”

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

beaker.x + 1

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

beaker.x + 1

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

beaker.x + 1

ge

t(“x

”)

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

beaker.x + 1

10

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

beaker.x + 1

10

Problem:

the request and

the reply are in

different threads

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

beaker.x + 1

1010

Problem:

the request and

the reply are in

different threads

Solution:

use a

java.util.concurrent.

SynchronousQueue

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

beaker.x + 1

10

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

⇒ 11

11

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1

// Groovy

beaker.x = 5+5

# Python

⇒ 11

Core Server // Groovy

beaker: { x: 10}# Python

beaker.x + 1 ⇒ 11

// Groovy

beaker.x = 5+5

# Python

Copyright © 2022 FDOCUMENTS