McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web...
Transcript of McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web...
![Page 1: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/1.jpg)
McLab tools on the webDeepanjan Roy
Supervisor: Prof. Laurie Hendren
![Page 2: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/2.jpg)
Brief overview of the McLab project
![Page 3: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/3.jpg)
Dynamic Scientific Programming Languages(Especially MATLAB)
LANGUAGES, COMPILERS, AND VIRTUAL MACHINES
![Page 4: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/4.jpg)
Analysis tools ➔ McLab Static Analysis Framework
➔ Tamer: collection of even more awesome static
analysis tools
Static compilers ➔ Fortran
➔ X10: IBM language for high performance
computing
JIT compiler ➔ McVM and McJIT: interpreter and optimizing
Just-In-Time compiler supporting a non-trivial
subset of MATLAB
![Page 5: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/5.jpg)
Distributed and parallel computing
➔ McNumJS: High perf JS library for
numeric computations
➔ Pando: Large scale distributed numerical
computing through a web browser
➔ Several other libraries and toolkits that
help generate efficient parallel code for
CPUs and GPUs
Programmer tools ➔ McLab Web (My COMP 400 project!)
![Page 6: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/6.jpg)
Introducing MCLAB WEB
![Page 7: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/7.jpg)
Right now all the tools are geared towards compiler researchers / highly technical programmers
Majority of MATLAB users are scientists and engineers who want friendly tools
![Page 8: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/8.jpg)
➔ Clean and clutter-free interface
➔ It lives on the web: no installing packages, no OS incompatibility
➔ Instant updates - everyone is always on the latest version
➔ Auxiliary advantage: If people actually start using this tool widely, we can give
them an option to share their code with us. That way we will build up a large
repository of MATLAB code actually written by scientists and engineers, which is
very useful for research purposes
We are building a web based tool for McLab
![Page 9: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/9.jpg)
McLab WEB: Version 0.1
➔ Runs McLab tools and relays
analysis results
➔ Handles user uploaded zip files
and serves file contents /
directory tree
➔ Provides persistence and
lightweight sessions
➔ File explorer
➔ Code viewer with syntax
highlighting
➔ Message terminal
➔ API for code highlighting using
analysis results
➔ Base classes for panels
Backend - Python/Django Frontend - React/Flux
![Page 10: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/10.jpg)
Two McLab functionalities as proof of concept
Kind Analysis
Compilation to Fortran
![Page 11: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/11.jpg)
Analysis tools ➔ McLab Static Analysis Framework
➔ Tamer: collection of even more awesome static
analysis tools
Static compilers ➔ Fortran
➔ X10: IBM language for high performance
computing
JIT compiler ➔ McVM and McJIT: interpreter and optimizing
Just-In-Time compiler supporting a non-trivial
subset of MATLAB
![Page 12: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/12.jpg)
Live Demo
![Page 13: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/13.jpg)
Implementation Details
![Page 14: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/14.jpg)
Built on modern open source technologies
React.js Rendering application state to DOMFlux Managing data flowAce Code Editor/Viewer
Django Backend Web FrameworkSuperAgent Ajax library
![Page 15: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/15.jpg)
The challenge, for the most part, is not hard algorithms and data structures
It is writing code that is easy to reason about, and consequently easy to extend
![Page 16: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/16.jpg)
Built on modern open source technologies
React.js Rendering application state to DOMFlux Managing data flowAce Code Editor/Viewer
Django Backend Web FrameworkSuperAgent Ajax library
![Page 17: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/17.jpg)
Built on modern open source technologies
React.js Rendering application state to DOMFlux Managing data flow
![Page 18: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/18.jpg)
React.js
UI library built by Facebook
https://facebook.github.io/react/
![Page 19: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/19.jpg)
React.js philosophy
Component
![Page 20: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/20.jpg)
React.js philosophy
PropsState
Component
![Page 21: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/21.jpg)
React.js philosophy
render : (state, props) ⇒ UI Fragment
PropsState
render()Component
![Page 22: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/22.jpg)
React.js philosophy
render : (state, props) ⇒ UI Fragment
props: colorstate: animal
Component: AnimalBox
let box = <AnimalBox color=”blue”>
box.setState({animal: “cat”})
![Page 23: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/23.jpg)
React.js philosophy
render : (state, props) ⇒ UI Fragment
props: colorstate: animal
Component: AnimalBox
let box = <AnimalBox color=”green”>
box.setState({animal: “dog”})
![Page 24: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/24.jpg)
render : (state, props) ⇒ UI Fragment
Note how we’re mixing HTML with Javascript:JSX syntax
![Page 25: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/25.jpg)
React acts as if at each render cycle, the part of DOM under its control is destroyed and recreated
![Page 26: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/26.jpg)
If you change the state of the Root Level React Component, this is equivalent to doing a full refresh of the page
![Page 27: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/27.jpg)
Of course this makes the code much easier to reason about, butdestroying and recreating the whole DOM is slow
![Page 28: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/28.jpg)
React solution: render to Virtual DOMCalculate the diff to get minimal set of changesBatch update the DOM with those changes
![Page 29: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/29.jpg)
Data Flow
How do you respond to user interaction?
How do you communicate with the server, and render asynchronously loaded data?
![Page 30: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/30.jpg)
Flux
Architectural Pattern
https://facebook.github.io/flux/
Introduced by Facebook, but many implementations
![Page 31: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/31.jpg)
Flux philosophyEvery user action creates an action object
![Page 32: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/32.jpg)
The action object is passed on to the Dispatcher
DISPATCHER
![Page 33: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/33.jpg)
The Dispatcher passes on the actions to different stores
DISPATCHERTerminalBufferStore
SelectedFileStore
FileContentsStore
![Page 34: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/34.jpg)
When a store changes its internal state in response to an action, it raises a change event
DISPATCHERTerminalBufferStore
SelectedFileStore
FileContentsStore
![Page 35: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/35.jpg)
React components listen to these emit events, and recalculates their state when a store they are interested in changes
DISPATCHER
TerminalBufferStore
SelectedFileStore
FileContentsStore
<Terminal />
![Page 36: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/36.jpg)
React components listen to these emit events, and recalculates their state when a store they are interested in changes
DISPATCHER
Store 1.1
Store 1.2
Store 1.3
<TopComponent> <Child1> <GrandChild1 /> </Child1></TopComponent>
Store 2.1
Store 2.2
![Page 37: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/37.jpg)
DISPATCHER
Store 1.1
Store 1.2
Store 1.3
<TopComponent> <Child1> <GrandChild1 /> </Child1></TopComponent>
Store 2.1
Store 2.2
User interactions create new action objects and sends it back to the dispatcher to start another cycle
![Page 38: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/38.jpg)
Developer Demo
![Page 39: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/39.jpg)
ChallengesUsing non-react technologies with react components are not straightforward.
Fortunately, react provides a lot of lifecycle methods that let you plug into different phases of the render cycle.
![Page 40: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/40.jpg)
What I would do differently
➔ Implement backend in Javascript instead of Python
◆ Easy to move logic between client and server
◆ Much easier to do asynchronous processing (very hard to do non-blocking I/O in Python)
![Page 41: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/41.jpg)
Future
![Page 42: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/42.jpg)
Analysis tools ➔ McLab Static Analysis Framework
➔ Tamer: collection of even more awesome static
analysis tools
Static compilers ➔ Fortran
➔ X10: IBM language for high performance
computing
JIT compiler ➔ McVM and McJIT: interpreter and optimizing
Just-In-Time compiler supporting a non-trivial
subset of MATLAB
![Page 43: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/43.jpg)
Coming up next
➔ Linters for MATLAB code
➔ Code Profiling tools
➔ Integrate with McLab’s Javascript numeric computation libraries
![Page 44: McLab tools on the web - GitHub · Pando: Large scale distributed numerical computing through a web browser Several other libraries and toolkits that help generate efficient parallel](https://reader034.fdocuments.in/reader034/viewer/2022042102/5e7f3d68dc5b11274279caff/html5/thumbnails/44.jpg)
Thank you!
McLab tools on the Web | Deepanjan Roy