Technical Road Map Morten Kromberg 0. 1 (Same Old) Focus Areas -Performance -(includes parallel...

Technical Road MapMorten Kromberg

1

2

(Same Old) Focus Areas

- Performance- (includes parallel computing)

- Portability- (includes user interfaces)

- Language- (includes increased focus on functional programming)

- Growing: Tools for Application Building

Technical Road Map - Dyalog'15

(Big) Bets We Are Placing

1. Parallel Processing is getting Critical

2. New Platforms will soon be Important

3. Most new User Interfaces will useeither HTML5/JS or WPF/XAML

1. Parallel Processing

Bang for the buck, here and now• Futures and Isolates

– Allow developer to easily identify and manage parallel sections of code

• Dyalog Compiler– Minimise interpreter overhead– [Parallel] Optimisations to be added over

time• Designed to require minimal re-coding

of existing applications

Optimised Bytecode Execution Engine

1. Parallel Processing, …

• Co-Dfns Compiler– Externally funded research

• Performed by Aaron Hsu at Indiana University– Focus on purely functional code (d-fns only)

• But happy to work on scalars ;-)– Will require application refactoring

• Few system functions and ”external” interfaces– If successful, will enable very highly

parallel applications– Targets GPUs / massively parallel

architectures

6

Version 14.1 Core Performance


7

Version 14.0 Core Performance


The “Hui Effect”: v14.0 runs apps 15-30% faster

8

14.0 + 14.1


9

V14.1 Performance Features

• “Special Code” added or enhanced– Idioms: ≢⍴ ⊣/ ⊢/ (?⍴) – ∧.= ∨.≠ ∘.= n⍴scalar

bool⊂ bool⍋• Bytecode Execution Engine enhanced to handle

global names and control structures:– Now compiles 59% of functions in “a major

application” (up from ~1% in v14.0)• Compiled code speeds up by factor of 2 on small args

– NB: A *lot* of arguments ARE small



10

V15.0 Performance Work

• Compiler projects ongoing• New C compilers on all platforms• More work on core algorithms• Some integration of Futures and Isolates into the

interpreter (no workspace required)Roger Hui and Jay FoadD04 Core PerformanceMonday 13:30-14:15

Aaron Hsu (Indiana University):Using Co-dfns to Accelerate APL CodeWednesday 11:00-11:45

Jay Foad:Compilation and Bytecode ExecutionWednesday 09:45-10:30

Roger HuiTP4 Writing Efficient Code in DyalogThursday 14:00-17:30

2. New Platforms• UNIX variants

– ARM Linux (Raspberry Pi), Apple OSX, Android

• Windows variants without Win32– Universal Windows Platform (UWP),

New Servers, Windows for ARM• Our new tools are designed to be cross-

platform– RIDE, MiServer, SAWS, R-Connect, DyaCrypt– Some old ones too: Conga, SQAPL, …

• Our goal is to allow future apps to be developed on (any) one platform and deployed on (any) other

12

Introducing Dyalog 14.1 for OS X

• Complete 64-bit Unicode Dyalog engine• 100% compatible and inter-operable with other

Dyalog variants• Includes CONGA, SAWS, MiServer, RConnect,

(SQAPL to come soon) • Development environment is RIDE 2.0.• GUI via MiServer - no emulation of WC⎕ .

• 70 copies downloaded to date


Nick Nickolov:A New Way to Interact with DyalogMonday 11:00-11:30

13

Introducing Dyalog for OS X


3. User Interfaces

• Cross-platform apps will use HTML5 andJavascript to define the userinterface (for at least 5 years)

• New Microsoft Desktop applications will use Windows Presentation Foundation (WPF)– Or its new equivalent on the mobile platforms

• We will support data binding and ”MVVM” style application building with both these tools

• We are looking at embedded HTML5/JS rendering engines to allow the same UI to service Web and Desktop or ”Phablet” applications.

• Some New Microsoft Desktop and UWP apps will use WPF. UWP is also XAML-based.– WPF and ”UWP” will morph into a single toolset

15

Model View ViewModel (MVVM)

List←'Andy Shiers' 'Bjørn Christensen' 'Brian Becker' …


Filter←'R'

FilteredList

Jason Rivers Richard Smith Roger Hui

Dyalog APL Server Dyalog APL + RIDE(Windows, UNIX, Mac,

Android, etc)

16

Portable User Interfaces


MiServer or otherHTML/JS Generator

Web Browser

Internet

MiServer or otherHTML/JS Generator

HTML Engine


17

V15.0 User Interface Work

• Data Binding features and performance

• (+ MiServer 3.0 and then 4.0)

John DaintreeD12 Data Binding – The MatrixTuesday 15:30-16:15

Morten Kromberg:D07 Cross-Platform User InterfacesMonday 15:30-16:00

Dan Baronet and Adám BrudzewskyD10 Miserver UI ControlsTuesday 09:00-09:45

(Big) Bets We Are Placing

1. Parallel Processing is getting Critical2. New Platforms will soon be Important3. Most new User Interfaces will use

either HTML5/JS or WPF/XAML

19

Language

• There are still many ideas worth adding to the (Dyalog) APL Language

• We want to move in a functional direction– Parallel computing REQUIRES it– Puts APL ”back on the map” – John Scholes and I have

both felt very welcome at functional events• We want to add features for analysis of ”big

data”


20

New Language Features• Version 14.0

– Rank (⍤), Key (⌸), Tally (≢)– Trains: (avg←+/ ÷ ≢) (flipenc←⍉⊤)– Dyadic Iota on higher-rank arrays (and 8⌶ for inverted tables)– Parallel: Futures and Isolates– Microsoft.Net data binding

• Version 14.1: No real ”language” Features– :Disposable– Experimental JSON Parser– External Workspace Files (Memory Mappable)


Morten Kromberg & Stig NielsenD13 External WorkspacesWednesday 16:30-16:45


21

New Frontiers: LanguageStill “one or two” things left to do:• Operators Cut / Tesselate, Merge and Dual• Notations for Namespace and Array Constants

Potential New Data Types and Related Functions:• High Precision Floats (Rounding)• Closures (“functional objects”)• Mechanisms for sharing data between processes in real time• Rational Numbers (Prime Number Functions)• Scalar String Type (0=≡"Hello World")• Objects with Array Semantics (sparse / inverted)


22

Active Language Research• New operators will target v15 plus 1

Likely v15.0 language features:• We are looking for a mechanism to replace lookup←keys∘⍳... By marking an array as ”searchable”

• Hope to allow selected arrays (”in memory databases”) to reside outside the main workspace.

We will publish proposals for new featuresFollow us on Twitter, Like us on Facebook, etc

Roger Hui & John ScholesD16 Proposals: Cut, Under & MergeThursday 09:15-10:00


23

Tools for Application Building

Tried and Tested• CONGA – TCP client and server library• SQAPL – SQL / ODBC access on all platforms• Quad-WC GUI – Win32 applications

Recent Additions• LoadData – Data import tools for Excel, XML, CSV, SQL/ODBC• RConnect – Integrate R statistical framework w/APL• Databound WPF – MVVM style Windows desktop applications• MiServer – Stand-alone Web Server

– Build HTML5/JS UI in APL– Also Web Services: SAWS become part of MiServer

24

New Frontiers: Tools• DyaCrypt – Cryptographic Library

– Available for testing– Design of v2.0 in process

• Databound MiServer– MVVM style programming in MiServer 4.0

• The Dyalog Project Project (DP2)– Project mechanism for managing source code, dependencies and

building runtime environments• Utility Libraries and Sample Applications

– To go with DP2• NewLeaf integrated with SharpPlot (SharpLeaf)

– D3, Syncfusion and other Javascript-based tools via MiServer


Nic DelcrosD17 SharpLeaf – Flowing Reports…Thursday 10:00-10:30

Morten KrombergD11 The Dyalog Project ProjectWednesday 14:15-14:45

Dan Baronet and Adám BrudzewskyD05 User Command UpdateMonday 14:15-14:40

25

New Frontiers: Cross Platform• ALL previously mentioned tools

– Every new tool is designed to be cross platform if at all possible– CONGA, SQAPL, MiServer, Rconnect, DP2, RIDE, ...

• File Functions – List files and folders, delete files and create folders. Rationalisation of file name handling.

• RIDE v3.0– Published JSON-based API for 3rd party developers

• Also more directly support EMACS/vi or other external code editors– ”Zero footprint” RIDE – run direct from a web browser

• RIDE Process Manager– Launch, monitor and debug [large] collections server processes

• Android, UWP, iOS interpreters to come (in some order)


Richard SmithD08 Cross-Platform File FunctionsMonday 16:00-16:30

26

Evangelism

“Dyalog is a modern, array-first, multi-paradigm programming language,

which supports functional, object-oriented and imperative programming,

based on an APL language kernel.”

Functional Conf’14, BangaloreArray’15 Workshop, Portland OregonDyalog on Talks@Google, GooglePlex Mountain ViewFunctional Conf’15, Bangalore


Fiona SmithD09 Dya(b)logMonday 17:30-18:15

http://www.sable.mcgill.ca/array/


https://www.youtube.com/watch?v=PlM9BXfu7UY






http://functionalconf.com/schedule.html


27

Evangelism

“Dyalog is a modern, array-first, multi-paradigm programming language,

which supports functional, object-oriented and imperative programming,

based on an APL language kernel.”

Functional Conf’14, BangaloreArray’15 Workshop, Portland OregonDyalog on Talks@Google, GooglePlex Mountain ViewFunctional Conf’15, Bangalore


Fiona SmithD09 Dya(b)logMonday 17:30-18:15












28

Conclusion

• Sustained Focus on Performance and Quality• Cross Platform IDE now available • Cross Platform UI Tools very nearly there• Increasing Focus on Samples and Tools

• New users on new platforms in sight

Andy ShiersD06 MyDyalog and Backroom processesThursday 10:00-10:30

Technical Road Map Morten Kromberg 0. 1 (Same Old) Focus Areas -Performance -(includes parallel...

Documents

Transcript of Technical Road Map Morten Kromberg 0. 1 (Same Old) Focus Areas -Performance -(includes parallel...