Alan Bateman Java Platform Group, Oracle November 2018
Transcript of Alan Bateman Java Platform Group, Oracle November 2018
![Page 1: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/1.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !1
Alan Bateman Java Platform Group, Oracle November 2018
![Page 2: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/2.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !2
Project Loom
• Continuations • Fibers • Tail-calls
![Page 3: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/3.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !3
Why Fibers
Today, developers choose between
App
Connections
simple (blocking / synchronous), but less scalable code (with threads)
App
Connections
complex, non-legacy-interoperable, but scalable code (asynchronous)
and
![Page 4: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/4.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !4
Why Fibers
With fibers, devs have both: simple, familiar, maintainable, interoperable code, that is also scalable
App
Connections
Fibers make even existing server applications consume fewer machines (by increasing utilization), significantly reducing costs
![Page 5: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/5.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !5
Continuations
![Page 6: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/6.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !6
A continuation (precisely: delimited continuation) is a program object representing a computation that may be suspended and resumed (also, possibly, cloned or even serialized).
![Page 7: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/7.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !7
Prototype Continuation API
package java.lang;
public class Continuation implements Runnable { public Continuation(ContinuationScope scope, Runnable target) public final void run() public static void yield(ContinuationScope scope) public boolean isDone()
:
}
![Page 8: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/8.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !8
Fibers
![Page 9: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/9.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !9
A Fiber light weight or user mode thread, scheduled by the Java virtual machine, not the operating system
Fibers are low footprint and have negilgible task-switching overhead. You can have millions of them!
![Page 10: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/10.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !10
• The runtime is well positioned to manage and schedule application threads, esp. if they interleave computation and I/O and interact very often (exactly how server threads behave)
• Make concurrency simple again
Why fibers?
![Page 11: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/11.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !11
fiber = continuation + scheduler
![Page 12: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/12.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !12
• A fiber wraps a task in a continuation
• The continuation yields when the task needs to block
• The continuation is continued when the task is ready to continue
• Scheduler executes tasks on a pool of carrier threads
• java.util.concurrent.Executor in the current prototype
• Default/built-in scheduler is a ForkJoinPool
fiber = continuation + scheduler
![Page 13: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/13.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !13
• Focus to date has been on the control flow and concepts, not the API
• Minimal java.lang.Fiber in current prototype that supports scheduling, park/unpark, and waiting for a fiber to terminate
• java.util.concurrent APIs can park/unpark fibers
• Socket and pipe APIs park fiber rather than block threads in syscalls
Fiber prototype
![Page 14: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/14.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !14
• A big question, lots of trade-offs
• Do we completely re-imagine threads?
• Can we run all existing code in the context of a fiber?
• Likely to wrestle with these questions for a long time
• Current prototype can run existing code but with some limitations
How much existing code can fibers run?
![Page 15: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/15.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !15
• Example uses Jetty and Jersey
Example using existing code/libraries
![Page 16: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/16.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !16
• Assume servlet or REST service that spends a long time waiting
Example with existing code/libraries
assume this takes 100ms
![Page 17: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/17.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !17
Default configuration (maxThreads = 200), load = 5000 HTTP request/s
![Page 18: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/18.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !18
maxThreads = 400, load = 5000 HTTP request/s
![Page 19: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/19.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !19
fiber per request, load = 5000 HTTP request/s
![Page 20: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/20.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !20
• Can’t yield with native frames on continuation stack
• Can’t yield while holding a monitor
• In both cases, parking pins the carrier thread
• monitorenter/Object.wait may park carrier thread
Current limitations
![Page 21: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/21.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !21
Back to the big questions
• Will fibers be able to run all existing code?
• Should we completely re-imagine threads?
![Page 22: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/22.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !22
• A lot of existing code uses the Thread API and Thread.currentThread() (maybe indirectly)
• For now, current prototype can run in a mode that emulates Thread.currentThread() and most of the Thread API. That allows fibers to run existing code.
• Project Loom is the opportunity to re-imagine threads
Thread.currentThread() and Thread API
![Page 23: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/23.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !23
• ThreadGroup
• Context ClassLoader
• Inheritance: TCCL, ACC, InheritedThreadLocals
• suspend/resume, deprecated for 20+ years
• Thread interrupt problematic with threads pools
• Thread locals …
What is wrong with java.lang.Thread
![Page 24: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/24.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !24
Thread locals
• e.g. container managed cache of connection or credentials context
• Long-standing source of memory leaks in thread pools
• Often used because because isn’t anything better
• Sometimes used to make context available to callees
• Sometimes used as approximation to “processor locals”
![Page 25: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/25.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !25
Locals (exploring)
• Frame/scope locals
• Locals that are accessible to calleese.g. Clojure dynamic binding, special variables in Lisp
• Semantics TDB
• Maybe tied with Structured Concurrency
• Processor locals
• Locals keyed on cpu ID rather than Thread
• Potential users are Striped64/LongAddr to avoid needing fields in Thread
![Page 26: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/26.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !26
Structured Concurrency (exploring)
• Core idea“every time that control splits into multiple concurrent paths, we want to make sure that they join up again”.
• Background reading and motivations:
• Nathaniel J Smith blogs:
• Notes on structured concurrency, or: Go statement considered harmful
• Timeouts and cancellation for humans
• Also Martin Sustrik blogs on state machines and structured concurrency in high-level languages
• Implemented as Nurseries in Python Trio library
![Page 27: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/27.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !27
Structured Concurrency
Instant deadline = Instant.now().plusSeconds(1);FiberScope.withDeadline(deadline).run(() -> {
Fiber<?> fiber1 = … Fiber<?> fiber2 = …
});
fiber1 and fiber2 guaranteed to have terminated
• Early prototype, but not in loom/loom yet
![Page 28: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/28.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !28
• Current prototype executes tasks as Runnable or Callables
• j.u.concurrent just works so can share objects or share by communicating
• Not an explicit goal at this time to introduce Channels or other concurrency APIs but new APIs may emerge
Communication between fibers
![Page 29: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/29.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !29
Current status
• Initial prototype with Continuation and Fiber support
• Current focused on
• Performance
• Fiber API
• Debugger support
• Several other topics under exploration
![Page 30: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/30.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !30
• Thread sleep, join
• java.util.concurrent and LockSupport.park
• Networking socket read/write/connect/accept
• Pipe read/write
APIs that potentially park in current prototype
![Page 31: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/31.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !31
Footprint
• Thread
• Typically 1MB reserved for stack + 16KB of kernel data structures
• ~2300 bytes per started Thread, includes VM meta data
• Fiber
• Continuation stack: hundreds of bytes to KBs
• 200-240 bytes per fiber in current prototype
![Page 32: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/32.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !32
Debugging and serviceability
• Basic support in JVM TI to track fiber scheduling, mount and unmount
• Hope to have some basic debugger support soon
• No investigation yet on JMX/java.lang.management and other tool APIs
![Page 33: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/33.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !33
Other topics to explore
• Tail calls • Forced preemption • Serialization and cloning
![Page 34: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/34.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !34
More information
• Project Loom page: http://openjdk.java.net/projects/loom/
• Mailing list: [email protected]
• Repo: http://hg.openjdk.java/net/loom/loom (fibers branch)
![Page 35: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/35.jpg)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. !35
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Safe Harbor Statement
![Page 36: Alan Bateman Java Platform Group, Oracle November 2018](https://reader031.fdocuments.in/reader031/viewer/2022012414/616f38b053204b6f4e78df15/html5/thumbnails/36.jpg)