JavaOne PPT 16x9 - RainFocus€“Coherence, Enterprise Manager, other potential users within Oracle...

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |

JDBC NextA new non-blocking API for connecting to adatabase

Douglas SurberJDBC ArchitectDatabase Server TechnologiesSeptember 13, 2016


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended forinformation purposes only, and may not be incorporated into any contract. It is not acommitment to deliver any material, code, or functionality, and should not be relied uponin making purchasing decisions. The development, release, and timing of any features orfunctionality described for Oracle’s products remains at the sole discretion of Oracle.

4


Program Agenda

OverviewCodeWrap-upQ&A

1

2

3

4

5


Non-Blocking JDBC API

6


Non-Blocking Database Access

• What: A database access API for Java that does not block user threads• Who: Initial development by Oracle–Oracle Database, MySQL, Derby– Coherence, Enterprise Manager, other potential users within Oracle–Moving to the JDBC Expert Group

• Why: Non-blocking apps have better throughput– Fewer threads means less thread scheduling, less thread contention– Database access is slow so blocked threads leave resources idle for a long time– Simultaneous access to multiple databases, eg map/reduce, sharded database– Fire and forget, eg DML, stored procedures

7


Goals

• Absolutely no blocking–Minimize the number of threads used for database access–No user thread blocks ever

• Alternate API for database access–Not an extension to the current JDBC API–Not a replacement for the current JDBC API

• Target high throughput apps–Not a completely general purpose database access API– At least version 1 will have a limited feature set

• Builds on the Java SE class library

8


Design Choices

• Minimal or no reference to java.sql• Rigorous use of types• Builder pattern• Fluent API• Immutable after initialization • One way to do something is enough• Avoid complex SQL processing• Avoid callback hell

9


What About … ?

• Streams– Java streams are inherently synchronous

• ReactiveStreams–Not integrated into the Java SE class library

• NodeJS• ADBCJ

There are already multiple non-blocking JDBC APIs

10


Code

11

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 12

public void trivialInsert(DataSource ds) { String sql = "insert into tab values (<>, <>, <>)"; try (Connection conn = ds.getConnection()) { conn.countOperation(sql) .set("id", 1, JdbcType.NUMERIC) .set("name", "Deep Thought", JdbcType.VARCHAR) .set("answer", 42, JdbcType.NUMERIC) .submit(); }}

Trivial Insert

<>

<<o out_parameter>>

<<io inout_parameter>>

<<column_name>> same as column_name <<o column_name>>

Example:

select <<id>>, concat(first_name, ‘ ‘, family_name) <<o name>> from tab where id = <> /* \<<not_a_marker>> */

Any occurrence of the parameter marker pattern that is not a parameter marker must beescaped, eg in literals or in comments.

Parameter Markers(Very Preliminary)

public void trivialSelect(DataSource ds, List<Integer> result) { String sql = "select <<id>>, <<name>>, <<answer>> " + "from tab where answer = <>"; try (Connection conn = ds.getConnection()) { conn.<List<Integer>>rowOperation(sql) .set("target", 42, JdbcType.NUMERIC) .rowAggregator( (ignore, row) -> { result.add(row.get("id", Integer.class)); return null; } ) .submit(); }}

Trivial Select


Execution Model

• Operations consist of– SQL– Parameter assignments– Result handling– Submission and CompletableFuture

• User thread creates and submits Operations– Creating and submitting never blocks; user thread never blocks

• Implementation executes those Operations asynchronously– Performs round trip(s) to the database–Handles results– Completes CompletableFutures

Everything is an Operation

15


public DataSource getDataSource(String url, String user, String pass) { return DataSourceFactory.forVendor("Oracle Database") .builder() .url("jdbc:oracle:nonblocking:@//javaone.oracle.com:5521/javaone") .username("scott") .password("tiger") .connectionProperty(JdbcConnectionProperty.TRANSACTION_ISOLATION, TransactionIsolation.SERIALIZABLE) .build();}

getDataSource


DataSource:

public default Connection getConnection() { return builder().build().connect();}

Connection:

public default Connection connect() { holdForMoreMembers(); submit(); connectOperation().submit(); return this;}

Getting a Connection

public Future<Integer> selectIdForAnswer(DataSource ds, int answer) { String sql = "select <<id>>, <<name>>, <<answer>> from tab " + "where answer = <<target>>"; try (Connection conn = ds.getConnection()) { return conn.<List<Integer>>rowOperation(sql) .set("target", 42, JdbcType.NUMERIC) .initialValue( () -> new ArrayList<>() ) .rowAggregator( (list, row) -> { list.add(row.get("id", Integer.class)); return list; } ) .submit() .toCompletableFuture() .thenApply( l -> l.get(0) ); }}

Basic SELECT

public void insertListIndependent(List<Item> list, DataSource ds) { String sql = "insert into tab values " + "(<>, <>, <>)"; try (Connection conn = ds.getConnection()) { OperationGroup group = conn.operationGroup() .independent(); for (Item elem : list) { group.countOperation(sql) .set("elem_", elem) .submit() .toCompletableFuture() .exceptionally( t -> { System.out.println(elem.getId()); return null; }); } group.submit(); } }

Big INSERT v1


public static class Item { protected int id; protected String name; protected int answer; public Item( @SqlMarker int id, @SqlMarker String name, @SqlMarker int answer) { this.id = id; this.name = name; this.answer = answer; }

(cont.)

POJOs


@SqlMarker("NUMERIC") public int getId() { return id; } @SqlMarker("VARCHAR") public String getName() { return name; } @SqlMarker("NUMERIC") public int getAnswer() { return answer; }}

POJOs (cont.)


OperationGroup

• OperationGroup has its own result handling and CompletableFuture• Members submitted to group. OperationGroup is submitted as a unit• Order of execution– Sequential in order submitted– Parallel, any order

• Error response– Dependent: remaining group members skipped– Independent: remaining group members unaffected

• Conditional or unconditional• Connection is an OperationGroup– Sequential, dependent, unconditional by default

Operations can be grouped

22

public void insertListHold(List<Item> list, DataSource ds) { String sql = "insert into tab " + "values (<>, <>, <>)"; try (Connection conn = ds.getConnection()) { OperationGroup group = conn.operationGroup() .holdForMoreMembers() .independent(); group.submit();

(cont.)

Big INSERT v2


for (Item elem : list) { group.countOperation(sql) .set("elem_", elem) .submit() .toCompletableFuture() .exceptionally( t -> { System.out.println(elem.getId()); return null; }); } group.releaseProhibitingMoreMembers(); }}

Big INSERT v2 (cont.)

public void updateListParallel(List<Item> list, DataSource ds) { String query = "select <<id>> from tab where answer = <"; String update = "update tab set answer = <> where id = <>"; try (Connection conn = ds.getConnection()) { OperationGroup<Object, Object> group = conn.operationGroup() .holdForMoreMembers() .independent() .parallel(); group.submit(); for (Item elem : list) { CompletableFuture<Integer> idF = group.<Integer>rowOperation(query) .set("answer", elem.getAnswer(), JdbcType.NUMERIC) .rowAggregator( (Integer ignore, Result.Row row) -> row.get("id", Integer.class) ) .submit() .toCompletableFuture();(cont.)

Parallel UPDATE


group.countOperation(update) .set("id", idF) .set("answer", "42") .submit() .toCompletableFuture() .exceptionally( t -> { System.out.println("Update failed: " + elem.getId()); return null; }); } group.releaseProhibitingMoreMembers(); }}

Parallel UPDATE (cont.)


public default void close() { closeOperation().submit(); releaseProhibitingMoreMembers();}

Note: A CloseOperation is never skipped.

Close


Wrap-Up

28


Status

• Everything subject to change• Prototype Oracle Database driver implements much of the API–Uses nio Selector so non-blocking all the way down

• The JDBC Expert Group will continue development• Submit through the Java Community Process• Java 10?

29


Q & A

30

JavaOne PPT 16x9 - RainFocus€“Coherence, Enterprise Manager, other potential users within Oracle...

Documents

Transcript of JavaOne PPT 16x9 - RainFocus€“Coherence, Enterprise Manager, other potential users within Oracle...