Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

47
Jonas Bonér Terracotta, Inc. [email protected] http://terracotta.org http://jonasboner.com Clustering the JVM using AOP

description

Clustering the JVM using AOP. Jonas Bonér Terracotta, Inc. [email protected] http://terracotta.org http://jonasboner.com. Let’s start with a demo – since a picture says more than a thousand words. Agenda. Overview of Terracotta Get started The inner workings of a clustered JVM - PowerPoint PPT Presentation

Transcript of Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Page 1: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Jonas BonérTerracotta, [email protected]://terracotta.orghttp://jonasboner.com

Clustering the JVM using AOP

Page 2: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Let’s start with a demo

– since a picture says more than a thousand

words

Page 3: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Agenda

Overview of Terracotta

Get started

The inner workings of a clustered JVM

Real-world examples

Q&A

Page 4: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Agenda

Overview of Terracotta

Get started

The inner workings of a clustered JVM

Real-world examples

Q&A

Page 5: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

What is Terracotta?

Network-Attached MemoryOpen Source Scalability and

High-Availability for the JVM

Page 6: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

What is Terracotta?

Network-Attached Memory

No API: declarative configuration to selectively share object

graphs across the cluster

No serialization: plain POJO clustering

Fine-grained replication: field-level, heap-level replication

Cross-JVM coordination: Java Memory Model maintained across

the cluster – e.g. cluster-wide wait/notify and synchronized

Large virtual heaps: that exceeds the capacity of a single JVM

Distributed Method Invocations

Runtime monitoring and control

Page 7: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Use-cases

Relieving Database Overload• Distributed Caching

• Hibernate Clustering

• HTTP Session Clustering

Simplifying Application Architecture and

Development• Virtual Heap for Large Datasets

• Clustering OSS Frameworks (Spring, Struts, Lucene, Wicket, EHCache etc.)

• Master/Worker – Managing Large Workloads

• POJO Clustering

• Messaging, Event-based Systems and Coordination-related Tasks

Page 8: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Terracotta approach

Today's Reality• Scale out is complex

• Requires custom Java code

Our different

approach• Cluster the JVM

• Eliminate need for custom

codeScale-out

Terracotta ServerClustering the JVM

Terracotta ServerClustering the JVM

App Server

Web App

JVM

Frameworks

Frameworks

Business Logic

App Server

Web App

JVM

Frameworks

Frameworks

Business Logic

App Server

Web App

JVM

Frameworks

Frameworks

Business Logic

Custom DevelopmentCustom DevelopmentJMSJMSRMIRMISerializationSerialization

Page 9: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

The architecture of Terracotta

Terracotta Server• 100% Pure Java

• HA Active / Passive Pair

Local JVM Client• Transparent

• Pure Java Libraries

Coordinator “traffic cop”• Coordinates resource access

• Runtime optimizations

Central Storage • Maintains state across restarts

Terracotta Server (PASSIVE)Clustering the JVM

Terracotta Server (PASSIVE)Clustering the JVM

Scale-out

Terracotta Server (ACTIVE)Clustering the JVM

Terracotta Server (ACTIVE)Clustering the JVM

App Server

Web App

JVM

Any Java Objects

Any Framework

Sessions Objects

App Server

Web App

JVM

Any Java Objects

Any Framework

Sessions Objects

App Server

Web App

JVM

Any Java Objects

Any Framework

Sessions Objects

Terracotta LibraryTerracotta LibraryTerracotta Library

Terracotta ServerClustering the JVM

Terracotta ServerClustering the JVM

Page 10: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Heap Level Replication• Declarative• No Serialization• Fine Grained / Field Level

GET_FIELD - PUT_FIELD• Only Where Resident

JVM Coordination• Distributed Synchronized

Block • Distributed wait()/notify()

• Fine Grained Locking MONITOR_ENTRY -MONITOR_EXIT

Large Virtual Heaps• As large as available disk• Dynamic paging

Scale-out

App Server

Web App

JVM TC Libraries

SharedObjects

Terracotta ServerClustering the JVM

Terracotta ServerClustering the JVM

App Server

Web App

JVM TC Libraries

SharedObjects

App Server

Web App

JVM TC Libraries

SharedObjects

Network-Attached Memory

TC

Man

agem

ent

Co

nso

le

TC

Man

agem

ent

Co

nso

le

Management Console• Runtime visibility• Data introspection• Cluster monitoring

Page 11: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Agenda

Overview of Terracotta

Get started

The inner workings of a clustered JVM

Real-world examples

Q&A

Page 12: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Hello World

Page 13: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Hello World - tutorial/HelloWorld.javapackage tutorial;

import java.util.*;

public class HelloWorld {   private List<String> hellos = new ArrayList<String>();

   public void sayHello() {      synchronized(hellos) {         hellos.add("Hello, World " + new Date());         for (String hello : hellos) {            System.out.println(hello);         }      }   }

   public static void main(String[] args) {      new HelloWorld().sayHello();   }}

Page 14: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Hello World - tc-config.xml<?xml version="1.0" encoding="UTF-8"?><tc:tc-config xmlns:tc="http://www.terracotta.org/config">  <application>    <dso>      <roots>        <root><field-name>tutorial.HelloWorld.hellos</field-name></root>      </roots>      <locks>        <autolock>          <method-expression>* tutorial.HelloWorld*.*(..)</method-expression>          <lock-level>write</lock-level>        </autolock>      </locks>      <instrumented-classes>        <include><class-expression>tutorial..*</class-expression></include>      </instrumented-classes>    </dso>  </application></tc:tc-config>

Page 15: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Demo

Page 16: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Enhanced Hello WorldUsing java.util.concurrent

Page 17: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Coordination - HelloWorldConcurrent.java

package tutorial;

import java.util.*;import java.util.concurrent.CyclicBarrier;

public class HelloWorldConcurrent {   private List<String> hellos = new ArrayList<String>();   private CyclicBarrier barrier;

   public void sayHello() throws Exception {      barrier = new CyclicBarrier(2);      barrier.await();            synchronized(hellos) {         hellos.add("Hello, World " + new Date());         for(String hello : hellos) {            System.out.println(hello);         }      }   }

   public static void main(String[] args) throws Exception {      new HelloWorldConcurrent().sayHello();   }}

Page 18: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Coordination - tc-config-concurrent.xml<?xml version="1.0" encoding="UTF-8"?><tc:tc-config xmlns:tc="http://www.terracotta.org/config">  <application>    <dso>      <roots>        <root><field-name>tutorial.HelloWorldConcurrent.hellos</field-name></root>        <root><field-name>tutorial.HelloWorldConcurrent.barrier</field-name></root>      </roots>      <locks>        <autolock>          <method-expression>* tutorial.HelloWorldConcurrent*.*(..)</method-expression>          <lock-level>write</lock-level>        </autolock>      </locks>      <instrumented-classes>        <include><class-expression>tutorial..*</class-expression></include>      </instrumented-classes>    </dso>  </application></tc:tc-config>

Page 19: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Demo

Page 20: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Agenda

Overview of Terracotta

Get started

The inner workings of a clustered JVM

Real-world examples

Q&A

Page 21: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Inner workings of a clustered JVM

Maintain the semantics of the application (e.g Java

Language Specification (JLS) and the Java Memory

Model (JMM)) across the cluster

Load-time bytecode instrumentation• Hook in java.lang.ClassLoader and -javaagent

• Transparent to the user -- classes are woven on-the-fly

Aspect-Oriented Programming techniques• AspectWerkz aspects

• Custom bytecode instrumentation using ASM

• Declarative configuration (XML) Simplified pointcut pattern language (based on AspectWerkz)

Page 22: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

What do we need to maintain across the cluster?

1.State sharing• Share data in the Java heap for the instances reachable

from a shared top-level “root” reference

• Make sure each JVMs local heap are in sync (in regards

to the clustered parts of the heap)

2.Thread coordination• Resource access and guarding

• Thread signaling

Page 23: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Sample application@Clustered public class Counter {

@Root public final static Counter instance = new Counter();

private final Object lock = new Object();

private volatile int counter = 0;

public void increment() {

synchronized(lock) { this.counter++; lock.notifyAll(); }

}

public void waitFor(int expected) {

synchronized(lock) {

while(this.counter < expected) {

try { lock.wait(); } catch(InterruptedException ex) {}

}

}

}

put top level root in shared space

lock and start txcommit tx and unlock

reconcile pending changes record field change

intercept notifyAll

instrument clusterable classes

short circuit root creation

Page 24: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

State sharing - field-level replication and managementWe need to break down the shared object graph into

“literals”• int, long, float, String, boolean etc.

Need to be able to detect changes to the object graph• New object reference is added/removed

• Literals changes value

Every reference has a “shadow” that is “managing” the

reference• Knows if the reference has been changed

• Can lazily page in and out the actual value

• Page in: Reconcile changes done in another node

• Page out: Null out the reference

Page 25: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Field-level replication

Child2Child1 Child3

Parent2

500 bytes

500 bytes each

16 bytes each

Total object graph size: 1548 bytes

Grandparent

Parent1

each instance has a shadow that is managing access and modification

Child3

Terracotta replicates: 16 bytes

Page 26: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Manage field access and modification

Intercept the PUTFIELD and GETFIELD bytecode instructions

Create two advice for managing get and set shared field

• before(TransparentAccess o, Object value) :

• set(* *.*) && isClusteredTarget(o) && args(value) {

• … // register changed field value in shadow

• }

• before(TransparentAccess o) :

• get(* *.*) && isClusteredTarget(o) {

• … // reconcile any pending changes

• }

Page 27: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

BCI Example – original byte codepublic demo.sharededitor.controls.Dispatcher(demo.sharededitor.models.ObjectManager,

demo.sharededitor.ui.Renderer);

Code:

0: aload_0

1: invokespecial #1; //Method javax/swing/event/MouseInputAdapter."<init>":()V

4: aload_0

5: aload_2

… snip

33: aload_0

34: aload_1

35: putfield #6; //Field objmgr:Ldemo/sharededitor/models/ObjectManager;

38: aload_0

39: getfield #6; //Field objmgr:Ldemo/sharededitor/models/ObjectManager;

42: aload_0

43: getfield #2; //Field renderer:Ldemo/sharededitor/ui/Renderer;

46: invokevirtual #7; //Method demo/sharededitor/models/ObjectManager.addListener:(Ldemo/sharededitor/events/IListListener;)V

49: return

Page 28: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

BCI Example – instrumented byte codepublic demo.sharededitor.controls.Dispatcher(demo.sharededitor.models.ObjectManager,

demo.sharededitor.ui.Renderer);

Code:

0: aload_0

1: invokespecial #99; //Method javax/swing/event/MouseInputAdapter."<init>":()V

4: aload_0

5: aload_2

… snip

33: aload_0

34: aload_1

35: invokevirtual #117; //Method __tc_setobjmgr:(Ldemo/sharededitor/models/ObjectManager;)V

38: aload_0

39: invokevirtual #119; //Method __tc_getobjmgr:()Ldemo/sharededitor/models/ObjectManager;

42: aload_0

43: getfield #101; //Field renderer:Ldemo/sharededitor/ui/Renderer;

46: invokevirtual #123; //Method demo/sharededitor/models/ObjectManager.addListener:(Ldemo/sharededitor/events/IListListener;)V

49: return

Page 29: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Thread coordination

We need to maintain the semantics of the Java

Memory Model (JMM) across multiple JVMs

Maintain the semantics of:

• synchronized(object) {..}

• wait()/notify()/notifyAll()

• Flushing of changes

Locking optimizations

• User defined

• Runtime optimized

Page 30: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Manage synchronized blocks

Intercept MONITORENTER and MONITOREXIT bytecode

instructions

before(Object o) :

isClustered() && lock() && args(o) {

ClusterManager.lock(o); // lock o in cluster

}

after(Object o) : // after finally

isClustered() && unlock() && args(o) {

ClusterManager.unlock(o); //unlock o in cluster + flush

changes

}

Page 31: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Manage thread coordination (wait/notify)

Intercept the wait(), notify() and notifyAll() methods in java.lang.Object

void around(Object o) :

isClusteredTarget(o) && call(void Object.wait()) {

ClusterManager.objectWait(o);

}

void around(Object o) :

isClusteredTarget(o) && call(void Object.notify()) {

ClusterManager.objectNotify(o);

}

… // remaining methods omitted

Page 32: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Agenda

Overview of Terracotta

Get started

The inner workings of a clustered JVM

Real-world examples

Q&A

Page 33: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

HTTP Session Clusteringwithout Java Serialization

Page 34: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

HTTP Session Clustering - Benefits

Terracotta Sessions gives you:• Near-Linear Scale

• No java.io.Serializable

• Large Sessions - MBs

• Higher Throughput

Supported Platforms:• Jetty,

• JBoss 4.x,

• Tomcat 5.0 & 5.5,

• WebLogic 8.1, WebLogic 9.2,

• WebSphere CE, Geronimo Alpha, WebSphere 6.1

Page 35: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

HTTP Session Clustering - DummyCart.java

package demo.cart; import java.util.*; public class DummyCart {    private List items = new ArrayList();     public List getItems() {        return Collections.unmodifiableList(items);    }     public void addItem(String name) {        items.add(name);    }     public void removeItem(String name) {        items.remove(name);    }}

Page 36: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

HTTP Session Clustering - carts.jsp<%@ page import="java.util.Iterator" %> <html> <jsp:useBean id="cart" scope="session" class="demo.cart.DummyCart" /> <% String submit = request.getParameter("submit"); String item = request.getParameter("item"); if (submit != null && item != null) { if (submit.equals("add")) { cart.addItem(item); } else if (submit.equals("remove")) { cart.removeItem(item); } }%> <body> <p>You have the following items in your cart:</p> <ol><% Iterator it = cart.getItems().iterator(); while (it.hasNext()) { %><li><% out.print(it.next()); %></li> <% } %> </ol> <form method=“get" action="<%=request.getContextPath()%>/cart.jsp"> <p>Item to add or remove: <input type="text" name="item" /></p> <input type="submit" name="submit" value="add" /> <input type="submit" name="submit" value="remove" /> </form> </body> </html>

Page 37: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

HTTP Session Clustering - tc-config.xml

<?xml version="1.0" encoding="UTF-8"?><tc:tc-config xmlns:tc="http://www.terracotta.org/config">  <application>    <dso>    <web-applications>    <web-application>cart</web-application>      </web-applications>      <instrumented-classes>        <include>          <class-expression>demo.*</class-expression>        </include>      </instrumented-classes>    </dso>  </application></tc:tc-config>

Page 38: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Demo

Page 39: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Cluster Spring andother OSS frameworks

Page 40: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

'Supported' by Terracotta means:• Just include a ‘Configuration Module’ (OSGi bundle) in your config to

cluster your application (a one liner)

• Plugs in underneath without any setup

• Technically, Terracotta supports all integrations as long as it runs on

the JVM (f.e. JRuby, PHP)

Glimpse of supported integrations

Page 41: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Example Configuration Module- EHCache clustering

<?xml version="1.0" encoding="UTF-8"?><tc:tc-config xmlns:tc="http://www.terracotta.org/config">  <clients>    <modules>      <module name="clustered-ehcache-1.3" version="1.0.0"/>    </modules>  </clients>  <application>    <dso>      <instrumented-classes>        <include>          <class-expression>tutorial.*..*</class-expression>        </include>      </instrumented-classes>    </dso>  </application></tc:tc-config>

Page 42: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Example - Spring clustering

Terracotta can declaratively cluster Spring beans (Singleton

+ Session and Custom scoped) with zero code changes

Can also cluster Spring ApplicationContext events, JMX

State and Spring Web Flow

<spring>  <application name="tc-jmx">    <application-contexts>        <application-context>        <paths>          <path>*/applicationContext.xml</path>        </paths>        <beans>          <bean name="clusteredCounter"/>          <bean name="clusteredHistory"/>        </beans>      </application-context>    </application-contexts>        </application></spring>

<bean id="localCounter"              class="demo.jmx.Counter"/> <bean id="clusteredCounter"       class="demo.jmx.Counter"/>       <bean id="localHistory"              class="demo.jmx.HistoryQueue"/> <bean id="clusteredHistory"       class="demo.jmx.HistoryQueue"/>

Terracotta config Spring config

Page 43: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Demo

Cluster Spring Web Flow’s Web Continuations

(conversational/workflow state)

Page 44: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Agenda

Overview of Terracotta

Get started

Real-world examples

Q&A

Page 45: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Wrapping up

Page 46: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Wrapping upTerracotta is Network-Attached Memory for the JVM

Turns Scalability and High-Availability into a deployment artifact

Keep the simplicity of POJO-based development

– get Scale-Out with Simplicity

Makes mission-critical applications simpler to:

• Write

• Understand

• Test

• Maintain

Endless possibilities for clustering and distributed programming

– these were just a few

Be creative, use your imagination and have fun…

Page 47: Jonas Bonér Terracotta, Inc. jonas@terracotta terracotta jonasboner

Questions?

http://[email protected]