Distributed Computing with Hazelcast - Brazil Tour

www.hazelcast.com@noctarius2k

/** * Distributed Computing * with Hazelcast */

Interactive!

Disclaimer# A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00002aaaaaf6d9c3, pid=2185, tid=1086892352 # # JRE version: 6.0_21-b06 # Java VM: Java HotSpot(TM) 64-Bit Server VM (17.0-b16 mixed mode linux-amd64 ) # # An error report file with more information is saved as: # /path/to/hs_err_pid2185.log # # This presentation may include certain forward-looking statements and projections provided by the Company. Any such # statements and projections reflect various estimates and assumptions by the Company concerning anticipated # results. These statements involve risks, uncertainties and assumptions and are based on the current estimates and # assumptions of the management of the Company as of the date of this presentation and are subject to uncertainty and # changes. No representations or warranties are made by the Company as to the accuracy of any such statements or # projections. Whether or not any such forward-looking statements or projections are in fact achieved will depend upon # future events some of which are not within the control of the Company. Accordingly, actual results may vary from the # projected results and such variations may be material. Statements contained herein describing documents and # agreements are summaries only and such summaries are qualified in their entirety by reference to such documents # and agreements.

Who’s that dude?

• Chris(toph) Engelbert • Twitter fanatic: @noctarius2k • Weird 9+ Java-Years • Performance, GC, Fairytales • Apache Committer • Gaming, Travel Management, …

The www.hazelcast.ninja

Who’s that dude?

DistributedComputing…

ToasterServer 1

TV-Server 1Power

CoffeeMakerServer 1

Kettle Server 1

StoveServer 1

Power GridServer 1, 2

DistributedComputing…

… but what meansDistributed Computing?OOO

“In-Memory Computing”

‘In-Memory Computing’

L1 Cache Reference L2 Cache Reference Main Memory Reference 1K send over 1 Gbps Network 4K read from SSD Seek on Spinning Disk Send Packet CA->AMS->CA

0.5 ns 7 ns

100 ns 10,000 ns

150,000 ns 10,000,000 ns

150,000,000 ns

0.01 ms 0.15 ms

10 ms 150 ms

Evolution of:- Memory-Price- Memory-Size

Data Distribution(Partitioning / Sharding)

Data Distribution

String code = “JCP-TOUR-2015”

Map<String, TShirtDelivery> shirts = Maps.create()

shirts.put(code, new TShirtDelivery(“Your Address”))

Data Distribution

String code = “JCP-TOUR-2015”

int hashcode = hash(code)

partitionId = hashcode % partitionCount

cluster.send(partitionId, code, delivery)

Data Distribution

int hashcode = hash(code)

partitionId = hashcode % partitionCount

cluster.send(partitionId, code, delivery)

Must be a consistent hash function

Partition count must be constant

Can be calculated, smart routing possible

Demo Time

That was easy!But don’t tell anyone!

Parallelize Processing

int[] values = [0..100000].map(randomInt())

int[][] blocks = split(values, 10000)

int[] subresult = blocks.foreach(_.sum())

int result = subresult.sum()

int[][] blocks = split(values, 10000)

int[] subresult = blocks.foreach(_.sum())

int result = subresult.sum()

No mutation, only new values

Independently computable

Might split further down (fork-join, work-stealing)

Demo Time2

I told you:Easy Peasy!

Distributed Caching!

Collect, store …

…access fast when necessary!…access fast when necessary!

Caches

Evolution Of Caching

Custom

http://gadizmo.com/roll-your-own-with-the-ham-dogger.php

Roll Your Own!Evolution Of Caching

Caches Are Simple!public class Cache<K, V> { private final Map<K, V> cache = new CHM<>();

public void put(K key, V value) { cache.put(key, value); }

public V get(K key) { return cache.get(key); } }

Even With TTL!

And Auto-Cleanup

What Is Wrong?

O(n)5 mio

Entries

It was a trap!

Custom

Proprietary

Custom

Proprietary

Open Source

Custom

Proprietary

Open Source

Commercial

Custom

Proprietary

Open Source

Commercial

Stupidly ExpensiveCustom

http://technoodling.net/mili-power-crystal-battery-pack-review/

Terracotta /EHcache

OracleCoherence

Infinispan

JavaCollections

Hazelcast

JCache

Custom

Success!Feels like

Quick Shameless Plug :)Hazelcast!

Hazelcast in 5 Facts• Java Collections API• Java Concurrency API

• Transparent Data Distribution• Drop-In Replacement

• Disruptively Simple(c) Rod Johnson (@springrod)

Time to be Happy!

Apache License 2

Time to be Happy!

www.hazelcast.org

You can do it!

Thank You!Any Questions?

@noctarius2k http://www.sourceprojects.org

http://github.com/noctarius

@hazelcast http://www.hazelcast.com http://www.hazelcast.org

http://github.com/hazelcast

Free Hazelcast T-Shirthttp://bit.ly/tshirtcast

We’re hiring!

www.hazelcast.com/careers

Distributed Computing with Hazelcast - Brazil Tour

Technology

Transcript of Distributed Computing with Hazelcast - Brazil Tour

Brazil By Grace Today your tour guide will be Grace. WELCOME TO BRAZIL !

Hazelcast Management Center Reference Manual · directory, i.e., hazelcast-management-center-3.12.5, contains the hazelcast-mancenter-3.12.5.war file. You can start this file directly

Replication Distilled: Hazelcast Deep Dive - Ensar... · Hazelcast is PA/EC Consistency is usually traded to availability and latency together. Hazelcast works in memory and mostly

Introduction to hazelcast€¦ · hazelcast code samples demo internals q/a 2 agenda. about me 3 core developer at hazelcast holds bsc. computer engineering started programming some

Wonderful Tropical Brazil Tour

Squeezing Performance out of Hazelcast

In-Memory Data Grid - Hazelcast | Documentation - …docs.hazelcast.org/docs/3.1.9/manual/pdf/hazelcast-documentation.pdf1 Chapter 1. Introduction Hazelcast is a clustering and highly

Hazelcast Documentation 3.3

Hazelcast - In-Memory DataGrid

Exclusive Brazil Tour Packages

Hazelcast Documentation - version 1.9docs.hazelcast.org/docs/1.9/manual/pdf/hazelcast-1.9-documentation… · Hazelcast Documentation iv 15.3. Internals 3: ... When a JVM's local

Introduction to hazelcast

Brazil (2011 study tour)

Spring Integration Hazelcast Extension

Devoxx 2013 - Hazelcast

Edu on Tour 2013: Brazil WTF lecture

Brazil World Cup tour - OperatorS

Itinerary LSU Brazil Study Tour - 2014

Brazil Study Tour 2010

Hazelcast 101