Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ......
Transcript of Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ......
![Page 1: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/1.jpg)
@PeterVeentjer
Distributed Systems Using Hazelcast
Peter Veentjer
![Page 2: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/2.jpg)
@PeterVeentjer
• Working for Hazelcast
• Senior developer
• Solution architect
• Author of ‘Mastering Hazelcast 3'
• 14 years Java experience
• Big love
• Concurrency control
• Distributed computing
Whoami
![Page 3: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/3.jpg)
@PeterVeentjer
![Page 4: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/4.jpg)
@PeterVeentjer
What is Hazelcast?
![Page 5: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/5.jpg)
@PeterVeentjer
What is Hazelcast?
• Leading Open Source Java In Memory Data/Compute Grid
• Out main goal is to simplify development of:
• Scalable systems
• Highly available systems
![Page 6: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/6.jpg)
@PeterVeentjer
Why Hazelcast?
• 2.5 MByte JAR
• no other dependencies!
• no need to install software!
• Its a library, not an application framework
• Apache 2 License
• Free to use
• no limitations
![Page 7: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/7.jpg)
@PeterVeentjer
Distributed Data-structures
• IAtomicLong/Ref
• IdGenerator
• Lock/Condition
• CountDownLatch
• Semaphore
• Queue
• Map
• MultiMap
• Set
• List
• Topic
• Executor
• TransactionalMap/Q/S…
• Write your own!
![Page 8: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/8.jpg)
@PeterVeentjer
So what can I do with it?
![Page 9: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/9.jpg)
@PeterVeentjer
Scaling
![Page 10: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/10.jpg)
@PeterVeentjer
Scale up
![Page 11: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/11.jpg)
@PeterVeentjer
Scale Out
![Page 12: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/12.jpg)
@PeterVeentjer
High Availability
![Page 13: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/13.jpg)
@PeterVeentjer
Raspberry Pi Cluster
![Page 14: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/14.jpg)
@PeterVeentjer
When?• Caching
• Messaging Solution
• Event processing
• Clustered Scheduling
• Job Processing
• Cluster management
• HTTP session clustering
![Page 15: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/15.jpg)
@PeterVeentjer
Which companies
• E-Commerce
• Apple, eBay
• Financials
• JP Morgan, Morgan Stanley, HSBC, Deutsche Bank
• Telco’s
• AT&T, Ericsson
• Gaming
• Ubisoft/Blue Byte
![Page 16: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/16.jpg)
@PeterVeentjer
Which Open Source Projects
• WSO2 Carbon
• Mule ESB
• Vert.x
• Apache Camel
• Apache Shiro
• OrientDB
• Alfresco
• Karaf
![Page 17: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/17.jpg)
@PeterVeentjer
How Often
• In October 2013
• 3M startups
• 48K unique
![Page 18: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/18.jpg)
@PeterVeentjer
Where is the code!
![Page 19: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/19.jpg)
@PeterVeentjer
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
Creating Hazelcast Cluster
![Page 20: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/20.jpg)
@PeterVeentjer
<hazelcast> <network>…</network> <map name=“m”>…</map> <queue name=“q”>…</queue> <… </hazelcast>
XML Configuration
![Page 21: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/21.jpg)
@PeterVeentjer
Config config = new Config(); … make config modifications HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);
Programmatic Configuration
![Page 22: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/22.jpg)
Demo
@PeterVeentjer
![Page 23: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/23.jpg)
@PeterVeentjer
Map<String,String> products = new HashMap(); map.put("1","IPhone");
Map
![Page 24: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/24.jpg)
@PeterVeentjer
Map<String,String> products = new ConcurrentHashMap(); map.put("1","IPhone");
Map
![Page 25: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/25.jpg)
@PeterVeentjer
HazelcastInstance hz = Hazelcast.newHazelcastInstance(); Map<String,String> products = hz.getMap("products"); cities.put("1","IPhone");
Map
![Page 26: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/26.jpg)
Demo
@PeterVeentjer
![Page 27: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/27.jpg)
@PeterVeentjer
!<map name="products"> <time-to-live-seconds>4</time-to-live-seconds> <indexes> <index>name</index> </indexes> <..> </map>
Map Configuration
![Page 28: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/28.jpg)
@PeterVeentjer
Map Scalability and High Availability
271 Partitions
partitionid = hash(key) % partitioncount
…
![Page 29: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/29.jpg)
@PeterVeentjer
Map Scalability and High Availability
Member 1
![Page 30: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/30.jpg)
@PeterVeentjer
Map Scalability and High Availability
Member 1
Member 2
![Page 31: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/31.jpg)
@PeterVeentjer
Map Scalability and High Availability
Member 1
Member 2
Member 3
![Page 32: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/32.jpg)
@PeterVeentjer
Map Scalability and High Availability
Member 1
Member 2
Member 3
![Page 33: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/33.jpg)
@PeterVeentjer
Map Scalability and High Availability
Member 1
Member 2
Member 3
![Page 34: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/34.jpg)
@PeterVeentjer
Map Scalability and High Availability
Member 1
Member 2
![Page 35: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/35.jpg)
Demo
@PeterVeentjer
![Page 36: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/36.jpg)
@PeterVeentjer
Multimap
• Collection as value
• Serialization Overhead
• Lost update
![Page 37: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/37.jpg)
Demo
@PeterVeentjer
![Page 38: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/38.jpg)
@PeterVeentjer
BlockingQueue queue = new LinkedBlockingQueue(); queue.offer("1"); Object item = queue.take();
Queue
![Page 39: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/39.jpg)
@PeterVeentjer
!HazelcastInstance hz = Hazelcast.newHazelcastInstance(); BlockingQueue queue = hz.getQueue("queue"); queue.offer("1"); Object item = queue.take();
Queue
![Page 40: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/40.jpg)
Demo
@PeterVeentjer
![Page 41: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/41.jpg)
@PeterVeentjer
ITopic topic = hz.getTopic("topic"); //publishing topic.publish(msg); !//subscribing topic.addMessageListener(new TopicSubscriber()); !public class TopicSubscriber implements MessageListener<String> { public void onMessage(Message<String> m) { System.out.println(m.getMessageObject()); } }
Topic
![Page 42: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/42.jpg)
@PeterVeentjer
Client
• Simplified
• Encryption
• Load Balance Policy
• C++ and C# version
![Page 43: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/43.jpg)
@PeterVeentjer
HazelcastInstance client = HazelcastClient.newHazelcastClient();
Client
![Page 44: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/44.jpg)
@PeterVeentjer
Network Communication
• Cluster Discovery
• Multicast
• TCP/IP Cluster
• AWS Cluster
• Normal Network Communication
• TCP/IP
![Page 45: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/45.jpg)
@PeterVeentjer
Advanced Hazelcast
![Page 46: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/46.jpg)
@PeterVeentjer
!ILock lock = hz.getLock(“someLock”); IAtomicLong counter = getAtomicLong(“someCounter”); IMap map = hz.getMap(“someMap”); map.put(“somePerson”,new Person()); map.get(“somePerson”);
Data Locality: Problem
![Page 47: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/47.jpg)
@PeterVeentjer
Member 1
Member 2
Member 3
SomeLock
SomeCounter
SomePerson
![Page 48: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/48.jpg)
@PeterVeentjer
name@partitionkey
name
![Page 49: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/49.jpg)
@PeterVeentjer
ILock lock = hz.getLock(“someLock@foo”); IAtomicLong counter = hz.getAtomicLong(“someCounter@foo”); IMap map = hz.getMap(“someMap”); map.put(“somePerson@foo”, new Person()); Person p = map.get(“somePerson@foo”);
Data Locality: Fixed
![Page 50: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/50.jpg)
@PeterVeentjer
Member 1
Member 2
Member 3
SomeLock
SomeCounter
SomePerson
Foo Partition
![Page 51: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/51.jpg)
@PeterVeentjer
IAtomicLong c1 = … IAtomicLong c2 = hz.getAtomicLong(“c2@“+c1.getPartitionKey());
Adding object in same partition
![Page 52: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/52.jpg)
@PeterVeentjer
Executor
• Execute task anywhere
• Execute task on key owner
• Execute task
• one/all/subset members
• Synchronisation
• Future
• ExecutionCallback
• CompletableFuture Hazelcast 3.3
![Page 53: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/53.jpg)
Demo
@PeterVeentjer
![Page 54: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/54.jpg)
@PeterVeentjer
Locking
• Locks
• TransactionalMap.getForUpdate
• Map.Lock
![Page 55: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/55.jpg)
Demo
@PeterVeentjer
![Page 56: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/56.jpg)
@PeterVeentjer
public void increment(int accountId,int amount){ Account account = accounts.get(accountId); account.balance+=amount; accounts.put(accountId, account); }
Race problem
![Page 57: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/57.jpg)
@PeterVeentjer
public void increment(int accountId,int amount){ accounts.lock(accountId); try{ Account account = accounts.get(accountId); account.balance+=amount; accounts.put(accountId, account); }finally{ accounts.unlock(accountId); } }
Pessimistic Increment
![Page 58: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/58.jpg)
@PeterVeentjer
public void increment(int accountId,int amount){ accounts.lock(accountId); try{ Account account = accounts.get(accountId); account.balance+=amount; accounts.put(accountId, account); }finally{ accounts.unlock(accountId); } }
Pessimistic Increment
![Page 59: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/59.jpg)
@PeterVeentjer
public void increment(int accountId,int amount){ for(;;){ Account oldAccount = accounts.get(accountId); Account newAccount = new Account(oldAccount); newAccount.balance+=amount; if(accounts.replace(accountId, oldAccount,newAccount)){ return; } } }
Optimistic Increment
![Page 60: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/60.jpg)
@PeterVeentjer
public void increment(int accountId,int amount){ for(;;){ Account oldAccount = accounts.get(accountId); Account newAccount = new Account(oldAccount); newAccount.balance+=amount; if(accounts.replace(accountId, oldAccount,newAccount)){ return; } } }
Optimistic Increment
![Page 61: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/61.jpg)
@PeterVeentjer
DATA
Bad: Send Data to Function
DATA
Good: Send Function to Data
Read Data
Write Data
Write Function
![Page 62: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/62.jpg)
@PeterVeentjer
private class BalanceTask implements Runnable,Serializable{ private int accountId, amount; ! public void run() { for(;;){ Account oldAccount = accounts.get(accountId); Account newAccount = new Account(oldAccount); newAccount.balance+=amount; if(accounts.replace(accountId, oldAccount,newAccount)){ return; } } } }
Increment with runnable
![Page 63: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/63.jpg)
@PeterVeentjer
public void increment(int accountId, int amount){ BalanceTask task = new BalanceTask(accountId,amount); executorService.executeOnKeyOwner(task,accountId); }
Increment with runnable
![Page 64: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/64.jpg)
@PeterVeentjer
class BalanceProcessor extends AbstractEntryProcessor<Integer,Account>{ int amount; BalanceProcessor(int amount) { this.amount = amount; } ! @Override public Object process(Map.Entry<Integer, Account> entry) { entry.getValue().balance+=amount; return null; } }
Increment with EntryProcessor
![Page 65: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/65.jpg)
@PeterVeentjer
public void increment(int accountId, int amount){ BalanceProcessor processor = new BalanceProcessor(amount); accounts.executeOnKey(accountId, processor); }
Using entry processor
![Page 66: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/66.jpg)
@PeterVeentjer
Map: In Memory Format
• 2 Options
• BINARY
• OBJECT
• Predicates
• EntryProcessor
![Page 67: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/67.jpg)
@PeterVeentjer
SPI• You can write your own distributed data-structures
• Example
• Distributed Actors implementation
![Page 68: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/68.jpg)
@PeterVeentjer
Serialization API
• Serializable/Externalizable
• Portable
• ByteArraySerializable
• StreamSerializer
• Kryo
• Jackson Smile
• Protobuf
![Page 69: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/69.jpg)
@PeterVeentjer
Kryo SerializationSi
ze in
Byt
es
0
5,000
10,000
15,000
20,000
Serialization Kryo Kryp + Compr
![Page 70: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/70.jpg)
@PeterVeentjer
!SerializerConfig productSerializer = new SerializerConfig() .setTypeClass(Product.class) .setImplementation(new ProductSerializer()); !Config config = new Config(); config.getSerializationConfig().addSerializerConfig(productSerializer); HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);
Pluggable Serialization
![Page 71: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/71.jpg)
@PeterVeentjer
IMap<Integer,Account> accounts = hz.getMap(“accounts”); Map<Integer,Integer> result = accounts .map(new AccountSumMapper()) .reduce(new AccountSumReducer()) .submit()
Hazelcast 3.2 Map Reduce
![Page 72: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/72.jpg)
@PeterVeentjer
Enterprise vs Community edition
• Support contracts
• Elastic Memory
• Security
• Management Center
![Page 73: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/73.jpg)
@PeterVeentjer
The Future
• Topologies
• Hosted Management Center
• Dynamic Creation
• Map of Maps
• Tiered storage
![Page 74: Distributed Systems Using Hazelcast - Jfokus · PDF file• Working for Hazelcast! ... • Vert.x! • Apache Camel! • ... Creating Hazelcast Cluster @PeterVeentjer](https://reader034.fdocuments.in/reader034/viewer/2022050917/5ab312fb7f8b9aea528deb87/html5/thumbnails/74.jpg)
@PeterVeentjer
Questions?
You can find us at the Hazelcast booth!
Good Question? Get a Hazelcast
book!