Scale Out and Conquer: Architectural Decisions Behind Distributed...

© 2018 GridGain Systems, Inc. GridGain Company Confidential

Scale Out and Conquer: Architectural

Decisions Behind Distributed In-Memory

Systems

Denis Magda

Apache Ignite PMC Chair

GridGain Product Management


Agenda

• Partitioning – Pitfalls of Even Distribution

• Affinity Collocation – Laid Out in Numbers

• Multi-threading of Distributed Systems – Things to Know


Memory-centric distributed

database, caching, and processing platform

Memory-centric platform based on Apache Ignite

adds enterprise features and support

for mission critical deployments


Among Top 5 Apache Projects

Total number of Apache Projects: 318Top-Level Apache Projects: 193

Top 5 Apache Projects by Commits

1. Hadoop2. Ambari3. Lucene-Solr4. Camel5. Ignite

Top 10 most active Apache mailing lists 1. Flex2. Lucene3. Ignite4. Kafka5. Geode6. Flink7. Tomcat8. Cassandra9. Beam10. Sentry


Partitioning – Pitfalls of Even Distribution


Where request goes?


Affinity Function

Key Partition

Server Node

ON-DISK


Affinity Function


Naïve Affinity


Naïve Function: New Node


Naïve Function: What’s Wrong?


Naïve Affinity: Redundant Partitions Re-shuffling!


Productized Affinity Functions

• Consistent hashing [1]

• Rendezvous hashing (HRW) [2]

[1] https://en.wikipedia.org/wiki/Consistent_hashing

[2] https://en.wikipedia.org/wiki/Rendezvous_hashing


Rendezvous Affinity: Weight Calculation

WEIGHT(partition, node)


Rendezvous Affinity: Two Nodes to Three


Rendezvous Affinity: Distribute Evenly at Best


Rendezvous Affinity: Possible Distribution


Affinity Collocation – Laid Out in Numbers


Running Transaction: No Collocation

1: class Account {

2: long id;

3: long city;

4: }


Running Transaction: No Collocation


Non Collocated Data: Number of Operations

2 (2 nodes)



2 (2 nodes)

2 (primary + backup)



2 (2 nodes)


2 (two-phase commit)



2 (2 nodes)


2 (two-phase commit)

2 (request-response)

--

16 network operations


Network Operations!


Running Transaction: Collocated Data

1: class Account{

2: long id;

3:

4: @AffinityKeyMapped

5: long city;

6: }


Running Transaction: Collocated Data!


Collocated Data: Number of Operations

1 (1 node)



1 (1 node)




1 (1 node)


1 (one-phase commit)



1 (1 node)


1 (one-phase commit)

2 (request-response)

--

4 network operations


Let’s Run an SQL Query – No Collocation


Executing SQL: Full Scan

• 1/3x latency

• 3x throughput


What About Indexes?


Index Search Complexity

log 1_000_000 ≈ 20

vs.

log 333_333 ≈ 18

log 333_333 ≈ 18

log 333_333 ≈ 18


Executing SQL: Indexes Search

• ~same latency

• ~same throughput


Let’s Collocate and Run SQL Again


Executing SQL: Indexed Search and Collocation

• same latency

• 3x throughput


Multi-threading of Distributed Systems – Things to Know


How Requests/Tasks/Queries Processed?


Requests Processing: Bottleneck!


Making Situation Worse


Cluster Node Chokes!


Tread-Per-Partition and Workers


Thread-Per-Partition and I/O Queues


Thread-Per-Partition: Not a Magic Bullet


Upshot


Upshot

• Partitioning – it’s not about even distribution only

• Affinity Collocation – a golden concept of distributed systems

• Business Data Model should be adopted/reconsidered

• Thread-per-partition – speeds up most but not all usage scenarios


These Days in New York

Docker NYC Meetup, Tomorrow, 6:00 PM

• Location: PeerSpace, 1740 Broadway, 15th Floor

• Distributed Database DevOps Dilemmas?

Kubernetes to the rescue

https://ignite.apache.org/events.html

https://ignite.apache.org/events.html


Any Questions?

#apacheignite

#TheASF

Thank you for joining us. Follow the conversation.https://ignite.apache.org

#gridgain

#denismagda

https://www.meetup.com/NYC-In-Memory-Computing-Meetup/

https://ignite.apache.org/

https://www.meetup.com/NYC-In-Memory-Computing-Meetup/

Scale Out and Conquer: Architectural Decisions Behind Distributed...

Documents

Transcript of Scale Out and Conquer: Architectural Decisions Behind Distributed...