Building distributed systems - PhpDay 2013
-
Upload
juozas-kaziukenas -
Category
Technology
-
view
1.293 -
download
0
description
Transcript of Building distributed systems - PhpDay 2013
![Page 1: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/1.jpg)
BUILDING DISTRIBUTED SYSTEMS
Juozas Kaziukėnas // juokaz.com // @juokaz
![Page 2: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/2.jpg)
300’000’000 products/ 24 hours = 12’500’00 products/ 3600 seconds = 3’472 products
/ 3000 nodes = 1.1 sec. per product
![Page 3: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/3.jpg)
BUICNOSCNLKSANCLAS
![Page 4: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/4.jpg)
![Page 5: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/5.jpg)
Juozas Kaziukėnas, Lithuanian
You can call me Joe
More info http://juokaz.com
![Page 6: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/6.jpg)
ONE SERVER IS NOT ENOUGH
![Page 7: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/7.jpg)
SCALING TO MULTIPLE SERVERS
![Page 8: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/8.jpg)
INDEPENDENT NODES
![Page 9: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/9.jpg)
1 TO 2 SERVERS = HARD2 TO 100 SERVERS = EASY .
![Page 10: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/10.jpg)
TWO GENERALS PROBLEM
![Page 11: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/11.jpg)
LOCKING... LOL
![Page 12: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/12.jpg)
MYSQL TRANSACTIONS
![Page 13: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/13.jpg)
STATELESS
![Page 14: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/14.jpg)
USE TIMEOUTS
![Page 15: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/15.jpg)
ASYNCHRONOUS PROCESSING
![Page 16: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/16.jpg)
MESSAGING AND QUEUES
![Page 17: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/17.jpg)
COMMUNICATION BETWEEN PROCESSES
![Page 18: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/18.jpg)
MESSAGE SIZE
![Page 19: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/19.jpg)
GEARMAN
![Page 20: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/20.jpg)
NO RUNNING ORDER OR RELIABILITY
![Page 21: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/21.jpg)
DATA
![Page 22: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/22.jpg)
LOCATION
![Page 23: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/23.jpg)
DE-NORMALIZATION
![Page 24: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/24.jpg)
CAP THEOREM
![Page 25: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/25.jpg)
EVENTUAL CONSISTENCY
![Page 26: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/26.jpg)
ATOMIC OPERATIONS
![Page 27: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/27.jpg)
CACHE
![Page 28: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/28.jpg)
SETTING SERVERS UP
![Page 29: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/29.jpg)
CONFIGURATION
![Page 30: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/30.jpg)
PUPPET FOR SERVERS MANAGEMENT
![Page 31: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/31.jpg)
DYNAMIC SCALING
![Page 32: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/32.jpg)
SELF DISCOVERY
![Page 33: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/33.jpg)
AMAZON EC2 SPOT INSTANCES
10x cheaper than normal on-demand instances
![Page 34: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/34.jpg)
![Page 35: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/35.jpg)
24’000 CORE CLUSTER = $300/H
![Page 36: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/36.jpg)
PROTOTYPING
![Page 37: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/37.jpg)
START SMALL
![Page 38: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/38.jpg)
VAGRANT FOR VMS MANAGEMENT
![Page 39: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/39.jpg)
WHAT IS GOING ON?
![Page 40: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/40.jpg)
HARD
![Page 41: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/41.jpg)
DEBUGGING DISTRIBUTED SYSTEMS IS HARD
![Page 42: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/42.jpg)
ALL STACK LEVELS FAIL
![Page 43: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/43.jpg)
MONITORING
![Page 44: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/44.jpg)
COUNT EVERYTHING
![Page 45: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/45.jpg)
UPTIME, VMSTAT, NETSTAT, IOSTAT, MPSTAT, TCPDUMP, GDB, VALGRIND, ULIMIT,
SYSCTL ETC...
![Page 46: Building distributed systems - PhpDay 2013](https://reader033.fdocuments.in/reader033/viewer/2022061202/546b8e4eb4af9f662c8b4d65/html5/thumbnails/46.jpg)
THANKS!Juozas Kaziukėnas
@juokaz