1.Developing highly scalable applications By Afkham Azeez twitter.com/afkham_azeez blog.afkham.org

2. About Me PMC member Apache Axis, Apache Stratos, Committer Synapse & Web Services Member, Apache Software Foundation Co-author, Axis2 Web Services Director of Architecture, WSO2 Inc Electronics, Arduino, 4x4 hobbyist Blog: http://blog.afkham.org Twitter: afkham_azeez2 3. Agenda Some core concepts Vertical/horizontal scaling techniques Capacity planning Show me the code! Q&A, Discussion 4. Scalability "The ability of the of a system to continue to operate correctly even when it is scaled to a larger size 5. Availability5 6. Availability6 7. High Availability A system that is designed for continuous operation in the event of a failure of one or more components. However, the system may display some degradation of service, but will continue to perform correctly. High Availability: The proportion of time during which the service is accessible with reasonable response times should be close to 100%. 8. How to decide required scale (capacity) & availability? Average throughput (TPS) Max throughput (TPS) Monetary value of a transaction Average loss & max loss per second of downtime Decide on how much to invest based on cost vs. benefit tradeoff8 9. Vertical Scaling Get the maximum out of each allocated JVM or resource Increase CPU size Increase memory 10. Horizontal Scaling 11. Load Balancing Load balancing algorithms Round robin Weighted Response based Health check Failover-only 12. Clustering for scalability12 13. Clustering for availabilityGroup Communication Channel/State replication13 14. ReentrantReadWriteLock 15. ReentrantReadWriteLock 16. ReentrantReadWriteLock 17. http://bit.ly/1a8uu7n REENTRANT READWRITE LOCK EXERCISE 18. Thread Pooling 19. ThreadLocal 20. http://bit.ly/1a8uu7n THREADLOCAL EXERCISE 21. ThreadPooling + ThreadLocal CAUTION: Make sure that ThreadLocal variables are reset prior to Threads being returned to the pool 22. Object Pooling Some objects would be expensive to create e.g. Connection objects Reuse objects State stored in those objects need to be cleared before being returned to the pool 23. Lazy Loading1. Lazy Initialization A null field indicates that there is no data. When the value is requested, a null check is performed to see if the actual data needs to be loaded. If(this.value== null) this.value = loadValue(); return this.value; 24. 2. Virtual ProxyLazy LoadingThe virtual proxy implements the same interface as the real object, and when called for the very first time, it loads the real object & delegates to that object. class VirtualProxy implements Interface { private RealObject real; private RealObject getReal(){ if(real == null) real = new RealObject (); return real; } public void justDoIt(){ getReal().justDoIt() } } class RealObject implements Interface { public void justDoIt(){ System.out.println(DONE!); } } 25. Lazy Loading 3. Value HolderA value holder is an object with a getValue method, which the clients will invoke in order to obtain a reference to the real object. Note that the method may not necessarily be named getValue. private ValueHolder valueHolder; public Widget getWidget { return valueHolder.getValue(); } 26. Lazy Loading 4. Ghost The real object without any data. The data is loaded as and when required. UUID id = UUID.getUuid(); String name; Connection dbConn; public Connection getDbConnection(){ If(dbConn == null) dbConn = loadDatabaseConnection(); } 27. Lazy Loading - performance 28. Asynchrony Callback Dual channel Queing 29. Queuing Producer - Consumer 30. Capacity PlanningSource: http://srinathsview.blogspot.com/2012/05/how-to-measure-performance-ofserver.html 31. Capacity PlanningThroughput = number of completed requests / time to complete the requestsNo. of servers = (projected max load * 1.3) / max throughput of one servermax throughput of one server = max throughput of that server for the slowest scenario in the set of use cases 32. Static membership Predefined members Other (non-predefined) nodes cannot join Static group M1M2M3NM432 33. Dynamic membership No predefined members Nodes can join & leave Dynamic group M1M2N JoinM3M433 34. Hybrid membership Some predefined (well-known) members, and some dynamic membersNodes can join & leaveMembership revolves around the static members Hybrid group Dynamic membersM6M5M7NStatic members M1M2M3Join (IP, Port)M434 35. Multicast based membership managementM4NM1Join (IP, Port)M2M335 36. Well-known Address (WKA) based membership managementHybrid group Dynamic membersStatic membersM6NWK1M5WK2 NotifyM7Join (IP, Port) WK3WK436 37. Multicast vs. WKA MulticastWKAAll nodes should be in the same subnetNodes can be in different networksAll nodes should be in the same multicast domainNo multicasting requirementMulticasting should not be blocked No fixed IP addresses or hosts requiredAt least one well-known IP address or host requiredFailure of any member does not affect membership discoveryNew members can join with some WKA nodes down, but not if all WKA nodes are downDoes not work on IaaSs such as Amazon EC2IaaS-friendly Requires keepalived, elastic IPs or some other mechanism for remapping IP addresses of WK members in cases of failure37 38. Auto-scaling Scale-out when load increases Scale-in when load decreases Always use the optimum amount of resources Try out AWS ELB Apache Stratos Load Balancer WSO2 ELB 39. Auto-scaling steady load 40. Auto-scaling load increasing 41. Auto-scaling load increasing 42. Auto-scaling steady load 43. Auto-scaling decreased load 44. Auto-scaling decreased load 45. Auto-scaling steady load 46. Autoscaling - Analysis & Results 47. Autoscaling - Analysis & Results 48. Single nodeCostLOWESTAvailabilityHIGHEST48 49. Primary-secondaryCostLOWESTAvailabilityHIGHESTPrimarySecondary 49 50. Primary-secondary, multiple LB HIGHESTCostLOWESTAvailabilitykeepalivedPrimarySecondary 50 51. Active cluster, multiple LB HIGHESTCostLOWESTAvailabilitykeepalivedActiveActiveActive 51 52. Multi-zone or multi-datacenter Deployment HIGHESTCloud ControllerZone 1CostLOWESTAvailabilityZone 2Region X 52 53. Multi-region deployment HIGHESTZone 1Zone 2Region XZone 2CostLOWESTAvailabilityZone 1Region Y53 54. Multiple IaaS (hybrid) Deployment HIGHESTZone 1Private cloud (data center)Zone 2Zone 1Zone 2Amazon EC2CostLOWESTAvailabilityZone 1Zone 2Rackspace Cloud54 55. Multi-region Multi-IaaSMulti-zoneMulti-node active cluster - Single zonePrimary-Secondary, with multiple LBsPrimary-Secondary, single LBSingle NodeCost of Availability55 56. Hazelcast Clustering and highly scalable data distribution platform for Java, which is: Lightning-fast; thousands of operations/sec. Fail-safe; no losing data after crashes. Dynamically scales as new servers added.Open source 57. Hazelcast In-memory data grid 58. Hazelcast Management Center 59. http://bit.ly/1a8uu7n HAZELCAST EXERCISE 60. JSR 107 JCache (javax.cache.*) CacheManager cacheManager = Caching.getCacheManagerFactory().getCacheManager(manger"); Cache cache = cacheManager.getCache("sampleCache"); int value1 = 9876; cache.put(key, value1); int value = cache.get(key).intValue() 61. JSR 107 JCache (javax.cache.*) CacheManager cacheManager Caching.getCacheManagerFactory().getCacheManager("test"); String cacheName = "cacheXXX"; Cache cache = cacheManager.createCacheBuilder(cacheName). setExpiry(CacheConfiguration.ExpiryType.MODIFIED, new CacheConfiguration.Duration(TimeUnit.SECONDS, 10)). setStoreByValue(false).build(); int value = 9876; cache.put(key, value); assertEquals(cache.get(key).intValue(), value); 62. http://bit.ly/1a8uu7n JCACHE EXERCISE 63. DISCUSSION 64. Thanks!