1. Double Auction-based Scheduling ofScientific Applications in Distributed Grid and Cloud Environments Francesc Lordan Gomis Metodologia de Recerca en la Informtica 25/06/2012

2. OutlineIntroductionAuction TheoryModel and strategiesResultsConclusionsDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 1 3. IntroductionGrid and Cloud Computing allow user to execute applications on a third-party infrastructure by paying for the access to the remote resourcesDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 2 4. IntroductionUsers are represented by Schedulers that picks which resource a job will be runThe providers are represented by Resource Managers that try to maximize the incomes of the provider by optimizing the resource usage Resource Resource ManagerManager Scheduler Resource Resource ManagerManager Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 3 5. IntroductionWhen many users try to obtain the same resources, they compete for them.Since access to these resources is paid, it can be seen as a market governed bythe supply-demand lawEconomy-based negotiation techniques are attractive from bussiness point of viewScheduler ResourceManagerSchedulerSchedulerMarketResource?ManagerSchedulerScheduler ResourceManagerSchedulerScheduler ResourceManagerSchedulerDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 4 6. Auction TheoryAuctions are an economic mechanism used for allocating a set of resources among a group of bidders.Models: Single-sided auctions: the seller submits a resource and the bidders submit their offers to buy (bid) - English auction: buyers constantly increase their offers until the highest one wins - Dutch auction: the seller decreases the price of the resource until a buyer decides to pay the proposed price - Sealed auctions: buyers give their bids on an envelope First-price: the highest bid wins Vickrey second-price: the second highest bid winsDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 5 7. Auction TheoryAuctions are an economic mechanism used for allocating a set of resources among a group of bidders.Models: Double-sided auctions: all the participant submit their offers to sell (asks) and buy (bids) - Call Auctions: during a timeframe all the participants submit their offers and finally they try to match - Continuous Double Auction (CDA): offers are continously submitted until a match between the lowest ask and the highest bid is achieved or the auction times out. Combinatorial auctions: a single bid for multiple resource (NP-complete)Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 6 8. Model and strategiesDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 7 9. Model and strategiesThe limitation on resources converts the schedulers into competitors with their own negotiation strategy.Any strategy is based on 4 factors Resource selection Resource valuation Auction participation Bidding Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 8 10. Model and strategiesResource SelectionBefore bidding, the scheduler decides for which resources it will participate in an auction.When a task is sumitted the scheduler looks for a pending auction. If there is no such auction the scheduler postpones the bidding until there is one.The selection is done with a lottery where probabilities take into account:Availability of the resourceExecution timeBudget cost Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 9 11. Model and strategiesResource ValuationUser Valuation 2 steps: - Update the value of the resource depending on the final price of itslast auction and a smoothing factor . - Increases the value depending on the number of lost auctions and arevaluation factor .Provider ValuationDepending on the success ratio on the last auctions for the resourcethe price is increased or decreased by a resource revaluation factor Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 10 12. Model and strategiesAuction participationScheduler participates when there is at least one task waiting for that resource. Tasks are kept in a priority queue sorted by the bottom levelsResource ManagerThe time between two auctions of the same resource are createddepend on the result for the last auctions. If it was won, it wait until the resource has been used and adds a delay If it was lost, the delay is incremented exponentially with a delaying factor Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 11 13. Model and strategiesBiddingZero Intelligence: participants submit their offers periodicallyScheduler tries to reduce the value of the resource by decreasing the distancebetween its valuation and the highest bidResource Manager tries to increase the value of the resource by reducing the distancebetween its valuation and the lowest askDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 12 14. Model and strategiesModifications to the base strategy:Self-limitation Reduce the price by reducing the competence limit the number of auctions where the scheduler participates in based on:The average number of retriesA resource limitation factor Pricing AgressivenessChange the revaluation factor of a single schedulerValue of the resource are increased faster or slower Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 13 15. ResultsResource ValuationDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 14 16. ResultsResource ValuationDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 15 17. ResultsSelf-limitationDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 16 18. ResultsPricing aggressiveness Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 117 19. ConclusionsCreation of an open market of resources implemented as a CDA instanceIdentification the general behaviour pattersEvaluation of strategies: Self-limitation: - has marginal improvements on specific configuration - NOT RECOMENDED Concessive strategy: - very good for the common-drift case Aggressive-strategy: - reduces execution time with low impact on the budget on the single-userdeviation case - higher budget and similar execution time on the common-drift case - RISKY Double Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 18 20. QuestionsDouble Auction-based Scheduling of Scientific applications in Distributed Grid and Cloud Environment 19 21. Resource SelectionavailabilityExpected execution time and cost at any resource Expected execution timeResource value0