Cloudifying Source Code Repositories: How much does it cost?
Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems...
-
Upload
irene-williams -
Category
Documents
-
view
216 -
download
0
Transcript of Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems...
![Page 1: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/1.jpg)
Cloudifying Source Code Repositories:Cloudifying Source Code Repositories:How much does it cost?How much does it cost?
1
Hadi Salimi,
Distributed Systems Labaratory,
School of Computer Engineering,
Iran University of Science and Technology
Fall 2010
![Page 2: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/2.jpg)
What’s the Cloud Computing ???• Large scale• Application-specific
architectures• Developed for in-house
use
• Available for
general usage• Inexpensive,
even for small
or medium scale deployments
2
![Page 3: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/3.jpg)
What is Revision Control?
• Repository for data (source code)– All changes are tracked by date and author– Branching and merging
• Why move it to the cloud?– Resilient storage– No physical server to administrate– Scale to larger communities (SourceForge)
3
![Page 4: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/4.jpg)
Available Tools
• Subversion, revision control system– Free, open-source– Very popular– Rigid consistency model
4
![Page 5: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/5.jpg)
Available Tools (Cont’d)
• Amazon S3, cloud storage service– Eventual consistency
• Yahoo ZooKeeper, coordination service– Free, open-source
5
![Page 6: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/6.jpg)
Alternative solutions
Cloud Computing P2P
• Subversion etc.• Repository stored
persistently in the cloud• One true, consistent
repository exists
• GIT etc.• Repository stored at
every client• Many repository copies,
converging eventually
6
![Page 7: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/7.jpg)
Outline
• Costs of using cloud storage for revision control• Architecture of a simple solution• Performance evaluation
7
![Page 8: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/8.jpg)
How to Measure Costs• Each revision stored as two files on disk
– Revision data– Revision properties
• Calculate bandwidth, per-transaction, and storage costs of pushing each revision into S3 over time
8
![Page 9: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/9.jpg)
Storage Costs
9
![Page 10: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/10.jpg)
Storage Trends
10
![Page 11: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/11.jpg)
Outline
• Costs of using cloud storage for revision control• Architecture of a simple solution• Performance evaluation
11
![Page 12: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/12.jpg)
AsynchronousReplication
Primary Backup
Clients Today’s architecture for source code revision control...
12
![Page 13: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/13.jpg)
A cloud-basedarchitecture...
EC2 EC2
S3S3S3
13
![Page 14: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/14.jpg)
Two simultaneous commits…
EC2 EC2
S3S3S3
Rev. 31337Rev. 31337
Rev. 31337
Followed by an update…Leads to data loss!
14
![Page 15: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/15.jpg)
Coordination Coordination
EC2 EC2
S3S3S3
15
![Page 16: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/16.jpg)
Commit Process
ZooKeeper
16
![Page 17: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/17.jpg)
17
![Page 18: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/18.jpg)
Outline
• Costs of using cloud storage for revision control• Architecture of a simple solution• Performance evaluation
18
![Page 19: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/19.jpg)
Usage Observations
• Apache Foundation– 1 repository, 74 projects– Average 1.10 commits per minute– Maximum 7 commits per minute
• Debian community– 506 repositories– Average 1.12 commits per minute– Maximum 6 commits per minute
19
![Page 20: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/20.jpg)
Results
Checkouts (Reads) Commits (Writes)
• Adding servers improves the user experience20
![Page 21: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/21.jpg)
Conclusion
• Storing source code repositories in the cloud is feasible…
• …and very inexpensive
• Only minor changes to existing revision control systems are necessary to robustly take advantage of cloud storage
21
![Page 22: Cloudifying Source Code Repositories: How much does it cost? 1 Hadi Salimi, Distributed Systems Labaratory, School of Computer Engineering, Iran University.](https://reader036.fdocuments.in/reader036/viewer/2022062518/56649ec05503460f94bcac47/html5/thumbnails/22.jpg)
Questions or Comment
24