Docker Hub: Past, Present and Future by Ken Cochrane & BC Wong
-
Upload
docker-inc -
Category
Technology
-
view
3.656 -
download
1
Transcript of Docker Hub: Past, Present and Future by Ken Cochrane & BC Wong
Docker Hub
Docker Hub (hub.docker.com) is a cloud registry service forsharing application and automating workflows.
Major features:• Public and Private Docker repositories• Official Repositories• Collaborators, Organizations and Groups• Automated builds• Webhooks
3
Hub by the Numbers
• 240,000 Users
• 150,000 Repositories
• 1,250,000 Automated Builds
• 500,000,000 Pulls
5
Organizations & Groups
• Best suited for businesses• Collaborate on repos with members of your organization• Fine-grain permissions• Add users to groups• Public and Private repos
20
Official Repositories
• 80 Official Repos that you can use to build your Docker applications• Repos are updated and maintained by Docker
21
Private Repositories
• Not searchable or browsable• Can add collaborators to push or pull• Only you or a collaborator can access• 1 free private repo with your docker hub user account
22
Coupon for 3 free months of the micro plan
dockercon15
Automated Builds
• Link to your Github or Bitbucket accounts• Keep your source code in sync with your docker repos• You write the code, we handle the build, and keep your Docker hub
repos up to date.• Public and Private repos
23
Repository Links
What if your base ubuntu:14.04 image is updated for a security fix?
• Link your automated build repository to another repository• Allows you to keep your automated build up to date
24
Build Triggers
• Trigger an automated build from an external system• Useful for integrating with CI systems
25
Webhooks
• Get notified when a docker repo is updated• Ability to chain together webhooks to create a build pipeline• Combine with a continuous integration and deployment solution to
easily move your application from development to server.
26
Upcoming Features
• Show image sizes• GitLab support for automated builds• Dynamic tagging for automated builds
28
Hub 2.0
• New, Easier to use UI• Re-architected, isomorphic javascript application based on react.js,
react-router and fluxible• A modern, modular approach using ES2015 enabling faster iterations
30
Data stores
Web UI
API Gateway
Postgresql Redis KafkaRabbitMQElastic Search
Micro Services
S3
New Architecture
Live demo time: Hub 2.0hub-beta.docker.com
Left brain: What could go wrong?Right brain: Some people never learn.
Architecture Upgrade - Registry v2
• Registry v2 protocol is far superior to v1• Supported by Docker >=1.6• Content-addressable manifest• More secure
36
80% fewer requests
60% less bandwitdth
Architecture Upgrade - Disentangle
• Clean up inter-service dependencies:
• Hub 2.0 uses a gateway with finer grain services.• Reduces cascading failures.
37
Infrastructure Upgrade
• Aka “throw money at the problem”• The difficult parts are when (i.e. monitoring), and the execution.• E.g. database upgrade:
38
External ping latency (worldwide average)DB response time (before vs after)
Automated Builds (v1 current)
Architecture• A set of long-running build servers w/o auto-scaling• One build per server at a time (docker in docker)• Thousand’s of builds per day
40
Challenges• # of builds <= # of servers, which
means pending builds• Security concerns• Build server issue == lost capacity
Automated Builds (v2 upcoming)
Speed and efficiency• Auto-scale on-demand builds• Per-user queuing• Better scheduling
Security• Use VM for isolation between users• TLS everywhere• Ephemeral build servers
41
Scalability & Reliability Work: Myth vs Reality
43
Need a team that can do the first 80% and the last 20%.Need a culture that understands quality.
Hey, it’s still fun
Why is my sink not draining?
From our post-mortems & bug reports:• DB capacity• ORM layer makes slow queries• Networking flakiness• RabbitMQ failures (and we didn’t understand its failure models)• Supervisor failed to restart appserver, but NGINX keeps accepting calls• Expired intermediate cert• Docker btrfs driver bug• ...
Know when to tighten a bolt, and when to get a new sink.
44
High Velocity Cycle (invest in tooling)
45
Monitoring& logging(what’s slow)
Features & fixes
QA, scale test,perf regression test(prove that things work)
Future Performance Work
• Continue to eliminate bottlenecks as Hub grows• Speed up push/pull outside the US• Scalable pipeline into data warehouse
46
Credits: The Hub Team
Ryan AbramsChris BiscardiKen CochraneJohn CostaRoberto HashiokaDaniel HiltgenNathan HsiehNick KralyAdrian KreherToli KuznetsDustin Lacewell
47
Nick LangSean LiAmy LindburgMarcus MartinsChinmayee NirmalAtul PhadnisMario PonticelloArunan RabindranJosh SouthJusten Stepkabc Wong