Docker in Production: Lessons Learned While Building the Self-Service Semantic Suite (S4)
12
Docker in Production Lessons Learned While Building the Self-Service Semantic Suite (S4) Alexander Simov July 7 th , 2015 Docker in production: the Self-Service Semantic Suite (S4) #1 July 2015
-
Upload
ontotext -
Category
Technology
-
view
152 -
download
1
Transcript of Docker in Production: Lessons Learned While Building the Self-Service Semantic Suite (S4)
Docker in Production Lessons Learned While Building the
Self-Service Semantic Suite (S4)
Alexander Simov
July 7th, 2015
Docker in production: the Self-Service Semantic Suite (S4) #1 July 2015
• Self-Service Semantic Suite (S4)
• S4 Cloud architecture
• DBaaS architecture & Docker integration
• Future work
Contents
#2 Docker in production: the Self-Service Semantic Suite (S4) July 2015
Semantic Technologies for Smart Data Management
• Semantic Search
– Semantic, exploratory search
– Metadata driven content
• Text Mining & Interlinking
– People, locations, organisations, topics
– Reuse open knowledge graphs
• Graph Database
– Flexible RDF graph data model
#3 Docker in production: the Self-Service Semantic Suite (S4) July 2015
What is S4?
#4 Docker in production: the Self-Service Semantic Suite (S4) July 2015
S4 Architecture
#5 Docker in production: the Self-Service Semantic Suite (S4) July 2015
applications
applications
Web UI
routing nodes
data nodes
coordinator
storage notifications
Docker repository
account / quota management
monitoring & logging metadata
store
text analytics
document queue
FactForge semantic
warehouse
S4 DBaaS architecture
#6 Docker in production: the Self-Service Semantic Suite (S4) July 2015
REST apps
3rd party RDF tools
Qu
ota
& A
cce
ss C
on
trol
routers
data nodes
coordinator
EBS
backups
SNS
Docker Registry
Account management
Quota management
reporting
Monitoring & Logging Dynamo
Amazon S3
images
• CRUD
– Router node receives a request
– Routes it to the proper data node & container
– Receives a response, forwards it back to client app
• Routing updates
– Data nodes push notification via SNS – “hearbeats” + changes regarding the hosted DBs (if any)
– Each routing node receives the notifications (via SNS) and updates its routing tables
– Coordinator also receives notifications, learns which DBs are operational / down for maintenance
Normal operations
#7 Docker in production: the Self-Service Semantic Suite (S4) July 2015
…
DBaaS data node
#8 Docker in production: the Self-Service Semantic Suite (S4) July 2015
EBS
Docker Registry
Host state manager
App logs
User data GraphDB engine
App logs
User data GraphDB engine
App logs
User data GraphDB engine
Data node / EC2 host
Docker daemon
Amazon S3
• Flexible processes management
• Quick deployment/migration across VMs
• Isolation
• Usage quota enforcement
• Automated recovery
• Incremental upgrades (new database versions)
Why Docker ?
#9 Docker in production: the Self-Service Semantic Suite (S4) July 2015
• Runs on Ubuntu 14.04 LTS
• Docker 1.6
• Images packaging – GraphDB running in Tomcat 7
• Usage quota enforcement – RAM consumption limitation, based on database size
• Containers restart on failure – Up to 100 failures (configurable)
Docker setup
#10 Docker in production: the Self-Service Semantic Suite (S4) July 2015
• Detailed resources usage metering
• Historical usage data & statistics collection
• Aggregation and visualization
• Experiment with AWS EC2 Container Service (ECS)
Docker related TODOs
#11 Docker in production: the Self-Service Semantic Suite (S4) July 2015
Thank you!
#12 Docker in production: the Self-Service Semantic Suite (S4) July 2015
s4.ontotext.com
