PuppetConf 2016: Service Discovery and Puppet – Marc Cluet, Ukon Cherry
-
Upload
puppet -
Category
Technology
-
view
38 -
download
0
Transcript of PuppetConf 2016: Service Discovery and Puppet – Marc Cluet, Ukon Cherry
Who am I?
@lynxman
Engineer based in London
Co-Founder of Ukon Cherry
Working at TrainLine
19 years of experience as a SysAdmin
Founding member of Juju and MAAS while at Canonical
Built a DevOps Engineering Team at Rackspace
Been DevOps’in for the last 7 years
2
The Problem
Cloud is Hard!
Old Style DNS
TTL a problem for auto-scaling
Non-reactive health checks
Where do I keep my metadata?
5
https://www.flickr.com/photos/qchristopher/5038229030/
Service Discovery
Automatically define your services
Active Health Checking
Dynamically updated service lists
Can be DNS accessible (if needed)
API accessible (win!)
7
https://www.flickr.com/photos/marknye/12992319144/
Service Discovery
9
Service Publication
Service Node A
Health Check
Discovery Agent
Discovery Agent
Service Node B
Health Check
Service Discovery
10
Service Publication
Service Node A
Health Check
Discovery Agent
Discovery Agent
Service Node B
Health Check
Service Discovery
11
Service Publication
Service Node A
Health Check
Discovery Agent
Discovery Agent
Service Node B
Health Check
Service Discovery
12
Service Publication
Service Node A
Health Check
Discovery Agent
Discovery Agent
Service Node B
Health Check
Service Discovery
15
Service Node A
Service: web
10.10.10.110.10.10.210.10.10.3
Service Node B
Service Node C
Service Discovery
16
Service Node A
Service: web
10.10.10.110.10.10.210.10.10.3
Service Node B
Service Node C
Service Discovery
17
Service Node A
Service: web
10.10.10.110.10.10.210.10.10.3
Service Node B
Service Node C
Service Discovery
19
Agent Master Agent Agent Agent Agent
Agent Agent Agent Agent Master Agent
Agent Agent Master Agent Agent Agent
Agent Agent Agent Agent Agent Agent
Service Discovery
20
Agent Master Agent Agent Agent Agent
Agent Agent Agent Agent Master Agent
Agent Agent Master Agent Agent Agent
Agent Agent Agent Agent Agent Agent
Service Discovery
21
Agent Master Agent Agent Agent Agent
Agent Agent Agent Agent Master Agent
Agent Agent Master Agent Agent Agent
Agent Agent Agent Agent Agent Agent
Service Discovery
22
Agent Master Agent Agent Agent Agent
Agent Agent Agent Agent Master Agent
Agent Agent Master Agent Agent
Agent Agent Agent Agent Agent Agent
Service Discovery
23
Agent Master Agent Agent Agent Agent
Agent Agent Agent Agent Master Agent
Agent Agent Master Agent Agent
Agent Agent Agent Agent Agent Agent
! !
!
Service Discovery
24
Agent Master Agent Agent Agent Agent
Agent Agent Agent Agent Master Agent
Agent Agent Master Agent Agent Agent
Agent Agent Agent Agent Agent Agent
Service Discovery Solutions
CoreOS Fleet Uses the sidekick
model, uses separate agents to orchestrate
service discovery.
Kubernetes Service All pods declare services to be
discoverable by the cluster.
27
Consul All services are declared by the
agents, the agents themselves are
responsible.
Service Discovery Solutions
CoreOS Fleet API publication
k/v Strongly Consistent
Container checks
Kubernetes Service API publication
Container checks
Auto-Heal
28
Consul API + DNS publication
k/v Strongly Consistent
Host + Service checks
Puppet modulePuppet module Hiera access
Puppet module Hiera access
What is Consul?
It's a Service Discovery System (duh!)
Service Publications (DNS + API)
k/v Storage (strongly consistent)
Health Checks
With encryption! (whaaaaaat)
30
Consul Architecture
34
Consul Master
Consul Master
Consul Master
Consul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul Agent
Consul Architecture
35
Consul Master
Consul Master
Consul Master
Consul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul Agent
Consul Master
WAN DC
Consul DNS publisher
service name: web
zone: yourDC
<servicename>.service.<zone>.consulweb.service.yourDC.consul
37
Consul Catalog API
API http://localhost:8500
/v1/catalog//v1/catalog/datacenters//v1/catalog/nodes//v1/catalog/services/<servicename>/v1/catalog/service/<name>/v1/catalog/node/<nodename>
38
Consul Health Checks
{ "check": { "id": "mem-util", "name": "Memory utilization", "script": "/usr/local/bin/check_mem.py", "interval": "10s" }}
39
Consul Restful API
API http://localhost:8500
/v1/kv/
/v1/agent/
/v1/catalog//v1/health/
/v1/session/
/v1/acl/
/v1/status/
43
Consul Restful API
API http://localhost:8500
/v1/kv/
/v1/agent/
/v1/catalog//v1/health/
/v1/session/
/v1/acl/
/v1/status/
44
You can do all this after the talk
https://github.com/lynxman/consul-first-steps
46
https://www.flickr.com/photos/hortlander/6245707871/
Puppet + Consul
47
https://www.flickr.com/photos/hortlander/6245707871/in/
You can integrate at several levels
Puppet Module
KyleAnderson-consul
Hiera
lynxman-hiera_consul
Puppet + Consul
48
https://www.flickr.com/photos/hortlander/6245707871/in/
You can integrate at several levels
k/v access
venmo-consulr
Templates with Consul
ghdbaston-consul_template
Puppet Module - Install
49
https://www.flickr.com/photos/hortlander/6245707871/in/
Puppet Forge makes it very easy to install
$ puppet module install KyleAnderson-consul
Puppet Module - Dnsmasq Install
50
https://www.flickr.com/photos/hortlander/6245707871/in/
$ puppet module install saz-dnsmasqinclude dnsmasq dnsmasq::conf { 'consul': ensure => present, content => 'server=/consul/127.0.0.1#8600', }
Puppet Module - Health Checks
51
https://www.flickr.com/photos/hortlander/6245707871/in/
consul::check { 'disk_space': script => 'check_disk -w 5% -c 1%',
interval => '30s', }
Puppet Module - Services
52
https://www.flickr.com/photos/hortlander/6245707871/in/
consul::service { 'nginx': port => '80', checks => [ { script => 'check_http -H localhost -w 20 -c 60', interval => '30s', }, ], }
Puppet Hiera Module - Install
54
https://www.flickr.com/photos/hortlander/6245707871/in/
Puppet Forge makes it very easy to install
$ puppet module install lynxman-hiera_consul
Puppet Hiera Module - Config
55
https://www.flickr.com/photos/hortlander/6245707871/in/
:backends: - yaml - consul
:yaml: :datadir: /etc/puppetlabs/hieradata
:consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/catalog/service - /v1/catalog/node
Puppet Hiera Module - Config
56
https://www.flickr.com/photos/hortlander/6245707871/in/
:consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/catalog/service - /v1/catalog/node
Puppet Hiera Module - Arrays
57
https://www.flickr.com/photos/hortlander/6245707871/in/
notice('Generating rabbitmq cluster members based on Consul information')
$consul_service_array = hiera('rabbitmq',[])
$mq_cluster_nodes = consul_info($consul_service_array, 'Address')
notice("Result: ${mq_cluster_nodes}")
Puppet Hiera Module - Arrays
58
https://www.flickr.com/photos/hortlander/6245707871/in/
notice("Generating neo4j_ha cluster members based on Consul information")
$consul_service_array = hiera('neo4j_ha',[])
$consul_fields = [ 'Address', 'ServicePort' ]
$consul_ha_initial_hosts = consul_info($consul_service_array, $consul_fields, ':')
$ha_initial_hosts = join($consul_ha_initial_hosts, ',')
notice("Result: ${ha_initial_hosts}")
Puppet Hiera Module - Accessing the k/v
59
https://www.flickr.com/photos/hortlander/6245707871/in/
:consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/kv/
Puppet Hiera Module - Accessing the k/v
60
https://www.flickr.com/photos/hortlander/6245707871/in/
:consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/kv/mystuff/
Puppet Hiera Module - Accessing the k/v
61
https://www.flickr.com/photos/hortlander/6245707871/in/
:consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - "/v1/kv/%{env}/"
Puppet Security - Github
63
https://www.flickr.com/photos/hortlander/6245707871/in/
$ git commit -m "All my passwords"$ git push
Puppet Hiera Module - Arrays
64
https://www.flickr.com/photos/hortlander/6245707871/in/
https://www.flickr.com/photos/tomukas/3554360505/
Puppet Security - Hiera Modules
65
https://www.flickr.com/photos/hortlander/6245707871/in/
hiera-eyamlhiera-gpg
Puppet Security - Module Install
67
https://www.flickr.com/photos/hortlander/6245707871/in/
Puppet Forge makes it very easy to install
$ puppet module install hiera-eyaml
Puppet Security - Hiera eyaml
68
https://www.flickr.com/photos/hortlander/6245707871/in/
$ eyaml createkeys
keys/public_key.pkcs7.pem
keys/private_key.pkcs7.pem
Puppet Security - Hiera eyaml
69
https://www.flickr.com/photos/hortlander/6245707871/in/
Usage: eyaml <subcommand>Please use one of the following subcommands or help for more help:
createkeys, decrypt, edit, encrypt, recrypt, version
Puppet Security - Hiera eyaml
70
https://www.flickr.com/photos/hortlander/6245707871/in/
mysecret: DEC::PKCS7[mypassword]!
Puppet Security - Hiera eyaml
71
https://www.flickr.com/photos/hortlander/6245707871/in/
mysecret: ENC[PKCS7,MIIBmQYJKoZIhvcNAQcDoIIBijCCAYYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAAikKizydVh0wXQrvtoMC7vM9NxfksqwOX2jtajDYMMJwXXP/5zKHjnnGmr+LSXFVkL52FuGentCdityjF0zZEvbZ2D95TWnRTinO9htteC8ZiwqpTeuNJkTJikOEEZvHbNlp6eX381ppKoatV1k0EmIHwsnqeRJN5T9TVScoXOb/1Fre4H7TxSvvaFqo02MWUBaKkWECoEu2PLiuXWEoiLrkDq8pxhjYADvGUJLWC8PUSWT/94075z5UKHYBQgLlFrzG+89Rhm5keTy/cuHsOK9d0nUScjd4m6duCEsvRT5SG/n6GwTEk/cDMqIuvAwNETv2fdepu4z5nR383zlngDBcBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBAJCDkds8PbXeBUMZhFPxWTgDDH1pvUCbCLtWDNVFkW2yZ1NYF06RuqsSTxofHfMwajC+BSPcTu7heMKQnbKP/KE6o=]
Puppet Security - Extra
Friday, October 21 • 11:15am - 12:00pm
Using HashiCorp's Vault With Puppet
Seth Vargo, HashiCorp
74