Building  a  Private  Supermarket  for  your  Organization

Justin  Dossey

Who  I  am  (and  why  you  should  listen  to  me)

• I’m  Justin  Dossey.  • Twenty  years  as  a  professional  sysadmin  

• Ten  years  as  a  Rubyist  • Seven  years  with  CM  • I  just  built  a  bunch  of  supermarkets  a  couple  of  months  ago  

• We’re  hiring!    Talk  to  me  after.

A  Brief  History  of  the  Supermarket

A  Brief  History  of  the  Supermarket• Built  for  Chef  as  a  simple  Ruby  on  Rails  app  • Hosted  in  AWS  • First  commit  (on  Github):  October  2013  • Originally  designed  as  something  to  be  hosted  for  Chef,  by  Chef,  as  a  public  thing  

• This  means  that  stuff  like  CLA  signing  was  baked  in  early  

• It  also  means  some  things  about  private  are  funky

Why  Build  Private?• The  organization  makes  a  lot  of  its  own  cookbooks  

• Public  supermarket  may  provide  cookbooks  of  suspect  quality  

• The  organization  is  worried  that  the  public  supermarkets  could  be  closed  when  it  needs  to  shop  

• Worries  that  the  public  supermarket  could  be  robbed  or  infiltrated

The  Public  Supermarket  Codemakes  the  private  supermarket  possible

The  Supermarket  is  Open-­‐Source• Chef  makes  the  supermarket  code  available  to  everyone  

• If  they  can  run  one,  so  can  we!  • …but  we’re  not  Chef,  so  we  might  have  to  change  some  things  about  how  it  works.  

• There  are  an  awful  lot  of  build  dependencies  • …  but  there’s  a  cookbook,  so  it’s  easy,  right?

Moving  Partshow  the  supermarket  works

Supermarket  Architecture

• Supermarket  is  a  Ruby  on  Rails  application.  • It  talks  to  a  Postgres  Database  • It  uses  dotenv  to  store  environment  variables  (usually  related  to  config)  • It  uses  omniauth  +  devise  • Redis  is  in  there  too

9

Supermarket  Architecture• Lots  of  integrations  with  external  services:  

• oc-­‐id  on  the  Chef  server  • Github  • StatsD  • Sentry  • Amazon  S3  • Fieri  • Curry  • Pubsubhubbub  • The  SMTP  relay  • New  Relic  • Google  Analytics

10

Our  Minimum  Working  Set• Lots  of  integrations  with  external  services:  

• oc-­‐id  on  the  Chef  server  • Github  • StatsD  • Sentry  • Amazon  S3  • Fieri  • Curry  • Pubsubhubbub  • The  SMTP  relay  • New  Relic  • Google  Analytics

11

ConfiguringFun  with  Data  Bags

Yes,  Data  Bags.• For  whatever  reason,  Chef’s  Supermarket  developers  decided  that  the  best  place  to  put  the  application  configuration  was  a  data  bag.      

• Back  in  2013,  this  probably  made  sense.  • That  decision  remains  with  us  today  and  feels  weird,  because  we  all  write  wrapper  cookbooks  to  accomplish  the  same  thing  nowadays.  

• The  data  bag  we  need  is  called  “apps/supermarket.”

Stuff  in  the  Data  Bag• redis  URL  • SMTP  server  info  • github  info  • oauth2  tokens  for  oc-­‐id  • curry  info  • sentry  url  • which  revision  of  supermarket  to  clone  • a  bunch  of  URLs

Stuff  not  in  the  Data  Bag• SSL  stuff  • supermarket  hostname  • deploy  location  • some  database  config  stuff  • some  sidekiq  stuff

Doing  this  Ourselves

Write  a  Wrapper  Cookbook• Put  attributes  and  a  working  data  bag  in  place.      • Get  the  .kitchen.yml  written  and  the  thing  building  in  dev,  then  build  the  production  system.  

• Read  the  docs—  this  stuff  is  changing  pretty  frequently.  

• When  stuff  breaks,  look  carefully  at  the  production.log,  the  .env  file,  and  the  data  bag.

Authentication  to  the  Chef  Server• The  Chef  server  needs  to  know  who  is  allowed  to  register  apps  for  oc-­‐id  auth.  

• Last  I  checked,  that  was  a  line  in  /etc/opscode/chef-server.rb  like  this:

• Once  that’s  in  and  we  run  chef-server-ctl reconfigure,  we  can  get  the  oauth  stuff  we  need  for  that  data  bag.

oc_id[‘administrators’] = [‘chef_username’]

Continuous  Integration  and  Continuous  Deployment  for  

Cookbooks

A  brief  overview  of  pipeline  design  for  infrastructure  code

How  it’s  Usually  Done

Doing  it  with  Supermarket

Sticky  Situations

Cookbook  Naming

• Berkshelf  has  a  flat  namespace.  • We  can  specify  multiple  sources  for  Berkshelf.  

• If  our  (in-­‐house)  cookbook  is  named  the  same  as  one  on  the  public  supermarket,  the  version  is  the  only  differentiator.    This  can  also  collide.  

• Switching  to  a  single  origin  means  someone  has  to  upload  community  cookbooks  to  the  private  supermarket  

• Doing  this  trades  namespace  collisions  for  pipeline  and  maintenance  overhead

Cookbook  Ownership

• With  the  pipeline  proposed,  the  only  uploader  of  cookbooks  is  a  role  user—Jenkins.      

• Therefore,  connecting  supermarket  accounts  to  github  accounts  is  weird.  

• We  wind  up  assigning  ownership  outside  Supermarket.

The  Future  of  Private  Supermarkets

Supermarket  Omnibus  Installer• Increasingly,  Chef  is  pushing  installation  via  pre-­‐built  packages  instead  of  the  Supermarket  cookbook.  

• This  means  that  instead  of  the  data  bag  and  such,  we  will  write  a  simple  wrapper  cookbook  to  install  a  package  and  configure  components  as  necessary.  

• It  is  unlikely  that  this  newer  method  will  affect  any  of  the  integration  points.

Review

Review• Running  a  private  supermarket  means  adapting  the  public  code  (which  performs  tasks  relevant  to  the  public  supermarket)  to  our  needs  • We  need  our  own  chef  server  to  host  a  private  supermarket  • The  supermarket  cookbook  uses  a  data  bag  for  some  configuration  data  • There  are  many  external  integration  points  which  we  can  select  if  we  like,  but  most  are  not  required  for  basic  operation  • We  have  to  decide  how  to  approach  namespace  and  ownership  issues  early  on

28

Questions

Thank  you!

(Did  I  mention  we  are  hiring?)

Find  me!  blog:  www.justindossey.com  twitter:  @justindossey  irc:  jdossey  on  #chef  (freenode)  email:  justin@newcontext.com