StoreconfigsAll About

Brice Figureaumasterzen (#puppet)

Who am I?

Puppet Community Developerplanetpuppet.org maintainer

Whatʼs Storeconfigs?

Storeconfigs stores to an RDBMSuses Railsʼ Active Record

2 Possible Usages

Inventory Export/Collect

Export/CollectNode collaborations

Syntax: export

@@resource { ...}

Syntax: collect

Resource <<| query |>>

Collect: Query•Resource Parameters•Tags•and, or

Tip

Always tag exported resources

The BIG Picture

masterDB

collector 1

Exporter

collector 2

Simple exampleWeb server replication, deployment server

Simple exampleWeb server replication, web server part

Intermediate exampleBuilding /etc/resolv.conf from exported snippets

Intermediate exampleBuilding /etc/resolv.conf, aggregating snippets

Advanced ExampleTinydns master

Advanced ExampleTinydns master

Advanced ExampleTinydns slave

Limitations

Inner Workings

Inner Workings1. Read all resources, tags for a node (slow)

Inner Workings2. Compare with in-memory catalog (fast)

Inner Workings3. Store/remove missing/extraneous resources/tags (slow)

Inner WorkingsDB schema

Performance Impact

Performance ImpactCollecting is fast

Performance ImpactStoreconfigs write can be slow

Performance Impact

0

6

13

19

25

Time (seconds)

Off On1st run (1950 resources)

Performance Impact

0

2

4

6

8

Time (seconds)

Off OnSubsequent runs (1950 resources)

First Run Impact

First Run ImpactFailure

err: Could not retrieve catalog: execution expired

Scaling: some rough math1 HDD = 250 IOP/s

Scaling: some rough mathAll nodes have 500 resources4 params/ resources4 tags/resources=> 4500 writes=> 1000 IOP

Scaling: some rough mathMaster concurrency: 5

Scaling: some rough mathIOP/s left per node:

250/5 = 50 IOP/s

Scaling: some rough mathTime per node

1000 IOP = 20s

Scaling: some rough mathAnd for a 500 nodes setup...

500/5 = 100 “runs”

Scaling: some rough mathWith a sleep time of 30min

30 min = 1800s1800s/20s = 90

90 nodes per 30 min

Scaling: some rough mathHouston we have a problem

90 nodes when we needed 100

Scaling the back-endDedicated RDBMS server(s)

Scaling the back-endPowerful I/O subsystem (RAID,BBU...)

Scaling the back-endDB setup tuning

Scaling the back-endMySQL, scaling for insertsinnodb_buffer_pool_size = 70% of physical RAM

Scaling the back-endMySQL, scaling for insertsinnodb_log_file_size = up to 5% of physical RAM

Scaling the back-endMySQL, scaling for insertsinnodb_flush_method = O_DIRECT

Scaling the back-endMySQL, scaling for insertsinnodb_flush_log_at_trx_commit = 2

0.25 to the help!

0.25 to the help!Async Storeconfigs - no more thundering herd

0.25: Async Storeconfigs

masterDB

node

puppetqdMessage Queue

0.25: Thin StoreconfigsOnly exported resources, tags and facts

0.25: Thin Storeconfigs

0

1

3

4

6

Time (seconds)

Off On1st run (1950 resources)

Storeconfigs future?

More AR adapters

More AR tuning (pooling)

Non RDBMS back-ends

Non RDBMS back-ends•Tokyo Cabinet/Tyrant

•Mongo DB

Questions ?Please speak loud and slowly and excuse me

by advance if I donʼt understand a word of your question.

Being French has some advantages (like health care access or a cute if not stupid accent)it also has drawbacks, especially our educational system for

learning foreign languages simply sucks :-D