Top 10 Tips for an Effective Postgres Deployment
-
Upload
enterprisedb -
Category
Software
-
view
564 -
download
2
Transcript of Top 10 Tips for an Effective Postgres Deployment
© 2015 EDB All rights reserved. 1
10 Tips for an Effective Postgres Deployment
© 2015 EnterpriseDB Corporation. All rights reserved. 2
• EnterpriseDB Overview • Preparation, Planning, and Postgres • Top 10 Questions to Ask
• Ongoing Questions and Dealing with Change • Summary and Resources • Q&A
Agenda
We will take 2 quick-polls during the webinar to gather group information
© 2015 EnterpriseDB Corporation. All rights reserved. 3
Brief EDB Overview
© 2015 EnterpriseDB Corporation. All rights reserved. 4
EDB is a Recognized Leader Magic Quadrant for Operational DBMS, Q4 ‘15
• Gartner Comments − “EnterpriseDB is responsible for
many features of PostgreSQL, contributing to JSON, materialized views and partitioning. Increasing numbers of users of Gartner’s inquiry services recognize the EnterpriseDB brand.”
− “Clients report that the functionality of EnterpriseDB's Postgres Plus Oracle compatibility feature is more than sufficient to run both mission-critical and non-mission critical applications.”
− “Reference customers rated EnterpriseDB highly for the stability of its DBMS, and were satisfied with its HAS/DR features.”
© 2015 EnterpriseDB Corporation. All rights reserved. 5
POSTGRES innovation
ENTERPRISE reliability
24/7 support
Services & training
Enterprise-class features & tools
Indemnification
Product road-map
Control
Thousands of developers
Fast development
cycles
Low cost
No vendor lock-in
Advanced features
Enabling commercial adoption of Postgres
© 2015 EnterpriseDB Corporation. All rights reserved. 6
Postgres Plus Advanced Server Postgres Plus
Cloud Database
High Availability Performance Management
REMOTE DBA 24x7
SUPPORT PROFESSIONAL
SERVICES
TRAINING
EDB Serves All Your Postgres Needs
PostgreSQL
Security
© 2015 EnterpriseDB Corporation. All rights reserved. 7
EDB Customers EDB currently has over 3,000 total customers including 63 of the Fortune 500 and 137 of the Forbes Global 2000
© 2015 EnterpriseDB Corporation. All rights reserved. 8
Preparation, Planning, and Postgres
© 2015 EnterpriseDB Corporation. All rights reserved. 9
Recovery (Objectives and Continuity)
Upgrades
Availability Requirements
Application (Usage Pattern, Response Expectations)
Hardware (Acquire and Configure)
Deployment Planning ‘Timeline’
© 2015 EnterpriseDB Corporation. All rights reserved. 10
• Why ask these questions? − Keep everyone on the same page − Not everyone knows what you know − Your answers might mean something unforeseen by others
• When to ask these questions? − Early − Often
Why? When?
© 2011 EnterpriseDB . All rights reserved.
© 2015 EnterpriseDB Corporation. All rights reserved. 11
Quick Poll Postgres Database Maturity Stage
© 2015 EnterpriseDB Corporation. All rights reserved. 12
• Question: What stage are you currently in with your Postgres project?
• Answer choices: a. Evaluation or Proof of Concept b. Development – active development c. Deployment – planning for production or recently launched d. Maintenance
Audience Quick Poll #1
© 2015 EnterpriseDB Corporation. All rights reserved. 13
The Top 10 List
© 2015 EnterpriseDB Corporation. All rights reserved. 14
1. Where should I focus my money? 2. How should I partition the I/O? 3. What does my app want?
4. Can my app handle what the DB offers? 5. Highly available or highly recoverable? 6. Is ASYNC enough or do I need SYNC?
7. Cascaded replication or fanned? 8. How do I upgrade? 9. Why upgrade?
10. How much data can I afford to lose?
10 Questions to Ask Yourself and Others
© 2015 EnterpriseDB Corporation. All rights reserved. 15
Hardware
© 2015 EnterpriseDB Corporation. All rights reserved. 16
1. Acquiring Hardware – Where should I focus my money?
• Where should I focus my money? − RAM
− Max it out − ECC
− I/O − Low latency − Battery backed − Local, not network
− CPU − More cache == more better − Less CPUs, more cores − One 12-core CPU > Two 6-core CPU
© 2015 EnterpriseDB Corporation. All rights reserved. 17 © 2011 EnterpriseDB . All rights reserved.
2. How Should I Partition the I/O ?
http://www.enterprisedb.com/ahc
• $PGDATA/pg_xlog − SSDs − ext2
• $PGDATA/pg_log − syslog
• $PGDATA/pg_stat_tmp − tmpfs
• $PGDATA/base − zfs, ext4, xfs - YES − btrfs - not yet − ext3 - NO
© 2015 EnterpriseDB Corporation. All rights reserved. 18
I/O Partitioning (cont.)
• Tablespaces (you should be using them) − Group like objects into a tablespace of their own − Indexes − Reporting/historical tables − zfs, ext4, xfs − Indexes on SSD
• Things to avoid − Hardware deduplication (just don't) − Hybrid SSD/HDD (use tablespaces instead) − NFS, GlusterFS, FUSE, remote* storage
© 2015 EnterpriseDB Corporation. All rights reserved. 19
File System and Storage Subsystem
• Types of Storage Systems − Direct Attached Storage (DAS) − Storage Area Network (SAN) w. Fiber Channel − Storage Area Network (SAN) w. iSCSI − Network File System (NFS)
• DAS: Fast and low latency • SAN: Fast, expensive (w. Fiber Channel), scalable, can
include redundancy and smart file system operations • NFS: Not an optimal solution for Postgres
© 2015 EnterpriseDB Corporation. All rights reserved. 20
Your Application
© 2015 EnterpriseDB Corporation. All rights reserved. 21
• A few long-running connections crunching lots of data? − large work_mem − Emphasize effective_cache over shared_buffers
• Lots of 'quick hit' connections? − connection pooling − larger shared_buffers
• More read? Or more write? − autovacuum aggressiveness − Checkpoint spreading
3. What Does My App Want?
© 2011 EnterpriseDB . All rights reserved.
© 2015 EnterpriseDB Corporation. All rights reserved. 22
• Can the app recover from a failed transaction? − During failover, for example
• Can the app re-request if the answer isn't up-to-date? − Read-only slave, slightly behind the
master
• Can the app handle 'eventual consistency'? − Multi-master replication
4. Can My App Handle what the DB Offers?
© 2011 EnterpriseDB . All rights reserved.
© 2015 EnterpriseDB Corporation. All rights reserved. 23
Availability
© 2015 EnterpriseDB Corporation. All rights reserved. 24
• Highly available − Enterprise Failover Manager (EFM) − xDB
• Highly recoverable − Warm standby − WAL archiving − PITR
• Geographically durable − xDB − Cascaded Replication
5. High Availability or High Recoverability?
© 2011 EnterpriseDB . All rights reserved.
© 2015 EnterpriseDB Corporation. All rights reserved. 25
• ASYNC Replication − Default − Eventually consistent − May be 1+ transactions behind − Unlimited* number of ASYNC slaves
• SYNC Replication − Atypical − Immediately consistent − Performance implications on the master − Only ever one SYNC slave
6. Is ASYNC Enough or Do I Need SYNC?
© 2011 EnterpriseDB . All rights reserved.
© 2015 EnterpriseDB Corporation. All rights reserved. 26
• Cascaded − Can survive death of the master − May introduce replication lag
• Fanned − Does not survive death of the master − No additional replication lag
• Both − No reason you can't mix-n-match
7. Cascaded or Fanned Replication?
© 2011 EnterpriseDB . All rights reserved.
OR
© 2015 EnterpriseDB Corporation. All rights reserved. 27
Upgrades
© 2015 EnterpriseDB Corporation. All rights reserved. 28
Quick Poll Upgrade Policies
© 2015 EnterpriseDB Corporation. All rights reserved. 29
• Question: What is your current upgrade policy?
• Answer choices: a. Fully defined patch mgmt. policy w/ standing maintenance
windows b. We apply security fixes only as needed c. We apply patches/upgrades whenever the mood strikes us d. We apply patches/upgrades when the vendor forces it e. Upgrades?
Audience Quick Poll #2
© 2015 EnterpriseDB Corporation. All rights reserved. 30
8. How Do I Upgrade?
• Postgres version number: xx.yy.zz − xx.yy = major version (8.4, 9.2, 9.3) − zz = minor version (9.1.14, 9.3.5)
• Current version: − 9.3.x → 9.3.5 − 9.2.x → 9.2.9
• Major version changes − Not backwards compatible for stored data − Dump/load − pg_upgrade
• Minor version changes − Replace the binaries, restart
© 2015 EnterpriseDB Corporation. All rights reserved. 31
• Use replication ! • Upgrade the slave first
− If a minor release upgrade, streaming replication − If a major release upgrade, logical replication
• Failover to the upgraded slave • Upgrade the old master
• Slave the old master to the new master • Fail back (optional)
How Do I Upgrade (cont.)?
© 2015 EnterpriseDB Corporation. All rights reserved. 32
9. Why Upgrade?
• Postgres minor releases fix only: − Frequently-encountered bugs − Security bugs − Data corruption bugs
• As such, you are more at risk by not upgrading
• Major versions are supported for 5 years which lessens the upgrade burden significantly
© 2015 EnterpriseDB Corporation. All rights reserved. 33
Recovery
© 2015 EnterpriseDB Corporation. All rights reserved. 34
• What’s the loss to the business per minute of the app being down? − High cost == design for high availability − Think multi-master replication − Think offsite replication − Think automatic failover
• What’s the loss to the business for every transaction you can’t recover? − High cost == design for high recoverability − Think SYNC replication − Think continuous WAL archiving − Think outside the db (zfs snapshots, RHCS, Veritas clustering)
10. How Much Data Can I Afford to Lose?
© 2015 EnterpriseDB Corporation. All rights reserved. 35
• Backup and Recovery Strategies protect you in case of: − Catastrophic device failure − Site failure − Maintenance − Operator error − Compliance − Data corruption
• Consider − Allowable PITR timeframe − Data retention policy − Test, test, test − Periodic backup validation
Why Do I Need Backup and Recovery?
© 2015 EnterpriseDB Corporation. All rights reserved. 36
• Various recovery options are available: − PITR − BART / Barman − pg_basebackup − pg_dump
• How long does restoration take? • Do I know how to restore using my chosen method?
What Are My Recovery Options?
© 2015 EnterpriseDB Corporation. All rights reserved. 37
Dealing with Change – Ongoing Questions
© 2015 EnterpriseDB Corporation. All rights reserved. 38
• When things change − Database load − Hardware − Software versions − Business needs − User base grows
And • At minimum, annually
=> Reconsider all questions
Moving Forward
© 2015 EnterpriseDB Corporation. All rights reserved. 39
Summary and Useful Resources
© 2015 EnterpriseDB Corporation. All rights reserved. 40
• Preparation, Planning, and Postgres − Most problems occur after moving to production − Many times these problems were foreshadowed − Try to think of “what's next” as much as “what's now”
• Communicate − Nothing exists in a vacuum − Seek input from other teams − Proactively “push” info to others
• Evaluate, Adjust, and Repeat − Monitor all the things − Define your 'line in the sand'
Summary
© 2015 EnterpriseDB Corporation. All rights reserved. 41
• Get the team ready for Postgres - Developer, DBA, Infrastructure
• EDB Knowledge Base
• Get Certified - http://www.enterprisedb.com/training
• Follow the conversation: − #postgres on Twitter − @enterprisedb on Twitter − Planet PostgreSQL, @planetpostgres , http://planet.postgresql.org
• User groups and Meetups – almost all major cities − http://www.postgresql.org/community/user-groups/ − http://postgresql.meetup.com/
What You Can Do
© 2015 EnterpriseDB Corporation. All rights reserved. 42
• Architectural Health Check − Review of database, OS, env.
settings − Guidance on tuning, backup,
high availability strategies
• Kick Start − Training − Flex consulting − Developer subscription
How We Can Help
Whitepapers and Recorded Webinars http://www.enterprisedb.com/resources-community
• Training and Certification − Flexible formats:
on-demand, live virtual, on-site − Continuing education and
Postgres certification
• Remote DBA Services − 24 x 7 monitoring − Supplemental staff − Architectural health check − Access to Postgres expertise