PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host...
Transcript of PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host...
![Page 2: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/2.jpg)
Who is Steve
PostgreSQL User since 2000
Author of contrib/dbmirror replication engine (7.x)
A Maintainer of Slony-I replication system
Blog http://scanningpages.wordpress.com
![Page 3: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/3.jpg)
PostgreSQL
BSD Licensed SQL Database
Standards Compliant, Transactions
Large Open Community
![Page 4: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/4.jpg)
Why Replicate
?
![Page 5: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/5.jpg)
Failover
![Page 6: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/6.jpg)
Load Balancing
![Page 7: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/7.jpg)
Thinkarchitecture
![Page 8: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/8.jpg)
database Application
![Page 9: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/9.jpg)
Db1(master) Application
Db3(slave)
Db2(slave)
Read only
![Page 10: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/10.jpg)
Db1
Application
Db2
sales
Application
customers
sales
![Page 11: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/11.jpg)
Db1 Application
Db2Reports
![Page 12: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/12.jpg)
What is important to you
![Page 13: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/13.jpg)
Slony-I
http://www.slony.info
![Page 14: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/14.jpg)
Slony-I
Key Features
Multiple Origin nodes (for different tables)
customers
sales
customers
sales
![Page 15: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/15.jpg)
Slony-I
Key Features
Cascaded Replication
![Page 16: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/16.jpg)
Slony-I
Key Features
sales sales
history
trigger
![Page 17: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/17.jpg)
6 Steps to replicate with Slony
h1 h2
Node 1 Node 2
slon slon
![Page 18: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/18.jpg)
Step 1
Write a preamble file
/tmp/preamble
cluster name=mycluster;node 1 admin conninfo='host=h1 dbname=mydb user=postgres';node 2 admin conninfo='host=h2 dbname=mydb user=postgres';
![Page 19: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/19.jpg)
Step 2
Create The Slave
pg_dump -h h1 -s mydb | psql -h h2 mydb
![Page 20: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/20.jpg)
Step 3
Add Slony to the Database
slonik <<_EOF_include</tmp/preamble>init cluster(id=1);store node(id=2,event node=1);store path(server=1,client=2,
conninfo='host=h1 dbname=mydb user=postgres');store path(server=2,client=1,
conninfo='host=h2 dbname=mydb user=postgres');_EOF_
![Page 21: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/21.jpg)
Step 4
Start the slon daemons
$ slon mycluster 'host=h1 dbname=mydb'
$ slon mycluster 'host=h2 dbname=mydb'
![Page 22: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/22.jpg)
Step 5
Create a replication set
slonik<<_EOF_include</tmp/preamble>create set(id=1,origin=1);set add table(set id=1, origin=1, id=100,
fully qualified name='public.sales');set add table(set id=1, origin=1, id=101,
fully qualified name='public.orders');
![Page 23: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/23.jpg)
Step 6
Subscribe the set
slonik << _EOF_include<'/tmp/preamble'>subscribe set(id=1,provider=1,receiver=2);sync(id=1);wait for event(id=1,confirmed=all,wait on=1);_EOF_
![Page 24: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/24.jpg)
Limitations
DDL Changes are not automatically replicated
10-30% performance impact
Can be complicated
Easy to shoot yourself in the foot
![Page 25: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/25.jpg)
Streaming Replication
![Page 26: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/26.jpg)
PostgreSQL Write Ahead Log (WAL)
base pg_clog global pg_xlog
WAL Log
data directory
![Page 27: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/27.jpg)
data
WAL
Archive
Master
WAL
WAL
Postgres
WAL WAL WAL WAL
data
WAL
Slave
WAL
WAL
Postgres
![Page 28: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/28.jpg)
Streaming ReplicationKey Features
Easy To Setup
![Page 29: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/29.jpg)
Streaming ReplicationKey Features
All SQL is replicated (DDL,BLOGS,...)
![Page 30: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/30.jpg)
Streaming ReplicationKey Features
Limited Performance Impact
![Page 31: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/31.jpg)
5 Steps to Setting up Streaming Replication
1. Edit postgresql.conf (on master)
wal_level=hot_standbyarchive_mode=onarchive_command='rsync %p h2:/home/arch_dir/%f</dev/null'
![Page 32: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/32.jpg)
5 Steps to Setting up Streaming Replication
2. Edit pg_hba.conf on master
host replication myuser 192.168.1.100/32 md5
![Page 33: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/33.jpg)
5 Steps to Setting up Streaming Replication
3. Create a base pg backup
select pg_start_backup('myb');
rsync -r pgsql_data h2:/home/mydata
select pg_stop_backup();
![Page 34: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/34.jpg)
5 Steps to Setting up Streaming Replication
4. Configure slaverecovery.conf
standby_mode='on'primary_conninfo='host=h1 port=5432 user=myuser password=foo'restore_command='cp /home/arch_dir/%f %p'trigger_file=/home/promote_standbyarchive_cleanup_command='pg_archivecleanup /home/arch_dir %r'
postgresql.conf
hot_standby=on
![Page 35: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/35.jpg)
5 Steps to Setting up Streaming Replication
rm /home/mydata/postmaster.pid
5 Start slave
pg_ctl -D /home/mydata start
![Page 36: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/36.jpg)
Avoid Long Running Transactions
![Page 37: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/37.jpg)
Decide what you need to do
![Page 38: PostgreSQL Replication SC2011 Aug 2011 · Streaming Replication 2. Edit pg_hba.conf on master host replication myuser 192.168.1.100/32 md5 ...](https://reader033.fdocuments.in/reader033/viewer/2022060603/6057d6371784f66bec177899/html5/thumbnails/38.jpg)
Questions ?
Steve Singerhttp://[email protected]: stevenSn
Photo Credits
http://www.flickr.com/photos/toolstop/4325151224
http://www.flickr.com/photos/danielaineurope/990977254
http://www.flickr.com/photos/frazzledjen/148709599
CC-BY-SA