Some Conference

OSCO

N 2011

PostgreSQL 9.1Selena Deckelmann

PostgreSQL Global Development Group

Stronger

Harder Better

Faster

OSCO

N 2011

Some Conference

OSCO

N 2011

Say “Postgres.”

Some Conference

OSCO

N 2011

Some Conference

OSCO

N 2011

“World’s most advanced open source database.”

Some Conference

OSCO

N 2011

Some Conference

OSCO

N 2011

Some Conference

OSCO

N 2011

Some Conference

OSCO

N 2011

Postgres disrupts the commercial relational

database industry.

Some Conference

OSCO

N 2011

Licensing

“Enterprise” Web development

Proprietary DBA Career Path

Disruptive forces

Some Conference

OSCO

N 2011

Licensing

Drop-in replacement for custom application development.

Cheap Data Warehousing

Ecosystem!

Some Conference

OSCO

N 2011

“Enterprise”Web development

Postgres is a “Developers database” - Extension is encouraged!

Database of choice for Django

Oracle wants PHP developers

Some Conference

OSCO

N 2011

“Visit the Ruby and Python Topicfor more information pertainingto Ruby/RoR, Python and other

non-PHP scripting languages.”

Some Conference

OSCO

N 2011

DBA Career path

Companies can’t hire Postgres DBAs fast enough.

Postgres is close enough to Oracle, that Oracle DBAs can jump in.

What we do better: Data loading, automation, TUI and GUI admin tools

Some Conference

OSCO

N 2011

And... Stability

Support major releases for at least 5 years.

Commits to HEAD are stable.

Bugs reported are fixed in <48 hrs.

Some Conference

OSCO

N 2011

Stronger

Harder Better

Faster

More secure & robust:SE PostgresReplication Tools

We made us better: Writeable CTEPer-column collationsExtensions

Performance:KNN IndexesUnlogged Tables

First in class:Serializable Snapshot IsolationForeign Data WrappersSynchronous Replication

10

features

Some Conference

OSCO

N 2011

Harder

Some Conference

OSCO

N 2011

Replication Tools

Harder

WAL-based replication:File-based (default 16-MB files)Streaming (per-WAL record) (9.0)AsynchronousWarm standbyHot standby (9.0)Synchronous (9.1)Cascading replication (just committed... 9.2)

http://www.pgcon.org/2010/schedule/attachments/149_PGCon2010-Built-in-Replication.pdf

new!

new!

Some Conference

OSCO

N 2011

Harder Replication Tools

pg_stat_replication viewshows all replicas and their statuses

pg_basebackupsingle-command cloning over port 5432 http://www.postgresql.org/docs/9.1/static/app-pgbasebackup.html

promote to masterpg_ctl promoteformerly only could do this by creating a trigger file

new functions pg_last_xact_replay_timestamp() pg_xlog_replay_pause()

Some Conference

OSCO

N 2011

HarderSE-Postgres

SE-Linux security label support!Originally a NSA-funded Linux kernel extension

For tablespace, schema, table, column, sequence, view and procedure object classes

Will have integrated support in pgAdmin!http://rhaas.blogspot.com/2010/09/se-linux-for-postgresql-part-1.html

http://rhaas.blogspot.com/2010/09/se-linux-for-postgresql-part-2.html

http://wiki.postgresql.org/wiki/SEPostgreSQL_Documentation

Some Conference

OSCO

N 2011

BetterWriteable CTE

But first - What’s a CTE? Common Table Expressions A temporary table or VIEW that exists for just a single query.

http://developer.postgresql.org/pgdocs/postgres/queries-with.html#QUERIES-WITH-MODIFYING

Some Conference

OSCO

N 2011

Better

Some Conference

OSCO

N 2011

Better Common TableExpression Example

WITH RECURSIVE t1(a, b) AS ( VALUES(0,0) UNION ALL SELECT CASE CAST(b AS BOOLEAN) WHEN b % 3 = 0 THEN b WHEN b % 5 = 0 THEN b END, b + 1 FROM t1 WHERE b < 1000)SELECT sum(a) FROM t1;

Euler Project, question 1: If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23. Find the sum of all the multiples of 3 or 5 below 1000.

http://wiki.postgresql.org/wiki/Euler_Project,_Question_1

Some Conference

OSCO

N 2011

BetterWriteable CTE

WITH deleted_posts AS (

DELETE FROM posts WHERE created < now()

- '6 months'::INTERVAL RETURNING * )SELECT user_id, count(*) FROM deleted_posts group BY 1;

Makes maintaining partitions easier:

http://www.depesz.com/index.php/2011/03/16/waiting-for-9-1-writable-cte/

http://xzilla.net/blog/2011/Mar/Upserting-via-Writeable-CTE.html

Some Conference

OSCO

N 2011

Better Per-columncollation

Supports multiple languages in the same table, and have ORDER BY sort by language order (assuming you’ve loaded the correct collation library)BEFORE:

$ select * from normal_polish order by some_text; some_text----------- alfa car ćma coś lama

Some Conference

OSCO

N 2011

Better Per-columncollation

AFTER:

$ select * from normal_polish order by some_text; some_text----------- alfa car coś ćma lama

Some Conference

OSCO

N 2011

BetterExtensions

Postgres-specific package management for contrib/, PgFoundry projects, tools.

Like Oracle "packages"

End user view: http://facility9.com/2011/03/postgresql-extensions/

Docs: http://developer.postgresql.org/pgdocs/postgres/sql-createextension.html

Some Conference

OSCO

N 2011

BetterExtensions

Some Conference

OSCO

N 2011

Faster

Some Conference

OSCO

N 2011

Faster K-Nearest Neighbor Indexes

Incredibly fast search for things that are "near" a particular object

Operator is: <->Datatypes supported: pg_trgm contrib and btree_gist datatypes (point)

Wiki: http://wiki.postgresql.org/wiki/What's_new_in_PostgreSQL_9.1#K-Nearest-Neighbor_IndexingDeveloper: http://www.sai.msu.su/~megera/postgres/talks/pgday-2010.pdf

Some Conference

OSCO

N 2011

Faster K-Nearest Neighbor Indexes

$ create table test ( position point );$ insert into test (position) select point( random() * 1000, random() * 1000) from generate_series(1,1000000);

$ create index q on test using gist ( position );$ select *, position <-> point(500,500) from test order by position <-> point(500,500) limit 10;

$ explain analyze select *, position <-> point(500,500) FROM test ORDER BY position <-> point(500,500) limit 10;

QUERY PLAN Limit (cost=0.00..0.77 rows=10 width=16) (actual time=0.164..0.475 rows=10 loops=1) -> Index Scan using q on test (cost=0.00..76512.60 rows=1000000 width=16) (actual time=0.163..0.473 rows=10 loops=1) Order By: ("position" <-> '(500,500)'::point)

Some Conference

OSCO

N 2011

FasterUnlogged Tables

Background: every write to a table in Postgres is essentially *two* writes because of the Write Ahead Log (basis for replication, and durability guarantees)

Unlogged tables are FREED of this durability constraint!

Could think of them as Global Temporary tables (see Haas' post)

http://www.depesz.com/index.php/2011/01/03/waiting-for-9-1-unlogged-tables/http://rhaas.blogspot.com/2010/05/global-temporary-and-unlogged-tables.html

Example:=$ perl -le 'my $x = "aaaaaaaa"; for (1..450000) {print $x++}' | \ perl -pe 's/.*/(\047$&\047)/;s/\s*\z/,/ if $.%3' | \ sed 's/.*/insert into test (some_text) values &;/'

Some Conference

OSCO

N 2011

Stronger Serializable Snapshot Isolation

PostgreSQL's is the first production implementationIncludes: per transaction support

Why use it?* No more blocking on table locks!* No more "select for update"* Simplifies programming

Wiki: http://wiki.postgresql.org/wiki/SSIDeveloper:My notes from Kevin's version of the talk: http://www.chesnok.com/daily/2011/03/24/raw-notes-from-kevin-grittners-talk-on-ssi/

Some Conference

OSCO

N 2011

Stronger

Some Conference

OSCO

N 2011

Stronger Foreign Data Wrappers

* Remote datasource access* Initially implemented TEXT, CSV data sources* Twitter: http://pgxn.org/dist/twitter_fdw/* Google Summer of Code student is working on other data sources* Underway currently: Oracle and MySQL sources

Awesome because:* 6-function API for implementing* Good for implementing imports, things which would otherwise fail if you just used COPY (imporoving COPY is a whole other rant...)

Stronger

Some Conference

OSCO

N 2011

Stronger Foreign Data Wrappers

Not awesome because:* Nothing other than sequential scans are possible for query plans* Not that many data sources implemented yet

Expect tons of FDWs to be implemented once we get 9.1 to production release

Stronger

Some Conference

OSCO

N 2011

Stronger Syncronous Replication

This is *the* most important use case for WAL-level replication.

User:http://developer.postgresql.org/pgdocs/postgres/warm-standby.html#SYNCHRONOUS-REPLICATION(kind of terrible for a person trying to understand how it works)

Somewhat better:http://wiki.postgresql.org/wiki/What%27s_new_in_PostgreSQL_9.1#Synchronous_replication_and_other_replication_features

Wiki: http://wiki.postgresql.org/wiki/Synchronous_replication

Stronger

Some Conference

OSCO

N 2011

Stronger Syncronous Replication

WAL stands for "write ahead log" and it is what ensures durability for all write operations

* WAL-shipping 16-MB chunks of these records was the pre-9.0 method for built-in replication* 9.0 and later have Streaming rep - per WAL record (per-commit) shipping of changes* Logical changes to DB, NOT statement-based* All-or-nothing wrt databases and tables

Some Conference

OSCO

N 2011

Stronger Syncronous Replication

* Create a replication role (no longer a forced role creation like in 9.0)* pg_hba.conf (host based authorization) update to allow replication user to access system* Five key configuration options: wal_level, max_wal_senders, archive_mode, archive_command, synchronous_standby_names* Create a base backup that will be the slave* recovery.conf config: restore_command, standby_mode, primary_conninfo, trigger_file

Some Conference

OSCO

N 2011

Terrifyingly live demo.

Some Conference

OSCO

N 2011

Thanks toJosh Berkushttp://www.pgexperts.com/document.html?id=50

Depeszhttp://www.depesz.com

Stronger

Harder Better

Faster

Some Conference

OSCO

N 2011

Other Postgres talks: Pro PostgreSQLRobert Treat, 1:40pm Wednesday

Unbreaking your Django ApplicationChristophe Pettus, 9am (tutorial happening NOW)

Stronger

Harder Better

Faster

Some Conference

OSCO

N 2011

Selena Deckelmann @selenamariehttp://chesnok.com

For hire! :)

Stronger

Harder Better

Faster