PostgreSQL 9.0 & The Future

49
PostgreSQL and the future Aaron Thul

Transcript of PostgreSQL 9.0 & The Future

Page 1: PostgreSQL 9.0 & The Future

PostgreSQL and the future Aaron Thul

Page 2: PostgreSQL 9.0 & The Future

Who am I?

• Computer & Database Geek •  Formerly a Debian SysAdmin •  Presently Vice President of Technology at a

EMOL •  PostgreSQL Evangelist •  Penguicon Organizer

Page 3: PostgreSQL 9.0 & The Future
Page 4: PostgreSQL 9.0 & The Future
Page 5: PostgreSQL 9.0 & The Future

Ground rules

• Questions are good • Arguments are bad

Page 6: PostgreSQL 9.0 & The Future

Agenda

• Big News • New Features •  State of PostgreSQL Community • Evil •  Profit

Page 7: PostgreSQL 9.0 & The Future

First Things First

•  There is some big news

Page 8: PostgreSQL 9.0 & The Future

First Things First

•  There is some big news •  PostgreSQL 8.5 is…

Page 9: PostgreSQL 9.0 & The Future

First Things First

•  There is some big news •  PostgreSQL 8.5 is…

• PostgreSQL 9.0 !

Page 10: PostgreSQL 9.0 & The Future

Why 8.5 became 9.0

• Built-in log-streaming replication • Hot standby

Page 11: PostgreSQL 9.0 & The Future

New Features

Page 12: PostgreSQL 9.0 & The Future

Streaming replication

• Warm Standby 8.x to Hot Standby • New kinds of postmaster processes, walsenders

and walreceiver •  Thanks to Streaming Replication, lag should be

nearly zero

Page 13: PostgreSQL 9.0 & The Future

DROP IF EXISTS

• Columns • Constraints

Page 14: PostgreSQL 9.0 & The Future

Better messages for unique violation

•  Improve unique-constraint-violation error messages to include the exact values being complained of.

Page 15: PostgreSQL 9.0 & The Future

Deferrable Uniqueness

• Massive unique-key reassignments

Page 16: PostgreSQL 9.0 & The Future

MOVE FORWARD or BACKWARD

• MOVE {FORWARD,BACKWARD} X • More easily move around in a curser

Page 17: PostgreSQL 9.0 & The Future

’samehost’ and ’samenet’

• Does anyone know what HBA stands for? •  To lazy to list all the hosts on your network • CIDR-ADDRESS

Page 18: PostgreSQL 9.0 & The Future

GRANT ALL

• Grant privileges on all existing tables • Automatically grant privileges on all tables that

will be created in this database in the future

Page 19: PostgreSQL 9.0 & The Future

TRIGGERS on columns

•  SQL-compliant triggers on columns, ie fire only if certain columns are named in the UPDATE's SET list

• Help resolve circular TRIGGER issues

Page 20: PostgreSQL 9.0 & The Future

Conditional TRIGGERS

CREATE TRIGGER test_u AFTER UPDATE ON test FOR EACH ROW WHEN ( NEW.i <= 0 ) EXECUTE PROCEDURE test_u();

Page 21: PostgreSQL 9.0 & The Future

VACUUM FULL

• VACUUM FULL will start to behave like CLUSTER

• Will not actually be reordering rows • VACUUM FULL INPLACE

Page 22: PostgreSQL 9.0 & The Future

Buffers info for explain explain ( analyze on, buffers on ) select count(*) from pg_attribute ;

QUERY PLAN --------------------------------------------------------------------------

Aggregate (cost=64.70..64.71 rows=1 width=0) (actual time=0.466..0.466 rows=1 loops=1)

Buffers: shared hit=18 read=21 -> Seq Scan on pg_attribute (cost=0.00..59.56 rows=2056 width=0) (actual time=0.002..0.301 rows=2002 loops=1)

Buffers: shared hit=18 read=21 Total runtime: 0.492 ms

Page 23: PostgreSQL 9.0 & The Future

The list goes on

• Multi-threaded pgbench • Hinting for number of distinct values • Machine readable EXPLAIN • Checking password strength •  PL/pgSQL by default

Page 24: PostgreSQL 9.0 & The Future

PostgreSQL Community

Page 25: PostgreSQL 9.0 & The Future

Who makes database Software

• Oracle/SUN •  IBM • Microsoft

Page 26: PostgreSQL 9.0 & The Future

PostgreSQL Community

• Community not company • Growing •  International •  PostgreSQL Association • Nothing happening with SUN/MySQL/Oracle is

hurting the PostgreSQL community

Page 27: PostgreSQL 9.0 & The Future

www.postgresql.org

•  downloads •  documentation •  bug reports •  security alerts • wiki •  support companies

Page 28: PostgreSQL 9.0 & The Future

www.pgfoundry.org

• Modules •  Programs • Resources

Page 29: PostgreSQL 9.0 & The Future

www.planetpostgresql.org

•  Project News •  Community News •  Helpful Tips / Examples

Page 30: PostgreSQL 9.0 & The Future

archives.postgresql.org

•  mailing list archives back to 1997 •  full text search via postgtresql 8.3 •  keyword search suggestions

Page 31: PostgreSQL 9.0 & The Future

#postgresql

•  irc.freenode.net •  real time help •  rtfm_please - ??help

Page 32: PostgreSQL 9.0 & The Future

PostgreSQL Community Events

•  PGCon •  PostgreSQL Conference 2009 Japan • European PGDay •  PgWest •  PgEast •  PGCon Brazil •  Local PUGs •  http://wiki.postgresql.org/wiki/Events

Page 33: PostgreSQL 9.0 & The Future

Project Management

• Core team • Committers •  -hackers • Roadmap • Web team

Page 34: PostgreSQL 9.0 & The Future

Threats to PostgreSQL

•  Patent attacks • Hiring of volunteers to work on unrelated

projects

Page 35: PostgreSQL 9.0 & The Future

Evil

Page 36: PostgreSQL 9.0 & The Future

Data Types

•  Just use text ▫  char/varchar/text the same under the hood ▫  avoid artificial limits

•  Focus on functions ▫  Phone numbers often require string manipulation ▫  Unix timestamp vs. Date arithmetic

• Minimize typecasts

Page 37: PostgreSQL 9.0 & The Future

Take advantage of strong data typing

• CHECK limits input at column level • ENUM limits specific values at type level ▫  Allows you to define a custom order, provides

compact storage • DOMAIN defines a data type within constraint

boundaries • Often outperforms JOIN on lookup tables • Allows for simpler schema design

Page 38: PostgreSQL 9.0 & The Future

Normalization (3NF) •  All non-key columns must be directly dependent on PK ▫  Head is only dependent on the Name through the

Workgroup column

Page 39: PostgreSQL 9.0 & The Future

Surrogate Keys

• Natural Key (NK) is a CK with a natural relationship to that row

•  Surrogate Key (SK) is an artificially added unique identifier ▫  Since they are artificial they make queries harder

to readand can lead to more joins •  Integers do not significantly improve JOIN

performance or reduce file I/O for many data sets

Page 40: PostgreSQL 9.0 & The Future

Bareword ids • Makes SQL less obvious to read ▫  SELECT id, id, id FROM foo, bar, baz WHERE … ▫  Makes ANSI JOIN syntax more cumbersome

JOIN foo USING (bar_id) ▫  JOIN foo ON (foo.bar_id = bar.id)

• Often resort to alias columns to add clarity, scoping

•  Some ORMs really like this (can be overridden • Use verbose id names instead ▫  Create table actor (actor_id, full_name text);

Page 41: PostgreSQL 9.0 & The Future

Use standard definitions

• Often data has been designed in a standard way ▫  Country Code ▫  Email address ▫  Zip Code ▫  VIN ▫  SEX (ISO 5218)

• Helps eliminate short-sightedness •  Increases commonality across projects

Page 42: PostgreSQL 9.0 & The Future

Images in the database

• Replication • Backups • Access control •  Transactions • OS Portability

Page 43: PostgreSQL 9.0 & The Future

Over indexing

•  Indexes must be updated when data changes occur ▫  INSERT/UPDATE/DELETE all touch indexes ▫  Some like it HOT, pg_stat_all_tables

• BitMap vs. Multi-Column Indexes ▫  Combine index on (a) and (b) in memory ▫  Index on (x,y,z) implies index on (x) and (x,y)

• Make sure indexes are used ▫  pg_stat_all_indexes

Page 44: PostgreSQL 9.0 & The Future

Covering indexes

• Creating indexes to avoid accessing data in the table ▫  TOAST makes this less necessary ▫  Visibility information stored in the table

Page 45: PostgreSQL 9.0 & The Future

Full text indexing

•  IT ROCKS! • Add search engine style functionality to DBMS ▫  LIKE '%foo%' and LIKE '%foo' cannot use index ▫  Regex searching has similar issues ▫  Built-in tsearch functionality in 8.3+   GIN, expensive to update, very fast for searching   GIST, cheaper to update, not as fast for searching

• Database Specific Syntax

Page 46: PostgreSQL 9.0 & The Future

Profit

Page 47: PostgreSQL 9.0 & The Future

Lessons To Take Away

•  PostgreSQL 9.0 is going to rock • Never confuse a company with community •  Some mistakes are just to much fun to make

only once • Never ask for directions from a two-headed

tourist! -Big Bird

Page 48: PostgreSQL 9.0 & The Future

Thanks to

•  The PostgreSQL Community! • Robert Treat • Hubert Lubaczewski • More info: ▫  http://www.depesz.com/ ▫  http://www.xzilla.net/

Page 49: PostgreSQL 9.0 & The Future

Questions

• Web: http://www.chasingnuts.com • Email: [email protected] •  IRC: AaronThul on irc.freenode.org •  Jabber: [email protected] •  Twitter: @AaronThul • AIM: AaronThul