Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released...
Transcript of Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released...
![Page 1: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/1.jpg)
Performance Enhancements In PostgreSQL 8.4
PGDay.EU 2009Paris, France
Magnus HaganderRedpill Linpro AB
![Page 2: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/2.jpg)
PostgreSQL 8.4● Released July 2009
– 8.4.1 released September 2009
● Major upgrade from 8.3● New features and enhancements
of existing ones
![Page 3: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/3.jpg)
Using PostgreSQL performance● “ORM-like queries” only get you so
far● Application specific optimizations● Don't be afraid to let the database
work!
![Page 4: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/4.jpg)
Performance enhancements● Some are application transparent
– Possibly even DBA transparent
● Some require application changes
![Page 5: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/5.jpg)
Let's get started● Query execution optimizations
![Page 6: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/6.jpg)
Anti-joins and Semi-joins● Formalized JOIN methods for
inequality joins● Better performance for EXISTS /
NOT EXISTS
![Page 7: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/7.jpg)
Anti-joins and Semi-joins● 8.3pagila=# EXPLAIN SELECT * FROM actor a WHERE NOT EXISTS (SELECT * FROM film_actor fa WHERE fa.actor_id=a.actor_id);
Seq Scan on actor (cost=0.00..288.99 rows=100 width=25) Filter: (NOT (subplan)) SubPlan -> Index Scan using film_actor_pkey on film_actor (cost=0.00..38.47 rows=27 width=12) Index Cond: (actor_id = $0)
![Page 8: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/8.jpg)
Anti-joins and Semi-joins● 8.3pagila=# EXPLAIN SELECT * FROM actor a WHERE NOT EXISTS (SELECT * FROM film_actor fa WHERE fa.actor_id=a.actor_id);
Nested Loop Anti Join (cost=0.00..30.57 rows=1 width=25) -> Seq Scan on actor (cost=0.00..4.00 rows=200 width=25) -> Index Scan using film_actor_pkey on film_actor (cost=0.00..1.54 rows=27 width=2) Index Cond: (film_actor.actor_id = actor.actor_id)
![Page 9: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/9.jpg)
Anti-joins and Semi-joins● 8.3pagila=# EXPLAIN SELECT * FROM actor a WHERE EXISTS (SELECT * FROM film_actor fa WHERE fa.actor_id=a.actor_id);
Nested Loop Semi Join (cost=0.00..30.57 rows=200 width=25) -> Seq Scan on actor (cost=0.00..4.00 rows=200 width=25) -> Index Scan using film_actor_pkey on film_actor (cost=0.00..1.54 rows=27 width=2) Index Cond: (film_actor.actor_id = actor.actor_id)
![Page 10: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/10.jpg)
Hash for DISTINCT/UNION● Previously, always a sort+unique● No longer guaranteed sorted!
– Add ORDER BY– Both plans will be considered
● Also affects EXCEPT & INTERSECT
![Page 11: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/11.jpg)
Hash improvements● Faster algorithms
– WARNING! New hash values!
● Also faster hash indexes– Still not WAL-logged
● And optimizations of HASH joins– Particularly around large joins
![Page 12: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/12.jpg)
Moving on● DBA optimizations
![Page 13: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/13.jpg)
Function level statistics● pg_stat_user_functions● Controlled by “track_functions”
– none, pl or all
● Tracks calls, time, and internal time
![Page 14: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/14.jpg)
postgres=# select * from pg_stat_user_functions ;-[ RECORD 1 ]------funcid | 101414schemaname | publicfuncname | foocalls | 1003total_time | 6self_time | 6
![Page 15: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/15.jpg)
Free Space Map (FSM)● Stores list of free blocks in
relations– Caused by DELETE and UPDATE
● Used by INSERT & UPDATE
![Page 16: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/16.jpg)
New Free Space Map (FSM)● No more max_fsm_pages!● Dynamically tuned● Uses normal buffer cache
![Page 17: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/17.jpg)
New Free Space Map (FSM)● No global lock● Not lost on crash
![Page 18: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/18.jpg)
New Free Space Map (FSM)● No global lock● Not lost on crash
● VACUUM is still needed, of course...
![Page 19: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/19.jpg)
Visibility Map● Tracks pages that are “visible to all
transactions” in bitmap● Set by VACUUM● Cleared by
INSERT/UPDATE/DELETE
![Page 20: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/20.jpg)
Partial VACUUM● “Visible to all” pages skipped by
VACUUM● Only heap tables, not indexes● Still requires freezing
![Page 21: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/21.jpg)
VACUUM snapshot tracking● Snapshot tracking for idle sessions● Makes VACUUM clean up better
with long running transactions● <IDLE> In Transaction
![Page 22: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/22.jpg)
Stats temp file improvements● Previously, unconditionally
written twice/sec in data dir● Now, written only on demand● And in configurable location
(tmpfs!)
![Page 23: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/23.jpg)
Parallel pg_restore● Restore from dump was single
threaded● Can now load in <n> sessions● At least one table per session● No single-transaction!
![Page 24: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/24.jpg)
int8 pass by value● 64-bit integers finally take
advantage of 64-bit CPUs
![Page 25: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/25.jpg)
Moving on● Application features
![Page 26: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/26.jpg)
Subselects in LIMIT/OFFSET● Previously, only constants allowed● Required two queries / roundtrips
– Or cursor in function
● SELECT * FROM … LIMIT ( SELECT something FROM other)
![Page 27: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/27.jpg)
WINDOW aggregates● Perform aggregates over parts of
data● Avoid requiring multiple queries● Avoid multiple scans
![Page 28: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/28.jpg)
SELECT name, department, salary,rank() OVER (PARTITION BY departmentORDER BY salary DESC
)FROM employees
![Page 29: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/29.jpg)
name | department | salary | rank -------+------------+--------+------ Berra | Ekonomi | 29400 | 1 Åke | Ekonomi | 29400 | 1 Sune | Ekonomi | 24000 | 3 Arne | IT | 24000 | 1 Pelle | IT | 22000 | 2 Kalle | IT | 18000 | 3(6 rows)
![Page 30: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/30.jpg)
SELECT name, department, salary,rank() OVER (PARTITION BY departmentORDER BY salary DESC
),rank() OVER (ORDER BY salary DESC)
FROM employees
![Page 31: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/31.jpg)
name | department | salary | rank | rank -------+------------+--------+------+------ Åke | Ekonomi | 29400 | 1 | 1 Berra | Ekonomi | 29400 | 1 | 1 Sune | Ekonomi | 24000 | 3 | 3 Arne | IT | 24000 | 1 | 3 Pelle | IT | 22000 | 2 | 5 Kalle | IT | 18000 | 3 | 6(6 rows)
![Page 32: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/32.jpg)
Common Table Expressions● WITH RECURSIVE● Traverse trees and graphs in SQL● .. avoid multiple queries
– (also makes your life easier)
![Page 33: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/33.jpg)
WITH RECURSIVE t(id, department, name, manager) AS ( SELECT id, department, name, manager FROM emp WHERE name='Kalle' UNION ALL SELECT emp.id,emp.department,emp.name,emp.manager FROM emp JOIN t ON t.manager=emp.id)SELECT * FROM t;
![Page 34: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/34.jpg)
id | department | name | manager ----+------------+-------+--------- 1 | IT | Kalle | 3 3 | IT | Arne | 5 5 | Ekonomi | Berra | (3 rows)
![Page 35: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/35.jpg)
id | department | name | manager ----+------------+-------+--------- 1 | IT | Kalle | 3 3 | IT | Arne | 5 5 | Ekonomi | Berra | (3 rows)
Very important!
![Page 36: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/36.jpg)
Lots of more improvements!● But that's it for now..● Go download and test!
![Page 37: Performance Enhancements In PostgreSQL 8 · PostgreSQL 8.4 Released July 2009 – 8.4.1 released September 2009 Major upgrade from 8.3 New features and enhancements of existing ones](https://reader033.fdocuments.in/reader033/viewer/2022060802/60865bbb1ef0af10397c03e6/html5/thumbnails/37.jpg)
Performance Enhancements In PostgreSQL 8.4
http://2009.pgday.eu/feedbackQuestions?
[email protected]: @magnushagander
http://blog.hagander.net