The PostgreSQL Global Development Group · 8.5. Date/Time Types.....126 8.5.1. Date/Time...

3301
PostgreSQL 9.6.0 Documentation The PostgreSQL Global Development Group

Transcript of The PostgreSQL Global Development Group · 8.5. Date/Time Types.....126 8.5.1. Date/Time...

  • PostgreSQL 9.6.0 Documentation

    The PostgreSQL Global Development Group

  • PostgreSQL 9.6.0 Documentationby The PostgreSQL Global Development GroupCopyright © 1996-2016 The PostgreSQL Global Development Group

    Legal Notice

    PostgreSQL is Copyright © 1996-2016 by the PostgreSQL Global Development Group.

    Postgres95 is Copyright © 1994-5 by the Regents of the University of California.

    Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a writtenagreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in allcopies.

    IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL,INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWAREAND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OFSUCH DAMAGE.

    THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO,THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PRO-VIDED HEREUNDER IS ON AN “AS-IS” BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDEMAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

  • Table of ContentsPreface ...........................................................................................................................................lxviii

    1. What is PostgreSQL? ........................................................................................................lxviii2. A Brief History of PostgreSQL...........................................................................................lxix

    2.1. The Berkeley POSTGRES Project .........................................................................lxix2.2. Postgres95...............................................................................................................lxix2.3. PostgreSQL.............................................................................................................. lxx

    3. Conventions.......................................................................................................................... lxx4. Further Information.............................................................................................................lxxi5. Bug Reporting Guidelines...................................................................................................lxxi

    5.1. Identifying Bugs ....................................................................................................lxxii5.2. What to Report ......................................................................................................lxxii5.3. Where to Report Bugs .......................................................................................... lxxiv

    I. Tutorial ............................................................................................................................................. 11. Getting Started ........................................................................................................................ 1

    1.1. Installation .................................................................................................................. 11.2. Architectural Fundamentals........................................................................................ 11.3. Creating a Database.................................................................................................... 21.4. Accessing a Database ................................................................................................. 3

    2. The SQL Language ................................................................................................................. 52.1. Introduction ................................................................................................................ 52.2. Concepts ..................................................................................................................... 52.3. Creating a New Table ................................................................................................. 52.4. Populating a Table With Rows ................................................................................... 62.5. Querying a Table ........................................................................................................ 72.6. Joins Between Tables.................................................................................................. 92.7. Aggregate Functions................................................................................................. 112.8. Updates ..................................................................................................................... 122.9. Deletions................................................................................................................... 13

    3. Advanced Features ................................................................................................................ 143.1. Introduction .............................................................................................................. 143.2. Views ........................................................................................................................ 143.3. Foreign Keys............................................................................................................. 143.4. Transactions.............................................................................................................. 153.5. Window Functions.................................................................................................... 173.6. Inheritance ................................................................................................................ 203.7. Conclusion................................................................................................................ 21

    II. The SQL Language...................................................................................................................... 224. SQL Syntax ........................................................................................................................... 24

    4.1. Lexical Structure....................................................................................................... 244.1.1. Identifiers and Key Words............................................................................ 244.1.2. Constants...................................................................................................... 26

    4.1.2.1. String Constants .............................................................................. 264.1.2.2. String Constants with C-style Escapes............................................ 264.1.2.3. String Constants with Unicode Escapes.......................................... 284.1.2.4. Dollar-quoted String Constants ....................................................... 284.1.2.5. Bit-string Constants......................................................................... 294.1.2.6. Numeric Constants .......................................................................... 294.1.2.7. Constants of Other Types ................................................................ 30

    iii

  • 4.1.3. Operators...................................................................................................... 314.1.4. Special Characters........................................................................................ 314.1.5. Comments .................................................................................................... 324.1.6. Operator Precedence .................................................................................... 32

    4.2. Value Expressions..................................................................................................... 334.2.1. Column References...................................................................................... 344.2.2. Positional Parameters................................................................................... 344.2.3. Subscripts..................................................................................................... 354.2.4. Field Selection ............................................................................................. 354.2.5. Operator Invocations.................................................................................... 364.2.6. Function Calls .............................................................................................. 364.2.7. Aggregate Expressions................................................................................. 374.2.8. Window Function Calls................................................................................ 394.2.9. Type Casts .................................................................................................... 414.2.10. Collation Expressions ................................................................................ 414.2.11. Scalar Subqueries....................................................................................... 424.2.12. Array Constructors..................................................................................... 424.2.13. Row Constructors....................................................................................... 444.2.14. Expression Evaluation Rules ..................................................................... 45

    4.3. Calling Functions...................................................................................................... 474.3.1. Using Positional Notation ............................................................................ 474.3.2. Using Named Notation ................................................................................ 484.3.3. Using Mixed Notation.................................................................................. 48

    5. Data Definition ...................................................................................................................... 505.1. Table Basics.............................................................................................................. 505.2. Default Values .......................................................................................................... 515.3. Constraints ................................................................................................................ 52

    5.3.1. Check Constraints ........................................................................................ 525.3.2. Not-Null Constraints.................................................................................... 545.3.3. Unique Constraints....................................................................................... 555.3.4. Primary Keys................................................................................................ 565.3.5. Foreign Keys ................................................................................................ 565.3.6. Exclusion Constraints .................................................................................. 59

    5.4. System Columns....................................................................................................... 595.5. Modifying Tables...................................................................................................... 61

    5.5.1. Adding a Column......................................................................................... 615.5.2. Removing a Column .................................................................................... 625.5.3. Adding a Constraint ..................................................................................... 625.5.4. Removing a Constraint ................................................................................ 625.5.5. Changing a Column’s Default Value............................................................ 635.5.6. Changing a Column’s Data Type ................................................................. 635.5.7. Renaming a Column .................................................................................... 635.5.8. Renaming a Table ........................................................................................ 63

    5.6. Privileges .................................................................................................................. 645.7. Row Security Policies............................................................................................... 645.8. Schemas.................................................................................................................... 69

    5.8.1. Creating a Schema ....................................................................................... 705.8.2. The Public Schema ...................................................................................... 715.8.3. The Schema Search Path.............................................................................. 715.8.4. Schemas and Privileges................................................................................ 725.8.5. The System Catalog Schema ....................................................................... 725.8.6. Usage Patterns.............................................................................................. 73

    iv

  • 5.8.7. Portability..................................................................................................... 735.9. Inheritance ................................................................................................................ 74

    5.9.1. Caveats ......................................................................................................... 775.10. Partitioning ............................................................................................................. 77

    5.10.1. Overview.................................................................................................... 775.10.2. Implementing Partitioning ......................................................................... 785.10.3. Managing Partitions ................................................................................... 815.10.4. Partitioning and Constraint Exclusion ....................................................... 825.10.5. Alternative Partitioning Methods............................................................... 835.10.6. Caveats ....................................................................................................... 84

    5.11. Foreign Data ........................................................................................................... 845.12. Other Database Objects .......................................................................................... 855.13. Dependency Tracking............................................................................................. 85

    6. Data Manipulation................................................................................................................. 876.1. Inserting Data ........................................................................................................... 876.2. Updating Data........................................................................................................... 886.3. Deleting Data............................................................................................................ 89

    7. Queries .................................................................................................................................. 907.1. Overview .................................................................................................................. 907.2. Table Expressions ..................................................................................................... 90

    7.2.1. The FROM Clause.......................................................................................... 917.2.1.1. Joined Tables ................................................................................... 917.2.1.2. Table and Column Aliases............................................................... 957.2.1.3. Subqueries ....................................................................................... 967.2.1.4. Table Functions ............................................................................... 967.2.1.5. LATERAL Subqueries ....................................................................... 98

    7.2.2. The WHERE Clause........................................................................................ 997.2.3. The GROUP BY and HAVING Clauses......................................................... 1007.2.4. GROUPING SETS, CUBE, and ROLLUP ....................................................... 1027.2.5. Window Function Processing .................................................................... 104

    7.3. Select Lists.............................................................................................................. 1047.3.1. Select-List Items ........................................................................................ 1057.3.2. Column Labels ........................................................................................... 1057.3.3. DISTINCT .................................................................................................. 106

    7.4. Combining Queries................................................................................................. 1067.5. Sorting Rows .......................................................................................................... 1077.6. LIMIT and OFFSET................................................................................................. 1087.7. VALUES Lists .......................................................................................................... 1087.8. WITH Queries (Common Table Expressions) ......................................................... 109

    7.8.1. SELECT in WITH......................................................................................... 1097.8.2. Data-Modifying Statements in WITH ......................................................... 113

    8. Data Types........................................................................................................................... 1158.1. Numeric Types........................................................................................................ 116

    8.1.1. Integer Types.............................................................................................. 1178.1.2. Arbitrary Precision Numbers ..................................................................... 1178.1.3. Floating-Point Types .................................................................................. 1198.1.4. Serial Types................................................................................................ 120

    8.2. Monetary Types ...................................................................................................... 1218.3. Character Types ...................................................................................................... 1228.4. Binary Data Types .................................................................................................. 124

    8.4.1. bytea Hex Format..................................................................................... 1248.4.2. bytea Escape Format................................................................................ 124

    v

  • 8.5. Date/Time Types..................................................................................................... 1268.5.1. Date/Time Input ......................................................................................... 128

    8.5.1.1. Dates.............................................................................................. 1288.5.1.2. Times ............................................................................................. 1298.5.1.3. Time Stamps.................................................................................. 1308.5.1.4. Special Values ............................................................................... 131

    8.5.2. Date/Time Output ...................................................................................... 1318.5.3. Time Zones ................................................................................................ 1328.5.4. Interval Input.............................................................................................. 1348.5.5. Interval Output ........................................................................................... 136

    8.6. Boolean Type.......................................................................................................... 1368.7. Enumerated Types .................................................................................................. 137

    8.7.1. Declaration of Enumerated Types.............................................................. 1378.7.2. Ordering ..................................................................................................... 1388.7.3. Type Safety ................................................................................................ 1388.7.4. Implementation Details.............................................................................. 139

    8.8. Geometric Types..................................................................................................... 1398.8.1. Points ......................................................................................................... 1408.8.2. Lines........................................................................................................... 1408.8.3. Line Segments............................................................................................ 1408.8.4. Boxes.......................................................................................................... 1418.8.5. Paths........................................................................................................... 1418.8.6. Polygons..................................................................................................... 1418.8.7. Circles ........................................................................................................ 142

    8.9. Network Address Types.......................................................................................... 1428.9.1. inet ........................................................................................................... 1428.9.2. cidr ........................................................................................................... 1438.9.3. inet vs. cidr............................................................................................ 1438.9.4. macaddr .................................................................................................... 143

    8.10. Bit String Types .................................................................................................... 1448.11. Text Search Types................................................................................................. 145

    8.11.1. tsvector ................................................................................................ 1458.11.2. tsquery .................................................................................................. 146

    8.12. UUID Type ........................................................................................................... 1478.13. XML Type ............................................................................................................ 148

    8.13.1. Creating XML Values .............................................................................. 1488.13.2. Encoding Handling .................................................................................. 1498.13.3. Accessing XML Values............................................................................ 150

    8.14. JSON Types .......................................................................................................... 1508.14.1. JSON Input and Output Syntax................................................................ 1528.14.2. Designing JSON documents effectively .................................................. 1538.14.3. jsonb Containment and Existence.......................................................... 1538.14.4. jsonb Indexing........................................................................................ 155

    8.15. Arrays ................................................................................................................... 1578.15.1. Declaration of Array Types...................................................................... 1578.15.2. Array Value Input..................................................................................... 1588.15.3. Accessing Arrays ..................................................................................... 1598.15.4. Modifying Arrays..................................................................................... 1618.15.5. Searching in Arrays.................................................................................. 1648.15.6. Array Input and Output Syntax................................................................ 165

    8.16. Composite Types .................................................................................................. 1668.16.1. Declaration of Composite Types.............................................................. 167

    vi

  • 8.16.2. Composite Value Input............................................................................. 1688.16.3. Accessing Composite Types .................................................................... 1688.16.4. Modifying Composite Types.................................................................... 1698.16.5. Composite Type Input and Output Syntax............................................... 169

    8.17. Range Types ......................................................................................................... 1708.17.1. Built-in Range Types ............................................................................... 1718.17.2. Examples.................................................................................................. 1718.17.3. Inclusive and Exclusive Bounds .............................................................. 1718.17.4. Infinite (Unbounded) Ranges................................................................... 1728.17.5. Range Input/Output.................................................................................. 1728.17.6. Constructing Ranges ................................................................................ 1738.17.7. Discrete Range Types .............................................................................. 1738.17.8. Defining New Range Types ..................................................................... 1748.17.9. Indexing ................................................................................................... 1758.17.10. Constraints on Ranges............................................................................ 175

    8.18. Object Identifier Types ......................................................................................... 1768.19. pg_lsn Type........................................................................................................... 1788.20. Pseudo-Types........................................................................................................ 178

    9. Functions and Operators ..................................................................................................... 1809.1. Logical Operators ................................................................................................... 1809.2. Comparison Functions and Operators .................................................................... 1809.3. Mathematical Functions and Operators.................................................................. 1839.4. String Functions and Operators .............................................................................. 187

    9.4.1. format....................................................................................................... 2029.5. Binary String Functions and Operators .................................................................. 2049.6. Bit String Functions and Operators ........................................................................ 2069.7. Pattern Matching .................................................................................................... 207

    9.7.1. LIKE ........................................................................................................... 2079.7.2. SIMILAR TO Regular Expressions ............................................................ 2089.7.3. POSIX Regular Expressions ...................................................................... 209

    9.7.3.1. Regular Expression Details ........................................................... 2139.7.3.2. Bracket Expressions ...................................................................... 2159.7.3.3. Regular Expression Escapes.......................................................... 2169.7.3.4. Regular Expression Metasyntax.................................................... 2189.7.3.5. Regular Expression Matching Rules ............................................. 2209.7.3.6. Limits and Compatibility .............................................................. 2219.7.3.7. Basic Regular Expressions ............................................................ 222

    9.8. Data Type Formatting Functions ............................................................................ 2229.9. Date/Time Functions and Operators....................................................................... 229

    9.9.1. EXTRACT, date_part ............................................................................... 2349.9.2. date_trunc .............................................................................................. 2389.9.3. AT TIME ZONE.......................................................................................... 2399.9.4. Current Date/Time ..................................................................................... 2399.9.5. Delaying Execution.................................................................................... 241

    9.10. Enum Support Functions ...................................................................................... 2429.11. Geometric Functions and Operators ..................................................................... 2439.12. Network Address Functions and Operators.......................................................... 2479.13. Text Search Functions and Operators ................................................................... 2499.14. XML Functions .................................................................................................... 254

    9.14.1. Producing XML Content.......................................................................... 2559.14.1.1. xmlcomment ............................................................................... 2559.14.1.2. xmlconcat ................................................................................. 255

    vii

  • 9.14.1.3. xmlelement ............................................................................... 2569.14.1.4. xmlforest ................................................................................. 2579.14.1.5. xmlpi .......................................................................................... 2589.14.1.6. xmlroot...................................................................................... 2589.14.1.7. xmlagg ........................................................................................ 258

    9.14.2. XML Predicates ....................................................................................... 2599.14.2.1. IS DOCUMENT ............................................................................. 2599.14.2.2. XMLEXISTS ................................................................................. 2599.14.2.3. xml_is_well_formed .............................................................. 260

    9.14.3. Processing XML ...................................................................................... 2619.14.4. Mapping Tables to XML.......................................................................... 262

    9.15. JSON Functions and Operators ............................................................................ 2659.16. Sequence Manipulation Functions ....................................................................... 2749.17. Conditional Expressions....................................................................................... 277

    9.17.1. CASE ......................................................................................................... 2779.17.2. COALESCE ................................................................................................ 2789.17.3. NULLIF..................................................................................................... 2799.17.4. GREATEST and LEAST.............................................................................. 279

    9.18. Array Functions and Operators ............................................................................ 2799.19. Range Functions and Operators............................................................................ 2839.20. Aggregate Functions............................................................................................. 2859.21. Window Functions................................................................................................ 2949.22. Subquery Expressions .......................................................................................... 296

    9.22.1. EXISTS..................................................................................................... 2969.22.2. IN ............................................................................................................. 2969.22.3. NOT IN..................................................................................................... 2979.22.4. ANY/SOME ................................................................................................. 2979.22.5. ALL ........................................................................................................... 2989.22.6. Single-row Comparison ........................................................................... 298

    9.23. Row and Array Comparisons ............................................................................... 2999.23.1. IN ............................................................................................................. 2999.23.2. NOT IN..................................................................................................... 2999.23.3. ANY/SOME (array) ..................................................................................... 3009.23.4. ALL (array) ............................................................................................... 3009.23.5. Row Constructor Comparison.................................................................. 3009.23.6. Composite Type Comparison................................................................... 301

    9.24. Set Returning Functions ....................................................................................... 3029.25. System Information Functions ............................................................................. 3059.26. System Administration Functions ........................................................................ 320

    9.26.1. Configuration Settings Functions............................................................. 3219.26.2. Server Signaling Functions ...................................................................... 3219.26.3. Backup Control Functions ....................................................................... 3229.26.4. Recovery Control Functions .................................................................... 3259.26.5. Snapshot Synchronization Functions....................................................... 3269.26.6. Replication Functions .............................................................................. 3279.26.7. Database Object Management Functions................................................. 3319.26.8. Index Maintenance Functions .................................................................. 3349.26.9. Generic File Access Functions................................................................. 3349.26.10. Advisory Lock Functions....................................................................... 335

    9.27. Trigger Functions ................................................................................................. 3389.28. Event Trigger Functions ....................................................................................... 338

    9.28.1. Capturing Changes at Command End...................................................... 338

    viii

  • 9.28.2. Processing Objects Dropped by a DDL Command ................................. 3399.28.3. Handling a Table Rewrite Event .............................................................. 340

    10. Type Conversion................................................................................................................ 34210.1. Overview .............................................................................................................. 34210.2. Operators .............................................................................................................. 34310.3. Functions .............................................................................................................. 34710.4. Value Storage........................................................................................................ 35010.5. UNION, CASE, and Related Constructs.................................................................. 351

    11. Indexes .............................................................................................................................. 35311.1. Introduction .......................................................................................................... 35311.2. Index Types........................................................................................................... 35411.3. Multicolumn Indexes............................................................................................ 35611.4. Indexes and ORDER BY......................................................................................... 35711.5. Combining Multiple Indexes ................................................................................ 35811.6. Unique Indexes ..................................................................................................... 35911.7. Indexes on Expressions ........................................................................................ 35911.8. Partial Indexes ...................................................................................................... 36011.9. Operator Classes and Operator Families .............................................................. 36211.10. Indexes and Collations........................................................................................ 36411.11. Index-Only Scans ............................................................................................... 36411.12. Examining Index Usage...................................................................................... 366

    12. Full Text Search ................................................................................................................ 36812.1. Introduction .......................................................................................................... 368

    12.1.1. What Is a Document?............................................................................... 36912.1.2. Basic Text Matching ................................................................................ 36912.1.3. Configurations.......................................................................................... 371

    12.2. Tables and Indexes................................................................................................ 37212.2.1. Searching a Table ..................................................................................... 37212.2.2. Creating Indexes ...................................................................................... 373

    12.3. Controlling Text Search........................................................................................ 37412.3.1. Parsing Documents .................................................................................. 37412.3.2. Parsing Queries ........................................................................................ 37512.3.3. Ranking Search Results ........................................................................... 37712.3.4. Highlighting Results ................................................................................ 379

    12.4. Additional Features .............................................................................................. 38012.4.1. Manipulating Documents......................................................................... 38012.4.2. Manipulating Queries............................................................................... 381

    12.4.2.1. Query Rewriting .......................................................................... 38212.4.3. Triggers for Automatic Updates .............................................................. 38412.4.4. Gathering Document Statistics ................................................................ 385

    12.5. Parsers................................................................................................................... 38612.6. Dictionaries........................................................................................................... 387

    12.6.1. Stop Words............................................................................................... 38812.6.2. Simple Dictionary .................................................................................... 38912.6.3. Synonym Dictionary ................................................................................ 39012.6.4. Thesaurus Dictionary ............................................................................... 392

    12.6.4.1. Thesaurus Configuration ............................................................. 39312.6.4.2. Thesaurus Example ..................................................................... 393

    12.6.5. Ispell Dictionary....................................................................................... 39412.6.6. Snowball Dictionary ................................................................................ 397

    12.7. Configuration Example......................................................................................... 39712.8. Testing and Debugging Text Search ..................................................................... 399

    ix

  • 12.8.1. Configuration Testing............................................................................... 39912.8.2. Parser Testing........................................................................................... 40112.8.3. Dictionary Testing.................................................................................... 402

    12.9. GIN and GiST Index Types .................................................................................. 40312.10. psql Support........................................................................................................ 40312.11. Limitations.......................................................................................................... 40612.12. Migration from Pre-8.3 Text Search................................................................... 406

    13. Concurrency Control ......................................................................................................... 40813.1. Introduction .......................................................................................................... 40813.2. Transaction Isolation ............................................................................................ 408

    13.2.1. Read Committed Isolation Level ............................................................. 40913.2.2. Repeatable Read Isolation Level.............................................................. 41113.2.3. Serializable Isolation Level...................................................................... 412

    13.3. Explicit Locking ................................................................................................... 41413.3.1. Table-level Locks ..................................................................................... 41413.3.2. Row-level Locks ...................................................................................... 41713.3.3. Page-level Locks ...................................................................................... 41813.3.4. Deadlocks................................................................................................. 41813.3.5. Advisory Locks........................................................................................ 419

    13.4. Data Consistency Checks at the Application Level.............................................. 42013.4.1. Enforcing Consistency With Serializable Transactions ........................... 42013.4.2. Enforcing Consistency With Explicit Blocking Locks ............................ 421

    13.5. Caveats.................................................................................................................. 42213.6. Locking and Indexes............................................................................................. 422

    14. Performance Tips .............................................................................................................. 42414.1. Using EXPLAIN .................................................................................................... 424

    14.1.1. EXPLAIN Basics ....................................................................................... 42414.1.2. EXPLAIN ANALYZE ................................................................................. 43014.1.3. Caveats ..................................................................................................... 433

    14.2. Statistics Used by the Planner .............................................................................. 43414.3. Controlling the Planner with Explicit JOIN Clauses............................................ 43614.4. Populating a Database .......................................................................................... 438

    14.4.1. Disable Autocommit ................................................................................ 43814.4.2. Use COPY.................................................................................................. 43814.4.3. Remove Indexes ....................................................................................... 43814.4.4. Remove Foreign Key Constraints ............................................................ 43914.4.5. Increase maintenance_work_mem ........................................................ 43914.4.6. Increase max_wal_size ......................................................................... 43914.4.7. Disable WAL Archival and Streaming Replication ................................. 43914.4.8. Run ANALYZE Afterwards........................................................................ 44014.4.9. Some Notes About pg_dump................................................................... 440

    14.5. Non-Durable Settings ........................................................................................... 44115. Parallel Query.................................................................................................................... 442

    15.1. How Parallel Query Works ................................................................................... 44215.2. When Can Parallel Query Be Used?..................................................................... 44315.3. Parallel Plans ........................................................................................................ 444

    15.3.1. Parallel Scans ........................................................................................... 44415.3.2. Parallel Joins ............................................................................................ 44415.3.3. Parallel Aggregation ................................................................................ 44415.3.4. Parallel Plan Tips ..................................................................................... 445

    15.4. Parallel Safety....................................................................................................... 44515.4.1. Parallel Labeling for Functions and Aggregates...................................... 445

    x

  • III. Server Administration ............................................................................................................. 44716. Installation from Source Code .......................................................................................... 449

    16.1. Short Version ........................................................................................................ 44916.2. Requirements........................................................................................................ 44916.3. Getting The Source............................................................................................... 45116.4. Installation Procedure........................................................................................... 45116.5. Post-Installation Setup.......................................................................................... 461

    16.5.1. Shared Libraries ....................................................................................... 46116.5.2. Environment Variables ............................................................................. 462

    16.6. Supported Platforms ............................................................................................. 46316.7. Platform-specific Notes ........................................................................................ 463

    16.7.1. AIX .......................................................................................................... 46316.7.1.1. GCC Issues.................................................................................. 46416.7.1.2. Unix-Domain Sockets Broken..................................................... 46416.7.1.3. Internet Address Issues................................................................ 46416.7.1.4. Memory Management ................................................................. 465

    References and Resources ................................................................ 46616.7.2. Cygwin..................................................................................................... 46616.7.3. HP-UX ..................................................................................................... 46716.7.4. MinGW/Native Windows ........................................................................ 468

    16.7.4.1. Collecting Crash Dumps on Windows ........................................ 46816.7.5. SCO OpenServer and SCO UnixWare..................................................... 468

    16.7.5.1. Skunkware ................................................................................... 46916.7.5.2. GNU Make .................................................................................. 46916.7.5.3. Readline....................................................................................... 46916.7.5.4. Using the UDK on OpenServer................................................... 46916.7.5.5. Reading the PostgreSQL Man Pages........................................... 46916.7.5.6. C99 Issues with the 7.1.1b Feature Supplement ......................... 47016.7.5.7. Threading on UnixWare .............................................................. 470

    16.7.6. Solaris ...................................................................................................... 47016.7.6.1. Required Tools ............................................................................ 47016.7.6.2. Problems with OpenSSL ............................................................. 47016.7.6.3. configure Complains About a Failed Test Program .................... 47116.7.6.4. 64-bit Build Sometimes Crashes ................................................. 47116.7.6.5. Compiling for Optimal Performance........................................... 47116.7.6.6. Using DTrace for Tracing PostgreSQL....................................... 471

    17. Installation from Source Code on Windows ..................................................................... 47317.1. Building with Visual C++ or the Microsoft Windows SDK................................. 473

    17.1.1. Requirements ........................................................................................... 47417.1.2. Special Considerations for 64-bit Windows ............................................ 47617.1.3. Building ................................................................................................... 47617.1.4. Cleaning and Installing ............................................................................ 47617.1.5. Running the Regression Tests .................................................................. 47717.1.6. Building the Documentation .................................................................... 477

    17.2. Building libpq with Visual C++ or Borland C++................................................. 47817.2.1. Generated Files ........................................................................................ 478

    18. Server Setup and Operation .............................................................................................. 48018.1. The PostgreSQL User Account ............................................................................ 48018.2. Creating a Database Cluster ................................................................................. 480

    18.2.1. Use of Secondary File Systems................................................................ 48118.2.2. Use of Network File Systems .................................................................. 481

    xi

  • 18.3. Starting the Database Server................................................................................. 48218.3.1. Server Start-up Failures ........................................................................... 48418.3.2. Client Connection Problems .................................................................... 484

    18.4. Managing Kernel Resources................................................................................. 48518.4.1. Shared Memory and Semaphores ............................................................ 48518.4.2. Resource Limits ....................................................................................... 49018.4.3. Linux Memory Overcommit .................................................................... 49118.4.4. Linux huge pages ..................................................................................... 493

    18.5. Shutting Down the Server..................................................................................... 49318.6. Upgrading a PostgreSQL Cluster ......................................................................... 494

    18.6.1. Upgrading Data via pg_dumpall.............................................................. 49518.6.2. Upgrading Data via pg_upgrade .............................................................. 49618.6.3. Upgrading Data via Replication............................................................... 496

    18.7. Preventing Server Spoofing .................................................................................. 49718.8. Encryption Options............................................................................................... 49718.9. Secure TCP/IP Connections with SSL ................................................................. 498

    18.9.1. Using Client Certificates .......................................................................... 49918.9.2. SSL Server File Usage ............................................................................. 50018.9.3. Creating a Self-signed Certificate ............................................................ 500

    18.10. Secure TCP/IP Connections with SSH Tunnels ................................................. 50118.11. Registering Event Log on Windows................................................................... 502

    19. Server Configuration ......................................................................................................... 50319.1. Setting Parameters ................................................................................................ 503

    19.1.1. Parameter Names and Values................................................................... 50319.1.2. Parameter Interaction via the Configuration File..................................... 50319.1.3. Parameter Interaction via SQL................................................................. 50419.1.4. Parameter Interaction via the Shell .......................................................... 50519.1.5. Managing Configuration File Contents.................................................... 505

    19.2. File Locations ....................................................................................................... 50619.3. Connections and Authentication........................................................................... 507

    19.3.1. Connection Settings ................................................................................. 50719.3.2. Security and Authentication..................................................................... 510

    19.4. Resource Consumption......................................................................................... 51219.4.1. Memory.................................................................................................... 51219.4.2. Disk .......................................................................................................... 51419.4.3. Kernel Resource Usage............................................................................ 51519.4.4. Cost-based Vacuum Delay ....................................................................... 51519.4.5. Background Writer................................................................................... 51619.4.6. Asynchronous Behavior........................................................................... 517

    19.5. Write Ahead Log .................................................................................................. 51919.5.1. Settings..................................................................................................... 51919.5.2. Checkpoints.............................................................................................. 52319.5.3. Archiving ................................................................................................. 524

    19.6. Replication............................................................................................................ 52519.6.1. Sending Server(s)..................................................................................... 52519.6.2. Master Server ........................................................................................... 52619.6.3. Standby Servers ....................................................................................... 527

    19.7. Query Planning..................................................................................................... 52919.7.1. Planner Method Configuration................................................................. 52919.7.2. Planner Cost Constants ............................................................................ 53019.7.3. Genetic Query Optimizer......................................................................... 53119.7.4. Other Planner Options.............................................................................. 532

    xii

  • 19.8. Error Reporting and Logging ............................................................................... 53419.8.1. Where To Log .......................................................................................... 53419.8.2. When To Log ........................................................................................... 53719.8.3. What To Log ............................................................................................ 53919.8.4. Using CSV-Format Log Output ............................................................... 54219.8.5. Process Title............................................................................................. 543

    19.9. Run-time Statistics................................................................................................ 54419.9.1. Query and Index Statistics Collector ....................................................... 54419.9.2. Statistics Monitoring................................................................................ 545

    19.10. Automatic Vacuuming ........................................................................................ 54519.11. Client Connection Defaults ................................................................................ 547

    19.11.1. Statement Behavior ................................................................................ 54719.11.2. Locale and Formatting ........................................................................... 55119.11.3. Shared Library Preloading ..................................................................... 55319.11.4. Other Defaults ........................................................................................ 554

    19.12. Lock Management .............................................................................................. 55519.13. Version and Platform Compatibility................................................................... 556

    19.13.1. Previous PostgreSQL Versions .............................................................. 55619.13.2. Platform and Client Compatibility......................................................... 557

    19.14. Error Handling.................................................................................................... 55819.15. Preset Options..................................................................................................... 55819.16. Customized Options ........................................................................................... 56019.17. Developer Options .............................................................................................. 56019.18. Short Options...................................................................................................... 563

    20. Client Authentication ........................................................................................................ 56420.1. The pg_hba.conf File ........................................................................................ 56420.2. User Name Maps .................................................................................................. 57020.3. Authentication Methods ....................................................................................... 572

    20.3.1. Trust Authentication ................................................................................ 57220.3.2. Password Authentication ......................................................................... 57220.3.3. GSSAPI Authentication ........................................................................... 57320.3.4. SSPI Authentication................................................................................. 57420.3.5. Ident Authentication................................................................................. 57520.3.6. Peer Authentication.................................................................................. 57620.3.7. LDAP Authentication .............................................................................. 57620.3.8. RADIUS Authentication.......................................................................... 57820.3.9. Certificate Authentication ........................................................................ 57920.3.10. PAM Authentication .............................................................................. 57920.3.11. BSD Authentication ............................................................................... 580

    20.4. Authentication Problems ...................................................................................... 58021. Database Roles .................................................................................................................. 582

    21.1. Database Roles ..................................................................................................... 58221.2. Role Attributes...................................................................................................... 58321.3. Role Membership ................................................................................................. 58421.4. Dropping Roles..................................................................................................... 58521.5. Default Roles ........................................................................................................ 58621.6. Function and Trigger Security .............................................................................. 587

    22. Managing Databases ......................................................................................................... 58822.1. Overview .............................................................................................................. 58822.2. Creating a Database.............................................................................................. 58822.3. Template Databases .............................................................................................. 58922.4. Database Configuration ........................................................................................ 590

    xiii

  • 22.5. Destroying a Database .......................................................................................... 59122.6. Tablespaces........................................................................................................... 591

    23. Localization....................................................................................................................... 59423.1. Locale Support...................................................................................................... 594

    23.1.1. Overview.................................................................................................. 59423.1.2. Behavior ................................................................................................... 59523.1.3. Problems .................................................................................................. 596

    23.2. Collation Support.................................................................................................. 59623.2.1. Concepts................................................................................................... 59623.2.2. Managing Collations................................................................................ 598

    23.3. Character Set Support........................................................................................... 59923.3.1. Supported Character Sets......................................................................... 59923.3.2. Setting the Character Set.......................................................................... 60223.3.3. Automatic Character Set Conversion Between Server and Client........... 60323.3.4. Further Reading ....................................................................................... 605

    24. Routine Database Maintenance Tasks............................................................................... 60624.1. Routine Vacuuming .............................................................................................. 606

    24.1.1. Vacuuming Basics.................................................................................... 60624.1.2. Recovering Disk Space ............................................................................ 60724.1.3. Updating Planner Statistics ...................................................................... 60824.1.4. Updating The Visibility Map ................................................................... 60924.1.5. Preventing Transaction ID Wraparound Failures..................................... 609

    24.1.5.1. Multixacts and Wraparound ........................................................ 61224.1.6. The Autovacuum Daemon ....................................................................... 612

    24.2. Routine Reindexing .............................................................................................. 61324.3. Log File Maintenance........................................................................................... 614

    25. Backup and Restore .......................................................................................................... 61625.1. SQL Dump............................................................................................................ 616

    25.1.1. Restoring the Dump ................................................................................. 61625.1.2. Using pg_dumpall.................................................................................... 61725.1.3. Handling Large Databases ....................................................................... 618

    25.2. File System Level Backup.................................................................................... 61925.3. Continuous Archiving and Point-in-Time Recovery (PITR)................................ 620

    25.3.1. Setting Up WAL Archiving...................................................................... 62125.3.2. Making a Base Backup ............................................................................ 62325.3.3. Making a Base Backup Using the Low Level API .................................. 624

    25.3.3.1. Making a non-exclusive low level backup .................................. 62425.3.3.2. Making an exclusive low level backup........................................ 62525.3.3.3. Backing up the data directory...................................................... 626

    25.3.4. Recovering Using a Continuous Archive Backup ................................... 62725.3.5. Timelines.................................................................................................. 62825.3.6. Tips and Examples ................................................................................... 629

    25.3.6.1. Standalone Hot Backups ............................................................. 62925.3.6.2. Compressed Archive Logs .......................................................... 63025.3.6.3. archive_command Scripts ........................................................ 630

    25.3.7. Caveats ..................................................................................................... 63126. High Availability, Load Balancing, and Replication......................................................... 632

    26.1. Comparison of Different Solutions....................................................................... 63226.2. Log-Shipping Standby Servers............................................................................. 635

    26.2.1. Planning ................................................................................................... 63626.2.2. Standby Server Operation ........................................................................ 63626.2.3. Preparing the Master for Standby Servers ............................................... 637

    xiv

  • 26.2.4. Setting Up a Standby Server .................................................................... 63726.2.5. Streaming Replication.............................................................................. 638

    26.2.5.1. Authentication ............................................................................. 63926.2.5.2. Monitoring................................................................................... 639

    26.2.6. Replication Slots ...................................................................................... 64026.2.6.1. Querying and manipulating replication slots .............................. 64026.2.6.2. Configuration Example ............................................................... 640

    26.2.7. Cascading Replication ............................................................................. 64126.2.8. Synchronous Replication ......................................................................... 641

    26.2.8.1. Basic Configuration..................................................................... 64226.2.8.2. Multiple Synchronous Standbys.................................................. 64226.2.8.3. Planning for Performance............................................................ 64326.2.8.4. Planning for High Availability .................................................... 643

    26.2.9. Continuous archiving in standby ............................................................. 64426.3. Failover ................................................................................................................. 64426.4. Alternative Method for Log Shipping .................................................................. 645

    26.4.1. Implementation ........................................................................................ 64626.4.2. Record-based Log Shipping..................................................................... 647

    26.5. Hot Standby .......................................................................................................... 64726.5.1. User’s Overview....................................................................................... 64726.5.2. Handling Query Conflicts ........................................................................ 64926.5.3. Administrator’s Overview........................................................................ 65126.5.4. Hot Standby Parameter Reference ........................................................... 65426.5.5. Caveats ..................................................................................................... 654

    27. Recovery Configuration .................................................................................................... 65527.1. Archive Recovery Settings ................................................................................... 65527.2. Recovery Target Settings ...................................................................................... 65627.3. Standby Server Settings........................................................................................ 657

    28. Monitoring Database Activity........................................................................................... 65928.1. Standard Unix Tools ............................................................................................. 65928.2. The Statistics Collector......................................................................................... 660

    28.2.1. Statistics Collection Configuration .......................................................... 66028.2.2. Viewing Statistics..................................................................................... 66128.2.3. Statistics Functions .................................................................................. 680

    28.3. Viewing Locks...................................................................................................... 68328.4. Progress Reporting ............................................................................................... 683

    28.4.1. VACUUM Progress Reporting................................................................. 68328.5. Dynamic Tracing .................................................................................................. 685

    28.5.1. Compiling for Dynamic Tracing.............................................................. 68628.5.2. Built-in Probes ......................................................................................... 68628.5.3. Using Probes ............................................................................................ 69428.5.4. Defining New Probes ............................................................................... 695

    29. Monitoring Disk Usage..................................................................................................... 69729.1. Determining Disk Usage ...................................................................................... 69729.2. Disk Full Failure................................................................................................... 698

    30. Reliability and the Write-Ahead Log................................................................................ 69930.1. Reliability ............................................................................................................. 69930.2. Write-Ahead Logging (WAL) .............................................................................. 70130.3. Asynchronous Commit......................................................................................... 70130.4. WAL Configuration .............................................................................................. 70330.5. WAL Internals ...................................................................................................... 706

    31. Regression Tests................................................................................................................ 707

    xv

  • 31.1. Running the Tests ................................................................................................. 70731.1.1. Running the Tests Against a Temporary Installation ............................... 70731.1.2. Running the Tests Against an Existing Installation ................................. 70731.1.3. Additional Test Suites .............................................................................. 70831.1.4. Locale and Encoding................................................................................ 70831.1.5. Extra Tests................................................................................................ 70931.1.6. Testing Hot Standby................................................................................. 709

    31.2. Test Evaluation ..................................................................................................... 71031.2.1. Error Message Differences....................................................................... 71031.2.2. Locale Differences ................................................................................... 71031.2.3. Date and Time Differences ...................................................................... 71131.2.4. Floating-Point Differences....................................................................... 71131.2.5. Row Ordering Differences ....................................................................... 71131.2.6. Insufficient Stack Depth........................................................................... 71231.2.7. The “random” Test ................................................................................... 71231.2.8. Configuration Parameters......................................................................... 712

    31.3. Variant Comparison Files ..................................................................................... 71231.4. TAP Tests.............................................................................................................. 71331.5. Test Coverage Examination.................................................................................. 714

    IV. Client Interfaces ....................................................................................................................... 71532. libpq - C Library ............................................................................................................... 717

    32.1. Database Connection Control Functions .............................................................. 71732.1.1. Connection Strings................................................................................... 723

    32.1.1.1. Keyword/Value Connection Strings ............................................ 72332.1.1.2. Connection URIs ......................................................................... 723

    32.1.2. Parameter Key Words .............................................................................. 72432.2. Connection Status Functions ................................................................................ 72832.3. Command Execution Functions ........................................................................... 733

    32.3.1. Main Functions ........................................................................................ 73332.3.2. Retrieving Query Result Information ...................................................... 74032.3.3. Retrieving Other Result Information ....................................................... 74432.3.4. Escaping Strings for Inclusion in SQL Commands ................................. 745

    32.4. Asynchronous Command Processing ................................................................... 74732.5. Retrieving Query Results Row-By-Row .............................................................. 75132.6. Canceling Queries in Progress.............................................................................. 75232.7. The Fast-Path Interface......................................................................................... 75332.8. Asynchronous Notification................................................................................... 75432.9. Functions Associated with the COPY Command .................................................. 755

    32.9.1. Functions for Sending COPY Data............................................................ 75632.9.2. Functions for Receiving COPY Data......................................................... 75732.9.3. Obsolete Functions for COPY ................................................................... 757

    32.10. Control Functions ............................................................................................... 75932.11. Miscellaneous Functions .................................................................................... 76132.12. Notice Processing ............................................................................................... 76332.13. Event System ...................................................................................................... 764

    32.13.1. Event Types............................................................................................ 76432.13.2. Event Callback Procedure...................................................................... 76632.13.3. Event Support Functions........................................................................ 76732.13.4. Event Example ....................................................................................... 768

    32.14. Environment Variables ....................................................................................... 77032.15. The Password File .............................................................................................. 772

    xvi

  • 32.16. The Connection Service File .............................................................................. 77232.17. LDAP Lookup of Connection Parameters .......................................................... 77332.18. SSL Support........................................................................................................ 774

    32.18.1. Client Verification of Server Certificates ............................................... 77432.18.2. Client Certificates................................................................................... 77532.18.3. Protection Provided in Different Modes ................................................ 77532.18.4. SSL Client File Usage............................................................................ 77732.18.5. SSL Library Initialization ...................................................................... 777

    32.19. Behavior in Threaded Programs ......................................................................... 77832.20. Building libpq Programs..................................................................................... 77932.21. Example Programs.............................................................................................. 780

    33. Large Objects .................................................................................................................... 78933.1. Introduction .......................................................................................................... 78933.2. Implementation Features ...................................................................................... 78933.3. Client Interfaces.................................................................................................... 789

    33.3.1. Creating a Large Object ........................................................................... 79033.3.2. Importing a Large Object......................................................................... 79033.3.3. Exporting a Large Object......................................................................... 79133.3.4. Opening an Existing Large Object........................................................... 79133.3.5. Writing Data to a Large Object................................................................ 79133.3.6. Reading Data from a Large Object .......................................................... 79233.3.7. Seeking in a Large Object........................................................................ 79233.3.8. Obtaining the Seek Position of a Large Object........................................ 79233.3.9. Truncating a Large Object ....................................................................... 79333.3.10. Closing a Large Object Descriptor ........................................................ 79333.3.11. Removing a Large Object ...................................................................... 793

    33.4. Server-side Functions ........................................................................................... 79433.5. Example Program ................................................................................................. 795

    34. ECPG - Embedded SQL in C............................................................................................ 80034.1. The Concept.......................................................................................................... 80034.2. Managing Database Connections ......................................................................... 800

    34.2.1. Connecting to the Database Server ..........................................