PostgreSQL - ArchWiki

# PostgreSQL

Related articles

- PhpPgAdmin

PostgreSQL is an open source, community driven, standard compliant object-relational database system.

This document describes how to set up PostgreSQL. It also describes how to configure PostgreSQL to be accessible from a remote client. Among other applications, PostgreSQL can be substituted for MySQL as part of the LAMP web stack.

## Installing PostgreSQL

Install the postgresql package. Then set a password for the newly created postgres user.

Then, switch to the default PostgreSQL user postgres by executing the following command:

- If you have sudo and your username is in `sudoers`:

      $ sudo -u postgres -i

- Otherwise:

      $ su
      # su -l postgres

See su(1) or sudo(8) for their usage.

Note: Commands that should be run as the postgres user are prefixed by `[postgres]$` in this article.

Before PostgreSQL can function correctly, the database cluster must be initialized:

    [postgres]$ initdb --locale $LANG -E UTF8 -D '/var/lib/postgres/data'

Where:

- the `--locale` is the one defined in the file `/etc/locale.conf`;
- the `-E` is the default encoding of the database that will be created in the future;
- and `-D` is the default location where the database cluster must be stored.

Many lines should now appear on the screen with several ending by `... ok`:

    The files belonging to this database system will be owned by user "postgres".
    This user must also own the server process.

    The database cluster will be initialized with locale "en_GB.UTF-8".
    The default text search configuration will be set to "english".

    Data page checksums are disabled.

    fixing permissions on existing directory /var/lib/postgres/data ... ok
    creating subdirectories ... ok
    selecting default max_connections ... 100
    selecting default shared_buffers ... 128MB
    selecting dynamic shared memory implementation ... posix
    creating configuration files ... ok
    creating template1 database in /var/lib/postgres/data/base/1 ... ok
    initializing pg_authid ... ok
    [...]

If these are the kind of lines you see, then the process succeeded. Return to the regular user using `exit`.

As root, start and enable `postgresql.service`.

Tip: If you change the root to something other than `/var/lib/postgres`, you will have to edit the service file. If the root is under `home`, make sure to set `ProtectHome` to false.
Warning: If the database resides on a Btrfs file system, you should consider disabling Copy-on-Write for the directory before creating any database. If the database resides on a ZFS file system, you should consult ZFS#Database before creating any database.

## Create your first database/user

Tip: If you create a PostgreSQL user with the same name as your Linux username, it allows you to access the PostgreSQL database shell without having to specify a user to login (which makes it quite convenient).

Become the postgres user. Add a new database user using the createuser command:

    [postgres]$ createuser --interactive

Create a new database over which the above user has read/write privileges using the createdb command (execute this command from your login shell if the database user has the same name as your Linux user, otherwise add `-U database-username` to the following command):

    $ createdb myDatabaseName

## Familiarize with PostgreSQL

### Access the database shell

Become the postgres user. Start the primary database shell, psql, where you can do all your creation of databases/tables, deletion, set permissions, and run raw SQL commands. Use the `-d` option to connect to the database you created (without specifying a database, `psql` will try to access a database that matches your username).

    [postgres]$ psql -d myDatabaseName

Some helpful commands:

Get help:

    => \help

Connect to a particular database:

    => \c <database>

List all users and their permission levels:

    => \du

Show summary information about all tables in the current database:

    => \dt

Exit/quit the `psql` shell:

    => \q or CTRL+d

There are of course many more meta-commands, but these should help you get started. To see all meta-commands run:

    => \?

## Optional configuration

### Configure PostgreSQL to be accessible from remote hosts

The PostgreSQL database server configuration file is `postgresql.conf`. This file is located in the data directory of the server, typically `/var/lib/postgres/data`. This folder also houses the other main configuration files, including the `pg_hba.conf`.

Note: By default, this folder will not be browsable or searchable by a regular user. This is why `find` and `locate` are not finding the configuration files.

Edit the file `/var/lib/postgres/data/postgresql.conf`. In the connections and authentications section, add the `listen_addresses` line to your needs:

    listen_addresses = 'localhost,my_local_ip_address'
    #You can use '*' to listen on all local addresses

Take a careful look at the other lines.

Host-based authentication is configured in `/var/lib/postgres/data/pg_hba.conf`. This file controls which hosts are allowed to connect. Note that the defaults allow any local user to connect as any database user, including the database superuser. Add a line like the following:

    # IPv4 local connections:
    host   all   all   my_remote_client_ip_address/32   md5

where `my_remote_client_ip_address` is the IP address of the client.

See the documentation for pg_hba.conf.

After this you should restart `postgresql.service` for the changes to take effect.

Note: PostgreSQL uses port `5432` by default for remote connections. Make sure this port is open and able to receive incoming connections.

For troubleshooting take a look in the server log file:

    $ journalctl -u postgresql

### Configure PostgreSQL authenticate against PAM

PostgreSQL offers a number of authentication methods. If you would like to allow users to authenticate with their system password, additional steps are necessary. First you need to enable PAM for the connection.

For example, the same configuration as above, but with PAM enabled:

    # IPv4 local connections:
    host   all   all   my_remote_client_ip_address/32   pam

The PostgreSQL server is however running without root privileges and will not be able to access `/etc/shadow`. We can work around that by allowing the postgres group to access this file:

    setfacl -m g:postgres:r /etc/shadow

### Change default data directory

The default directory where all your newly created databases will be stored is `/var/lib/postgres/data`. To change this, follow these steps:

Create the new directory and make the postgres user its owner:

    # mkdir -p /pathto/pgroot/data
    # chown -R postgres:postgres /pathto/pgroot

Become the postgres user, and initialize the new cluster:

    [postgres]$ initdb -D /pathto/pgroot/data

Some variables need to be overridden in the configuration, as described in Systemd#Editing provided units. First, run:

    # systemctl edit postgresql.service

Systemctl will open a drop-in configuration file in your editor. Write the following in that file:

    [Service]
    Environment=PGROOT=/pathto/pgroot
    PIDFile=/pathto/pgroot/data/postmaster.pid

If you want to use `/home` directory for default directory or for tablespaces, add one more line in this file:

    ProtectHome=false

### Change default encoding of new databases to UTF-8

Note: If you ran `initdb` with `-E UTF8` these steps are not required.

When creating a new database (e.g. with `createdb blog`) PostgreSQL actually copies a template database. There are two predefined templates: `template0` is vanilla, while `template1` is meant as an on-site template changeable by the administrator and is used by default. In order to change the encoding of a new database, one of the options is to change on-site `template1`. To do this, log into PostgreSQL shell (`psql`) and execute the following:

First, we need to drop `template1`. Templates cannot be dropped, so we first modify it so it is an ordinary database:

    UPDATE pg_database SET datistemplate = FALSE WHERE datname = 'template1';

Now we can drop it:

    DROP DATABASE template1;

The next step is to create a new database from `template0`, with a new default encoding:

    CREATE DATABASE template1 WITH TEMPLATE = template0 ENCODING = 'UNICODE';

Now modify `template1` so it is actually a template:

    UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template1';

Optionally, if you do not want anyone connecting to this template, set `datallowconn` to `FALSE`:

    UPDATE pg_database SET datallowconn = FALSE WHERE datname = 'template1';

Note: This last step can create problems when upgrading via `pg_upgrade`.

Now you can create a new database:

    [postgres]$ createdb blog

If you log back in to `psql` and check the databases, you should see the proper encoding of your new database:

    \l

                                  List of databases
      Name    |  Owner   | Encoding  | Collation | Ctype |   Access privileges
    -----------+----------+-----------+-----------+-------+----------------------
    blog      | postgres | UTF8      | C         | C     |
    postgres  | postgres | SQL_ASCII | C         | C     |
    template0 | postgres | SQL_ASCII | C         | C     | =c/postgres
                                                         : postgres=CTc/postgres
    template1 | postgres | UTF8      | C         | C     |

## Administration tools

- phpPgAdmin — Web-based administration tool for PostgreSQL.

  http://phppgadmin.sourceforge.net || phppgadmin

- pgAdmin — GUI-based administration tool for PostgreSQL.

  http://www.pgadmin.org/ || pgadmin3

## Setup HHVM to work with PostgreSQL

This article or section is out of date.

Reason: hhvm-pgsql fails to compile against HHVM 3.7.0, but upstream has not resolved the problem yet. See https://github.com/PocketRent/hhvm-pgsql/issues/82 (Discuss in Talk:PostgreSQL#Setting up HHVM)

    $ git clone https://github.com/PocketRent/hhvm-pgsql.git
    $ cd hhvm-pgsql

If you do not use a nightly build, then run this command (verified on HHVM 3.6.1) to avoid compile errors:

    $ git checkout tags/3.6.0

Then build the extension (if you do not need an improved support for Hack language, then remove -DHACK_FRIENDLY=ON):

    $ hphpize
    $ cmake -DHACK_FRIENDLY=ON .
    $ make

Then copy the built extension:

    # cp pgsql.so /etc/hhvm/

Add to /etc/hhvm/server.ini:

    extension_dir = /etc/hhvm
    hhvm.extensions[pgsql] = pgsql.so

## Upgrading PostgreSQL

Upgrading minor PostgreSQL versions (i.e. `9.2, 9.3, 9.4, 9.5, 9.6`) requires some extra maintenance.

Tip: If you already migrated from 9.2 to 9.3 and you want to migrate from 9.3 to 9.4, change versions before executing commands. If `/var/lib/postgres/data-9.2` already exists and you just copy-paste all commands, `pg_upgrade` will complain about the wrong version of the database, because your version 9.3 database will be stored in `/var/lib/postgres/data-9.2/data/`.

### Quick guide

If you had custom settings in configuration files like `pg_hba.conf` and `postgresql.conf`, merge them into the new ones. If you have set options such as `data_directory`, `hba_file` or `ident_file` in your `postgresql.conf`, please temporally unset them before the migration.

Warning: This quick script is critical. Please make sure you have dumped your complete database before embarking in such upgrade.

First step: dump your database and stop PostgreSQL

    # su -l postgres
    [postgres]$ cd /var/lib/postgres
    [postgres]$ mkdir dump
    [postgres]$ pg_dumpall | gzip > dump/pgdumpall.gz

Stop `postgresql.service`.

Second step: prepare the upgrade script

    upgrade_pg.sh

    # Usage: sh ./upgrade_pg.sh <x.y version to update from>

    FROM_VERSION="$1"

    su -l postgres -c "mv /var/lib/postgres/data /var/lib/postgres/data-${FROM_VERSION}"
    su -l postgres -c 'mkdir /var/lib/postgres/data'
    su -l postgres -c 'chmod 700 /var/lib/postgres/data'
    su -l postgres -c "initdb --locale $LANG -E UTF8 -D /var/lib/postgres/data"
    su -l postgres -c "pg_upgrade -b /opt/pgsql-${FROM_VERSION}/bin/ -B /usr/bin/ -d /var/lib/postgres/data-${FROM_VERSION} -D /var/lib/postgres/data"

Third step: update the packages and install postgresql-old-upgrade

    # pacman -Syu postgresql-old-upgrade postgresql postgresql-libs

Fourth step: run the upgrade script

The first time, it is advisable to run one by one the commands of this script.

Fith step: start the database

Start `postgresql.service`

#### Troubleshooting

This article or section needs language, wiki syntax or style improvements.

Reason: please use the first argument of the template to provide a brief explanation. (Discuss in Talk:PostgreSQL#)

If the `pg_upgrade` step fails with the following messages,

cannot write to log file pg_upgrade_internal.log
Failure, exiting

The factual accuracy of this article or section is disputed.

Reason: There's no need to switch from sudo to su to run a login shell. (Discuss in Talk:PostgreSQL#)

  Make sure you are in a directory that the postgres user has enough rights to write the log file to (`/tmp` for example), or use `su -l postgres` instead of `sudo -u postgres`.

  If you are in a directory that postgres user has enough rights to write the log file to however you still get this error then make sure `/var/lib/postgres` is owned by postgres

LC_COLLATE error that says that old and new values are different

  Figure out what the old locale was, `C` or `en_US.UTF-8` for example, and force it when calling `initdb`.

      sudo -u postgres LC_ALL=C initdb -D /var/lib/postgres/data

There seems to be a postmaster servicing the old cluster.
Please shutdown that postmaster and try again.

  Make sure postgres is not running. If you still get the error, then chances are there is an old PID file you need to clear out.

  Find the old pid in old pg data,

        $ sudo -u postgres ls -l /var/lib/postgres/data-9.X
        total 88
        -rw------- 1 postgres postgres     4 Mar 25  2012 PG_VERSION
        drwx------ 8 postgres postgres  4096 Jul 17 00:36 base
        drwx------ 2 postgres postgres  4096 Jul 17 00:38 global
        drwx------ 2 postgres postgres  4096 Mar 25  2012 pg_clog
        -rw------- 1 postgres postgres  4476 Mar 25  2012 pg_hba.conf
        -rw------- 1 postgres postgres  1636 Mar 25  2012 pg_ident.conf
        drwx------ 4 postgres postgres  4096 Mar 25  2012 pg_multixact
        drwx------ 2 postgres postgres  4096 Jul 17 00:05 pg_notify
        drwx------ 2 postgres postgres  4096 Mar 25  2012 pg_serial
        drwx------ 2 postgres postgres  4096 Jul 17 00:53 pg_stat_tmp
        drwx------ 2 postgres postgres  4096 Mar 25  2012 pg_subtrans
        drwx------ 2 postgres postgres  4096 Mar 25  2012 pg_tblspc
        drwx------ 2 postgres postgres  4096 Mar 25  2012 pg_twophase
        drwx------ 3 postgres postgres  4096 Mar 25  2012 pg_xlog
        -rw------- 1 postgres postgres 19169 Mar 25  2012 postgresql.conf
        -rw------- 1 postgres postgres    48 Jul 17 00:05 postmaster.opts
        -rw------- 1 postgres postgres    80 Jul 17 00:05 postmaster.pid   # <-- This is the problem

  Move the old pid to temporary directory,

      $ sudo -u postgres mv /var/lib/postgres/data-9.X/postmaster.pid /tmp

ERROR: could not access file "$libdir/postgis-2.0": No such file or directory

  Retrieve `postgis-2.0.so` from postgis for version postgresql 9.X () and copy it to `/opt/pgsql-9.X/lib` and make sure the privileges are readable by postgres user.

Your installation references loadable libraries that are missing from the new installation.
could not load library "$libdir/postgis-2.2":
ERROR: incompatible library "/usr/lib/postgresql/postgis-2.2.so": version mismatch
DETAIL: Server is version 9.6, library is version 9.5.

  You can manually compile the previous version of PostGIS against the new version of PostgreSQL, use it for the upgrade process, then install the new version back.

  In this example we upgrade from the old PostgreSQL 9.5 + PostGIS 2.2 to the new PostgreSQL 9.6 + PostGIS 2.3:

      # Uninstall PostGIS 2.3.
      sudo pacman -R postgis

      # Download and compile PostGIS 2.2 against PostgreSQL 5.6. The latest sources can be found at http://postgis.net/source/
      wget http://download.osgeo.org/postgis/source/postgis-2.2.3.tar.gz
      tar -xvf postgis-2.2.3.tar.gz
      cd postgis-2.2.3
      ./configure
      make

      # Copy just the files we need to upgrade the database.
      sudo cp postgis/postgis-2.2.so /usr/lib/postgresql
      sudo cp raster/rt_pg/rtpostgis-2.2.so /usr/lib/postgresql
      sudo cp topology/postgis_topology-2.2.so /usr/lib/postgresql

      # Run pg_upgrade as described above. It should now finish successfully.
      [...]

      # Remove PostGIS 2.2.
      sudo rm /usr/lib/postgresql/postgis-2.2.so
      sudo rm /usr/lib/postgresql/rtpostgis-2.2.so
      sudo rm /usr/lib/postgresql/postgis_topology-2.2.so

      # Install PostGIS 2.3 back again.
      sudo pacman -S postgis

Your installation references loadable libraries that are missing from the new installation.
You can add these libraries to the new installation, or remove the functions using them from the old installation.
A list of problem libraries is in the file
loadable_libraries.txt

Could not load library "$libdir/pg_upgrade_support"
ERROR: could not access file "$libdir/pg_upgrade_support": No such file or directory

  It means you have leftovers from old failed pg_upgrade attempts that you never completed. So you'll need to start postgres with old data and
  in each database execute `DROP SCHEMA IF EXISTS binary_upgrade CASCADE;`

### Detailed instructions

Note: Official PostgreSQL upgrade documentation should be followed.
Warning: The following instructions could cause data loss. Use at your own risk.

It is recommended to add the following to your `/etc/pacman.conf` file:

    IgnorePkg = postgresql postgresql-libs

This will ensure you do not accidentally upgrade the database to an incompatible version. When an upgrade is available, pacman will notify you that it is skipping the upgrade because of the entry in `pacman.conf`. Minor version upgrades (e.g. 9.0.3 to 9.0.4) are safe to perform. However, if you do an accidental upgrade to a different major version (e.g. 9.0.x to 9.1.x), you might not be able to access any of your data. Always check the PostgreSQL home page (http://www.postgresql.org/) to be sure of what steps are required for each upgrade. For a bit about why this is the case, see the versioning policy.

There are two main ways to upgrade your PostgreSQL database. Read the official documentation for details.

For those wishing to use `pg_upgrade`, a postgresql-old-upgrade package is available that will always run one major version behind the real PostgreSQL package. This can be installed side-by-side with the new version of PostgreSQL.

When you are ready, upgrade the following packages: postgresql, postgresql-libs, and postgresql-old-upgrade. Note that the data directory does not change from version to version, so before running `pg_upgrade`, it is necessary to rename your existing data directory and migrate into a new directory. The new database must be initialized, as described near the top of this page.

    # systemctl stop postgresql.service
    # su -l postgres -c 'mv /var/lib/postgres/data /var/lib/postgres/olddata'
    # su -l postgres -c 'initdb --locale en_US.UTF-8 -E UTF8 -D /var/lib/postgres/data'

The upgrade invocation will likely look something like the following. Do not run this command blindly without understanding what it does! Reference the upstream pg_upgrade documentation for details.

    # su -l postgres -c 'pg_upgrade -d /var/lib/postgres/olddata/ -D /var/lib/postgres/data/ -b /opt/pgsql-9.4/bin/ -B /usr/bin/'

#### Manual dump and reload

You could also do something like this (after the upgrade and install of postgresql-old-upgrade).

Note: Below are the commands for PostgreSQL 9.4. You can find similar commands in `/opt/` for PostgreSQL 9.2.

    # systemctl stop postgresql.service
    # /opt/pgsql-9.4/bin/pg_ctl -D /var/lib/postgres/olddata/ start
    # /opt/pgsql-9.4/bin/pg_dumpall >> old_backup.sql
    # /opt/pgsql-9.4/bin/pg_ctl -D /var/lib/postgres/olddata/ stop
    # systemctl start postgresql.service
    # psql -f old_backup.sql postgres

## Troubleshooting

### Improve performance of small transactions

If you are using PostgresSQL on a local machine for development and it seems slow, you could try turning synchronous_commit off in the configuration. Beware of the caveats, however.

    /var/lib/postgres/data/postgresql.conf

    synchronous_commit = off

### Prevent disk writes when idle

PostgreSQL periodically updates its internal "statistics" file. By default, this file is stored on disk, which prevents disks from spinning down on laptops and causes hard drive seek noise. It is simple and safe to relocate this file to a memory-only file system with the following configuration option:

    /var/lib/postgres/data/postgresql.conf

    stats_temp_directory = '/run/postgresql'

### Cannot connect to database through pg_connect()

Install php-pgsql and edit the `php.ini` file uncommenting the lines `extension=pdo_pgsql.so` and `extension=pgsql.so`, then restart `httpd`.

Category:

- Database management systems

Hidden categories:

- Pages or sections flagged with Template:Out of date
- Pages or sections flagged with Template:Style
- Pages or sections flagged with Template:Accuracy