Your country

Tools that support it use your country for local currency, number formats, units and paper size. Your choice is saved only in this browser.

Type a name or a two-letter code. Use the up and down arrow keys to move through the countries, Enter to choose one and Escape to close.

PostgreSQL in Depth  Module 2 – PostgreSQL data types, chosen well

Arrays, enums, domains and composite types

When a PostgreSQL array beats a join table and how GIN serves it, a domain that checks a PIN code once, safe enum changes, and composite row values.

  • Intermediate
  • 25 minutes
  • Examples run with PostgreSQL 18.6, Python 3.14.8 and Pyodide 314.0.7
  • By MySmartCoPilot
Free lesson, Pro answers: 25 minutes
Free: the whole lesson. With a pass: the model answers to its 2 interview questions.

What you will learn

  • Decide between an array column and a join table, and index the array with GIN
  • Query arrays with unnest WITH ORDINALITY and the array functions of PostgreSQL 18, and send lists as one array with = ANY
  • Validate a PIN code once with a domain that every column of the type shares
  • Add a value to an enum type safely, and name the changes ALTER TYPE cannot make
  • Use composite types and ROW values in SQL functions and hand clients plain columns

Before you start

What you’ll learn · Before you start

What you will learn

  • Decide between an array column and a join table, and index the array with GIN
  • Query arrays with unnest WITH ORDINALITY and the array functions of PostgreSQL 18, and send lists as one array with = ANY
  • Validate a PIN code once with a domain that every column of the type shares
  • Add a value to an enum type safely, and name the changes ALTER TYPE cannot make
  • Use composite types and ROW values in SQL functions and hand clients plain columns

Before you start

On this page

Finding the 1,428 of 50,000 products tagged both tea and organic takes one GIN index lookup when the tags are an array, and a text search in which LIKE '%tea%' also matches teapot when they are the string 'tea,organic'. A status column of plain text, likewise, stores shiped as readily as shipped. PostgreSQL has four kinds of type for values like these: arrays keep a short list inside the row, enums a fixed and ordered set of labels, domains a base type with a rule every value must pass, and composite types several named fields that travel together as one value.

On a PostgreSQL 18.6 server you will measure a GIN-indexed tag array against a join table of the same tags, put the rule of a six-digit PIN code into one domain for two tables, add a value to an enum around the commit it needs, and read a composite value’s fields in the forms that the parser and client drivers expect.

GIN answers containment on an array column

The first run gives each of 50,000 products two tags in a text[] column, ARRAY['tea', 'organic'] for product 35, and indexes the column with GIN: Generalized Inverted Index: an index that stores, for every element of a composite value (an array item, a jsonb key or value, a word of a tsvector), the rows that contain it. More in “GIN, GiST, SP-GiST, BRIN, hash and bloom”. GIN keeps every distinct element once, followed by the rows whose array contains it, so a question about elements inside the arrays becomes a lookup of a few lists. Which questions the index can take is set by its operator class: The part of an index definition that says which operators the index can search with for a column's type, such as jsonb_path_ops for GIN. The catalogs pg_opclass and pg_amop list them. More in “GIN, GiST, SP-GiST, BRIN, hash and bloom”: the run lists the operators of array_ops, the default class for arrays, and then asks for the products tagged both tea and organic:

Tags as a text array with a GIN index, on 50,000 products

tags_session.py

"""Tags as a text[] column with a GIN index, on PostgreSQL 18.6: runs tags.sql on a throwaway cluster."""

from pg_session import Server

# Plans that are the same on every run: no autovacuum, no parallel workers or JIT, statistics read from every row,
# and UTC as the time zone.
SETTINGS = {"autovacuum": "off", "max_parallel_workers_per_gather": 0, "jit": "off",
            "default_statistics_target": 1000, "timezone": "'UTC'"}

with Server(settings=SETTINGS) as pg:
    pg.up()
    pg.psql("tags.sql", errors=0)

Output

$ psql -X -a -f tags.sql
-- 50,000 products, each with a category tag and one more tag.
CREATE TABLE products (id int PRIMARY KEY, name text NOT NULL, tags text[] NOT NULL);
CREATE TABLE
INSERT INTO products
SELECT n, 'product ' || n,
       ARRAY[(ARRAY['tea', 'coffee', 'snacks', 'spices', 'sweets'])[1 + n % 5],
             (ARRAY['organic', 'gift', 'vegan', 'sale', 'new', 'local', 'imported'])[1 + n % 7]]
FROM generate_series(1, 50000) AS n;
INSERT 0 50000
CREATE INDEX products_tags ON products USING gin (tags);
CREATE INDEX
VACUUM ANALYZE products;
VACUUM
-- The operators of array_ops, the default GIN operator class for arrays.
SELECT amopopr::regoperator AS operator FROM pg_amop
WHERE amopfamily = (SELECT opcfamily FROM pg_opclass WHERE opcname = 'array_ops'
                    AND opcmethod = (SELECT oid FROM pg_am WHERE amname = 'gin'))
ORDER BY amopstrategy;
       operator        
-----------------------
 &&(anyarray,anyarray)
 @>(anyarray,anyarray)
 <@(anyarray,anyarray)
 =(anyarray,anyarray)
(4 rows)

-- Products tagged both tea and organic.
EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
SELECT count(*) FROM products WHERE tags @> ARRAY['tea', 'organic'];
                                  QUERY PLAN                                  
------------------------------------------------------------------------------
 Aggregate (actual rows=1.00 loops=1)
   ->  Bitmap Heap Scan on products (actual rows=1428.00 loops=1)
         Recheck Cond: (tags @> '{tea,organic}'::text[])
         Heap Blocks: exact=562
         ->  Bitmap Index Scan on products_tags (actual rows=1428.00 loops=1)
               Index Cond: (tags @> '{tea,organic}'::text[])
               Index Searches: 1
(7 rows)

-- The same question for one tag, written with = ANY.
EXPLAIN (COSTS OFF) SELECT count(*) FROM products WHERE 'tea' = ANY (tags);
                 QUERY PLAN                 
--------------------------------------------
 Aggregate
   ->  Seq Scan on products
         Filter: ('tea'::text = ANY (tags))
(3 rows)

EXPLAIN (COSTS OFF) SELECT count(*) FROM products WHERE tags @> ARRAY['tea'];
                     QUERY PLAN                      
-----------------------------------------------------
 Aggregate
   ->  Bitmap Heap Scan on products
         Recheck Cond: (tags @> '{tea}'::text[])
         ->  Bitmap Index Scan on products_tags
               Index Cond: (tags @> '{tea}'::text[])
(5 rows)

-- Every table and index of this design, and their total.
SELECT coalesce(relname, 'total') AS relation, pg_size_pretty(sum(pg_relation_size(oid))) AS size
FROM pg_class WHERE relnamespace = 'public'::regnamespace GROUP BY ROLLUP (relname) ORDER BY relname;
   relation    |  size   
---------------+---------
 products      | 4496 kB
 products_pkey | 1112 kB
 products_tags | 192 kB
 total         | 5800 kB
(4 rows)

-- The same tags as one comma-separated string per product: smaller, but searched by pattern.
CREATE TABLE products_csv AS SELECT id, name, array_to_string(tags, ',') AS tags FROM products;
SELECT 50000
SELECT pg_size_pretty(pg_relation_size('products_csv')) AS csv_table,
       'teapot,gift' LIKE '%tea%' AS teapot_matches_tea;
 csv_table | teapot_matches_tea 
-----------+--------------------
 3072 kB   | t
(1 row)

-- Renaming one tag rewrites every row that carries it.
UPDATE products SET tags = array_replace(tags, 'sale', 'offer') WHERE tags @> ARRAY['sale'];
UPDATE 7143

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 tags_session.py

tags.sql

-- 50,000 products, each with a category tag and one more tag.
CREATE TABLE products (id int PRIMARY KEY, name text NOT NULL, tags text[] NOT NULL);
INSERT INTO products
SELECT n, 'product ' || n,
       ARRAY[(ARRAY['tea', 'coffee', 'snacks', 'spices', 'sweets'])[1 + n % 5],
             (ARRAY['organic', 'gift', 'vegan', 'sale', 'new', 'local', 'imported'])[1 + n % 7]]
FROM generate_series(1, 50000) AS n;
CREATE INDEX products_tags ON products USING gin (tags);
VACUUM ANALYZE products;
-- The operators of array_ops, the default GIN operator class for arrays.
SELECT amopopr::regoperator AS operator FROM pg_amop
WHERE amopfamily = (SELECT opcfamily FROM pg_opclass WHERE opcname = 'array_ops'
                    AND opcmethod = (SELECT oid FROM pg_am WHERE amname = 'gin'))
ORDER BY amopstrategy;
-- Products tagged both tea and organic.
EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
SELECT count(*) FROM products WHERE tags @> ARRAY['tea', 'organic'];
-- The same question for one tag, written with = ANY.
EXPLAIN (COSTS OFF) SELECT count(*) FROM products WHERE 'tea' = ANY (tags);
EXPLAIN (COSTS OFF) SELECT count(*) FROM products WHERE tags @> ARRAY['tea'];
-- Every table and index of this design, and their total.
SELECT coalesce(relname, 'total') AS relation, pg_size_pretty(sum(pg_relation_size(oid))) AS size
FROM pg_class WHERE relnamespace = 'public'::regnamespace GROUP BY ROLLUP (relname) ORDER BY relname;
-- The same tags as one comma-separated string per product: smaller, but searched by pattern.
CREATE TABLE products_csv AS SELECT id, name, array_to_string(tags, ',') AS tags FROM products;
SELECT pg_size_pretty(pg_relation_size('products_csv')) AS csv_table,
       'teapot,gift' LIKE '%tea%' AS teapot_matches_tea;
-- Renaming one tag rewrites every row that carries it.
UPDATE products SET tags = array_replace(tags, 'sale', 'offer') WHERE tags @> ARRAY['sale'];

array_ops holds four operators: && (the two arrays share an element), @> (the left array contains every element of the right one), <@ (the reverse) and =. The containment query went through a Bitmap Index Scan on products_tags and found 1,428 products in 562 heap blocks, the products whose number is a multiple of 35. The whole design takes 5,800 kB: 4,496 kB of table, 1,112 kB of primary key and only 192 kB of GIN index, because the index stores each of the 12 distinct tags once with the rows that carry it. A copy with the tags as comma-separated text took 3,072 kB without any index, less again, but it can be searched only by pattern, and one check of the run shows why that misleads: 'teapot,gift' LIKE '%tea%' is true.

The same question for one tag, written as 'tea' = ANY (tags), scanned the whole table. That condition compares a text value with each element through the = of text, which no array operator class holds; tags @> ARRAY['tea'] asks for the same rows with an operator of array_ops, and the index served it.

= ANY on the array column

WHERE 'tea' = ANY (tags) reads every row even with a GIN index on tags. Write WHERE tags @> ARRAY['tea']: the same rows, asked with an operator of array_ops, which is the form the index serves.

The last statement renamed the tag sale to offer. array_replace() rewrote 7,143 rows, every product that carried the tag, and each new row version put its entries into the GIN index again. An array copies its values into every row that holds them, which costs nothing while the values stay as they are and a rewrite of each copy when one changes.

Checkpoint

Check: GIN on array columns

  1. Question 1 of 2 The run listed the four operators of array_ops, the default GIN class for arrays. Which condition can the index products_tags serve?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: tags @> ARRAY['tea']

    An index serves only the operators of its operator class. For arrays, GIN's array_ops serves && (overlap), @> (contains), <@ (is contained by) and = (equal), each comparing two arrays.

  2. Question 2 of 2 Product n has the tag tea when n % 5 = 0 and the tag organic when n % 7 = 0, for n from 1 to 50,000. How many products does tags @> ARRAY['tea', 'organic'] find?

    Type a number.

    A hint

    Both conditions hold when n is a multiple of 35.

    Show the answer: Question 2 of 2

    Answer: 1428 products

    A product carries both tags when its number is a multiple of both 5 and 7, that is of 35, and 50,000 ÷ 35 is 1,428 and a bit. The recorded plan read exactly those 1,428 rows from the GIN bitmap, in 562 heap blocks.

A join table wins once each tag has a life of its own

The PostgreSQL manual warns that arrays are not sets, and that a need to search them for single elements may point to a missing table with one row per element. The second run builds that design for the same 50,000 products, a tags table of 12 rows and a join table: A table with one row for each pairing of rows from two other tables, such as one row per product and tag, holding a foreign key to each side. Facts about the pairing itself can be columns of it., product_tags, with one row per product and tag, 100,000 rows with a foreign key to each side:

The same tags as a tags table and a join table

jointable_session.py

"""The tags of tags.sql as a join table, on PostgreSQL 18.6: runs jointable.sql on a throwaway cluster."""

from pg_session import Server

# The settings of tags_session.py, so the two plans are read on equal terms.
SETTINGS = {"autovacuum": "off", "max_parallel_workers_per_gather": 0, "jit": "off",
            "default_statistics_target": 1000, "timezone": "'UTC'"}

with Server(settings=SETTINGS) as pg:
    pg.up()
    # One error is expected: the foreign key refuses a tag id that is not in tags.
    pg.psql("jointable.sql", errors=1)

Output

$ psql -X -a -f jointable.sql
-- The same 50,000 products, with their tags as rows of a join table.
CREATE TABLE products (id int PRIMARY KEY, name text NOT NULL);
CREATE TABLE
CREATE TABLE tags (id int PRIMARY KEY, name text NOT NULL UNIQUE);
CREATE TABLE
CREATE TABLE product_tags (
  product_id int NOT NULL REFERENCES products,
  tag_id int NOT NULL REFERENCES tags,
  PRIMARY KEY (product_id, tag_id));
CREATE TABLE
INSERT INTO products SELECT n, 'product ' || n FROM generate_series(1, 50000) AS n;
INSERT 0 50000
INSERT INTO tags (id, name)
SELECT id, name FROM unnest(ARRAY['tea', 'coffee', 'snacks', 'spices', 'sweets', 'organic',
                                  'gift', 'vegan', 'sale', 'new', 'local', 'imported'])
                     WITH ORDINALITY AS t(name, id);
INSERT 0 12
INSERT INTO product_tags SELECT n, 1 + n % 5 FROM generate_series(1, 50000) AS n;
INSERT 0 50000
INSERT INTO product_tags SELECT n, 6 + n % 7 FROM generate_series(1, 50000) AS n;
INSERT 0 50000
CREATE INDEX product_tags_tag ON product_tags (tag_id, product_id);
CREATE INDEX
VACUUM ANALYZE products, tags, product_tags;
VACUUM
-- Products tagged both tea and organic: one row per tag, grouped back per product.
EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
SELECT count(*) FROM (
  SELECT pt.product_id FROM product_tags pt JOIN tags t ON t.id = pt.tag_id
  WHERE t.name IN ('tea', 'organic')
  GROUP BY pt.product_id HAVING count(*) = 2) AS both_tags;
                                                QUERY PLAN                                                 
-----------------------------------------------------------------------------------------------------------
 Aggregate (actual rows=1.00 loops=1)
   ->  HashAggregate (actual rows=1428.00 loops=1)
         Group Key: pt.product_id
         Filter: (count(*) = 2)
         Batches: 1  Memory Usage: 1305kB
         Rows Removed by Filter: 14286
         ->  Nested Loop (actual rows=17142.00 loops=1)
               ->  Seq Scan on tags t (actual rows=2.00 loops=1)
                     Filter: (name = ANY ('{tea,organic}'::text[]))
                     Rows Removed by Filter: 10
               ->  Index Only Scan using product_tags_tag on product_tags pt (actual rows=8571.00 loops=2)
                     Index Cond: (tag_id = t.id)
                     Heap Fetches: 0
                     Index Searches: 2
(14 rows)

-- Every table and index of this design, and their total.
SELECT coalesce(relname, 'total') AS relation, pg_size_pretty(sum(pg_relation_size(oid))) AS size
FROM pg_class WHERE relnamespace = 'public'::regnamespace GROUP BY ROLLUP (relname) ORDER BY relname;
     relation      |    size    
-------------------+------------
 product_tags      | 3544 kB
 product_tags_pkey | 3312 kB
 product_tags_tag  | 2208 kB
 products          | 2544 kB
 products_pkey     | 1112 kB
 tags              | 8192 bytes
 tags_name_key     | 16 kB
 tags_pkey         | 16 kB
 total             | 12 MB
(9 rows)

-- A tag that does not exist is refused by the foreign key.
INSERT INTO product_tags VALUES (1, 99);
psql:jointable.sql:27: ERROR:  insert or update on table "product_tags" violates foreign key constraint "product_tags_tag_id_fkey"
DETAIL:  Key (tag_id)=(99) is not present in table "tags".
-- Renaming a tag changes one row.
UPDATE tags SET name = 'offer' WHERE name = 'sale';
UPDATE 1

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 jointable_session.py

jointable.sql

-- The same 50,000 products, with their tags as rows of a join table.
CREATE TABLE products (id int PRIMARY KEY, name text NOT NULL);
CREATE TABLE tags (id int PRIMARY KEY, name text NOT NULL UNIQUE);
CREATE TABLE product_tags (
  product_id int NOT NULL REFERENCES products,
  tag_id int NOT NULL REFERENCES tags,
  PRIMARY KEY (product_id, tag_id));
INSERT INTO products SELECT n, 'product ' || n FROM generate_series(1, 50000) AS n;
INSERT INTO tags (id, name)
SELECT id, name FROM unnest(ARRAY['tea', 'coffee', 'snacks', 'spices', 'sweets', 'organic',
                                  'gift', 'vegan', 'sale', 'new', 'local', 'imported'])
                     WITH ORDINALITY AS t(name, id);
INSERT INTO product_tags SELECT n, 1 + n % 5 FROM generate_series(1, 50000) AS n;
INSERT INTO product_tags SELECT n, 6 + n % 7 FROM generate_series(1, 50000) AS n;
CREATE INDEX product_tags_tag ON product_tags (tag_id, product_id);
VACUUM ANALYZE products, tags, product_tags;
-- Products tagged both tea and organic: one row per tag, grouped back per product.
EXPLAIN (ANALYZE, COSTS OFF, TIMING OFF, SUMMARY OFF, BUFFERS OFF)
SELECT count(*) FROM (
  SELECT pt.product_id FROM product_tags pt JOIN tags t ON t.id = pt.tag_id
  WHERE t.name IN ('tea', 'organic')
  GROUP BY pt.product_id HAVING count(*) = 2) AS both_tags;
-- Every table and index of this design, and their total.
SELECT coalesce(relname, 'total') AS relation, pg_size_pretty(sum(pg_relation_size(oid))) AS size
FROM pg_class WHERE relnamespace = 'public'::regnamespace GROUP BY ROLLUP (relname) ORDER BY relname;
-- A tag that does not exist is refused by the foreign key.
INSERT INTO product_tags VALUES (1, 99);
-- Renaming a tag changes one row.
UPDATE tags SET name = 'offer' WHERE name = 'sale';

The same question now needs a grouping. The plan read 17,142 rows of product_tags through an index-only scan of product_tags_tag, 8,571 for each tag on average, grouped them by product and kept the 1,428 products that had both tags, after the filter removed 14,286 that had only one. The design takes 12 MB against the array’s 5,800 kB, because every pair is a row of its own with two B-trees over it.

In return it does three things the array could not. The foreign key refused tag 99, which does not exist, since a foreign key checks that the values of a column match a row of the other table, and an array’s value is the whole array. The rename was UPDATE 1 in tags instead of 7,143 rewritten products. And a tag can have columns of its own, a display name or the team that owns it, in one row that every product refers to.

Which design fits depends on what the application asks of its tags. Answer the five questions of the table for a list of your own:

Interactive · answer and compare Where should a product's tags live? Answer five questions about your list of tags and the table rules out the designs that cannot do what you need.
Do you look up the items that carry a given tag?
Does each tag need facts of its own, such as a display name or the team that owns it?
Do tags change apart from the rows that carry them, by being renamed or retired?
Must every tag come from a list of allowed tags that keeps growing?
How many tags can one row hold?

Answer the questions above: the table then marks what fits your case and says why the rest does not.

Three designs for the tags of 50,000 products, against five things a catalogue asks of them
Design Items with tags A and B Space on disk Only tags that exist Facts about each tag Renaming a tag
Comma-separated text Stored as: tags text, 'tea,organic' Poor only patterns, and LIKE '%tea%' matched teapot too Good 3,072 kB without an index, the smallest of the three Poor every spelling is stored as it was typed Poor nowhere to keep them Poor string surgery on every row that holds the tag
Array column Stored as: tags text[] with a GIN index Good tags @> ARRAY['tea', 'organic'] read 1,428 rows from the GIN bitmap Good 5,800 kB in all, the GIN index 192 kB of it Poor a foreign key compares whole values, never each element Poor an element is only a string Poor UPDATE 7143: every row that carries the tag
Join table Stored as: tags and product_tags Mixed GROUP BY with HAVING count(*) = 2 grouped 17,142 rows Mixed 12 MB in all: 100,000 pairs and two B-trees on them Good the foreign key refused tag 99 Good columns on tags, one row per tag Good UPDATE 1, in tags
How the answers decide
  • “Do you look up the items that carry a given tag?” Yes, by tag: rules out Comma-separated text. A string can be searched only by pattern, and LIKE '%tea%' matched teapot; an array or a join table answers exactly.
  • “Does each tag need facts of its own, such as a display name or the team that owns it?” Yes, facts per tag: rules out Comma-separated text and Array column. Facts about each tag need a row per tag, which a table of tags holds and an array element or a string cannot.
  • “Do tags change apart from the rows that carry them, by being renamed or retired?” Yes, they have a life of their own: rules out Comma-separated text and adds a caveat to Array column. Renaming one tag rewrote 7,143 product rows in the array design and one row of tags in the join design; a string needs surgery in every row.
  • “Must every tag come from a list of allowed tags that keeps growing?” Yes, a growing list: rules out Comma-separated text and Array column. Only a join table can point at a table of allowed tags with a foreign key; a CHECK with <@ holds an array to a list written into it.
  • “How many tags can one row hold?” Hundreds or more: rules out Comma-separated text and adds a caveat to Array column. Every change rewrites the whole value, an array or a string; for many elements the manual suggests a table with one row per item.
  • “Does each tag need facts of its own, such as a display name or the team that owns it?” No, the word is enough and “Do tags change apart from the rows that carry them, by being renamed or retired?” No, they change with the row: adds a caveat to Join table. Nothing here needs a row per tag, so a join table works but costs more: 12 MB against 5,800 kB, and a grouping for every two-tag search.

In short: a short list that is read and written with its row and searched by containment belongs in an array with a GIN index, and items with facts, references or changes of their own belong in rows. A comma-separated text column does neither job well.

Checkpoint

Check: Array column or join table

  1. Question 1 of 2 Each tag now needs a display name and the team that owns it. Which design holds that without a workaround?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: A table of tags with those columns, and a join table of product and tag ids

    When an item of a list has facts of its own, it becomes a row of its own table. That is the point at which a join table beats an array column, whatever the query sizes.

  2. Question 2 of 2 Renaming the tag sale to offer, how many rows did each design change in the runs?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: 7,143 product rows in the array design, 1 row of tags in the join design

    An array copies each value into every row that has it. A join table stores the value once and refers to it, so a change to the value is one row.

Arrays turn into rows, and PostgreSQL 18 sorts them

Arrays meet ordinary SQL in two directions. A list of values can go into a query as one array, WHERE id = ANY (ARRAY[17, 42, 99]), which a driver binds as a single parameter however long the list; psycopg 3 adapts a Python list to an array and recommends = ANY (%s) over IN, which also works for an empty list. In the other direction unnest() turns an array into rows, and WITH ORDINALITY adds each element’s position. The third run keeps the items of 10,000 baskets in the order they were added:

ANY, unnest and the new array functions of PostgreSQL 18

arrayfns_session.py

"""Array functions on PostgreSQL 18.6, including two new in 18: runs arrayfns.sql on a throwaway cluster."""

from pg_session import Server

# No autovacuum and no parallel workers, so the plans and counts are the same on every run; UTC as the time zone.
SETTINGS = {"autovacuum": "off", "max_parallel_workers_per_gather": 0, "jit": "off", "timezone": "'UTC'"}

with Server(settings=SETTINGS) as pg:
    pg.up()
    pg.psql("arrayfns.sql", errors=0)

Output

$ psql -X -a -f arrayfns.sql
-- 10,000 baskets, each with its items in the order they were added.
CREATE TABLE baskets (id int PRIMARY KEY, items text[] NOT NULL);
CREATE TABLE
INSERT INTO baskets
SELECT n, CASE n % 3 WHEN 0 THEN ARRAY['tea', 'spices', 'tea', 'sugar']
                     WHEN 1 THEN ARRAY['coffee', 'sweets'] ELSE '{}' END
FROM generate_series(1, 10000) AS n;
INSERT 0 10000
VACUUM ANALYZE baskets;
VACUUM
-- A list of ids: = ANY (array) and IN (list) use the same B-tree.
EXPLAIN (COSTS OFF) SELECT id FROM baskets WHERE id = ANY (ARRAY[17, 42, 99]);
                     QUERY PLAN                     
----------------------------------------------------
 Index Only Scan using baskets_pkey on baskets
   Index Cond: (id = ANY ('{17,42,99}'::integer[]))
(2 rows)

EXPLAIN (COSTS OFF) SELECT id FROM baskets WHERE id IN (17, 42, 99);
                     QUERY PLAN                     
----------------------------------------------------
 Index Only Scan using baskets_pkey on baskets
   Index Cond: (id = ANY ('{17,42,99}'::integer[]))
(2 rows)

-- An array as numbered rows.
SELECT b.id, i.item, i.pos
FROM baskets AS b, unnest(b.items) WITH ORDINALITY AS i(item, pos) WHERE b.id = 3;
 id |  item  | pos 
----+--------+-----
  3 | tea    |   1
  3 | spices |   2
  3 | tea    |   3
  3 | sugar  |   4
(4 rows)

-- New in PostgreSQL 18: array_sort(), array_reverse(), min() and max() of arrays.
SELECT items, array_sort(items) AS sorted, array_sort(items, true) AS descending,
       array_reverse(items) AS reversed FROM baskets WHERE id = 3;
         items          |         sorted         |       descending       |        reversed        
------------------------+------------------------+------------------------+------------------------
 {tea,spices,tea,sugar} | {spices,sugar,tea,tea} | {tea,tea,sugar,spices} | {sugar,tea,spices,tea}
(1 row)

SELECT min(items), max(items) FROM baskets;
 min |          max           
-----+------------------------
 {}  | {tea,spices,tea,sugar}
(1 row)

-- An empty array has no dimensions and so no length, but its cardinality is 0.
SELECT array_ndims('{}'::text[]) AS dimensions, array_length('{}'::text[], 1) AS length,
       cardinality('{}'::text[]) AS cardinality;
 dimensions | length | cardinality 
------------+--------+-------------
            |        |           0
(1 row)

SELECT count(*) FILTER (WHERE array_length(items, 1) = 0) AS by_length,
       count(*) FILTER (WHERE cardinality(items) = 0) AS by_cardinality FROM baskets;
 by_length | by_cardinality 
-----------+----------------
         0 |           3333
(1 row)

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 arrayfns_session.py

arrayfns.sql

-- 10,000 baskets, each with its items in the order they were added.
CREATE TABLE baskets (id int PRIMARY KEY, items text[] NOT NULL);
INSERT INTO baskets
SELECT n, CASE n % 3 WHEN 0 THEN ARRAY['tea', 'spices', 'tea', 'sugar']
                     WHEN 1 THEN ARRAY['coffee', 'sweets'] ELSE '{}' END
FROM generate_series(1, 10000) AS n;
VACUUM ANALYZE baskets;
-- A list of ids: = ANY (array) and IN (list) use the same B-tree.
EXPLAIN (COSTS OFF) SELECT id FROM baskets WHERE id = ANY (ARRAY[17, 42, 99]);
EXPLAIN (COSTS OFF) SELECT id FROM baskets WHERE id IN (17, 42, 99);
-- An array as numbered rows.
SELECT b.id, i.item, i.pos
FROM baskets AS b, unnest(b.items) WITH ORDINALITY AS i(item, pos) WHERE b.id = 3;
-- New in PostgreSQL 18: array_sort(), array_reverse(), min() and max() of arrays.
SELECT items, array_sort(items) AS sorted, array_sort(items, true) AS descending,
       array_reverse(items) AS reversed FROM baskets WHERE id = 3;
SELECT min(items), max(items) FROM baskets;
-- An empty array has no dimensions and so no length, but its cardinality is 0.
SELECT array_ndims('{}'::text[]) AS dimensions, array_length('{}'::text[], 1) AS length,
       cardinality('{}'::text[]) AS cardinality;
SELECT count(*) FILTER (WHERE array_length(items, 1) = 0) AS by_length,
       count(*) FILTER (WHERE cardinality(items) = 0) AS by_cardinality FROM baskets;

Both lists of ids ran as the same plan, an index-only scan of the primary key with Index Cond: (id = ANY ('{17,42,99}'::integer[])): PostgreSQL turns an IN list of constants into an array, and the B-tree searches it value by value. For basket 3, unnest() returned tea, spices, tea and sugar as four rows numbered 1 to 4, the repeated tea kept in both of its places.

PostgreSQL 18 adds array_sort(), which sorted that basket to {spices,sugar,tea,tea}, and with true as its second argument to {tea,tea,sugar,spices}, and array_reverse(), which only turned it round, to {sugar,tea,spices,tea}. min() and max() now take arrays too and compare them element by element: the smallest basket was the empty one, {}, and the largest the one that starts with tea.

New in PostgreSQL 18

array_sort(), array_reverse(), and min() and max() over arrays and composite types are new in PostgreSQL 18. On older servers, sort an array with unnest() and array_agg(... ORDER BY ...).

The last two queries show the trap of empty arrays. array_ndims() returned NULL for '{}': an empty array has no dimensions, so array_length(items, 1) returned NULL for it too, and the filter array_length(items, 1) = 0 counted none of the 3,333 empty baskets, while cardinality(items), the number of elements, returned 0 and counted all of them.

Testing for an empty array with array_length()

WHERE array_length(items, 1) = 0 never matches, because the length of a dimension that an empty array does not have is NULL. Write WHERE cardinality(items) = 0, or compare with the empty array, items = '{}'.

Checkpoint

Check: ANY, unnest and array functions

  1. Question 1 of 2 What did the planner turn WHERE id IN (17, 42, 99) into?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: id = ANY ('{17,42,99}'::integer[]), searched in the primary key's B-tree

    A list of constants after IN becomes an array, and = ANY (array) is a condition a B-tree can search, value by value, in one index scan.

  2. Question 2 of 2 3,333 baskets have an empty items array. Which condition finds them?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: cardinality(items) = 0

    Test emptiness with cardinality(items) = 0 or items = '{}'. array_length() answers NULL for a dimension the array does not have, and NULL = 0 is not true.

A domain checks a value once for every column of the type

A domain: A data type made from another type with constraints, NOT NULL and a default added. A value is checked when it is converted to the domain, so every column, cast and argument of the type follows one rule. is a data type made from another one with constraints added. CREATE DOMAIN pin_code AS text CHECK (VALUE ~ '^[1-9][0-9]{5}$') is text that must be six digits, the first of them 1 to 9, and every column declared as pin_code gets that rule, as does every cast to the type, so the rule is written once however many tables hold PIN codes. Domains suit any value with one rule and many homes: postal codes, phone numbers, amounts that must be positive.

A six-digit PIN code

India’s postal code, the PIN code, has six digits written after the name of the locality. In the Universal Postal Union’s sheet for India, the first digit stands for the state or province, and the digits after it lead on to the sorting and forwarding centre and then to the delivery office. pin_code also refuses a first digit of 0, as the contacts table of From app code: COPY, pipelining, RETURNING does with an integer column and a range check instead of a pattern.

The fourth run uses the domain in two tables, customers and stores, and writes one good code and one bad one:

One pin_code domain for customers and stores

pincode_session.py

"""A domain for PIN codes on PostgreSQL 18.6: runs pincode.sql on a throwaway cluster."""

from pg_session import Server

SETTINGS = {"autovacuum": "off", "timezone": "'UTC'"}

with Server(settings=SETTINGS) as pg:
    pg.up()
    # One error is expected: the store's PIN code starts with 0, which the domain's CHECK refuses.
    pg.psql("pincode.sql", errors=1)

Output

$ psql -X -a -f pincode.sql
-- A PIN code: six digits, the first of them 1 to 9.
CREATE DOMAIN pin_code AS text CHECK (VALUE ~ '^[1-9][0-9]{5}$');
CREATE DOMAIN
CREATE TABLE customers (id int PRIMARY KEY, name text NOT NULL, pin pin_code NOT NULL);
CREATE TABLE
CREATE TABLE stores (id int PRIMARY KEY, city text NOT NULL, pin pin_code NOT NULL);
CREATE TABLE
INSERT INTO customers VALUES (1, 'Asha Rao', '411001');
INSERT 0 1
INSERT INTO stores VALUES (1, 'Pune', '011001');
psql:pincode.sql:6: ERROR:  value for domain pin_code violates check constraint "pin_code_check"
-- Test candidate values without raising an error.
SELECT v AS value, pg_input_is_valid(v, 'pin_code') AS valid
FROM (VALUES ('411001'), ('011001'), ('41100'), ('4110012'), ('41l001'), (' 411001')) AS t(v);
  value  | valid 
---------+-------
 411001  | t
 011001  | f
 41100   | f
 4110012 | f
 41l001  | f
  411001 | f
(6 rows)

SELECT message, sql_error_code FROM pg_input_error_info('011001', 'pin_code');
                               message                                | sql_error_code 
----------------------------------------------------------------------+----------------
 value for domain pin_code violates check constraint "pin_code_check" | 23514
(1 row)

-- One rule, two columns: the catalog knows every column of the domain.
SELECT table_name, column_name FROM information_schema.columns
WHERE domain_name = 'pin_code' ORDER BY table_name;
 table_name | column_name 
------------+-------------
 customers  | pin
 stores     | pin
(2 rows)

SELECT conname, pg_get_constraintdef(oid) FROM pg_constraint WHERE contypid = 'pin_code'::regtype;
    conname     |           pg_get_constraintdef            
----------------+-------------------------------------------
 pin_code_check | CHECK ((VALUE ~ '^[1-9][0-9]{5}$'::text))
(1 row)

-- Text operators work on the domain, and their results are plain text.
SELECT pg_typeof(pin) AS stored, pg_typeof(pin || '-1') AS concatenated FROM customers;
  stored  | concatenated 
----------+--------------
 pin_code | text
(1 row)

-- The domain itself allows NULL; NOT NULL belongs on each column.
SELECT NULL::pin_code IS NULL AS null_is_allowed;
 null_is_allowed 
-----------------
 t
(1 row)

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 pincode_session.py

pincode.sql

-- A PIN code: six digits, the first of them 1 to 9.
CREATE DOMAIN pin_code AS text CHECK (VALUE ~ '^[1-9][0-9]{5}$');
CREATE TABLE customers (id int PRIMARY KEY, name text NOT NULL, pin pin_code NOT NULL);
CREATE TABLE stores (id int PRIMARY KEY, city text NOT NULL, pin pin_code NOT NULL);
INSERT INTO customers VALUES (1, 'Asha Rao', '411001');
INSERT INTO stores VALUES (1, 'Pune', '011001');
-- Test candidate values without raising an error.
SELECT v AS value, pg_input_is_valid(v, 'pin_code') AS valid
FROM (VALUES ('411001'), ('011001'), ('41100'), ('4110012'), ('41l001'), (' 411001')) AS t(v);
SELECT message, sql_error_code FROM pg_input_error_info('011001', 'pin_code');
-- One rule, two columns: the catalog knows every column of the domain.
SELECT table_name, column_name FROM information_schema.columns
WHERE domain_name = 'pin_code' ORDER BY table_name;
SELECT conname, pg_get_constraintdef(oid) FROM pg_constraint WHERE contypid = 'pin_code'::regtype;
-- Text operators work on the domain, and their results are plain text.
SELECT pg_typeof(pin) AS stored, pg_typeof(pin || '-1') AS concatenated FROM customers;
-- The domain itself allows NULL; NOT NULL belongs on each column.
SELECT NULL::pin_code IS NULL AS null_is_allowed;

411001 went into customers. 011001, with its leading 0, stopped the store’s insert with value for domain pin_code violates check constraint "pin_code_check", and the catalog lists one constraint, pin_code_check, for the two columns customers.pin and stores.pin. Follow both values through the conversion to the domain:

Interactive · step by step Two PIN codes meet the pin_code domain Step through what happens to a good and a bad PIN code on their way into a pin_code column, and how to test a value first.
Two PIN codes meet the pin_code domain, step 7 of 7. Showing A text value, pg_input_is_valid, Cast to domain, Domain CHECK, ERROR 23514, Row is written (both tables). In focus: Domain CHECK.true or NULLfalseA text value411001, 011001pg_input_is_validno error raisedCast to domainon INSERT or ::Domain CHECK^[1-9][0-9]{5}$ERROR 23514no row writtenRow is writtenboth tables

The drawing shows the whole scene; its 7 steps are listed below.

Every step as text
  1. Two values arrive. A form sends 411001 for a customer and 011001 for a store. Both are valid text, so nothing has checked them yet.
  2. Conversion runs the rule. Writing a value into a pin_code column converts it to the domain, and so does a cast with ::pin_code. Each conversion evaluates the domain's CHECK, with VALUE standing for the value being converted.
  3. 411001 matches. The pattern asks for six digits, the first of them 1 to 9. 411001 matches, the CHECK is true and the row is written, INSERT 0 1.
  4. 011001 is refused. 011001 starts with 0, so the CHECK is false. The INSERT stops with value for domain pin_code violates check constraint pin_code_check, SQLSTATE 23514, and the store's row is not written.
  5. NULL is no violation. A CHECK that comes out NULL counts as success, so NULL::pin_code is accepted. The manual advises allowing NULL in the domain and putting NOT NULL on each column, as both tables of the run do.
  6. Test a value first. pg_input_is_valid() runs the same conversion and returns false instead of raising the error, and pg_input_error_info() gives the message and the code 23514. Of six candidates, only 411001 came back true.
  7. One rule for two tables. customers.pin and stores.pin both use the one constraint pin_code_check, so a change to the rule is one ALTER DOMAIN, and every column of the type follows it.

Two details of the run decide how a domain is written. NULL::pin_code was accepted, because a CHECK that comes out NULL counts as success; the manual advises leaving NULL allowed in the domain and putting NOT NULL on each column, as both tables do, because an outer join or an empty subquery can produce a NULL of the domain type anyway. And pin || '-1' was plain text: an operator of the base type sees the domain value as text, and only a conversion back to the domain checks the rule again.

pg_input_is_valid() tests a candidate without raising an error, which suits a form that wants to say which field is wrong: of the six values the run tried, only 411001 was valid, and pg_input_error_info() returned the message and SQLSTATE 23514, check_violation. A later change of the rule is one ALTER DOMAIN ... ADD CONSTRAINT, which checks every stored value of every column of the type, or, added NOT VALID, leaves the stored values until ALTER DOMAIN ... VALIDATE CONSTRAINT reads them. One cost comes with the checks: adding a column of a domain that has constraints to an existing table rewrites the table and its indexes, where a plain column without a default does not.

Key idea

A domain moves a rule from each column to the type: every column, cast and function argument of the type checks it, and one ALTER DOMAIN changes it everywhere.

Checkpoint

Check: Domains and their checks

  1. Question 1 of 2 Which value does the domain pin_code, CHECK (VALUE ~ '^[1-9][0-9]{5}$'), accept?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: '560034'

    The pattern is anchored at both ends: one digit from 1 to 9, then five digits, and nothing else. pg_input_is_valid() gave true for 411001 and false for the five other candidates of the run.

  2. Question 2 of 2 A table has a column pin pin_code without NOT NULL, and a row is inserted with NULL in it. What happens?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: The row is written, because a CHECK that comes out NULL counts as success

    Write the domain so that it allows NULL and put NOT NULL on the columns that must have a value, as customers.pin and stores.pin do.

Enums keep their declared order, and a new value waits for a commit

An enum type: A data type with a fixed list of labels, made with CREATE TYPE ... AS ENUM. Values take 4 bytes and compare in the order the labels were declared; labels can be added and renamed, never removed. is a fixed list of labels, such as the stages of an order. Each value takes 4 bytes in a row, whatever the length of its label; labels are case-sensitive and, in a standard build, at most 63 bytes long, and values compare in the order they were declared. Enums suit a short list that changes rarely and mostly grows. The fifth run creates order_status with pending, paid, shipped and cancelled, then adds a value, renames one and tries to drop one:

Ordering and changing the order_status enum

enum_session.py

"""Changing an enum type on PostgreSQL 18.6: runs enum.sql on a throwaway cluster."""

from pg_session import Server

SETTINGS = {"autovacuum": "off", "timezone": "'UTC'"}

with Server(settings=SETTINGS) as pg:
    pg.up()
    # Two errors are expected: the new value used before its transaction commits, and DROP VALUE, which does not exist.
    pg.psql("enum.sql", errors=2)

Output

$ psql -X -a -f enum.sql
-- Four statuses, in the order an order moves through them.
CREATE TYPE order_status AS ENUM ('pending', 'paid', 'shipped', 'cancelled');
CREATE TYPE
CREATE TABLE orders (id int PRIMARY KEY, status order_status NOT NULL);
CREATE TABLE
INSERT INTO orders VALUES (1, 'shipped'), (2, 'pending'), (3, 'cancelled'), (4, 'paid');
INSERT 0 4
-- Enum values sort in the order they were declared, not alphabetically.
SELECT id, status FROM orders ORDER BY status;
 id |  status   
----+-----------
  2 | pending
  4 | paid
  1 | shipped
  3 | cancelled
(4 rows)

SELECT id, status FROM orders WHERE status > 'paid' ORDER BY id;
 id |  status   
----+-----------
  1 | shipped
  3 | cancelled
(2 rows)

-- A new value inside a transaction cannot be used before that transaction commits.
BEGIN;
BEGIN
ALTER TYPE order_status ADD VALUE 'refunded' BEFORE 'cancelled';
ALTER TYPE
UPDATE orders SET status = 'refunded' WHERE id = 3;
psql:enum.sql:11: ERROR:  unsafe use of new value "refunded" of enum type order_status
LINE 1: UPDATE orders SET status = 'refunded' WHERE id = 3;
                                   ^
HINT:  New enum values must be committed before they can be used.
ROLLBACK;
ROLLBACK
-- Committed first, it can be used at once, and the table keeps its data file.
SELECT pg_relation_filenode('orders') AS file_before \gset
ALTER TYPE order_status ADD VALUE 'refunded' BEFORE 'cancelled';
ALTER TYPE
SELECT pg_relation_filenode('orders') = :file_before AS same_file;
 same_file 
-----------
 t
(1 row)

UPDATE orders SET status = 'refunded' WHERE id = 3;
UPDATE 1
SELECT enum_range(NULL::order_status);
                enum_range                 
-------------------------------------------
 {pending,paid,shipped,refunded,cancelled}
(1 row)

SELECT enumlabel, enumsortorder FROM pg_enum
WHERE enumtypid = 'order_status'::regtype ORDER BY enumsortorder;
 enumlabel | enumsortorder 
-----------+---------------
 pending   |             1
 paid      |             2
 shipped   |             3
 refunded  |           3.5
 cancelled |             4
(5 rows)

-- A value can be renamed, but there is no command to drop one.
ALTER TYPE order_status RENAME VALUE 'shipped' TO 'dispatched';
ALTER TYPE
ALTER TYPE order_status DROP VALUE 'refunded';
psql:enum.sql:23: ERROR:  dropping an enum value is not implemented
LINE 1: ALTER TYPE order_status DROP VALUE 'refunded';
                                ^
SELECT enum_range(NULL::order_status);
                  enum_range                  
----------------------------------------------
 {pending,paid,dispatched,refunded,cancelled}
(1 row)

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 enum_session.py

enum.sql

-- Four statuses, in the order an order moves through them.
CREATE TYPE order_status AS ENUM ('pending', 'paid', 'shipped', 'cancelled');
CREATE TABLE orders (id int PRIMARY KEY, status order_status NOT NULL);
INSERT INTO orders VALUES (1, 'shipped'), (2, 'pending'), (3, 'cancelled'), (4, 'paid');
-- Enum values sort in the order they were declared, not alphabetically.
SELECT id, status FROM orders ORDER BY status;
SELECT id, status FROM orders WHERE status > 'paid' ORDER BY id;
-- A new value inside a transaction cannot be used before that transaction commits.
BEGIN;
ALTER TYPE order_status ADD VALUE 'refunded' BEFORE 'cancelled';
UPDATE orders SET status = 'refunded' WHERE id = 3;
ROLLBACK;
-- Committed first, it can be used at once, and the table keeps its data file.
SELECT pg_relation_filenode('orders') AS file_before \gset
ALTER TYPE order_status ADD VALUE 'refunded' BEFORE 'cancelled';
SELECT pg_relation_filenode('orders') = :file_before AS same_file;
UPDATE orders SET status = 'refunded' WHERE id = 3;
SELECT enum_range(NULL::order_status);
SELECT enumlabel, enumsortorder FROM pg_enum
WHERE enumtypid = 'order_status'::regtype ORDER BY enumsortorder;
-- A value can be renamed, but there is no command to drop one.
ALTER TYPE order_status RENAME VALUE 'shipped' TO 'dispatched';
ALTER TYPE order_status DROP VALUE 'refunded';
SELECT enum_range(NULL::order_status);

ORDER BY status returned pending, paid, shipped, cancelled, the order of the declaration and not of the alphabet, and status > 'paid' found the shipped and the cancelled order. Step through the changes that came next:

Interactive · step by step Adding refunded to order_status Step through the run's changes to the enum and watch the catalog row of the new label, the statement that uses it too early, and the one that works.
Adding refunded to order_status, step 6 of 6. Showing ALTER TYPE (committed alone), pg_enum (shipped renamed), UPDATE orders (next statement), Result (UPDATE 1). In focus: pg_enum.ALTER TYPEcommitted alonepg_enumshipped renamedUPDATE ordersnext statementResultUPDATE 1

The drawing shows the whole scene; its 6 steps are listed below.

Every step as text
  1. Four labels, in order. order_status starts with pending, paid, shipped and cancelled, stored in the pg_enum catalog with sort orders 1 to 4. ORDER BY status follows those numbers, not the alphabet, and status > paid found the shipped and the cancelled order.
  2. ADD VALUE inside BEGIN. Inside BEGIN, ADD VALUE 'refunded' BEFORE 'cancelled' writes a fifth pg_enum row with sort order 3.5, halfway between shipped and cancelled. Until the transaction commits, no other session sees it.
  3. Used too early. The UPDATE that moves order 3 to refunded runs in the same transaction and fails with unsafe use of new value, and the hint says that new enum values must be committed before they can be used.
  4. ROLLBACK takes it away. The error aborted the transaction, so ROLLBACK was the only way on, and it removed the new label as well. The type was back to its four values.
  5. Commit first, then use. Run on its own, outside BEGIN, ALTER TYPE commits at once and the table keeps its data file, so no row is rewritten. The next statement may use the label, UPDATE 1, and enum_range() then listed pending, paid, shipped, refunded, cancelled.
  6. Rename it, never drop it. RENAME VALUE turned shipped into dispatched in place. DROP VALUE stopped with dropping an enum value is not implemented, so a label stays in the type once it is added.

ADD VALUE 'refunded' BEFORE 'cancelled' put the new label between shipped and cancelled. Added inside a transaction block, a value cannot be used until that transaction has committed, so the UPDATE in the same transaction failed with unsafe use of new value, and the ROLLBACK that the error forced took the new label away as well. Run on its own, outside BEGIN, the ALTER TYPE committed at once, the table kept the same data file, so no row was rewritten, and the next statement could use refunded. The system catalog: One of the tables in the pg_catalog schema where PostgreSQL records its objects, such as pg_class for relations, pg_attribute for columns and pg_constraint for constraints. They can be queried like any table. More in “System catalogs, pages and TOAST” pg_enum shows how the place was found: the new label got sort order 3.5, halfway between its neighbours, and the manual notes that comparisons involving a value placed this way can be a little slower than comparisons of the original values.

Two changes cannot be made to an enum at all. A value can be renamed, as shipped became dispatched in its old place, but not removed: DROP VALUE stopped with dropping an enum value is not implemented. Nor can the existing values be put in another order, short of dropping and re-creating the type. A list that must also shrink therefore fits better as rows of a lookup table, referred to by a foreign key, or as text held to a CHECK constraint.

Drivers see enums as strings: psycopg 3, for one, loads an enum value as a Python string without being told about the type, but an array of an enum type only after the program registers the array’s type.

Checkpoint

Check: Enum order and new values

  1. Question 1 of 2 order_status was declared as ('pending', 'paid', 'shipped', 'cancelled'). Which statuses does status > 'paid' select?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: shipped and cancelled

    An enum's order is the order of its labels when the type was created, with any later value placed where ADD VALUE put it. ORDER BY status in the run gave pending, paid, shipped, cancelled.

  2. Question 2 of 2 Inside BEGIN, ALTER TYPE order_status ADD VALUE 'refunded' BEFORE 'cancelled' succeeded. What does the next statement of the same transaction, an UPDATE that sets a status to refunded, do?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: It fails with unsafe use of new value, and the transaction is aborted

    After the error the only way on was ROLLBACK, which removed the new value too. Run on its own, ALTER TYPE commits at once, and the next statement could use refunded.

A composite value travels as one value with named fields

A composite type: A data type made of named fields, each with its own type, like the columns of a row. Every table has one under its own name, and CREATE TYPE ... AS (...) makes a standalone one. is a list of named fields with their types, the shape of a row. Every table has one under its own name, and CREATE TYPE order_line AS (item text, price numeric(10,2), qty int) makes a standalone one. A row constructor: An expression that builds a composite value from its fields, written ROW(a, b, c) or, with two fields or more, (a, b, c), often cast to a named composite type. such as ROW('Masala tea', 120.00, 2) builds a value of it, and the key word ROW may be left out when there are two fields or more. The last run stores one line per order, with prices in rupees, and reads it in several ways:

An order_line composite in a table and in a function

composite_session.py

"""A composite type and ROW values on PostgreSQL 18.6: runs composite.sql on a throwaway cluster."""

from pg_session import Server

SETTINGS = {"autovacuum": "off", "timezone": "'UTC'"}

with Server(settings=SETTINGS) as pg:
    pg.up()
    # One error is expected: line.price without parentheses, which the parser reads as a table's column.
    pg.psql("composite.sql", errors=1)

Output

$ psql -X -a -f composite.sql
-- One line of an order as a composite type: an item, its price in rupees, a quantity.
CREATE TYPE order_line AS (item text, price numeric(10,2), qty int);
CREATE TYPE
CREATE TABLE orders (id int PRIMARY KEY, line order_line NOT NULL);
CREATE TABLE
INSERT INTO orders VALUES (1, ROW('Masala tea', 120.00, 2)), (2, ('Coffee, filter', 250, 1)),
                          (3, ROW('Cardamom', 90.50, 3));
INSERT 0 3
-- The whole value prints in its text form, with quotes where a field needs them.
SELECT id, line FROM orders ORDER BY id;
 id |            line             
----+-----------------------------
  1 | ("Masala tea",120.00,2)
  2 | ("Coffee, filter",250.00,1)
  3 | (Cardamom,90.50,3)
(3 rows)

-- Without parentheses, line is read as a table name.
SELECT line.price FROM orders;
psql:composite.sql:9: ERROR:  missing FROM-clause entry for table "line"
LINE 1: SELECT line.price FROM orders;
               ^
SELECT id, (line).item, (line).price * (line).qty AS total FROM orders ORDER BY id;
 id |      item      | total  
----+----------------+--------
  1 | Masala tea     | 240.00
  2 | Coffee, filter | 250.00
  3 | Cardamom       | 271.50
(3 rows)

-- A function takes the whole line as one argument.
CREATE FUNCTION line_total(l order_line) RETURNS numeric
LANGUAGE sql IMMUTABLE RETURN (l).price * (l).qty;
CREATE FUNCTION
SELECT line_total(ROW('Ginger', 40, 5)::order_line) AS ginger;
 ginger 
--------
 200.00
(1 row)

-- New in PostgreSQL 18: max() of a composite compares field by field.
SELECT max(line) FROM orders;
           max           
-------------------------
 ("Masala tea",120.00,2)
(1 row)

-- (line).* hands a client plain columns instead of one text value.
SELECT o.id, (o.line).* FROM orders AS o WHERE o.id = 2;
 id |      item      | price  | qty 
----+----------------+--------+-----
  2 | Coffee, filter | 250.00 |   1
(1 row)

Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 composite_session.py

composite.sql

-- One line of an order as a composite type: an item, its price in rupees, a quantity.
CREATE TYPE order_line AS (item text, price numeric(10,2), qty int);
CREATE TABLE orders (id int PRIMARY KEY, line order_line NOT NULL);
INSERT INTO orders VALUES (1, ROW('Masala tea', 120.00, 2)), (2, ('Coffee, filter', 250, 1)),
                          (3, ROW('Cardamom', 90.50, 3));
-- The whole value prints in its text form, with quotes where a field needs them.
SELECT id, line FROM orders ORDER BY id;
-- Without parentheses, line is read as a table name.
SELECT line.price FROM orders;
SELECT id, (line).item, (line).price * (line).qty AS total FROM orders ORDER BY id;
-- A function takes the whole line as one argument.
CREATE FUNCTION line_total(l order_line) RETURNS numeric
LANGUAGE sql IMMUTABLE RETURN (l).price * (l).qty;
SELECT line_total(ROW('Ginger', 40, 5)::order_line) AS ginger;
-- New in PostgreSQL 18: max() of a composite compares field by field.
SELECT max(line) FROM orders;
-- (line).* hands a client plain columns instead of one text value.
SELECT o.id, (o.line).* FROM orders AS o WHERE o.id = 2;

line.price failed with missing FROM-clause entry for table "line": a name followed by a dot reads as a table or an alias, so the field of a column, or of any other expression, is written with the expression in parentheses, (line).price. That form gave totals of 240.00, 250.00 and 271.50 rupees, and line_total() took a whole line as one argument and returned 200.00 for five units of ginger at 40 rupees each. max(line), new in PostgreSQL 18, compared the lines field by field and returned Masala tea, the largest item name. What a client receives depends on how the query asks for the value:

Interactive · step by step Four ways to read one order line Step through four queries on the same composite column and see which one fails, which one returns a field, and what a client receives from the other two.
Four ways to read one order line, step 4 of 4. Showing line.price, ERROR, (line).price, 250.00, SELECT line, One text value, (o.line).*, Three columns. In focus: (o.line).*, Three columns.line.priceno parenthesesERRORno table named line(line).pricefield selection250.00one numeric valueSELECT linethe whole valueOne text value("Coffee, filter",…)(o.line).*expandedThree columnsitem, price, qtyorder 2

The drawing shows the whole scene; its 4 steps are listed below.

Every step as text
  1. A dot alone names a table. line.price reads as the column price of a table or alias called line, and the query has none, so it fails with missing FROM-clause entry for table line. The parser never looks at the column of that name.
  2. Parentheses make it a value. (line).price first evaluates the column line, then selects the field price of the value it got. The run used it to compute 240.00, 250.00 and 271.50, the price times the quantity of each order.
  3. The whole value is one string. SELECT line returns one value per row, which a client receives as its text form, ("Coffee, filter",250.00,1). The comma inside the item made the server quote that field. A driver that does not know order_line gets this string and nothing more; psycopg 3, for one, turns it into a tuple only after register_composite().
  4. Expanded, it is plain columns. (o.line).* expands the value into item, price and qty, three columns with their own types, which every driver and object mapper reads without knowing the composite type.

Sent whole, the value arrives as its text form, ("Coffee, filter",250.00,1), quoted because the item holds a comma, and a driver that does not know the type gets only that string; psycopg 3 adapts a composite type to a Python tuple after the program registers it with register_composite(). Expanded with (o.line).*, the same value arrived as three ordinary columns. Composite types therefore do their best work inside SQL, as function arguments, results and row comparisons, with plain columns at the edge where a driver or an object mapper takes over.

Expanding a function's result with .*

SELECT (f(x)).* FROM t calls f() once for every field of its result, because PostgreSQL turns the expansion into one expression per field. Call the function once, as a LATERAL item in FROM, and expand that: SELECT m.* FROM t, LATERAL f(t.x) AS m.

Checkpoint

Check: Composite values and field access

  1. Question 1 of 2 Which query returns the price field of the composite column line?

    Choose one answer.

    Show the answer: Question 1 of 2

    Answer: SELECT (line).price FROM orders

    A name followed by a dot is a table or alias with a column. To select a field of a column, a function result or any other expression, put the expression in parentheses first.

  2. Question 2 of 2 An object mapper reads query columns into object fields and knows nothing about the type order_line. Which query should the application send for order 2?

    Choose one answer.

    Show the answer: Question 2 of 2

    Answer: SELECT o.id, (o.line).* FROM orders AS o WHERE o.id = 2

    Keep composite values inside SQL, in function arguments, results and comparisons, and hand clients plain columns with (value).* or with one expression per field.

Practise: write arrays out, and choose types

A program that writes an array into a file or a script, rather than passing it as a parameter, must spell it the way the server reads it: braces, commas, and double quotes around any element that holds a comma, a brace, a quote, a backslash or white space, is empty, or is the word NULL. Write that function against its sample tests:

Exercise · Easy · Python

Write a list as a PostgreSQL array value

A program that passes tags to PostgreSQL as a query parameter needs nothing more: the driver sends the list as an array. A program that writes the value into a file or an SQL script must spell it out in the array's text form, the one psql prints, such as {tea,organic}, and the server reads that text back with rules of its own.

Write array_literal(items) in array_literal.py. It takes a list whose items are strings or None and returns the text form of a one-dimensional text array:

  • The elements, in list order, separated by commas and wrapped in { and }. An empty list gives {}.
  • None becomes NULL, without quotes: the element is NULL.
  • A string goes in double quotes when it is empty, when it is the word NULL in any mix of upper and lower case, or when it contains a brace { or }, a comma, a double quote, a backslash or any white space. Every other string is written as it is.
  • Inside the double quotes, put a backslash before each double quote and before each backslash of the string.

So ["green tea", "", None] becomes {"green tea","",NULL}. The sample tests check plain tags, strings that need quotes, the word NULL against a real NULL, quotes and backslashes inside an element, and an empty list.

Starter code · array_literal.py

def array_literal(items):
    """Returns the text form of a PostgreSQL text array holding items (strings or None), such as {tea,organic}."""
    # Your code here: turn each item into NULL, a quoted string or the string as it is,
    # join them with commas and wrap the result in braces.
    return "{}"
The sample tests · test_array_literal.py
from array_literal import array_literal


def test_plain_tags_need_no_quotes():
    assert array_literal(["tea", "organic"]) == "{tea,organic}"


def test_white_space_commas_and_braces_are_quoted():
    assert array_literal(["green tea", "tea, masala", "{gift}"]) == '{"green tea","tea, masala","{gift}"}'


def test_empty_strings_and_the_word_null_are_quoted_but_none_is_not():
    assert array_literal(["", None, "NULL", "null"]) == '{"",NULL,"NULL","null"}'


def test_quotes_and_backslashes_get_a_backslash():
    assert array_literal(['say "chai"', "C:\\tea"]) == '{"say \\"chai\\"","C:\\\\tea"}'


def test_an_empty_list():
    assert array_literal([]) == "{}"
A hint

Decide for each item on its own, in this order: None first, then whether the string needs quotes (empty, the word NULL with .upper(), or any character from {},"\ or white space, which str.isspace() finds). Escape backslashes before double quotes, or the backslashes you add for the quotes get doubled too.

The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.

The questions below come back to the six runs: the GIN plans, the grouping of the join table, the empty baskets, the PIN codes, the enum changes and the order lines. Before you start, decide which of the four types you would give a column of delivery slots that is always morning, afternoon or evening and sorts in that order:

Practice that adapts to you

Practise this lesson

18 questions about 6 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.

  1. Question 1 of 18 Why can a GIN index answer tags @> ARRAY['tea', 'organic'] without reading every row?

    Choose one answer.

    Show the answer: Question 1 of 18

    Answer: It keeps, for each tag, the list of rows that contain it, and intersects the lists of tea and organic

    GIN indexes the elements of each value. For a containment query it looks up each element of the right-hand array and keeps the rows that appear in every list.

  2. Question 2 of 18 A query with WHERE 'vegan' = ANY (tags) runs a Seq Scan although products_tags exists. Which rewrite lets the GIN index serve it?

    Choose one answer.

    Show the answer: Question 2 of 18

    Answer: WHERE tags @> ARRAY['vegan']

    = ANY compares a scalar with each element, so the condition belongs to the text type's operators. Write containment as tags @> ARRAY[...] and the GIN class takes it.

  3. Question 3 of 18 A page lists the products tagged gift or vegan, or both. Which condition gives those rows and can use products_tags?

    Choose one answer.

    Show the answer: Question 3 of 18

    Answer: tags && ARRAY['gift', 'vegan']

    @> is "has all of", && is "has any of", and <@ is "has nothing but". All three, and =, are in array_ops.

  4. Question 4 of 18 When does a join table fit better than an array column?

    Choose one answer.

    Show the answer: Question 4 of 18

    Answer: When each item has facts of its own, must refer to another table, or changes apart from the row

    An array suits a short list that is read and written with its row. Once the items need facts, references or a life of their own, they become rows of a table.

  5. Question 5 of 18 A notes app lets each note carry up to five labels typed by its user. Labels are read with the note and searched by label; nothing else is known about a label. Which design is simplest?

    Choose one answer.

    Show the answer: Question 5 of 18

    Answer: A text[] column with a GIN index

    Choose the array while the items are plain values of the row. Move to a join table when the first fact about an item, or the first need for a foreign key, appears.

  6. Question 6 of 18 A recipe site stores each recipe's ingredients. In each recipe an ingredient has a quantity and a unit, and the site lists all recipes that use an ingredient. Which design fits?

    Choose one answer.

    Show the answer: Question 6 of 18

    Answer: A join table recipe_ingredients with recipe_id, ingredient_id, quantity and unit

    Facts that belong to the link between two things, here how much of an ingredient a recipe uses, live in the join table itself. That is a case no array can hold.

  7. Question 7 of 18 What does WITH ORDINALITY add to unnest(items)?

    Choose one answer.

    Show the answer: Question 7 of 18

    Answer: A column that numbers each element from 1, in the array's order

    unnest() turns an array into rows, and WITH ORDINALITY keeps each element's position as a bigint column, so the order can be used in joins, filters or an array_agg() that puts the array back together.

  8. Question 8 of 18 On PostgreSQL 18, what does array_sort(ARRAY['tea', 'spices', 'tea', 'sugar'], true) return?

    Choose one answer.

    Show the answer: Question 8 of 18

    Answer: {tea,tea,sugar,spices}

    array_sort(array, descending, nulls_first) sorts the first dimension by the element type's default order; true as the second argument makes it descending. array_reverse() only reverses.

  9. Question 9 of 18 A cleanup job runs DELETE FROM baskets WHERE array_length(items, 1) = 0 and reports DELETE 0, though 3,333 baskets are empty. Why, and what should it test?

    Choose one answer.

    Show the answer: Question 9 of 18

    Answer: array_length() returns NULL for an empty array, so the condition is never true; test cardinality(items) = 0

    NULL is neither equal nor unequal to 0, so a filter on array_length() silently drops empty arrays. cardinality() and the comparison items = '{}' treat them as the values they are.

  10. Question 10 of 18 What is a domain in PostgreSQL?

    Choose one answer.

    Show the answer: Question 10 of 18

    Answer: A data type based on another type, with constraints checked whenever a value is converted to it

    A domain behaves like its base type, so every operator and function of text works on pin_code, and it adds its constraints, NOT NULL and a default, to every column of the type.

  11. Question 11 of 18 pin is a pin_code column. What is the type of pin || '-1'?

    Choose one answer.

    Show the answer: Question 11 of 18

    Answer: text, because an operator of the base type gives the base type and runs no domain check

    The manual calls it a down-cast: a domain value used with an operator or function of its base type becomes the base type. Only a conversion back to the domain, such as a cast or a write into a pin_code column, checks the rule.

  12. Question 12 of 18 You add a second CHECK to pin_code with ALTER DOMAIN pin_code ADD CONSTRAINT …. Which stored values does PostgreSQL check?

    Choose one answer.

    Show the answer: Question 12 of 18

    Answer: Every value in every column of the domain, in customers and in stores, unless the constraint is added NOT VALID

    One ALTER DOMAIN reaches every column of the type, which is the point of a domain. For a big table, add the constraint NOT VALID first and run ALTER DOMAIN … VALIDATE CONSTRAINT later.

  13. Question 13 of 18 How much space does one enum value take on disk?

    Choose one answer.

    Show the answer: Question 13 of 18

    Answer: 4 bytes, whatever the length of its label

    Rows store a 4-byte reference, and pg_enum keeps each label once with its sort order, which is why sorting and comparing follow the declaration.

  14. Question 14 of 18 The run tried ALTER TYPE order_status DROP VALUE 'refunded'. What happened?

    Choose one answer.

    Show the answer: Question 14 of 18

    Answer: It failed: dropping an enum value is not implemented

    Existing enum values cannot be removed and their sort order cannot be changed, short of dropping and re-creating the type. Plan the list of labels before the first row uses it.

  15. Question 15 of 18 After ALTER TYPE order_status RENAME VALUE 'shipped' TO 'dispatched', where does dispatched sort among pending, paid, refunded and cancelled?

    Choose one answer.

    Show the answer: Question 15 of 18

    Answer: Third, in shipped's old place, because a rename keeps the sort order

    Rows refer to a pg_enum row, which holds the label and the sort order. RENAME VALUE changes only the label, so every row and every ordering follows it at once.

  16. Question 16 of 18 What is a composite type?

    Choose one answer.

    Show the answer: Question 16 of 18

    Answer: A type made of named fields, like a row; every table also has one under its own name

    A composite value holds several named fields as one value, so a function can take or return it whole. Read a field with (value).field and all fields with (value).*.

  17. Question 17 of 18 Why did order 2 print as ("Coffee, filter",250.00,1) while order 3 printed as (Cardamom,90.50,3)?

    Choose one answer.

    Show the answer: Question 17 of 18

    Answer: Order 2's item holds a comma and a space, and the output routine quotes such fields

    A client that reads the whole value gets this text form and must undo its quoting. Expanding the value into columns avoids that.

  18. Question 18 of 18 A report runs SELECT (price_breakdown(o.id)).* FROM orders AS o, where price_breakdown() returns a composite of three fields and takes 50 ms a call. Why is it three times slower than expected, and what is the fix?

    Choose one answer.

    Show the answer: Question 18 of 18

    Answer: The .* expansion calls the function once per field; call it once in FROM with LATERAL and expand that

    SELECT m.* FROM orders AS o, LATERAL price_breakdown(o.id) AS m calls the function once a row and still returns the three fields as columns.

Interview questions

Warm-up for freshers: What does a PostgreSQL domain add over a CHECK constraint written on the column?

A domain is a named type: a base type with constraints, an optional NOT NULL and a default, written once with CREATE DOMAIN. Every column, cast and function argument that uses the type gets the same rule, so a PIN code is six digits in customers, in stores and in a function’s parameter alike, and one ALTER DOMAIN changes the rule and checks every column of the type. A CHECK constraint on a column protects that column only, and each table repeats it. A domain’s CHECK may refer only to VALUE, never to other columns, so a rule that compares two columns stays a table constraint. A domain allows NULL unless told otherwise, and the manual advises putting NOT NULL on the columns instead. Operators of the base type return the base type, so pin || ‘-1’ is plain text and is not checked.

Two questions of this module go further from these runs: a hashtag array whose searches slow down under a stream of new posts, and a status enum on a busy orders table that has to gain one value and stop using another. Each has its first step and its follow-ups on the module’s question page:

Model answers open with any pass: ₹79 for 1 week ($2.99), one payment that never renews.

  1. Mid to senior

    A social app stores each post's hashtags in a text[] column with a GIN index. At 3,000 new posts a second, hashtag searches slow down at peak, and the product team now wants hourly trending hashtags. Do you keep the array or move to a join table, and how do you index what you keep?

    Model answer, Pro. Model answers open with any pass; prices are in the notice on this page. First step and follow-ups

  2. Mid to staff

    An orders table of 40 million rows has a status column of type order_status (pending, on_hold, paid, shipped, cancelled). This release adds partially_refunded between paid and shipped and retires on_hold, which nothing may write again. Your migration tool runs each file in one transaction. Plan both changes without blocking orders.

    Model answer, Pro. First step and follow-ups

Flashcards · 5 cards

Recall it, then check

Say the answer to yourself before you look: recalling it is what makes it stick.

  1. How do you search array elements, and how is it indexed?

    Write element searches as tags @> ARRAY[...], never = ANY. GIN's array_ops serves &&, @>, <@ and =, keeping a short list inside the row (the tag array took 5,800 kB against 12 MB for a join table).

  2. When does a join table beat an array column?

    Once the items have facts, foreign keys or changes of their own. A join table can refuse an unknown tag and rename one with UPDATE 1, where the array design had to rewrite every row that held it.

  3. Which array helpers matter for queries in PostgreSQL 18?

    IN lists run as = ANY over an array in one B-tree scan; unnest() WITH ORDINALITY numbers elements; PostgreSQL 18 adds array_sort(), array_reverse() and min()/max() over arrays; use cardinality(), not array_length(), to find empty arrays.

  4. What does a domain do, and how does it treat NULL?

    A domain puts one rule on a type, checked whenever a value is converted to it (e.g. pin_code accepts 411001, refuses 011001). It allows NULL by default and leaves NOT NULL to the columns.

  5. How do enum types sort and change?

    Enums sort in their declared order and take 4 bytes a value. A value added inside a transaction can be used only after that transaction commits, and values can be renamed but never dropped.

Key takeaways

  • Arrays keep a short list inside the row, and GIN’s array_ops serves &&, @>, <@ and =, so element searches are written as tags @> ARRAY[...], never = ANY; the tag array took 5,800 kB against 12 MB for the join table.
  • A join table wins once the items have facts, foreign keys or changes of their own: it refused an unknown tag and renamed one with UPDATE 1, where the array design rewrote 7,143 rows.
  • IN lists run as = ANY over an array in one B-tree scan, unnest() WITH ORDINALITY numbers the elements, PostgreSQL 18 adds array_sort(), array_reverse() and min() and max() over arrays, and cardinality(), not array_length(), finds empty arrays.
  • Domains put one rule on a type, checked whenever a value is converted to it: pin_code accepted 411001 and refused 011001 for two tables, allowed NULL, and left NOT NULL to the columns.
  • Enums sort in their declared order and take 4 bytes a value; a value added inside a transaction can be used only after that transaction commits, and values can be renamed but never dropped.
  • Composite types carry named fields as one value: read a field as (line).price, pass whole values to functions, and give clients plain columns with (value).*.

Terms in this lesson

composite type
A data type made of named fields, each with its own type, like the columns of a row. Every table has one under its own name, and CREATE TYPE ... AS (...) makes a standalone one.
domain
A data type made from another type with constraints, NOT NULL and a default added. A value is checked when it is converted to the domain, so every column, cast and argument of the type follows one rule.
enum type
A data type with a fixed list of labels, made with CREATE TYPE ... AS ENUM. Values take 4 bytes and compare in the order the labels were declared; labels can be added and renamed, never removed.
join table
A table with one row for each pairing of rows from two other tables, such as one row per product and tag, holding a foreign key to each side. Facts about the pairing itself can be columns of it.
row constructor
An expression that builds a composite value from its fields, written ROW(a, b, c) or, with two fields or more, (a, b, c), often cast to a named composite type.

References

Report a problem with this lesson

Quick answers and tool search

Type to search tools or to get a quick answer, for example 18% of 2500. Use the up and down arrow keys to move through the results, Enter to choose, and Escape to close.