Transparency, blending and alpha compositing

A pane of red glass that lets half the light through hangs in front of a blue-grey wall, so its pixel should be half the glass’s red and half the wall’s colour. Alpha compositing is that mix: each layer covers its alpha’s share of the pixel. Porter and Duff’s over operator computes it, and two decisions decide whether a renderer gets it right: whether colours are stored already multiplied by their alpha, and in which order transparent surfaces are drawn.

Every number below comes from programs on this page: three layers composited over a wall and a sprite’s edge after a resize, then the blend states and draw orders of a GPU, worked out on the CPU by the same rules.

Alpha says how much of a pixel a surface covers

A pixel’s alpha is the share of it that a surface covers, from 0, nothing, to 1, all of it. Thomas Porter and Tom Duff’s paper of 1984 on compositing digital images defined how images that carry alpha combine, and its over operatorover operator: Porter and Duff's rule for laying a source with alpha over a backdrop: the source takes its alpha's share of the pixel and the backdrop shows through the rest. lays a source over a backdrop. The W3C’s Compositing and Blending specification writes it for colours C and alphas α:

In words: the source takes its alpha’s share of the pixel, the backdrop shows through the rest, and the result’s coverage grows the same way. Transparency in a renderer comes down to this operator, applied pixel by pixel. Over an opaque wall, red glass of alpha 0.5 gives 0.5 × 0.8 + 0.5 × 0.2 = 0.5 in red. This is alpha compositingalpha compositing: Combining images or surfaces by their alpha, the share of each pixel they cover, with operators such as over, so that partly transparent layers show what lies behind them., and glTF uses it: a material with alphaMode BLEND is combined with the background by over, OPAQUE ignores alpha, and MASK shows each pixel fully or not at all, at a cutoff. Step through three layers laid over the wall:

Premultiplied alpha stores the colour already times its alpha

Store each colour already multiplied by its alpha, c = αC, and over becomes one multiply-add in every channel, alpha included:

That is premultiplied alphapremultiplied alpha: Storing a pixel's colour already multiplied by its alpha, so that half-transparent red is (0.5, 0, 0, 0.5). It makes over one multiply-add and keeps filtering correct., against the straight alphastraight alpha: Storing a pixel's colour and its alpha separately, without multiplying the colour by the alpha; also called unassociated or non-premultiplied alpha. of the formula above. The program composites the three layers both ways, merges them before they meet the wall, and halves a sprite’s edge:

// Porter and Duff's "over" on three invented layers above an opaque wall, with straight and with premultiplied
// alpha, then a fringe that only straight alpha makes when a sprite is resized. Colours are linear RGB.
const fmt = (c) => `(${c.map((x) => x.toFixed(3)).join(', ')})`;
// Straight alpha: [r, g, b, a] with the colour not multiplied by a.
const overStraight = (s, d) => {
  const a = s[3] + d[3] * (1 - s[3]);
  if (a === 0) return [0, 0, 0, 0];
  return [0, 1, 2].map((k) => (s[k] * s[3] + d[k] * d[3] * (1 - s[3])) / a).concat(a);
};
// Premultiplied alpha: the colour is already times a, so "over" is one multiply-add per channel.
const premultiply = (c) => [c[0] * c[3], c[1] * c[3], c[2] * c[3], c[3]];
const overPremultiplied = (s, d) => s.map((x, k) => x + d[k] * (1 - s[3]));

const wall = [0.2, 0.3, 0.45, 1];
const glass = [0.8, 0.1, 0.1, 0.5]; // red glass, half the light through
const smoke = [0.1, 0.7, 0.2, 0.25]; // green smoke
const label = [1, 1, 1, 0.8]; // white text

let straight = wall;
let premultiplied = premultiply(wall);
for (const [name, layer] of [['glass', glass], ['smoke', smoke], ['label', label]]) {
  straight = overStraight(layer, straight);
  premultiplied = overPremultiplied(premultiply(layer), premultiplied);
  console.log(`after the ${name.padEnd(5)}  straight ${fmt(straight)}  premultiplied ${fmt(premultiplied)}`);
}

// Grouping: the three layers merged first, then laid over the wall.
const merged = overPremultiplied(premultiply(label), overPremultiplied(premultiply(smoke), premultiply(glass)));
console.log(`\nlayers merged first, alpha ${merged[3].toFixed(3)}: over the wall ${fmt(overPremultiplied(merged, premultiply(wall)))}`);
// The same grouping with the opaque-backdrop formula, s a + d (1 - a), which ignores the lower layer's alpha.
const naive = (s, d) => [0, 1, 2].map((k) => s[k] * s[3] + d[k] * (1 - s[3])).concat(s[3] + d[3] * (1 - s[3]));
const naiveMerged = naive(label, naive(smoke, glass));
console.log(`merged with s a + d (1 - a), then over the wall: ${fmt(naive(naiveMerged, wall))}`);

// A sprite edge: an opaque white texel beside a transparent one that straight alpha stores as black, (0, 0, 0, 0).
// Halving the sprite averages the two texels; the result is drawn over a white page.
const page = [1, 1, 1, 1];
const avg = (p, q) => p.map((x, k) => (x + q[k]) / 2);
const s = avg([1, 1, 1, 1], [0, 0, 0, 0]); // the same numbers in both encodings, read two ways
const toCode = (l) => Math.round((l <= 0.0031308 ? 12.92 * l : 1.055 * l ** (1 / 2.4) - 0.055) * 255);
const asStraight = overStraight(s, page)[0];
const asPremultiplied = overPremultiplied(s, page)[0];
console.log(`\nhalved edge texel ${fmt(s)} over a white page:`);
console.log(`  read as straight alpha:      ${asStraight.toFixed(3)}, code ${toCode(asStraight)}`);
console.log(`  read as premultiplied alpha: ${asPremultiplied.toFixed(3)}, code ${toCode(asPremultiplied)}`);

after the glass  straight (0.500, 0.200, 0.275, 1.000)  premultiplied (0.500, 0.200, 0.275, 1.000)
after the smoke  straight (0.400, 0.325, 0.256, 1.000)  premultiplied (0.400, 0.325, 0.256, 1.000)
after the label  straight (0.880, 0.865, 0.851, 1.000)  premultiplied (0.880, 0.865, 0.851, 1.000)

layers merged first, alpha 0.925: over the wall (0.880, 0.865, 0.851, 1.000)
merged with s a + d (1 - a), then over the wall: (0.871, 0.809, 0.797, 1.000)

halved edge texel (0.500, 0.500, 0.500, 0.500) over a white page:
  read as straight alpha:      0.750, code 225
  read as premultiplied alpha: 1.000, code 255

Both ways give (0.880, 0.865, 0.851). Premultiplied over is associative: the three layers merged first, alpha 0.925, give the same pixel, while merging straight colours with the opaque-backdrop formula gives (0.871, 0.809, 0.797). Filtering is the second gain. Halving a sprite averages an opaque white texel with a transparent one, (0, 0, 0, 0): read as straight alpha, the average is grey at half coverage and shows 0.750 over a white page, code 225, a dark fringe; read as premultiplied, it is white at half coverage, 1.000, code 255.

The two ways of storing a colour with its alpha compare like this:

A WebGPU canvas takes the same choice: its alphaMode is “opaque” by default, and “premultiplied” reads colours so, where 100 % red at 50 % alpha is [0.5, 0, 0, 0.5]. On a GPU, the same arithmetic is set as a blend state. Alpha blending mixes straight colours by their alpha. Premultiplied blending adds the source to the destination times one minus its alpha. Additive blending adds light, for fire and glows. Transparent surfaces are drawn after the opaque ones, sorted from back to front, usually with depth writes off. Surfaces that cross need order-independent transparencyBlending transparent surfaces in the right order at each pixel separately, instead of sorting whole objects, so that surfaces which cross each other also come out right., which resolves the order per pixel.

Practise: over, premultiply and unpremultiply

The exercise writes the three operations this lesson uses. Its sample tests lay the glass, the smoke and the label over the wall, then check that merging them first changes nothing.

/** [r, g, b, a] with straight colour to premultiplied: the colour times its alpha. */
export function premultiply(c) {
  return [c[0] * c[3], c[1] * c[3], c[2] * c[3], c[3]];
}

/** Premultiplied back to straight: the colour divided by its alpha; a fully transparent pixel is [0, 0, 0, 0]. */
export function unpremultiply(c) {
  // Divides by zero for a fully transparent pixel.
  return [c[0] / c[3], c[1] / c[3], c[2] / c[3], c[3]];
}

/** Porter and Duff's "over" for premultiplied colours: src + dst x (1 - src alpha), in every channel. */
export function over(src, dst) {
  // The opaque-backdrop formula for straight colours: wrong for premultiplied ones, and for alpha.
  const a = src[3];
  return [src[0] * a + dst[0] * (1 - a), src[1] * a + dst[1] * (1 - a), src[2] * a + dst[2] * (1 - a), 1];
}

import { test, assert } from 'mysmartcopilot:test';
import { premultiply, unpremultiply, over } from './alpha.mjs';

const close = (got, want, what) => {
  for (let k = 0; k < 4; k++) assert.ok(Math.abs(got[k] - want[k]) <= 1e-9, `${what}: channel ${k} should be ${want[k]}, got ${got[k]}`);
};
const WALL = [0.2, 0.3, 0.45, 1];
const GLASS = [0.8, 0.1, 0.1, 0.5];
const SMOKE = [0.1, 0.7, 0.2, 0.25];
const LABEL = [1, 1, 1, 0.8];

test('premultiply and back, including a fully transparent pixel', () => {
  close(premultiply(GLASS), [0.4, 0.05, 0.05, 0.5], 'red glass at alpha 0.5');
  close(unpremultiply([0.4, 0.05, 0.05, 0.5]), GLASS, 'and back again');
  close(unpremultiply([0, 0, 0, 0]), [0, 0, 0, 0], 'alpha 0 has no colour to recover');
  close(premultiply(WALL), WALL, 'an opaque colour does not change');
});

test("the lesson's three layers over the wall", () => {
  let pixel = premultiply(WALL);
  pixel = over(premultiply(GLASS), pixel);
  close(pixel, [0.5, 0.2, 0.275, 1], 'after the glass');
  pixel = over(premultiply(SMOKE), pixel);
  pixel = over(premultiply(LABEL), pixel);
  close(pixel, [0.88, 0.865, 0.85125, 1], 'after the label');
});

test('over is associative for premultiplied colours, and keeps alpha', () => {
  const g = premultiply(GLASS);
  const s = premultiply(SMOKE);
  const l = premultiply(LABEL);
  const merged = over(l, over(s, g));
  close(merged, [0.865, 0.8425, 0.8175, 0.925], 'the layers merged first');
  close(over(merged, premultiply(WALL)), over(l, over(s, over(g, premultiply(WALL)))), 'same as one at a time');
  close(over(premultiply([1, 0, 0, 0]), g), g, 'a fully transparent layer changes nothing');
  close(over(premultiply([0, 0, 1, 1]), g), [0, 0, 1, 1], 'an opaque layer hides everything below');
});

WebGPU blend states, evaluated factor by factor Pro: opens with a Pro, Premium or Ultimate pass

Two quiet errors: the canvas's alpha and the sRGB target, reproduced Pro: opens with a Pro, Premium or Ultimate pass

Drawing transparent geometry in order, and where sorting fails Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

A warm-up comes first, answered in full, then three problems where transparency goes wrong. The question page of this module holds the first steps and the follow-up titles of all three:

Key takeaways

• Over gives the source its alpha’s share of the pixel and the backdrop the rest, and the result’s alpha grows the same way.
• Premultiplied alpha stores the colour times its alpha, which makes over one multiply-add, associative, and safe to filter: a halved sprite edge stays 1.000 instead of turning into a 0.750 grey fringe.
• On a GPU, blend states carry the same arithmetic, and transparent surfaces are drawn after opaque ones, from back to front.