Applied Cryptography for Developers Module 15 – End-to-end encryption and advanced protocols
End-to-end encryption: Signal protocols and MLS
How a key chain that deletes gives forward secrecy, how a fresh Diffie-Hellman key heals a copied phone, and what X3DH, the Double Ratchet and MLS each do.
What you will learn
- Explain forward secrecy and post-compromise security, and say which part of a ratchet gives each
- Implement a symmetric key ratchet from HMAC and test that a copied state opens no earlier message
- Describe the role of X3DH, PQXDH, the Double Ratchet and MLS in a messenger
- Estimate the ciphertexts one key update needs in a group of 1,000 members with pairwise sessions and with a ratchet tree
- List what end-to-end encryption does not hide
Before you start
What you’ll learn · Before you start
What you will learn
- Explain forward secrecy and post-compromise security, and say which part of a ratchet gives each
- Implement a symmetric key ratchet from HMAC and test that a copied state opens no earlier message
- Describe the role of X3DH, PQXDH, the Double Ratchet and MLS in a messenger
- Estimate the ciphertexts one key update needs in a group of 1,000 members with pairwise sessions and with a ratchet tree
- List what end-to-end encryption does not hide
Before you start
On this page
A thief copies a phone that holds 10,000 messages. If the chat used one key, all 10,000 open; if every message had its own key and each key was deleted after use, none of the earlier ones do. In this lesson’s test the numbers are 5 of 5 against 0 of 5 for a copy taken after message 5.
This lesson builds that key chain from HMAC, tests it with a thief of its own, and adds what a fresh Diffie-Hellman key does after a copy. Then it sets out the three layers of a messenger: X3DH and PQXDH to start a session, the Double Ratchet to run it, and MLS for groups. It closes with what none of them hide, and a code lab builds the receiving end of a chain.
A stolen phone tests two promises: forward secrecy and healing
Two questions decide how good a messenger’s encryption is once a device is lost. Can the thief read what was said before the copy? That is forward secrecy: The property that recorded sessions stay closed after a long-term key leaks, because each session's keys came from ephemeral keys that were deleted. More in “Key agreement with X25519 and ECDH”. And once the thief is out again, do the messages that follow close up, or does the copy keep working? That second promise is post-compromise security: The property that a device whose keys were copied once becomes secure again, because a fresh key exchange mixes in a secret the thief never saw. The Double Ratchet specification calls it break-in recovery., which the Double Ratchet specification calls break-in recovery. Throughout this lesson the thief copies the device once and afterwards only listens to the traffic.
The table gives the answers for three designs. Its first row is the lesson’s first test, its second and third come from the tests of the sections that follow:
| Design | Messages from before the copy | Messages from after the copy |
|---|---|---|
| One key for the whole chat | all open | all open |
| A key chain that deletes every key it has used | none open | all open, for as long as the chain runs |
| The chain plus a fresh Diffie-Hellman key whenever someone replies | none open | open only until the owner’s next reply with a new key |
No design in the table stops a thief who reads the screen of a phone that someone has left open, or one who stays on the device. End-to-end encryption, as the last section sets out, limits what a single copy of the keys is worth.
A key chain that forgets: one-way steps and deletion give forward secrecy
Start from a secret both phones share, the chain key: The secret a symmetric ratchet holds between messages. HMAC of it with one constant gives the next message key, and HMAC with another constant gives the next chain key.. For each message the sender computes two values from it with HMAC-SHA-256, as section 7.2 of the Double Ratchet specification recommends. The message key is the HMAC of the chain key with the single byte 0x01, and the next chain key is the HMAC of the chain key with the byte 0x02. The message key encrypts one message. Then the sender overwrites the chain key with the next one and discards the message key. The receiver takes the same steps and arrives at the same keys. This chain is the symmetric-key ratchet: A chain that turns one chain key into a message key and the next chain key with a one-way function, then deletes what it used, so earlier keys cannot be recomputed from the chain's current state.:
In words: each chain key makes the next chain key and one message key, and nothing flows back. With the lesson’s lab chain key, the first five message keys have the fingerprints 52808d97, 5ee9147c, 2caa5f1a, a00f5eec and 22b31b27 (the first four bytes of the SHA-256 of each key), and Alice’s and Bob’s chains produce them alike. Step through three messages and a copy of the device:
The drawing shows the whole scene; its 7 steps are listed below.
Every step as text
- A shared secret. Both phones hold the same 32-byte chain key. It is never used to encrypt anything; it only produces keys.
- Message 0. HMAC-SHA-256 of the chain key with the byte 0x01 gives the message key, and with 0x02 the next chain key. Alice and Bob compute the same two values.
- Use it once, then delete. The message is encrypted with message key 0. Then chain key 0 and message key 0 are deleted. Chain key 1 is the only secret the phone keeps.
- Messages 1 and 2. The same two steps run again for each message, and each time the old chain key and the message key are deleted.
- A thief copies the phone. After message 2 a thief copies everything the phone holds. That is chain key 3, and nothing else, because everything older was deleted.
- Forward is open. From chain key 3 the thief can run the same steps and compute the keys of messages 3, 4, 5 and every later one, for as long as nothing new is mixed in.
- Backward is closed. Keys 0, 1 and 2 are not in the copy, and HMAC cannot be run backwards, so the first three messages stay closed. Forward secrecy comes from the one-way step and the deletion together.
The program below runs the chain for eight messages, each sealed with AES-256-GCM under a key and nonce that HKDF derives from the message key. A message key encrypts one message, so a nonce taken from the same derivation cannot repeat under it. The program tests three designs: a thief copies each design’s state after message 5 and tries every key it can get from the copy, including the ones it can compute by running a chain forward.
// A symmetric-key ratchet as section 2.2 and section 7.2 of the Double Ratchet specification describe it: each message
// gets a key from HMAC-SHA-256 of the chain key with the byte 0x01, the chain key moves on with the byte 0x02, and the
// old chain key is overwritten. Three designs are then tested: a thief copies each design's state after message 5 and
// tries to open all eight ciphertexts. The chain key and the messages are lab values.
import { createCipheriv, createDecipheriv, createHash, createHmac, hkdfSync } from 'node:crypto';
const hmac = (key, byte) => createHmac('sha256', key).update(Buffer.from([byte])).digest();
const step = (ck) => ({ mk: hmac(ck, 0x01), ck: hmac(ck, 0x02) });
const fingerprint = (bytes) => createHash('sha256').update(bytes).digest('hex').slice(0, 8);
// All a thief can do with a chain key is run it forward.
function forward(ck, count) {
const out = [];
for (let i = 0; i < count; i++) {
const s = step(ck);
out.push(s.mk);
ck = s.ck;
}
return out;
}
// One message key encrypts one message, so the nonce can come from the same HKDF call as the AES key.
function seal(mk, plaintext, ad) {
const okm = Buffer.from(hkdfSync('sha256', mk, Buffer.alloc(32), 'msc ratchet demo v1', 44));
const cipher = createCipheriv('aes-256-gcm', okm.subarray(0, 32), okm.subarray(32, 44));
cipher.setAAD(Buffer.from(ad));
return Buffer.concat([cipher.update(plaintext, 'utf8'), cipher.final(), cipher.getAuthTag()]);
}
function open(mk, blob, ad) {
const okm = Buffer.from(hkdfSync('sha256', mk, Buffer.alloc(32), 'msc ratchet demo v1', 44));
const decipher = createDecipheriv('aes-256-gcm', okm.subarray(0, 32), okm.subarray(32, 44));
decipher.setAAD(Buffer.from(ad));
decipher.setAuthTag(blob.subarray(-16));
try {
return Buffer.concat([decipher.update(blob.subarray(0, -16)), decipher.final()]).toString('utf8');
} catch {
return null;
}
}
const ROOT_CK = Buffer.alloc(32, 0x5a);
const MESSAGES = ['hello', 'are you there', 'meet at six', 'bring the files', 'ok', 'new number is 98xx', 'delete this chat', 'bye'];
const THEFT_AFTER = 5; // the state is copied once five messages are sent
// Alice and Bob walk the same chain: the first five message keys agree, and each side forgets what it has used.
let a = ROOT_CK;
let b = ROOT_CK;
const keys = [];
for (let n = 0; n < 5; n++) {
const sa = step(a);
const sb = step(b);
a = sa.ck;
b = sb.ck;
keys.push(`#${n} ${fingerprint(sa.mk)}${fingerprint(sa.mk) === fingerprint(sb.mk) ? '' : ' MISMATCH'}`);
}
console.log(`first five message keys, Alice and Bob agree: ${keys.join(' ')}`);
// The three designs. send(n, text) returns a ciphertext, state() is what a thief copies from the device.
const designs = {
'one key for every message': () => {
const key = ROOT_CK;
return { send: (n, text) => seal(key, text, `msg ${n}`), state: () => ({ key }), derive: (s) => [s.key] };
},
'chain, used keys kept': () => {
let ck = ROOT_CK;
const used = [];
return {
send: (n, text) => {
const s = step(ck);
ck = s.ck;
used.push(s.mk); // kept "in case of a retry": the mistake
return seal(s.mk, text, `msg ${n}`);
},
state: () => ({ ck, used: [...used] }),
derive: (s) => [...s.used, ...forward(s.ck, 100)],
};
},
'chain, used keys deleted': () => {
let ck = ROOT_CK;
return {
send: (n, text) => {
const s = step(ck);
ck = s.ck; // the old chain key and the message key are not kept anywhere
return seal(s.mk, text, `msg ${n}`);
},
state: () => ({ ck }),
derive: (s) => forward(s.ck, 100),
};
},
};
console.log(`\nA thief copies each device after ${THEFT_AFTER} of ${MESSAGES.length} messages and tries every key it can get:`);
console.log('design before the theft after the theft');
for (const [name, make] of Object.entries(designs)) {
const d = make();
const sealed = [];
let stolen;
MESSAGES.forEach((text, n) => {
if (n === THEFT_AFTER) stolen = d.state();
sealed.push(d.send(n, text));
});
const candidates = d.derive(stolen);
const opened = sealed.map((blob, n) => candidates.some((mk) => open(mk, blob, `msg ${n}`) === MESSAGES[n]));
const before = opened.slice(0, THEFT_AFTER).filter(Boolean).length;
const after = opened.slice(THEFT_AFTER).filter(Boolean).length;
console.log(`${name.padEnd(28)} ${`${before} of ${THEFT_AFTER}`.padEnd(18)} ${after} of ${MESSAGES.length - THEFT_AFTER}`);
}
// The AEAD still refuses a changed ciphertext or a ciphertext moved to another message number.
const mk0 = step(ROOT_CK).mk;
const blob = seal(mk0, 'hello', 'msg 0');
const flipped = Buffer.from(blob);
flipped[0] ^= 1;
const show = (text) => text ?? 'refused';
console.log(`\nunchanged: ${show(open(mk0, blob, 'msg 0'))}, one bit changed: ${show(open(mk0, flipped, 'msg 0'))}, same bytes as message 1: ${show(open(mk0, blob, 'msg 1'))}`); Output
first five message keys, Alice and Bob agree: #0 52808d97 #1 5ee9147c #2 2caa5f1a #3 a00f5eec #4 22b31b27 A thief copies each device after 5 of 8 messages and tries every key it can get: design before the theft after the theft one key for every message 5 of 5 3 of 3 chain, used keys kept 5 of 5 3 of 3 chain, used keys deleted 0 of 5 3 of 3 unchanged: hello, one bit changed: refused, same bytes as message 1: refused
Recorded with Node.js 24.21.0 on macOS 26 arm64. To run it yourself: mise exec node@24.21.0 -- node symmetric_ratchet.mjs
One key for every message opens 5 of 5 earlier messages and 3 of 3 later ones. The chain that keeps its used keys “for a retry” opens just as many: the chain was right, and the key store ruined it. The chain that deletes opens 0 of the 5 earlier messages, because from chain key 5 a thief can only compute forward. It still opens all 3 later messages. The last line shows what the AEAD adds: a changed ciphertext, or the same bytes presented as another message number, is refused.
Keeping used keys for a retry
Wrong: store each message key until the other side confirms the message, “in case it must be sent again”. The first row of the program’s table shows the result, a copy that opens every earlier message. Right: keep the ciphertext or the plaintext for a retry, and delete the key as soon as the message is sealed.
A test like this belongs in the project that implements the ratchet. The next program runs a design for eight messages, copies its state after message 5 and searches the copy, and everything the copy can derive, for the key of any earlier message. It audits four designs, and one of them keeps a single chain key too many:
"""A forward-secrecy audit for a message-key ratchet: run a design for eight messages, copy its state after message 5,
and look for the key of any earlier message in that copy or in what the copy can derive. The chain key is a lab value.
Press Edit, add a design of your own to DESIGNS, and run it again."""
import hashlib
import hmac
ROOT_CK = bytes([0x5A] * 32)
SENT, THEFT_AFTER, FORWARD_STEPS = 8, 5, 1000
def step(ck: bytes) -> tuple[bytes, bytes]:
"""The Double Ratchet's KDF_CK with HMAC-SHA-256: (next chain key, message key)."""
return hmac.digest(ck, b"\x02", "sha256"), hmac.digest(ck, b"\x01", "sha256")
def fingerprint(key: bytes) -> str:
return hashlib.sha256(key).hexdigest()[:8]
class OneKey:
"""Every message under the root key."""
def __init__(self): self.key = ROOT_CK
def send(self): return self.key
def state(self): return [self.key]
def advance(self, state): return state # nothing to run forward
class KeepsUsedKeys:
"""A real chain, but every message key is kept for retries."""
def __init__(self): self.ck, self.used = ROOT_CK, []
def send(self):
self.ck, mk = step(self.ck)
self.used.append(mk)
return mk
def state(self): return [self.ck, *self.used]
def advance(self, state): return state[:1]
class KeepsPreviousChainKey:
"""Deletes message keys but keeps the chain key of the message before, to retry the last send."""
def __init__(self): self.ck, self.prev = ROOT_CK, b""
def send(self):
self.prev = self.ck
self.ck, mk = step(self.ck)
return mk
def state(self): return [self.ck, self.prev]
def advance(self, state): return state
class DeletesAsItGoes:
"""The ratchet: only the next chain key is kept."""
def __init__(self): self.ck = ROOT_CK
def send(self):
self.ck, mk = step(self.ck)
return mk
def state(self): return [self.ck]
def advance(self, state): return state
DESIGNS = [OneKey, KeepsUsedKeys, KeepsPreviousChainKey, DeletesAsItGoes]
def audit(design) -> list[int]:
"""Numbers of the messages sent before the theft whose keys the stolen copy holds or can derive."""
d = design()
keys, stolen = [], []
for n in range(SENT):
if n == THEFT_AFTER:
stolen = d.state()
keys.append(d.send())
earlier: dict[bytes, set[int]] = {}
for n in range(THEFT_AFTER):
earlier.setdefault(keys[n], set()).add(n) # one key may belong to several messages
found: set[int] = set()
for value in stolen: # keys sitting in the copy
found |= earlier.get(value, set())
for ck in d.advance(stolen): # keys the copy can reach by running a chain key forward
probe = ck
for _ in range(FORWARD_STEPS):
probe, mk = step(probe)
found |= earlier.get(mk, set())
return sorted(found)
first = []
ck = ROOT_CK
for n in range(5):
ck, mk = step(ck)
first.append(f"#{n} {fingerprint(mk)}")
print("first five message keys: " + " ".join(first))
print()
for design in DESIGNS:
leaked = audit(design)
noun = "message" if len(leaked) == 1 else "messages"
verdict = "PASS" if not leaked else f"FAIL: can recover the keys of {noun} " + ", ".join(map(str, leaked))
print(f"{design.__name__:<22} {verdict}") Output
first five message keys: #0 52808d97 #1 5ee9147c #2 2caa5f1a #3 a00f5eec #4 22b31b27 OneKey FAIL: can recover the keys of messages 0, 1, 2, 3, 4 KeepsUsedKeys FAIL: can recover the keys of messages 0, 1, 2, 3, 4 KeepsPreviousChainKey FAIL: can recover the keys of message 4 DeletesAsItGoes PASS
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 audit.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
The audit fails the first two designs for all five earlier messages and the third for message 4, the one whose chain key it kept. Only the design that keeps nothing but the next chain key clears the audit. An audit like this catches designs that keep or reuse keys; what it cannot prove is that HMAC is one-way, which the whole chain assumes. Press Edit and add a design of your own, for instance one that keeps the chain key of every fifth message, and run it again.
Key idea
Forward secrecy in a ratchet needs two things together: a step that cannot be run backwards, and the habit of deleting every message key and every old chain key as soon as it has been used.
Checkpoint
Check: A key chain that forgets
Messages arrive late and twice, so keys wait in a store with a limit
Networks drop, delay and reorder. If message 3 arrives before message 2, the receiver must step the chain past 2 to reach the key for 3, and keep the key for 2 until that message turns up. These skipped message key: A message key that a receiver computes ahead and stores because the message with that number has not arrived yet. The key is deleted as soon as its message opens, and the store has a limit. are the one place a ratchet keeps old keys on purpose, so sections 3.2 and 3.5 of the specification put rules on them. A stored key is deleted as soon as its message arrives. A number behind the chain with no stored key is a message already received, and is refused. And the store has a limit.
The limit matters because a sender controls the number in the header. A packet that claims message 1,000,000 would make the receiver compute a million chain steps and store a million 32-byte keys, about 32 MB, before it had checked a single tag. The specification calls the cap on a gap MAX_SKIP and leaves its value to the implementer. Its security considerations also ask for a limit on the skipped keys stored for a session, with 1,000 as an example. A second rule is as important as the cap: a message that fails its tag check must change nothing, or one forged packet would push the chain ahead and the real message could never open.
| The message number is | The receiver | What changes |
|---|---|---|
| Behind the chain, no key stored | refuses it as a replay or too old | nothing |
| Behind the chain, key stored | opens it with the stored key and deletes the key | the store shrinks |
| The next number | opens it and steps the chain on | one step |
| Ahead by at most MAX_SKIP | steps ahead, stores the keys it skips over, then opens it | the store grows |
| Ahead by more than MAX_SKIP | refuses it | nothing |
Every key in the store is a key a later copy of the device could use, so the store holds only what is waiting. The second step of the code lab at the end of this lesson writes this table as a receiving chain, with the limit and the rule that a message which does not open changes nothing, in the practice section.
Checkpoint
Check: Skipped keys and their limit
A fresh Diffie-Hellman key on every reply heals a copied phone
The chain alone leaves a thief with a permanent seat: whoever holds chain key 3 can compute keys 4, 5 and 6 as fast as the phone can. To lock the thief out, the two phones need a new secret that the copy never contained. The Double Ratchet: Signal's algorithm for an ongoing conversation: a symmetric-key ratchet for every message, and a Diffie-Hellman ratchet that adds a fresh shared secret whenever the direction of the conversation changes. adds it with a second ratchet that turns more slowly. Every message header carries the sender’s current ratchet public key. When the direction of the conversation changes, the receiver runs X25519 between that key and her own private ratchet key, feeds the result into HKDF with the root key as the salt, and gets a new root key and a new chain key. Then she makes a new ratchet key pair, does the same once more with it, and starts her own sending chain from the result. The old private key is overwritten. This is the Diffie-Hellman ratchet: The half of the Double Ratchet that mixes a new Diffie-Hellman output into the root key each time a party answers with a fresh ratchet key pair. That is what restores secrecy after a key copy.; step through one exchange with a thief in the picture:
The drawing shows the whole scene; its 6 steps are listed below.
Every step as text
- Alice starts. Alice sends messages 0 and 1 with her ratchet public key A0 in the header. The chain behind them started from a Diffie-Hellman value of A0 and Bob's B0.
- The copy. A thief copies Alice's whole state: the private key a0, the root key and the sending chain key. Everything older was deleted, so messages 0 and 1 are already closed.
- Bob answers with B1. Bob makes a new key pair B1 and encrypts messages 2 and 3 from a root key that mixes in X25519 of B1 and A0, the key Alice used.
- The thief can follow. The thief holds a0 and sees B1 in the header, so it computes the same X25519 value Alice does. Messages 2 and 3 open for the thief.
- Alice answers with A1. Alice makes a new key pair A1 and mixes X25519 of A1 and B1 into the root key. The thief never saw a1, and does not hold b1, so it cannot compute that value.
- Secrecy is back. Every later root key depends on that value. Bob's message 5 and Alice's message 6 stay closed to the thief, who would need a new copy to get back in.
The program below runs the algorithm of section 3 of the specification on X25519 for seven messages. It takes two copies of Alice’s state, one after message 1 and one after message 4, and gives each to a clone of Alice that tries to read every message. The clone draws its own random ratchet keys, as a thief would: it cannot know the ones Alice draws.
// A Double Ratchet on X25519 (sections 3.2 to 3.5 of the Double Ratchet specification, without header encryption):
// Alice and Bob talk for seven messages, and a thief copies Alice's state twice, once after message 1 and once after
// message 4. A thief clone of Alice then tries to read every message. Keys come from fixed lab seeds.
import { createCipheriv, createDecipheriv, createHmac, createPrivateKey, createPublicKey, diffieHellman, hkdfSync } from 'node:crypto';
const PKCS8 = Buffer.from('302e020100300506032b656e04220420', 'hex');
const SPKI = Buffer.from('302a300506032b656e032100', 'hex');
const privateKeyOf = (seed) => createPrivateKey({ key: Buffer.concat([PKCS8, seed]), format: 'der', type: 'pkcs8' });
const publicOf = (seed) => Buffer.from(createPublicKey(privateKeyOf(seed)).export({ type: 'spki', format: 'der' }).subarray(-32));
const dh = (seed, pub) => diffieHellman({ privateKey: privateKeyOf(seed), publicKey: createPublicKey({ key: Buffer.concat([SPKI, pub]), format: 'der', type: 'spki' }) });
const hmac = (key, byte) => createHmac('sha256', key).update(Buffer.from([byte])).digest();
const kdfCk = (ck) => ({ mk: hmac(ck, 0x01), ck: hmac(ck, 0x02) });
const kdfRk = (rk, out) => {
const okm = Buffer.from(hkdfSync('sha256', out, rk, 'msc double ratchet demo', 64));
return { rk: okm.subarray(0, 32), ck: okm.subarray(32) };
};
function seal(mk, text, ad) {
const okm = Buffer.from(hkdfSync('sha256', mk, Buffer.alloc(32), 'msc ratchet demo v1', 44));
const c = createCipheriv('aes-256-gcm', okm.subarray(0, 32), okm.subarray(32, 44));
c.setAAD(ad);
return Buffer.concat([c.update(text, 'utf8'), c.final(), c.getAuthTag()]);
}
function open(mk, blob, ad) {
const okm = Buffer.from(hkdfSync('sha256', mk, Buffer.alloc(32), 'msc ratchet demo v1', 44));
const d = createDecipheriv('aes-256-gcm', okm.subarray(0, 32), okm.subarray(32, 44));
d.setAAD(ad);
d.setAuthTag(blob.subarray(-16));
try {
return Buffer.concat([d.update(blob.subarray(0, -16)), d.final()]).toString('utf8');
} catch {
return null;
}
}
// Each party draws its fresh ratchet keys from its own stream of seeds. A thief has a different stream.
const stream = (label) => {
let i = 0;
return () => createHmac('sha256', label).update(String(i++)).digest();
};
class Party {
constructor(label, next) {
this.label = label;
this.next = next;
this.s = { dhs: null, dhr: null, rk: null, cks: null, ckr: null, ns: 0, nr: 0, pn: 0 };
}
static initAlice(next, sk, bobPub) {
const p = new Party('alice', next);
p.s.dhs = next();
p.s.dhr = bobPub;
({ rk: p.s.rk, ck: p.s.cks } = kdfRk(sk, dh(p.s.dhs, bobPub)));
return p;
}
static initBob(next, sk, bobSeed) {
const p = new Party('bob', next);
p.s.dhs = bobSeed;
p.s.rk = sk;
return p;
}
static stolen(state, next) {
const p = new Party('thief', next);
p.s = { ...state };
return p;
}
snapshot() {
return { ...this.s };
}
send(text) {
const { mk, ck } = kdfCk(this.s.cks);
this.s.cks = ck; // the old chain key is gone
const header = { pub: publicOf(this.s.dhs), n: this.s.ns++ };
return { header, blob: seal(mk, text, Buffer.concat([header.pub, Buffer.from([header.n])])) };
}
ratchet(pub) {
this.s.pn = this.s.ns;
this.s.ns = 0;
this.s.nr = 0;
this.s.dhr = pub;
({ rk: this.s.rk, ck: this.s.ckr } = kdfRk(this.s.rk, dh(this.s.dhs, pub)));
this.s.dhs = this.next(); // a fresh key pair; the old private key is overwritten
({ rk: this.s.rk, ck: this.s.cks } = kdfRk(this.s.rk, dh(this.s.dhs, pub)));
}
receive(msg) {
if (!this.s.dhr || !msg.header.pub.equals(this.s.dhr)) this.ratchet(msg.header.pub);
const { mk, ck } = kdfCk(this.s.ckr);
this.s.ckr = ck;
this.s.nr++;
return open(mk, msg.blob, Buffer.concat([msg.header.pub, Buffer.from([msg.header.n])]));
}
// What a copy of Alice can do with a message Alice herself sent: run the sending chain forward, if the key is the same.
readOwn(msg) {
if (!this.s.dhs || !publicOf(this.s.dhs).equals(msg.header.pub) || msg.header.n < this.s.ns) return null;
let ck = this.s.cks;
let mk;
for (let n = this.s.ns; n <= msg.header.n; n++) ({ mk, ck } = kdfCk(ck));
return open(mk, msg.blob, Buffer.concat([msg.header.pub, Buffer.from([msg.header.n])]));
}
}
const SK = Buffer.alloc(32, 0x42);
const bobSeed = stream('bob')(); // Bob's first ratchet key, known to Alice like a signed prekey
const alice = Party.initAlice(stream('alice'), SK, publicOf(bobSeed));
const bob = Party.initBob(stream('bob.next'), SK, bobSeed);
const script = [
['alice', 'hello'],
['alice', 'are you there'],
['bob', 'yes, I am here'],
['bob', 'send the file'],
['alice', 'sending now'],
['bob', 'got it'],
['alice', 'thanks'],
];
const transcript = [];
const thefts = { 1: null, 4: null };
script.forEach(([who, text], i) => {
const sender = who === 'alice' ? alice : bob;
const receiver = who === 'alice' ? bob : alice;
const msg = { from: who, header: null, blob: null, text };
Object.assign(msg, sender.send(text));
if (receiver.receive(msg) !== text) throw new Error(`message ${i} did not arrive`);
transcript.push(msg);
if (i in thefts) thefts[i] = alice.snapshot();
});
// A thief with Alice's copy tries every message: the earlier ones with a fresh clone each, the later ones in order.
function attempt(theftIndex) {
const state = thefts[theftIndex];
const result = [];
const thief = Party.stolen(state, stream(`thief ${theftIndex}`));
transcript.forEach((msg, i) => {
let text;
if (i <= theftIndex) {
const fresh = Party.stolen(state, stream(`thief ${theftIndex}`));
text = msg.from === 'alice' ? fresh.readOwn(msg) : fresh.receive(msg);
} else {
text = msg.from === 'alice' ? thief.readOwn(msg) : thief.receive(msg);
}
result.push(text === msg.text);
});
return result;
}
const first = attempt(1);
const second = attempt(4);
// Names for the ratchet public keys, in the order they first appear: A0, A1, A2 for Alice, B0 (never used to send), B1, B2 for Bob.
const names = new Map([[publicOf(bobSeed).toString('hex'), 'B0']]);
for (const who of ['alice', 'bob']) {
let count = who === 'alice' ? 0 : 1;
for (const msg of transcript) {
const hex = msg.header.pub.toString('hex');
if (msg.from === who && !names.has(hex)) names.set(hex, `${who === 'alice' ? 'A' : 'B'}${count++}`);
}
}
console.log('message sender ratchet key thief with the copy after message 1 thief with the copy after message 4');
transcript.forEach((msg, i) => {
const key = names.get(msg.header.pub.toString('hex'));
const col = (r, at) => `${i <= at ? '(before) ' : ''}${r[i] ? 'READS' : 'refused'}`;
console.log(`${String(i).padEnd(8)} ${msg.from.padEnd(7)} ${key.padEnd(12)} ${col(first, 1).padEnd(38)}${col(second, 4)}`);
}); Output
message sender ratchet key thief with the copy after message 1 thief with the copy after message 4 0 alice A0 (before) refused (before) refused 1 alice A0 (before) refused (before) refused 2 bob B1 READS (before) refused 3 bob B1 READS (before) refused 4 alice A1 refused (before) refused 5 bob B2 refused READS 6 alice A2 refused refused
Recorded with Node.js 24.21.0 on macOS 26 arm64. To run it yourself: mise exec node@24.21.0 -- node dh_ratchet.mjs
With the copy from after message 1, the thief reads messages 2 and 3: Bob made their keys from a value computed with Alice’s old ratchet public key, and the copy holds the private half, so it recomputes that value. It reads nothing from message 4 on: Alice replied with a new key pair, and the root key moved on with a value the copy cannot compute. With the copy from after message 4, the thief reads message 5 only; messages 2 and 3 stay closed, because the copy no longer holds the old private key or the old chain keys. So the healing window runs from the copy to the owner’s next reply under a new key, and the same data shows forward secrecy again.
Healing is for a thief who only listens
Section 8.2 of the specification says that recovery targets passive eavesdroppers. A thief who also holds the identity key and can send messages as Alice, or one who can predict her random numbers, can follow along, so a suspected copy calls for new keys and a new session.
Key idea
The symmetric chain forgets the past, and the Diffie-Hellman step shuts the thief out of the future, because each new root key needs a private key that the copy never held.
Checkpoint
Check: The Diffie-Hellman step that heals
Starting a session with someone who is offline: X3DH and PQXDH
A ratchet needs a shared secret and the other side’s first ratchet public key before it can start. Alice wants to message Bob while his phone is off, so Bob cannot take part in a handshake. X3DH: Extended Triple Diffie-Hellman, Signal's way to start a session with someone who is offline: the sender combines four Diffie-Hellman values from the receiver's published prekey bundle and her own keys into one secret. solves that with a prekey bundle: The public keys a user leaves on a server so that others can start a session while they are offline: an identity key, a signed prekey with its signature, and one-time prekeys. that Bob leaves on a server: his long-term identity key IK, a medium-term signed prekey SPK with a signature made by IK, and a batch of one-time prekeys OPK, each of which the server hands out once and deletes. Alice adds her own identity key and a fresh ephemeral key EK, and computes four Diffie-Hellman values:
In words: the first two values tie each side’s long-term key to the other’s, which authenticates them, and the last two use only keys that Alice or Bob later deletes, which gives forward secrecy. F is 32 bytes of 0xFF for X25519, which keeps this input apart from the one used for signatures, and the salt is 32 zero bytes. Step through the exchange:
The drawing shows the whole scene; its 6 steps are listed below.
Every step as text
- Bob leaves his keys. Bob uploads his identity key IK, a signed prekey SPK with a signature made by IK, and a batch of one-time prekeys. The private halves stay on his phone.
- Alice asks. Bob is offline, so he cannot take part in a handshake. Alice asks the server for his bundle.
- The bundle. The server returns IK, SPK, the signature and one one-time prekey OPK, and deletes that OPK so no one else gets it.
- Alice checks, then computes. Alice verifies the signature on SPK and aborts if it fails. Then she makes an ephemeral key EK and computes four values, DH(IK_A, SPK), DH(EK, IK), DH(EK, SPK) and DH(EK, OPK), and puts them through HKDF.
- The first message. Alice sends her identity key IK_A, her ephemeral key EK, the id of the OPK she used and a first message sealed under a key from SK, with both identity keys as associated data.
- Bob computes the same key. When Bob comes online he makes the same four values from his private keys and Alice's public keys, gets the same SK, and deletes the private key of the one-time prekey.
The program computes it on X25519 with Bob’s bundle, then adds the post-quantum step. PQXDH: X3DH with a post-quantum key-encapsulation secret added behind the Diffie-Hellman values, so a recording of the handshake stays closed against a future quantum computer. Authentication still uses classical keys. has Alice encapsulate a secret to a KEM public key in the bundle and put that secret behind the Diffie-Hellman values in the same HKDF call. The specification names CRYSTALS-Kyber-1024 as its example, and the program uses ML-KEM-1024, its standardised successor in FIPS 203. A copy of the handshake recorded today then stays closed to a future quantum computer, at a cost in bytes. The program also tests the check a client must make on the bundle:
// The X3DH key agreement of the Signal specification (four Diffie-Hellman values, then HKDF), then the PQXDH change that
// puts a KEM secret behind them (ML-KEM-1024), and a bundle check that a client must make. Bob's prekey signature is made
// with a separate Ed25519 key here; Signal's XEdDSA signs with the X25519 identity key itself. X25519 keys come from fixed
// lab seeds; the ML-KEM keys are random, so the example prints only what does not depend on them.
import {
createCipheriv, createDecipheriv, createHash, createPrivateKey, createPublicKey, decapsulate, diffieHellman, encapsulate,
generateKeyPairSync, hkdfSync, sign, verify,
} from 'node:crypto';
const seed = (label) => createHash('sha256').update(`msc x3dh demo ${label}`).digest();
const X_PKCS8 = Buffer.from('302e020100300506032b656e04220420', 'hex');
const X_SPKI = Buffer.from('302a300506032b656e032100', 'hex');
const E_PKCS8 = Buffer.from('302e020100300506032b657004220420', 'hex');
const xPrivate = (label) => createPrivateKey({ key: Buffer.concat([X_PKCS8, seed(label)]), format: 'der', type: 'pkcs8' });
const xPublic = (label) => Buffer.from(createPublicKey(xPrivate(label)).export({ type: 'spki', format: 'der' }).subarray(-32));
const dh = (privLabel, pub) => diffieHellman({ privateKey: xPrivate(privLabel), publicKey: createPublicKey({ key: Buffer.concat([X_SPKI, pub]), format: 'der', type: 'spki' }) });
const F = Buffer.alloc(32, 0xff); // keeps the KDF input apart from signature inputs, as the specification says
function kdf(parts, info) {
return Buffer.from(hkdfSync('sha256', Buffer.concat([F, ...parts]), Buffer.alloc(32), info, 32));
}
// Bob's prekey bundle, as the server hands it out.
const bobSigning = createPrivateKey({ key: Buffer.concat([E_PKCS8, seed('bob signing')]), format: 'der', type: 'pkcs8' });
const bobVerify = createPublicKey(bobSigning);
const pq = generateKeyPairSync('ml-kem-1024');
const pqRaw = pq.publicKey.export({ format: 'raw-public' });
const bundle = {
ik: xPublic('bob identity'),
spk: xPublic('bob signed prekey'),
opk: xPublic('bob one-time prekey'),
pqpk: pqRaw,
};
bundle.spkSig = sign(null, bundle.spk, bobSigning);
bundle.pqSig = sign(null, bundle.pqpk, bobSigning);
function checkBundle(b) {
if (!verify(null, b.spk, bobVerify, b.spkSig)) throw new Error('signed prekey does not verify');
if (b.pqpk && !verify(null, b.pqpk, bobVerify, b.pqSig)) throw new Error('post-quantum prekey does not verify');
}
// Alice's side: ephemeral key EK, then DH1..DH4 and, for PQXDH, the KEM secret.
function aliceAgrees(b, { verifyBundle = true, useOpk = true, usePq = false } = {}) {
if (verifyBundle) checkBundle(b);
const dhs = [dh('alice identity', b.spk), dh('alice ephemeral', b.ik), dh('alice ephemeral', b.spk)];
if (useOpk) dhs.push(dh('alice ephemeral', b.opk));
let ct = null;
if (usePq) {
const out = encapsulate(createPublicKey({ key: b.pqpk, format: 'raw-public', asymmetricKeyType: 'ml-kem-1024' }));
ct = out.ciphertext;
dhs.push(out.sharedKey);
}
return { sk: kdf(dhs, usePq ? 'msc-demo_CURVE25519_SHA-256_ML-KEM-1024' : 'msc-demo'), ct };
}
// Bob's side from Alice's first message: her identity key, her ephemeral key and which prekeys she used.
function bobAgrees({ useOpk = true, ct = null } = {}) {
const ikA = xPublic('alice identity');
const ekA = xPublic('alice ephemeral');
const dhs = [dh('bob signed prekey', ikA), dh('bob identity', ekA), dh('bob signed prekey', ekA)];
if (useOpk) dhs.push(dh('bob one-time prekey', ekA));
if (ct) dhs.push(decapsulate(pq.privateKey, ct));
return kdf(dhs, ct ? 'msc-demo_CURVE25519_SHA-256_ML-KEM-1024' : 'msc-demo');
}
const eq = (a, b) => a.equals(b);
const x3dh = aliceAgrees(bundle);
console.log(`X3DH, four DH values: both sides derive the same 32 bytes: ${eq(x3dh.sk, bobAgrees())}`);
const noOpk = aliceAgrees(bundle, { useOpk: false });
console.log(`no one-time prekey, three DH: same on both sides: ${eq(noOpk.sk, bobAgrees({ useOpk: false }))}, same as with four: ${eq(noOpk.sk, x3dh.sk)}`);
const pqxdh = aliceAgrees(bundle, { usePq: true });
console.log(`PQXDH, plus ML-KEM-1024: same on both sides: ${eq(pqxdh.sk, bobAgrees({ ct: pqxdh.ct }))}, same as X3DH: ${eq(pqxdh.sk, x3dh.sk)}`);
console.log(`ML-KEM-1024 sizes: public key ${pqRaw.length} bytes, ciphertext ${pqxdh.ct.length} bytes, secret 32 bytes`);
// The initial message is sealed under a key from SK, with both identity keys as associated data.
const ad = Buffer.concat([xPublic('alice identity'), bundle.ik]);
const okm = Buffer.from(hkdfSync('sha256', x3dh.sk, Buffer.alloc(32), 'msc-demo initial message', 44));
const c = createCipheriv('aes-256-gcm', okm.subarray(0, 32), okm.subarray(32));
c.setAAD(ad);
const blob = Buffer.concat([c.update('hello bob', 'utf8'), c.final(), c.getAuthTag()]);
function openInitial(sk, associated) {
const k = Buffer.from(hkdfSync('sha256', sk, Buffer.alloc(32), 'msc-demo initial message', 44));
const d = createDecipheriv('aes-256-gcm', k.subarray(0, 32), k.subarray(32));
d.setAAD(associated);
d.setAuthTag(blob.subarray(-16));
try {
return Buffer.concat([d.update(blob.subarray(0, -16)), d.final()]).toString('utf8');
} catch {
return 'refused';
}
}
console.log(`\ninitial message at Bob: ${openInitial(bobAgrees(), ad)}; with another identity in the associated data: ${openInitial(bobAgrees(), Buffer.concat([xPublic('mallory identity'), bundle.ik]))}`);
// The check a client must make: a bundle whose signed prekey was swapped must be refused. A server that swaps the prekey
// for one it holds, and withholds the one-time prekey, can derive the key of the session if Bob's identity key leaks later.
const swapped = { ...bundle, spk: xPublic('swapped prekey') }; // the old signature does not cover this key
for (const verifyBundle of [false, true]) {
try {
const r = aliceAgrees(swapped, { verifyBundle, useOpk: false });
const laterLeak = [dh('swapped prekey', xPublic('alice identity')), dh('bob identity', xPublic('alice ephemeral')), dh('swapped prekey', xPublic('alice ephemeral'))];
console.log(`client ${verifyBundle ? 'checks' : 'skips'} the signature: accepts the swapped prekey; if Bob's identity key leaks later, its holder derives the same SK: ${eq(r.sk, kdf(laterLeak, 'msc-demo'))}`);
} catch (e) {
console.log(`client ${verifyBundle ? 'checks' : 'skips'} the signature: refuses the swapped prekey (${e.message})`);
}
} Output
X3DH, four DH values: both sides derive the same 32 bytes: true no one-time prekey, three DH: same on both sides: true, same as with four: false PQXDH, plus ML-KEM-1024: same on both sides: true, same as X3DH: false ML-KEM-1024 sizes: public key 1568 bytes, ciphertext 1568 bytes, secret 32 bytes initial message at Bob: hello bob; with another identity in the associated data: refused client skips the signature: accepts the swapped prekey; if Bob's identity key leaks later, its holder derives the same SK: true client checks the signature: refuses the swapped prekey (signed prekey does not verify)
Recorded with Node.js 24.21.0 on macOS 26 arm64. To run it yourself: mise exec node@24.21.0 -- node x3dh.mjs
Both sides derive the same key in each variant. With no one-time prekey Alice uses three values and gets a different key, so her first message has to say which variant she used. The ML-KEM-1024 public key and ciphertext are 1,568 bytes each, against 32 bytes for an X25519 public key, and the bundle signs the KEM key like the others. The first message is sealed with both identity keys as associated data, so a message that names another identity is refused. The PQXDH specification adds that its authentication still rests on classical keys: it protects a recorded handshake, not a live one against an attacker with a quantum computer. The ML-KEM Braid specification carries the same idea into the running conversation, spreading the large KEM material over many messages.
Skipping the signature on the prekey
Wrong: use the bundle as the server sent it. In the program, a client that skips the check accepts a swapped prekey, and whoever holds that key, when the one-time prekey is withheld too, derives the session key if Bob’s identity key leaks later. Right: verify the signature on every prekey, and abort when it fails, as the second client does.
Checkpoint
Check: Starting a session with X3DH and PQXDH
Groups: a ratchet tree makes an update cost about log N ciphertexts, not N
Pairwise ratchets scale badly. In a group of 1,000 each member holds 999 sessions, and refreshing one member’s key means 999 ciphertexts. Section 1 of RFC 9420 adds that designs with a sender key per member need a number of messages that grows with the square of the group size to restore post-compromise security for everyone. Messaging Layer Security (MLS): The protocol of RFC 9420 for end-to-end encrypted groups of two to thousands of members, with forward secrecy, post-compromise security and key updates that cost about the logarithm of the group size. arranges the members as the leaves of a ratchet tree: The binary tree at the heart of MLS. Each member is a leaf and knows the secrets on its path to the root, so one update is encrypted to a few subtrees instead of to every member.. Each member knows the secrets of the nodes on its path to the root. To update, a member picks new secrets for its path and encrypts each one to the subtree that hangs off the path at that level. For eight members that is three ciphertexts, against seven for pairwise sessions:
One update by member C: three ciphertexts instead of seven
Text description of the diagram
The eight members A to H are the leaves at the bottom of a binary tree. Above them are the pairs AB, CD, EF and GH, then the groups ABCD and EFGH, then the root, which stands for all eight members. Each member knows the secrets of the nodes on the path from its leaf to the root.
Member C updates. It chooses new secrets for its own path, the nodes CD, ABCD and the root, which are drawn with a shaded fill together with C. Each new secret must reach the members under the node that is not on C's path. The secret for CD goes to D alone, the secret for ABCD goes to the pair AB, and the secret for the root goes to the group EFGH. These three nodes are drawn with a thick outline, and each receives one ciphertext. Everyone can then work out the new secrets above the node it was given. Pairwise sessions would need one ciphertext for each of the seven other members.
A Commit message applies a batch of proposals, such as Add, Update and Remove, and starts a new epoch: One period of an MLS group with a fixed set of members and one set of group secrets. Every Commit message starts the next epoch. with new group secrets. A Welcome message brings a new member into the epoch. A removed member does not receive the fresh secrets, which is how the group closes the door. The delivery service orders and stores messages. RFC 9750 says that even a malicious one cannot join a group by itself or recover the group key; it can still infer who is in a group, and it can reorder messages or refuse to relay them.
The program counts the ciphertexts of one update with the rules of sections 4.1 and 7.5 of RFC 9420, in a model that leaves out unmerged leaves and the Welcome message:
"""What one key update costs in an MLS ratchet tree, counted in HPKE ciphertexts (RFC 9420 sections 4.1, 7.5 and 12.1.3).
The model keeps the rules that decide the count: a perfect binary tree whose leaves hold the members, a node is blank
until a commit passes through it, the resolution of a blank node is the resolutions of its two children, and a commit
encrypts each new path secret once for every node in the resolution of the child that is not on the committer's path.
It leaves out unmerged leaves, the Welcome message and the tree growing; pairwise sessions need N - 1 ciphertexts."""
from math import ceil, log2
class Tree:
def __init__(self, members: int):
self.members = members
self.depth = max(1, ceil(log2(members)))
self.size = 2**self.depth # leaves; nodes are numbered like a heap: the root is 1, the leaves are size.. 2*size-1
self.keyed = [False] * (2 * self.size) # a leaf: a member is there; an inner node: it holds a key
for i in range(members):
self.keyed[self.size + i] = True
def resolution(self, node: int) -> int:
"""How many nodes must be encrypted to so that everyone under this node is reached."""
if self.keyed[node]:
return 1
if node >= self.size: # a blank leaf: nobody is there
return 0
return self.resolution(2 * node) + self.resolution(2 * node + 1)
def cost(self, leaf: int) -> int:
"""Ciphertexts in the update path of a commit by the member at this leaf."""
node, total = self.size + leaf, 0
while node > 1:
total += self.resolution(node ^ 1) # the child that is not on the committer's path
node >>= 1
return total
def commit(self, leaf: int) -> int:
"""The same commit, applied: every node on the path whose other side is not empty gets a new key."""
total = self.cost(leaf)
node = self.size + leaf
while node > 1:
self.keyed[node >> 1] = self.resolution(node ^ 1) > 0
node >>= 1
return total
def key_everything(self) -> None:
"""The state after every member has committed once: each node above a member holds a key."""
for node in range(self.size - 1, 0, -1):
self.keyed[node] = self.keyed[2 * node] or self.keyed[2 * node + 1]
def remove(self, leaf: int) -> None:
"""A removed member's leaf and every node above it are blanked (RFC 9420 section 12.1.3)."""
node = self.size + leaf
while node >= 1:
self.keyed[node] = False
node >>= 1
print(f"{'members':>8} {'depth':>5} {'pairwise':>9} {'first update':>13} {'update, tree keyed':>19}")
for n in (8, 100, 1000, 10000, 100000):
fresh, ready = Tree(n), Tree(n)
ready.key_everything()
print(f"{n:>8,} {fresh.depth:>5} {n - 1:>9,} {fresh.cost(0):>13,} {ready.cost(0):>19,}")
# 1,000 members with a keyed tree. A quarter of them leave, one commit each, by the member next to the leaver.
t = Tree(1000)
t.key_everything()
leavers = list(range(4, 1000, 4))
commit_costs = []
for leaver in leavers:
t.remove(leaver)
commit_costs.append(t.commit(leaver + 1))
present = [m for m in range(1000) if m not in set(leavers)]
update_costs = [t.cost(m) for m in present]
print(f"\n{len(leavers)} of 1,000 members leave, one commit each: those commits cost {min(commit_costs)} to {max(commit_costs)} ciphertexts")
print(f"then an update by each of the {len(present)} remaining members would cost {min(update_costs)} to {max(update_costs)}, {sum(update_costs) / len(update_costs):.1f} on average") Output
members depth pairwise first update update, tree keyed
8 3 7 7 3
100 7 99 99 7
1,000 10 999 999 10
10,000 14 9,999 9,999 14
100,000 17 99,999 99,999 17
249 of 1,000 members leave, one commit each: those commits cost 7 to 9 ciphertexts
then an update by each of the 751 remaining members would cost 7 to 10, 9.7 on average
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 ratchet_tree.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
Once the tree is keyed, an update costs the depth of the tree, 10 for 1,000 members and 17 for 100,000, against 999 and 99,999 for pairwise sessions. The “first update” column equals the pairwise cost: in a group where no member has committed yet every inner node is blank, the resolution of a blank node is its two children, and the update must reach every leaf. The tree pays once members have committed through it. RFC 9750 also advises asking idle members to refresh their keys, because an old key is a risk if its device is ever copied. A removal blanks the leaver’s path, and the member who commits it renews its own path at once: after 249 of 1,000 members left this way, an update by each of the 751 who remained would cost 7 to 10 ciphertexts.
Checkpoint
Check: Groups with a ratchet tree
What end-to-end encryption leaves visible
Everything so far protects the content of messages from the network and the server. The table lists what it leaves, with the exposure each item has in the specifications:
| What stays exposed | Why | What reduces it |
|---|---|---|
| metadata: Data about a message rather than its content: who sent it to whom, when, how large it is and from which address. End-to-end encryption protects the content, not these.: who talks to whom, when, how often | The server must route messages; MLS headers carry the group ID, the epoch and the type of message in the clear (RFC 9750 section 8.1) | Encrypt handshake messages too, hide network addresses with a relay, send less identifying information |
| Message length | Encryption hides content, not size; padding is optional in MLS (RFC 9750 section 8.2.1) | Pad messages to fixed sizes |
| Whose key it is | Without a comparison outside the channel, neither side has proof of whom the key belongs to (X3DH section 4.1) | key verification: Comparing a short code or QR code derived from two identity keys, in person or over another channel, so that two users can tell that nobody sits between them. Without it, neither side has proof of who holds the other key. by a code or QR code, and key transparency: A public, append-only log of which key belongs to which identity, so a user can check what the service says about their keys and anyone can detect a swapped key. logs (RFC 9750 section 4) |
| What a device holds now | A ratchet limits what one copy reveals about the past and the future, not what the phone shows or stores | Encrypt stored messages and remove copies (RFC 9750 section 8.3.4) |
| Authentication against a quantum attacker | The authentication of PQXDH is still classical | Watch for post-quantum authentication in later revisions of the specifications |
Read the first row with the group example in mind: a delivery service that cannot read a message can still count members and see when a Commit arrives. The ratchets decide what a stolen key is worth, not what a service can infer.
A teaching ratchet is not a messenger
The programs of this lesson are models that make each property testable. A messenger needs an audited implementation of the specifications, a plan for backups and linked devices, and a way for users to verify keys.
Checkpoint
Check: What stays visible
Practise: build a receiving chain
Two exercises with sample tests that run on this page build the receiving end of a ratchet. The first writes the HMAC step with its two constants and a chain that keeps nothing it has used. The second turns the table of the third section into a class: stored keys for late messages, a cap on gaps and on the store, and no change of state unless the message opens:
Code lab · 2 steps
Build the receiving end of a ratchet
Write the HMAC chain step, then a receiving chain that handles late, repeated and far-ahead messages within a limit.
- A chain that gives one key per message and forgets it To do
- A receiving chain with a bounded store of skipped keys To do
Exercise · Medium · Python
A chain that gives one key per message and forgets it
Write the two parts of chain.py, the chain that gives a messenger one key per message and forgets each key as it goes (sections 2.2 and 7.2 of the Double Ratchet specification):
kdf_ck(chain_key)returns(next_chain_key, message_key). The message key is the HMAC-SHA-256 of the chain key with the one byte0x01as the message, and the next chain key is the HMAC-SHA-256 of the chain key with the one byte0x02. Usehmac.digest(key, message, "sha256").Chain(chain_key)holds a 32-byte chain key and counts messages from 0. Any other key length, or a key that is notbytes, raisesValueError.next_key()returns(number, message_key)and moves the chain on. After it returns, the chain must keep neither the message key nor the chain key it came from: the only secret it holds is the next chain key, whichstate()returns, as a thief copying the device would find it.
The starter code uses the chain key itself as every message key and never moves the chain on, which is the design the lesson's thief opened in full.
Starter code · chain.py
import hmac
def kdf_ck(chain_key: bytes) -> tuple[bytes, bytes]:
"""(next chain key, message key): HMAC-SHA-256 of the chain key with b"\x02" and with b"\x01"."""
return chain_key, chain_key
class Chain:
def __init__(self, chain_key: bytes) -> None:
"""A chain from a 32-byte chain key; any other key raises ValueError."""
self._ck = chain_key
self.number = 0
def next_key(self) -> tuple[int, bytes]:
"""(the number of this message, its message key); the chain moves on and keeps nothing it used."""
return self.number, self._ck
def state(self) -> bytes:
"""What a thief copying the device would find: the chain key as it is now."""
return self._ck The sample tests · test_chain.py
import hmac
from chain import Chain, kdf_ck
CK = bytes([0x5A] * 32) # the lab chain key of the lesson's examples
# Message keys 0 and 1 and the chain key after two messages, as the lesson's examples compute them.
MK0 = bytes.fromhex("d135461faae19c2be32a6ae43db70d12e1c6666a5756c6cad05426e6217cc1c6")
MK1 = bytes.fromhex("bbe84a6f3a719cab532711278d227732238351a5f27e2e19e666de275238ac46")
CK2 = bytes.fromhex("4c0cfb42c33ed66e6edf5c5432baac1c2d35c45618cdbf56ac26571668736c78")
def raises_value_error(call):
try:
call()
except ValueError:
return True
return False
def test_kdf_ck_uses_the_two_constants():
"""the next chain key is HMAC(ck, 0x02) and the message key HMAC(ck, 0x01)"""
next_ck, mk = kdf_ck(CK)
assert mk == MK0
assert next_ck == hmac.digest(CK, b"\x02", "sha256")
assert next_ck != mk
def test_numbers_and_keys_follow_the_chain():
"""messages are numbered from 0 and each key comes from the chain key the last one left"""
chain = Chain(CK)
assert chain.next_key() == (0, MK0)
assert chain.next_key() == (1, MK1)
assert chain.state() == CK2
def test_two_chains_from_one_key_agree():
"""Alice's sending chain and Bob's receiving chain walk the same keys"""
a, b = Chain(CK), Chain(CK)
assert [a.next_key() for _ in range(5)] == [b.next_key() for _ in range(5)]
assert Chain(bytes([0x5B] * 32)).next_key() != Chain(CK).next_key()
def test_the_chain_keeps_nothing_it_used():
"""a thief who copies the state after five messages gets no earlier key, held or derivable"""
chain = Chain(CK)
used = [chain.next_key()[1] for _ in range(5)]
held = [v for v in vars(chain).values() if isinstance(v, (bytes, bytearray))]
assert all(v not in used for v in held)
assert all(not isinstance(v, (list, dict, set, tuple)) for v in vars(chain).values())
ck, forward = chain.state(), set()
for _ in range(200):
ck, mk = kdf_ck(ck)
forward.add(mk)
assert not forward & set(used)
assert chain.state() not in used
def test_a_key_of_the_wrong_size_is_refused():
"""16 and 33 bytes, text and nothing are not chain keys"""
for bad in (bytes(16), bytes(33), "k" * 32, None):
assert raises_value_error(lambda bad=bad: Chain(bad)) A hint
Compute both values from the same chain key before you replace it: message_key = hmac.digest(ck, b"\x01", "sha256") and next_ck = hmac.digest(ck, b"\x02", "sha256"). In next_key, take the number first, then self._ck, key = kdf_ck(self._ck) (the old chain key is overwritten by the assignment), then add 1 to the counter.
Results of the sample tests
| Test | Result | Details |
|---|
What your code printed
The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.
Exercise · Hard · Python
A receiving chain with a bounded store of skipped keys
Write ReceivingChain in receiving.py: the receiving end of a message-key chain that must cope with messages that arrive late, twice or far ahead, as the Double Ratchet specification asks (sections 3.2 and 3.5).
ReceivingChain(chain_key) starts at message 0. open_with(number, opens) returns the message key of message number if opens(key) says the message opens with it (the caller's tag check), and otherwise raises ValueError. The rules, with the message each error carries:
- A number behind the chain that has no stored key is a message already received, or one too old:
"replay or too old". - A number with a stored key is a late message. Use that key if
opensaccepts it, and delete it from the store as soon as it is used; ifopensrefuses it, raise"cannot open"and keep the key. - The next number opens with the next key and moves the chain on, only if
opensaccepts it. - A number ahead of the chain needs the keys in between, stored for later. The gap is
numberminus the next number: a gap aboveMAX_SKIPis"too far ahead", and a gap that would put more thanMAX_SKIPkeys in the store is"too many skipped keys". - Nothing changes unless
opensaccepts the message: a refused message, and any error, leaves the chain and the store exactly as they were.
stored() returns the numbers whose keys are in the store, in order. Message keys are kdf_ck of the previous chain key, as in the first exercise; the starter code already has it. The starter code ignores the store and the limit.
Starter code · receiving.py
import hmac
MAX_SKIP = 4
def kdf_ck(chain_key: bytes) -> tuple[bytes, bytes]:
"""(next chain key, message key), as in the first exercise."""
return hmac.digest(chain_key, b"\x02", "sha256"), hmac.digest(chain_key, b"\x01", "sha256")
class ReceivingChain:
def __init__(self, chain_key: bytes) -> None:
self._ck = chain_key
self._next = 0
self._skipped: dict[int, bytes] = {}
def stored(self) -> list[int]:
"""The numbers whose keys are in the store, in order."""
return sorted(self._skipped)
def open_with(self, number: int, opens) -> bytes:
"""The message key of message `number` if opens(key) accepts it; ValueError otherwise (see the prompt)."""
self._ck, key = kdf_ck(self._ck)
if not opens(key):
raise ValueError("cannot open")
self._next = number + 1
return key The sample tests · test_receiving.py
from receiving import MAX_SKIP, ReceivingChain, kdf_ck
CK = bytes([0x5A] * 32) # the lab chain key of the lesson's examples
def message_keys(count):
"""The keys of messages 0 .. count - 1, as the sender's chain makes them."""
ck, out = CK, []
for _ in range(count):
ck, mk = kdf_ck(ck)
out.append(mk)
return out
KEYS = message_keys(20)
def accepts(_key):
return True
def refuses(_key):
return False
def error_of(call):
try:
call()
except ValueError as e:
return str(e)
return None
def test_messages_in_order_store_nothing():
"""each number opens with its own key and no key is stored"""
chain = ReceivingChain(CK)
for n in range(4):
assert chain.open_with(n, accepts) == KEYS[n]
assert chain.stored() == []
def test_a_late_message_uses_its_stored_key_once():
"""3 arrives first, so 1 and 2 are stored; each opens when it comes, leaves the store, and cannot open twice"""
chain = ReceivingChain(CK)
assert chain.open_with(0, accepts) == KEYS[0]
assert chain.open_with(3, accepts) == KEYS[3]
assert chain.stored() == [1, 2]
assert chain.open_with(2, accepts) == KEYS[2]
assert chain.stored() == [1]
assert error_of(lambda: chain.open_with(2, accepts)) == "replay or too old"
assert chain.open_with(1, accepts) == KEYS[1]
assert chain.stored() == []
assert chain.open_with(4, accepts) == KEYS[4]
def test_a_message_already_received_is_a_replay():
"""number 0 again, after it was opened, has no key left"""
chain = ReceivingChain(CK)
chain.open_with(0, accepts)
chain.open_with(1, accepts)
assert error_of(lambda: chain.open_with(0, accepts)) == "replay or too old"
assert error_of(lambda: chain.open_with(1, accepts)) == "replay or too old"
def test_the_limit_on_a_gap():
"""a gap of MAX_SKIP is accepted, a gap of one more is refused and changes nothing"""
chain = ReceivingChain(CK)
chain.open_with(0, accepts)
far = 1 + MAX_SKIP + 1 # next is 1, so this number is MAX_SKIP + 1 ahead
assert error_of(lambda: chain.open_with(far, accepts)) == "too far ahead"
assert chain.stored() == []
assert chain.open_with(1, accepts) == KEYS[1]
ok = ReceivingChain(CK)
assert ok.open_with(MAX_SKIP, accepts) == KEYS[MAX_SKIP]
assert ok.stored() == list(range(MAX_SKIP))
def test_the_store_has_a_limit_too():
"""skipped keys that wait add up: more than MAX_SKIP in the store is refused"""
chain = ReceivingChain(CK)
chain.open_with(3, accepts) # keys 0, 1, 2 are stored
assert chain.stored() == [0, 1, 2]
assert error_of(lambda: chain.open_with(6, accepts)) == "too many skipped keys" # two more would make five
assert chain.stored() == [0, 1, 2]
assert chain.open_with(5, accepts) == KEYS[5] # one more is the fourth: allowed
assert chain.stored() == [0, 1, 2, 4]
def test_a_message_that_does_not_open_changes_nothing():
"""a forged number far ahead must not move the chain or fill the store"""
chain = ReceivingChain(CK)
chain.open_with(0, accepts)
assert error_of(lambda: chain.open_with(4, refuses)) == "cannot open"
assert chain.stored() == []
assert chain.open_with(1, accepts) == KEYS[1]
assert error_of(lambda: chain.open_with(3, refuses)) == "cannot open"
assert chain.stored() == []
assert chain.open_with(2, accepts) == KEYS[2]
def test_a_stored_key_that_fails_to_open_stays():
"""a bad copy of message 1 does not use up its key"""
chain = ReceivingChain(CK)
chain.open_with(2, accepts)
assert error_of(lambda: chain.open_with(1, refuses)) == "cannot open"
assert chain.stored() == [0, 1]
assert chain.open_with(1, accepts) == KEYS[1]
assert chain.stored() == [0] A hint
Work out everything on local variables first and write to self only at the end, once opens(key) has said yes. For a number ahead of the chain, loop from self._next up to number: the keys before number go into a new dictionary, the key at number is the one to test, and the chain key left after it is the new self._ck.
Results of the sample tests
| Test | Result | Details |
|---|
What your code printed
The sample tests run on this device, in your browser (Pyodide): nothing is sent to mysmartcopilot.com. The first run downloads Python (about 13.5 MB), which is kept for the next runs. A check in your browser is feedback for you, not proof that the code is right for every input.
Last, a practice session asks about all seven ideas in turn and comes back to the one you know least; each answer comes with its reason:
Practice that adapts to you
Practise this lesson
19 questions about 7 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
Interview questions
Model answers open with any pass: ₹79 for 1 week ($2.99), one payment that never renews.
-
Fresher to mid · Warm-up
What do forward secrecy and post-compromise security mean for a messaging app, and which part of the Double Ratchet gives each?
Forward secrecy means that a thief who copies a device today cannot read the messages sent before. The symmetric-key ratchet gives it: every message key comes from an HMAC step that cannot be run backwards, and each message key and old chain key is deleted once used. Post-compromise security means that messages sent after the copy close up again once the conversation has moved on. The Diffie-Hellman ratchet gives it: every reply carries a fresh ratchet public key, and the new Diffie-Hellman value is mixed into the root key, which the copy cannot compute. It works against a thief who only listens, not one who can also send as the victim. In the lesson's test a copy taken after message 5 opened 0 of the 5 earlier messages, and, under the chain alone, all 3 later ones.
-
Mid to senior
A product team wants to back up every device's ratchet state to the cloud each night, so that conversations survive a lost phone. Evaluate the idea for forward secrecy and post-compromise security, say what else it breaks, and propose a safer way to restore conversations.
-
Mid to senior
A receiver's chain is at message 12, and a packet's header claims message 4,000,000. Say what the receiver must do before and after checking the tag, what limits you would set, and which tests prove that the rule holds.
Flashcards · 8 cards
Recall it, then check
Say the answer to yourself before you look: recalling it is what makes it stick.
-
What do forward secrecy and post-compromise security promise after a device is copied?
Forward secrecy: a thief who copies the device cannot read what was said before the copy. Post-compromise security, called break-in recovery in the Double Ratchet specification: once the thief is out again, messages sent after a fresh key exchange are closed to a thief who only listens. One key for the whole chat gives neither: a copy opens every message.
-
How does the symmetric-key ratchet give forward secrecy?
Each chain key makes the next chain key and one message key with HMAC-SHA-256: the message key is HMAC(CK, 0x01) and the next chain key HMAC(CK, 0x02) (Double Ratchet specification 7.2). The message key encrypts one message; then the sender overwrites the chain key and discards the message key. A copy after message 5 opened 0 of 5 earlier messages but still all later ones.
-
Why is keeping used message keys "for a retry" a mistake?
The chain was right and the key store ruined it: a copy then opened 5 of 5 earlier messages, as with one key for everything. Keep the ciphertext or the plaintext for a retry, and delete the key as soon as the message is sealed.
-
How does a ratchet receiver handle late, repeated and far-ahead messages?
Behind the chain with no stored key: refuse. Behind with a stored key: open and delete the key. The next number: open and step on. Ahead by at most MAX_SKIP: step ahead, store the skipped keys, then open. Ahead by more: refuse. A message that fails its tag check must change nothing, or one forged packet would move the chain. A claim of message 1,000,000 would cost about 32 MB of stored keys before any tag check.
-
How does the Diffie-Hellman ratchet heal a copied phone?
Every header carries the sender's current ratchet public key. When the direction changes, the receiver runs X25519 between it and her own private ratchet key, feeds the result into HKDF with the root key as salt to get a new root and chain key, then makes a new ratchet key pair and repeats. The old private key is overwritten. A thief reads nothing after the owner's next reply. Recovery targets a thief who only listens (specification 8.2).
-
How do X3DH and PQXDH start a session with someone who is offline?
Bob leaves a prekey bundle on a server: his identity key IK, a signed prekey SPK (signed with IK) and one-time prekeys OPK, each handed out once. Alice adds her identity key and a fresh ephemeral key and computes four DH values; the first two authenticate, the last two give forward secrecy. PQXDH adds a KEM secret (ML-KEM-1024 keys and ciphertexts are 1,568 bytes). The client must verify the signature on every prekey.
-
How does MLS make a group update cheap?
Members are the leaves of a ratchet tree. An update picks new secrets for the member's path and encrypts each to the subtree that hangs off the path at that level: about log N ciphertexts, 10 for 1,000 members against 999 for pairwise sessions (3 against 7 for eight members). A Commit applies proposals and starts a new epoch, and a Welcome brings in a new member. The first update in a never-committed group costs as much as pairwise.
-
What does end-to-end encryption still leave visible?
Metadata: who talks to whom, when and how often (MLS headers carry the group ID, epoch and message type in the clear). Message length, since padding is optional. Whose key it is, unless keys are verified by a code or QR code or logged by key transparency. What a device holds now: a ratchet limits what one copy reveals, not what the phone stores. And PQXDH authentication, which is still classical.
Key takeaways
- Forward secrecy means a copy of the device opens no earlier message; post-compromise security means messages sent after a fresh key exchange are closed again to a thief who only listens.
- A chain that makes one message key per message with HMAC, and deletes every key and chain key once used, gave a thief 0 of 5 earlier messages in the test, while keeping used keys or using one key gave 5 of 5.
- A receiver keeps skipped message keys only up to a limit, deletes each one when its message arrives, and lets a message that fails its tag check change nothing.
- A new Diffie-Hellman key pair on every reply locks a thief out: in the run the thief read two of Bob’s messages and one more with a later copy, and nothing after Alice’s next reply.
- X3DH starts a session with an offline user from four Diffie-Hellman values; PQXDH adds a KEM secret; MLS cuts the cost of a group update from 999 ciphertexts to 10 for 1,000 members; none of them hides metadata, or whose key you hold.
Terms in this lesson
- chain key
- The secret a symmetric ratchet holds between messages. HMAC of it with one constant gives the next message key, and HMAC with another constant gives the next chain key.
- Diffie-Hellman ratchet
- The half of the Double Ratchet that mixes a new Diffie-Hellman output into the root key each time a party answers with a fresh ratchet key pair. That is what restores secrecy after a key copy.
- Double Ratchet
- Signal's algorithm for an ongoing conversation: a symmetric-key ratchet for every message, and a Diffie-Hellman ratchet that adds a fresh shared secret whenever the direction of the conversation changes.
- epoch
- One period of an MLS group with a fixed set of members and one set of group secrets. Every Commit message starts the next epoch.
- key transparency
- A public, append-only log of which key belongs to which identity, so a user can check what the service says about their keys and anyone can detect a swapped key.
- key verification
- Comparing a short code or QR code derived from two identity keys, in person or over another channel, so that two users can tell that nobody sits between them. Without it, neither side has proof of who holds the other key.
- Messaging Layer Security (MLS)
- The protocol of RFC 9420 for end-to-end encrypted groups of two to thousands of members, with forward secrecy, post-compromise security and key updates that cost about the logarithm of the group size.
- metadata
- Data about a message rather than its content: who sent it to whom, when, how large it is and from which address. End-to-end encryption protects the content, not these.
- post-compromise security
- The property that a device whose keys were copied once becomes secure again, because a fresh key exchange mixes in a secret the thief never saw. The Double Ratchet specification calls it break-in recovery.
- PQXDH
- X3DH with a post-quantum key-encapsulation secret added behind the Diffie-Hellman values, so a recording of the handshake stays closed against a future quantum computer. Authentication still uses classical keys.
- prekey bundle
- The public keys a user leaves on a server so that others can start a session while they are offline: an identity key, a signed prekey with its signature, and one-time prekeys.
- ratchet tree
- The binary tree at the heart of MLS. Each member is a leaf and knows the secrets on its path to the root, so one update is encrypted to a few subtrees instead of to every member.
- skipped message key
- A message key that a receiver computes ahead and stores because the message with that number has not arrived yet. The key is deleted as soon as its message opens, and the store has a limit.
- symmetric-key ratchet
- A chain that turns one chain key into a message key and the next chain key with a one-way function, then deletes what it used, so earlier keys cannot be recomputed from the chain's current state.
- X3DH
- Extended Triple Diffie-Hellman, Signal's way to start a session with someone who is offline: the sender combines four Diffie-Hellman values from the receiver's published prekey bundle and her own keys into one secret.
References
- The Double Ratchet Algorithm (Signal Messenger)
- The X3DH Key Agreement Protocol (Signal Messenger)
- The PQXDH Key Agreement Protocol (Signal Messenger)
- The ML-KEM Braid Protocol (Signal Messenger)
- RFC 9420: The Messaging Layer Security (MLS) Protocol (Internet Engineering Task Force (IETF))
- RFC 9750: The Messaging Layer Security (MLS) Architecture (Internet Engineering Task Force (IETF))
- RFC 5869: HMAC-based Extract-and-Expand Key Derivation Function (HKDF) (Internet Engineering Task Force (IETF))
- RFC 7748: Elliptic Curves for Security (X25519) (Internet Research Task Force (IRTF))
- FIPS 203: Module-Lattice-Based Key-Encapsulation Mechanism Standard (National Institute of Standards and Technology (NIST))
- Node.js 24 documentation: Crypto (hkdfSync, diffieHellman, encapsulate) (OpenJS Foundation (Node.js))
Report a problem with this lesson
Kept only in this browser. Your Learn progress