Hash functions: SHA-2, SHA-3 and BLAKE2

Flip one bit of a message and about half of the 256 bits of its SHA-256 digest flip with it: over the 1,000 one-bit changes this lesson measures, 127.7 on average. That behaviour, and three precise promises behind it, is what lets a 32-byte digest stand for a file of any size, a release, a stored token or a signed document.

This lesson makes the promises exact. You see what preimage, second-preimage and collision resistance mean and what each costs an attacker, compute SHA-2, SHA-3 and BLAKE2 digests in JavaScript and Python and check them against the official values, hash large inputs in pieces, and see where MD5 and SHA-1 still turn up and what replaces them. A two-step code lab at the end hashes a stream and finds a collision of a deliberately short hash.

Three promises: preimage, second-preimage and collision resistance

A cryptographic hash functioncryptographic hash function: A function that maps input of any length to a fixed-size digest, quickly, and resists preimages, second preimages and collisions. SHA-256, SHA-3 and BLAKE2 are current examples. maps input of any length to a digest of fixed length, quickly. The Handbook of Applied Cryptography lists three further properties, each a promise that some search is infeasible:

Each matters somewhere. A database that stores the SHA-256 of each reset token needs preimage resistance, so that a leaked table does not give back the tokens. A download page that publishes a file’s digest needs second-preimage resistance, so that nobody can make another file to match it. A signature on a digest needs collision resistance too: someone who can choose the message could prepare two that collide, have one signed and use the signature for the other. The Handbook proves that collision resistance implies second-preimage resistance (Fact 9.18).

Collisions are the cheap one because of the birthday boundbirthday bound: The rule that random values of b bits start to repeat after about 2^(b/2) of them. It is why a 256-bit hash gives about 128 bits of collision resistance and why random nonces have usage limits. More in “Security levels, key sizes and attack cost”: every pair of inputs tried is a chance, and pairs grow with the square of the inputs. Cut SHA-256 down to 16 bits and every search finishes in a moment, so the costs can be measured:

"""SHA-256 cut down to its first 16 bits, so that the three generic attacks finish in a moment: a collision, a second
preimage and a preimage, each searched 10 times with numbered messages, so the counts repeat on every run."""
import hashlib

BITS = 16


def h(message):
    """The first BITS bits of SHA-256: a toy hash that is weak only because it is short."""
    return int.from_bytes(hashlib.sha256(message).digest()[:4], "big") >> (32 - BITS)


def collision(run):
    """Hash new messages until two of them share a value; return how many were hashed."""
    seen = {}
    i = 0
    while True:
        value = h(f"run {run} message {i}".encode())
        if value in seen:
            return i + 1
        seen[value] = i
        i += 1


def matching(target, run):
    """Hash new messages until one has the given value; return how many were hashed."""
    i = 0
    while True:
        i += 1
        if h(f"run {run} try {i}".encode()) == target:
            return i


runs = range(10)
collisions = [collision(r) for r in runs]
second = [matching(h(f"the original file, version {r}".encode()), r) for r in runs]
preimage = [matching(r * 6553 % 2**BITS, r) for r in runs]

average = lambda xs: sum(xs) / len(xs)
print(f"a {BITS}-bit hash has {2**BITS:,} values")
print(f"collision:       {average(collisions):>9,.0f} hashes on average; about 1.25 x 2^{BITS // 2} = {1.25 * 2 ** (BITS / 2):,.0f}")
print(f"second preimage: {average(second):>9,.0f} hashes on average; about 2^{BITS} = {2**BITS:,}")
print(f"preimage:        {average(preimage):>9,.0f} hashes on average; about 2^{BITS} = {2**BITS:,}")
print("for the full 256 bits: collision about 2^128, second preimage and preimage about 2^256")

a 16-bit hash has 65,536 values
collision:             388 hashes on average; about 1.25 x 2^8 = 320
second preimage:    56,171 hashes on average; about 2^16 = 65,536
preimage:           60,564 hashes on average; about 2^16 = 65,536
for the full 256 bits: collision about 2^128, second preimage and preimage about 2^256

On ten runs each, a collision took 388 hashes on average, close to the predicted 320, while a second preimage took 56,171 and a preimage 60,564, close to 2¹⁶ = 65,536. The same arithmetic at full length is why SHA-256 gives 128 bits of collision resistance and 256 of preimage resistance, the two numbers of NIST SP 800-57’s Table 3 in the security-levels lesson. ASVS 5.0 turns that into a rule in requirement 11.4.3: an output of at least 256 bits where collision resistance is needed, and at least 128 bits where only second preimages matter.

SHA-2, SHA-3 and BLAKE2, checked against their official values

Three families cover nearly every modern use. SHA-2, in FIPS 180-4, has six functions: SHA-224 and SHA-256, SHA-384 and SHA-512, and two truncated forms of SHA-512, SHA-512/224 and SHA-512/256. SHA-3, in FIPS 202, has four hash functions, SHA3-224 to SHA3-512, and two extendable-output functionsextendable-output function (XOF): A hash function that gives as many output bytes as you ask for, such as SHAKE128 and SHAKE256 of FIPS 202., SHAKE128 and SHAKE256, all built on the Keccak permutation. BLAKE2, in RFC 7693, has BLAKE2b with digests of 1 to 64 bytes and BLAKE2s with 1 to 32, and can take a key directly, which makes it a MAC as well. ASVS 5.0’s Appendix C lists SHA-256 and longer, the 256-bit and longer SHA-3 functions, SHAKE256, BLAKE2b, BLAKE2s and BLAKE3 as acceptable for general use.

A digest you compute is only trusted once it matches a published value. NIST publishes example values for each of its algorithms, and the RFC gives its own. Web Crypto’s digest() supports SHA-1, SHA-256, SHA-384 and SHA-512, and this program checks all four against NIST:

// Web Crypto's digest() for SHA-1, SHA-256, SHA-384 and SHA-512, checked against the example values NIST publishes
// for the message "abc" and for the 448-bit two-block message.
const ABC = new TextEncoder().encode('abc');
const TWO_BLOCK = new TextEncoder().encode('abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq');
const hex = (buffer) => Array.from(new Uint8Array(buffer), (b) => b.toString(16).padStart(2, '0')).join('');

const NIST = [
  ['SHA-1', ABC, 'a9993e364706816aba3e25717850c26c9cd0d89d'],
  ['SHA-256', ABC, 'ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad'],
  ['SHA-384', ABC, 'cb00753f45a35e8bb5a03d699ac65007272c32ab0eded1631a8b605a43ff5bed8086072ba1e7cc2358baeca134c825a7'],
  ['SHA-512', ABC, 'ddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a9eeee64b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f'],
  ['SHA-1', TWO_BLOCK, '84983e441c3bd26ebaae4aa1f95129e5e54670f1'],
  ['SHA-256', TWO_BLOCK, '248d6a61d20638b8e5c026930c3e6039a33ce45964ff2167f6ecedd419db06c1'],
];

for (const [algorithm, message, expected] of NIST) {
  const digest = hex(await crypto.subtle.digest(algorithm, message));
  const which = message === ABC ? '"abc"' : 'two-block';
  console.log(`${algorithm.padEnd(8)}${which.padEnd(11)}${digest.slice(0, 16)}… ${digest.length * 4} bits, matches NIST: ${digest === expected}`);
}

SHA-1   "abc"      a9993e364706816a… 160 bits, matches NIST: true
SHA-256 "abc"      ba7816bf8f01cfea… 256 bits, matches NIST: true
SHA-384 "abc"      cb00753f45a35e8b… 384 bits, matches NIST: true
SHA-512 "abc"      ddaf35a193617aba… 512 bits, matches NIST: true
SHA-1   two-block  84983e441c3bd26e… 160 bits, matches NIST: true
SHA-256 two-block  248d6a61d20638b8… 256 bits, matches NIST: true

Every line matches, for abc and for the 448-bit message, which takes two blocks once it is padded. Python’s hashlib adds SHA-3, BLAKE2 and SHAKE, which Web Crypto does not offer in browsers. Run it, then change data to your own bytes:

"""SHA-256, SHA3-256, BLAKE2b and SHAKE128 of the same input with hashlib, two checked against their standards, and a
large input hashed in pieces."""
import hashlib

SHA3_256_ABC = "3a985da74fe225b2045c172d6bd390bd855f086e3e9d525b46bfe24511431532"  # NIST's example value
BLAKE2B_512_ABC = (  # RFC 7693, Appendix A
    "ba80a53f981c4d0d6a2797b69f12f6e94c212f14685ac4b74b12bb6fdbffa2d1"
    "7d87c5392aab792dc252d5de4533cc9518d38aa8dbf1925ab92386edd4009923"
)

data = b"abc"
sha256 = hashlib.sha256(data).hexdigest()
sha3 = hashlib.sha3_256(data).hexdigest()
blake2b_512 = hashlib.blake2b(data).hexdigest()
print(f"sha256:              {sha256}")
print(f"sha3_256:            {sha3}")
print(f"blake2b, 32 bytes:   {hashlib.blake2b(data, digest_size=32).hexdigest()}")
print(f"shake_128, 32 bytes: {hashlib.shake_128(data).hexdigest(32)}")
print(f"SHA3-256 as in FIPS 202's example: {sha3 == SHA3_256_ABC}")
print(f"BLAKE2b-512 as in RFC 7693 Appendix A: {blake2b_512 == BLAKE2B_512_ABC}")

big = bytes(range(256)) * 4096  # 1 MiB
pieces = hashlib.sha256()
for start in range(0, len(big), 65_536):
    pieces.update(big[start : start + 65_536])
print(f"1 MiB in 16 pieces equals 1 MiB at once: {pieces.hexdigest() == hashlib.sha256(big).hexdigest()}")

sha256:              ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad
sha3_256:            3a985da74fe225b2045c172d6bd390bd855f086e3e9d525b46bfe24511431532
blake2b, 32 bytes:   bddd813c634239723171ef3fee98579b94964e3bb1cb3e427262c8c068d52319
shake_128, 32 bytes: 5881092dd818bf5cf8a3ddb793fbcba74097d5c526a6d35f97b83351940f2cc8
SHA3-256 as in FIPS 202's example: True
BLAKE2b-512 as in RFC 7693 Appendix A: True
1 MiB in 16 pieces equals 1 MiB at once: True

SHA3-256 matches the FIPS 202 example and BLAKE2b-512 the value in RFC 7693’s Appendix A, and the three 256-bit digests share nothing although the input is the same. SHAKE128 gave 32 bytes because the call asked for 32: an extendable-output function gives as many bytes as you ask for. The last property worth seeing is the avalanche effectavalanche effect: The property that changing one bit of the input changes about half of the output bits, so that a digest gives no hint of how its input differs from another., the flip of about half the output for one changed input bit:

"""The avalanche effect, measured: flip one bit of a message and count how many of SHA-256's 256 output bits change."""
import hashlib


def bits(data):
    return int.from_bytes(hashlib.sha256(data).digest(), "big")


changes = []
for i in range(1000):
    message = bytearray(f"order {i}: pay 500 to account 1234".encode())
    position = i % (len(message) * 8)
    before = bits(bytes(message))
    message[position // 8] ^= 1 << (position % 8)  # flip one bit
    changes.append(bin(before ^ bits(bytes(message))).count("1"))

mean = sum(changes) / len(changes)
print(f"1,000 one-bit flips: on average {mean:.1f} of 256 output bits changed")
print(f"fewest {min(changes)}, most {max(changes)}; between 112 and 144 in {sum(112 <= c <= 144 for c in changes) / 10:.1f} % of cases")
print(f"one example: {changes[0]} bits changed when one bit of 'order 0: …' flipped")

1,000 one-bit flips: on average 127.7 of 256 output bits changed
fewest 101, most 153; between 112 and 144 in 96.7 % of cases
one example: 131 bits changed when one bit of 'order 0: …' flipped

Over 1,000 one-bit flips, 127.7 of the 256 output bits changed on average, never fewer than 101 and never more than 153, and 96.7 % of the flips changed between 112 and 144. A digest that changed in only a few places would leak what changed in the input; this one gives no hint.

Large inputs: hash in pieces, where the API allows it

SHA-256 never needs the whole input at once. It pads the message, cuts it into 512-bit blocks and mixes them one at a time into a running state, so the next block can wait until it has been read. Step through it for abc:

Whether you can use that depends on the API. Web Crypto’s digest() takes one buffer, and MDN notes that it does not support streaming input, so the whole input must be in memory. node:crypto has createHash(), whose update() takes the data piece by piece, and crypto.hash() for a one-shot digest, which is no longer experimental from Node 24.13.1. Python’s hashlib documents that repeated update() calls equal a single call on the joined data, and hashlib.file_digest(), from Python 3.11, reads a file for you; from 3.14 it raises BlockingIOError for a file opened in non-blocking mode. This program hashes 10 MiB all three ways:

// Hashing a large input in pieces with node:crypto, and why Web Crypto's digest() needs the whole input at once.
import { createHash, hash } from 'node:crypto';

const big = new Uint8Array(10 * 1024 * 1024); // 10 MiB
for (let i = 0; i < big.length; i++) big[i] = (i * 31) & 0xff;

const pieces = createHash('sha256');
for (let start = 0; start < big.length; start += 1 << 20) pieces.update(big.subarray(start, start + (1 << 20)));
const streamed = pieces.digest('hex');

const oneShot = hash('sha256', big); // the one-shot helper of node:crypto
const web = Array.from(new Uint8Array(await crypto.subtle.digest('SHA-256', big)), (b) => b.toString(16).padStart(2, '0')).join('');

console.log(`10 MiB in 10 pieces with createHash().update(): ${streamed.slice(0, 16)}…`);
console.log(`the same with crypto.hash() in one call:        ${oneShot === streamed}`);
console.log(`the same with Web Crypto digest(), all at once: ${web === streamed}`);
console.log(`createHash keeps ${pieces.constructor.name === 'Hash' ? 'a Hash object' : 'state'} between pieces; digest() takes one buffer and has no update()`);

10 MiB in 10 pieces with createHash().update(): 50a082ea7b8c57f5…
the same with crypto.hash() in one call:        true
the same with Web Crypto digest(), all at once: true
createHash keeps a Hash object between pieces; digest() takes one buffer and has no update()

All three digests agree. For a file of several gigabytes, createHash().update() or hashlib keeps the memory use to one piece; Web Crypto would need the whole file as one buffer.

The running state has a second consequence, shown in the last step of the figure. A SHA-256 digest is the complete state after the last block, so anyone who knows the digest of a message and its length can go on hashing more blocks from it without knowing the message. This is called length extensionlength extension: Continuing a hash from a known digest without knowing the message, possible with SHA-256 and SHA-512 because their digest is the full internal state. It is why SHA-256(secret ∥ message) is not a MAC., and it is why a tag made as SHA-256(secret ∥ message) can be extended to a longer message with a valid tag. HMAC, SHA-3 and SHA-512/256 are not open to it, and the next module builds message authentication on HMAC.

MD5 and SHA-1: retired for security, still met in old code

MD5 and SHA-1 were once the default choices, and both now fail at the promise that matters most for signatures. Their history in standards, from first publication to retirement:

RFC 6151 rules MD5 out wherever collision resistance is needed, digital signatures among them. For SHA-1 the first collision was demonstrated publicly: two different PDF files with the same SHA-1 digest, after about 6,500 CPU-years and 110 GPU-years of work, and NIST has set the end of 2030 for phasing SHA-1 out in favour of SHA-2 and SHA-3. ASVS 5.0 is blunter: requirement 11.4.1 says disallowed hash functions such as MD5 must not be used for any cryptographic purpose, and its Appendix C lists MD4, MD5 and every CRC as disallowed and SHA-1 as legacy, unsuitable for HMAC, key derivation, random bit generation and signatures.

You will still meet both. Download pages print MD5 or SHA-1 checksums, which catch a damaged download but never a deliberate swap. Older protocols use HMAC-SHA-1 or HMAC-MD5; RFC 6151 sees no practical attack on HMAC-MD5 as a MAC yet says new protocols should not use it. And code uses MD5 for cache keys and file names, where nothing is defended. Python makes that last use explicit: every hashlib constructor takes usedforsecurity=False, which tells a restricted build that the hash is not used in a security context.

Practise: hash a stream, find a short collision

Two exercises, with sample tests that run on this page, put the lesson to work in Python. The first hashes a stream of chunks with an allowed algorithm, held to NIST’s values split at awkward places, and refuses MD5 and SHA-1. The second finds a collision of SHA-256 cut down to 12, 16 and 20 bits within the birthday bound, where the starter code wastes its time on a second-preimage search:

import hashlib


def hash_chunks(chunks, algorithm="sha256"):
    """The hex digest of all chunks joined together, hashed one chunk at a time."""
    for chunk in chunks:
        h = hashlib.new(algorithm)  # a new hash for every chunk: only the last one counts
        h.update(chunk)
    return h.hexdigest()

import hashlib

from chunks import hash_chunks

ABC = "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"  # SHA-256("abc"), NIST
TWO_BLOCK = b"abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"
TWO_BLOCK_SHA256 = "248d6a61d20638b8e5c026930c3e6039a33ce45964ff2167f6ecedd419db06c1"
SHA3_ABC = "3a985da74fe225b2045c172d6bd390bd855f086e3e9d525b46bfe24511431532"  # FIPS 202 example


def raises_value_error(call):
    try:
        call()
    except ValueError:
        return True
    return False


def test_abc_split_every_way():
    """'abc' in one chunk, in two and in three gives NIST's SHA-256 value"""
    for chunks in [[b"abc"], [b"a", b"bc"], [b"ab", b"c"], [b"a", b"b", b"c"], [b"", b"abc", b""]]:
        assert hash_chunks(chunks) == ABC, chunks


def test_two_block_message_split_at_block_edges():
    """the 448-bit message split at 1, 55, 56 and 63 bytes"""
    for cut in [1, 55, 56, 63]:
        assert hash_chunks([TWO_BLOCK[:cut], TWO_BLOCK[cut:]]) == TWO_BLOCK_SHA256, cut


def test_a_generator_read_once():
    """1 MiB from a generator of 4,096 chunks equals hashing it at once"""
    data = bytes(range(256)) * 4096
    pieces = (data[i : i + 256] for i in range(0, len(data), 256))
    assert hash_chunks(pieces) == hashlib.sha256(data).hexdigest()


def test_empty_input():
    """no chunks at all is the digest of the empty message"""
    assert hash_chunks([]) == hashlib.sha256(b"").hexdigest()


def test_other_allowed_hashes():
    """SHA3-256 of 'abc' as in FIPS 202, and BLAKE2b"""
    assert hash_chunks([b"ab", b"c"], "sha3_256") == SHA3_ABC
    assert hash_chunks([b"a", b"bc"], "blake2b") == hashlib.blake2b(b"abc").hexdigest()


def test_retired_hashes_are_refused():
    """md5, sha1, and names that are not hashes"""
    for name in ["md5", "sha1", "MD5", "crc32", ""]:
        assert raises_value_error(lambda: hash_chunks([b"abc"], name)), name

import hashlib


def short_hash(message, bits):
    """The first `bits` bits of SHA-256, as a number: a toy hash that is weak only because it is short."""
    return int.from_bytes(hashlib.sha256(message).digest()[:4], "big") >> (32 - bits)


def find_collision(bits):
    """Two different messages with the same short hash, and how many messages were hashed."""
    first = b"0"
    target = short_hash(first, bits)
    i = 0
    while True:  # waits for a match with one fixed message: about 2**bits tries
        i += 1
        message = str(i).encode()
        if short_hash(message, bits) == target:
            return first, message, i + 1

import hashlib

from collision import find_collision, short_hash


def reference_short_hash(message, bits):
    """The first `bits` bits of SHA-256, computed here, independently of the code under test."""
    return int(hashlib.sha256(message).hexdigest(), 16) >> (256 - bits)


def raises_value_error(call):
    try:
        call()
    except ValueError:
        return True
    return False


def check(bits):
    earlier, later, hashed = find_collision(bits)
    assert earlier != later, "two different messages"
    assert reference_short_hash(earlier, bits) == reference_short_hash(later, bits), "with the same short hash"
    assert hashed <= 4 * 1.25 * 2 ** (bits / 2), f"{hashed:,} messages: about {1.25 * 2 ** (bits / 2):,.0f} should do"
    return hashed


def test_the_short_hash():
    """the first 16 bits of SHA-256('abc') are 0xba78"""
    assert short_hash(b"abc", 16) == 0xBA78


def test_12_bits():
    """a 12-bit hash repeats within about 80 messages"""
    check(12)


def test_16_and_20_bits():
    """16 and 20 bits: four more output bits, four times the work"""
    check(16)
    check(20)


def test_messages_in_order():
    """the messages are b'0', b'1', b'2' … and the count includes the later one"""
    earlier, later, hashed = find_collision(12)
    assert later == str(hashed - 1).encode()
    assert int(earlier) < int(later)


def test_bits_out_of_range():
    """7 and 33 bits are refused"""
    for bits in [7, 33]:
        assert raises_value_error(lambda: find_collision(bits)), bits

Finish with practice over the four ideas: your weakest idea is asked first, each right answer leads to a harder question, and the ones you miss return at the end:

Interview questions

Warm-up (fresher to mid level): how do second-preimage resistance and collision resistance differ?

Both are about two inputs with the same digest, but the attacker starts from different places. For a second preimage, the first input is given, say a published release file, and the attacker must find a different input with the same digest; for an n-bit hash that takes about 2ⁿ work. For a collision the attacker may choose both inputs freely, and the birthday bound brings the work down to about 2^(n/2), so SHA-256 gives 128 bits of collision resistance and 256 of second-preimage resistance. Collision resistance matters whenever an attacker can influence what gets hashed and signed, as with certificates and documents; that is exactly where MD5 and SHA-1 failed. Collision resistance also implies second-preimage resistance, so a hash that has it covers both.

Key takeaways

• A cryptographic hash gives preimage and second-preimage resistance up to about 2ⁿ work and collision resistance up to 2^(n/2), as the 16-bit searches measured: 388 hashes for a collision against about 60,000 for a preimage.
• SHA-2 (FIPS 180-4), SHA-3 and SHAKE (FIPS 202) and BLAKE2 (RFC 7693) are current, and every implementation is checked against the published values, as the programs here matched NIST’s and the RFC’s.
• Good hashes show the avalanche effect: one flipped input bit changed 127.7 of 256 output bits on average.
• SHA-256 keeps a running state, so createHash().update() and hashlib hash large inputs in pieces while Web Crypto’s digest() needs the whole input; the same running state is why SHA-256(secret ∥ message) is not a MAC.
• MD5 and SHA-1 are retired for security: RFC 6151 excludes MD5 where collisions matter, SHA-1 has a public collision, and NIST phases it out by the end of 2030.