Archives and compression: zip, tar, gzip, zstd

The three text files backed up in this lesson hold 5,135 bytes: zipped they take 568, and as a .tar.zst 287. An upload of 1,208 bytes, on the other hand, can unpack into a million, and a member named ../../settings.ini asks to be written outside the folder you chose. Python’s standard library builds and reads both kinds of archive, zip with zipfile and tar with tarfile, and compresses bytes with gzip, bz2, lzma and, new in Python 3.14, compression.zstd; shutil.make_archive() and shutil.unpack_archive() do the common jobs in one call.

You will build and read both kinds of archive, measure what each compressor does to real data, choose one for your own files, and unpack archives that someone else made without letting them write outside the destination or fill the disk.

An archive names files, a compressor shrinks bytes

The two words are often used as if they meant the same thing. An archive format answers “which files, under which names, with which metadata”; a lossless compressionlossless compression: Compression whose decompress step gives back exactly the original bytes, as gzip, bz2, lzma and zstd do. Text with repeated patterns shrinks a lot; data that is already compressed hardly shrinks at all. format answers “how do I store these bytes in less space and get them back exactly”. A zip does both at once: it compresses every member on its own and ends with a central directorycentral directory: The index at the end of a zip file that lists every member's name, its sizes and where its data starts, so that a program can read one member without decompressing the others. that lists every name and where its data starts. A tar only archives: it writes each file behind a header, one after the other, keeping Unix details such as permissions and links, and leaves the shrinking to a compressor that wraps the whole stream. That is why tar files have double extensions: .tar.gz (gzip), .tar.bz2, .tar.xz (lzma) and .tar.zst (Zstandard).

The layout decides what each format is good at. Step through one read of the last of the three files, shimla.txt, from the zip and from the .tar.gz that the next two sections write:

A program can open a zip, read its directory and decompress just one member, which is why many document and package formats are zip files underneath. A .tar.gz has no index: to reach the last file you decompress everything before it. In exchange, compressing the whole stream lets the compressor reuse patterns across files, so a folder of similar text files usually comes out smaller, here 335 bytes against 568.

zipfile writes a zip, and compresses only when asked

zipfile.ZipFile opens an archive like a file, with the mode "r" to read, "w" to write a new one, "a" to add to an existing one and "x" to create one only if it does not exist yet. The example makes a small folder of notes, backs it up, lists what the archive holds, checks it and restores it into another folder:

import shutil
import zipfile
from pathlib import Path

# A small folder to back up. The example writes its own files, so it runs anywhere.
notes = Path("notes")
files = {
    "trips/goa.txt": "Packing list: sunscreen, sandals, phone charger.\n" * 40,
    "trips/shimla.txt": "Packing list: jacket, gloves, thermos flask.\n" * 40,
    "recipes/dal.txt": "Rinse the lentils, boil them soft, then add the tadka.\n" * 25,
}
for name, text in files.items():
    path = notes / name
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(text, encoding="utf-8")


def backup(folder, dest):
    """Write every file under folder into dest/<folder name>-backup.zip and return its path."""
    dest.mkdir(parents=True, exist_ok=True)
    archive = dest / f"{folder.name}-backup.zip"
    with zipfile.ZipFile(archive, "w", compression=zipfile.ZIP_DEFLATED) as zf:
        for path in sorted(folder.rglob("*")):
            if path.is_file():
                # Names inside the archive are relative: notes/trips/goa.txt, never a full path.
                zf.write(path, arcname=path.relative_to(folder.parent).as_posix())
    return archive


archive = backup(notes, Path("backups"))
print(f"wrote {archive.as_posix()} ({archive.stat().st_size:,} bytes)")

with zipfile.ZipFile(archive) as zf:
    for info in zf.infolist():
        print(f"  {info.filename:<22} {info.file_size:>5,} bytes stored as {info.compress_size:>3} bytes")
    print("damaged members:", zf.testzip())  # None: every member's CRC-32 checksum matches
    zf.extractall("restored")

same = all((Path("restored") / "notes" / name).read_bytes() == (notes / name).read_bytes() for name in files)
print("restored copy matches the original:", same)

# The quick route: one call archives a whole folder (zip, tar, gztar, bztar, xztar or zstdtar).
quick = shutil.make_archive("backups/notes-quick", "gztar", root_dir=".", base_dir="notes")
print("make_archive wrote", Path(quick).name)

wrote backups/notes-backup.zip (568 bytes)
  notes/recipes/dal.txt  1,375 bytes stored as  63 bytes
  notes/trips/goa.txt    1,960 bytes stored as  69 bytes
  notes/trips/shimla.txt 1,800 bytes stored as  62 bytes
damaged members: None
restored copy matches the original: True
make_archive wrote notes-quick.tar.gz

• compression=zipfile.ZIP_DEFLATED matters. Without it, ZipFile uses ZIP_STORED and writes the members uncompressed. Here DEFLATE, the method every zip tool understands, turned 1,960 bytes of repetitive text into 69. Press Edit, delete the compression argument and run it again: every member is stored at its full size.
• arcname is the name inside the archive. Without it, write() stores the path you passed in, folders and all. A relative name such as notes/trips/goa.txt keeps the archive portable. Never store absolute paths.
• infolist() returns a ZipInfo for each member, with filename, file_size (the size once unpacked) and compress_size (the bytes it takes in the archive). namelist() gives the names alone.
• testzip() reads every member and checks its CRC-32 checksum, returning the name of the first damaged one, or None when all are intact.
• extractall(path) unpacks everything below path, and extract(name, path) unpacks one member. read(name) returns a member’s bytes without writing a file.

The last lines show the shortcut. shutil.make_archive(base_name, format, root_dir, base_dir) archives a folder in one call and returns the file name it chose, adding the extension for the format: zip, tar, gztar, bztar, xztar or, from Python 3.14, zstdtar. Paths in the archive are relative to root_dir, and base_dir picks the folder inside it to include. Its partner, shutil.unpack_archive(), guesses the format from the extension.

tarfile writes one stream that a compressor wraps

tarfile.open(name, mode) opens a tar archive. The mode says what to do and how the stream is compressed: "r:*" reads any of them and detects the compression itself, "w:gz" writes a gzip-compressed tar, and "w:bz2", "w:xz" and, since Python 3.14, "w:zst" choose the other compressors. tar.add(path) adds a file or a whole folder, and getmembers() returns a TarInfo for each member, with its name, size, type and permissions. The example archives the same notes in all five modes:

import tarfile
from pathlib import Path

# The folder of zip_backup.py: three short text files in two subfolders.
notes = Path("notes")
files = {
    "trips/goa.txt": "Packing list: sunscreen, sandals, phone charger.\n" * 40,
    "trips/shimla.txt": "Packing list: jacket, gloves, thermos flask.\n" * 40,
    "recipes/dal.txt": "Rinse the lentils, boil them soft, then add the tadka.\n" * 25,
}
for name, text in files.items():
    path = notes / name
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(text, encoding="utf-8")


def same_metadata(info):
    """add() calls this for each member it writes: the same owner, time and permissions on every run."""
    info.uid = info.gid = info.mtime = 0
    info.uname = info.gname = ""
    info.mode = 0o755 if info.isdir() else 0o644
    return info


# One tar archive of the folder in each compression mode.
for mode, suffix in [("w", ".tar"), ("w:gz", ".tar.gz"), ("w:bz2", ".tar.bz2"), ("w:xz", ".tar.xz"), ("w:zst", ".tar.zst")]:
    archive = Path("notes" + suffix)
    with tarfile.open(archive, mode) as tar:
        tar.add(notes, filter=same_metadata)  # the folder and everything in it, in sorted order
    print(f"{mode:<6} {archive.name:<15} {archive.stat().st_size:>6,} bytes")

with tarfile.open("notes.tar.zst", "r:*") as tar:  # "r:*" finds out the compression itself
    for member in tar.getmembers():
        kind = "folder" if member.isdir() else "file"
        print(f"  {member.name:<22} {kind:<6} {member.size:>5,} bytes")

w      notes.tar       10,240 bytes
w:gz   notes.tar.gz       335 bytes
w:bz2  notes.tar.bz2      439 bytes
w:xz   notes.tar.xz       368 bytes
w:zst  notes.tar.zst      287 bytes
  notes                  folder     0 bytes
  notes/recipes          folder     0 bytes
  notes/recipes/dal.txt  file   1,375 bytes
  notes/trips            folder     0 bytes
  notes/trips/goa.txt    file   1,960 bytes
  notes/trips/shimla.txt file   1,800 bytes

The plain tar takes 10,240 bytes for 5,135 bytes of text. The POSIX standard for the format explains why: a tar is a sequence of 512-byte records, with a header record before every member, every file’s data filling whole records, two records of zeros at the end, and the last block of records always written at its full size. Compressed, the same archive took between 287 and 439 bytes. add() adds a folder and everything in it in sorted order, and its filter argument, a function that add() calls for every member it writes, gave each member the same owner, time and permissions, so the sizes are the same on every run.

A tar member can be more than a regular file: a folder, a symbolic linksymbolic link: A file that only points to another path. Opening it opens what it points to, so a link inside a folder can lead anywhere on the computer; Path.resolve() follows links to the real location. More in “pathlib: paths, folders and file operations”, a hard link or a device file, each with an owner and permission bits. That is what makes tar the right format to back up a Unix system, and it is also what makes an unknown tar file risky to unpack, as the section on the 'data' filter shows.

Text shrinks to a quarter or less, random bytes not at all

The four compression modules of this lesson share the same core: compress(data) returns smaller bytes, decompress(data) returns the exact original, and open(filename, mode) gives a file object that compresses as you write, for data too large to hold in memory, such as gzip.open("server.log.gz", "wt", encoding="utf-8"). Python 3.14 groups them in the compression package (PEP 784): compression.gzip, compression.bz2, compression.lzma and compression.zlib are the existing modules under new names, which the documentation calls the preferred ones, and compression.zstd is new. The old names keep working.

The example compresses two inputs of exactly the same size: a CSV export of 4,000 orders and the same number of random bytes, which behave like a photo or a video that is already compressed:

import random
from compression import bz2, gzip, lzma, zstd

# Two inputs of the same size: a CSV export (text with repeated words) and random bytes,
# which behave like a photo or video that is already compressed.
rng = random.Random(7)
cities = ["Pune", "Kochi", "Indore", "Nagpur", "Mysuru", "Surat"]
rows = ["order_id,city,amount_inr,status"]
for n in range(1, 4001):
    rows.append(f"{100000 + n},{rng.choice(cities)},{rng.randint(99, 9999)},{rng.choice(['paid', 'refunded', 'pending'])}")
text = ("\n".join(rows) + "\n").encode("utf-8")
noise = rng.randbytes(len(text))

codecs = {
    "gzip (level 9)": (gzip.compress, gzip.decompress),
    "bz2 (level 9)": (bz2.compress, bz2.decompress),
    "lzma (preset 6)": (lzma.compress, lzma.decompress),
    "zstd (level 3)": (zstd.compress, zstd.decompress),
}

for label, data in [("CSV text", text), ("random bytes", noise)]:
    print(f"{label}: {len(data):,} bytes")
    for name, (pack, unpack) in codecs.items():
        packed = pack(data)
        assert unpack(packed) == data  # lossless: the exact bytes come back
        print(f"  {name:<16} {len(packed):>7,} bytes  {len(packed) / len(data):6.1%} of the original")

# A higher zstd level trades time for size; level 19 is near the top of the normal range.
for level in (1, 3, 19):
    print(f"zstd level {level:>2}: {len(zstd.compress(text, level=level)):,} bytes")

CSV text: 102,311 bytes
  gzip (level 9)    24,472 bytes   23.9% of the original
  bz2 (level 9)     19,590 bytes   19.1% of the original
  lzma (preset 6)   20,668 bytes   20.2% of the original
  zstd (level 3)    25,534 bytes   25.0% of the original
random bytes: 102,311 bytes
  gzip (level 9)   102,364 bytes  100.1% of the original
  bz2 (level 9)    103,127 bytes  100.8% of the original
  lzma (preset 6)  102,376 bytes  100.1% of the original
  zstd (level 3)   102,323 bytes  100.0% of the original
zstd level  1: 24,533 bytes
zstd level  3: 25,534 bytes
zstd level 19: 21,612 bytes

The table puts each method’s compression ratiocompression ratio: How much compression shrinks data, given in this track as the compressed size as a share of the original: 23.9 % for a CSV export with gzip at level 9, about 100 % for random bytes. on the two inputs side by side:

The CSV shrank to between 19 % and 25 % of its size with every method, because text repeats itself: the same city names, the same status words, the same digits. The random bytes did not shrink at all; each compressor added a few bytes of headers, and bz2 added 816. That is the rule to remember: compressing JPEG photos, MP4 videos or zip files again costs time and gains nothing. The last lines show the level knob: zstd at level 19 beat gzip and came within about 10 % of bz2, and level 1 was even a little smaller than the default level 3 on this input, a reminder that the effect of a level depends on the data. Measure on your own files before choosing.

Speed is the other half of the choice

Size is half of the trade. The second example times each compressor on about 0.4 MB of the same kind of text, taking the best of three runs:

import random
import time
from compression import bz2, gzip, lzma, zstd

# About 0.4 MB of CSV-like text, compressed and decompressed with each module.
rng = random.Random(7)
cities = ["Pune", "Kochi", "Indore", "Nagpur", "Mysuru", "Surat"]
lines = [f"{100000 + n},{rng.choice(cities)},{rng.randint(99, 9999)},paid" for n in range(16000)]
data = "\n".join(lines).encode("utf-8")

codecs = [
    ("gzip level 9", lambda d: gzip.compress(d), gzip.decompress),  # gzip.compress() defaults to 9
    ("gzip level 6", lambda d: gzip.compress(d, compresslevel=6), gzip.decompress),
    ("bz2 level 9", lambda d: bz2.compress(d), bz2.decompress),
    ("lzma preset 6", lambda d: lzma.compress(d), lzma.decompress),
    ("zstd level 3", lambda d: zstd.compress(d), zstd.decompress),
]


def best_ms(func, arg, repeat=3):
    """The fastest of several runs, in milliseconds (the slower ones were disturbed by something else)."""
    times = []
    for _ in range(repeat):
        start = time.perf_counter()
        func(arg)
        times.append(time.perf_counter() - start)
    return min(times) * 1000


print(f"input: {len(data):,} bytes")
for name, pack, unpack in codecs:
    packed = pack(data)
    print(f"{name:<13} {len(packed):>7,} bytes  compress {best_ms(pack, data):6.1f} ms  decompress {best_ms(unpack, packed):5.1f} ms")

input: 371,867 bytes
gzip level 9   87,961 bytes  compress   90.4 ms  decompress   0.8 ms
gzip level 6   88,585 bytes  compress   23.0 ms  decompress   0.5 ms
bz2 level 9    76,247 bytes  compress   29.4 ms  decompress  13.0 ms
lzma preset 6  66,084 bytes  compress  120.8 ms  decompress   5.2 ms
zstd level 3   96,849 bytes  compress    1.6 ms  decompress   0.5 ms

The sizes are the same on every computer; the times are this run’s. In the recorded run, zstd compressed the text in a few milliseconds, many times faster than any other method, while lzma made the smallest file and was the slowest to compress. gzip.compress() defaults to level 9, its slowest, which made the file less than 1 % smaller than level 6 for a lot of extra time. Decompressing was quickest with gzip and zstd, and slowest with bz2.

Which compressor fits depends on who reads the files, how often they are read back and what matters most. Answer the three questions in the table: it rules out what does not fit your case and says why.

In short: gzip for files anyone must read and for web responses, zstd for logs, backups and data you read back often on systems you control, lzma (xz) for the smallest download compressed once, and bz2 mainly to read the .bz2 files others made.

The ‘data’ filter refuses tar members that leave the folder

An archive is a list of names chosen by whoever made it, and extracting it means writing files under those names. A member named ../outside.txt, an absolute name such as /var/data/report.txt or a link that points elsewhere can make an extractor write outside the folder you gave it, the weakness called path traversalpath traversal: A weakness in which a name from outside, such as ../../etc/passwd, makes a program read or write a file outside the folder it meant to use. Resolving the joined path and checking that it is still inside the folder prevents it. More in “pathlib: paths, folders and file operations”. A tiny archive can also declare members that unpack to gigabytes, a decompression bombdecompression bomb: A small compressed file that unpacks into an enormous one, made to fill a disk or memory. Checking the sizes an archive declares and counting the bytes actually written stop it. that fills the disk. The documentation of zipfile, tarfile and shutil.unpack_archive() carries the same warning: never extract archives from untrusted sources without inspecting them first.

tar archives have extraction filtersextraction filter: In tarfile, a function that sees each member before it is extracted and may refuse or change it. The 'data' filter, the default from Python 3.14, refuses names and links that lead outside the destination., added in Python 3.12 (PEP 706). The filter argument of extract() and extractall() decides what each member may do. "fully_trusted" honours everything the archive says. "tar" blocks the worst features: it strips leading slashes, refuses members that would land outside the destination and clears the setuid, setgid and sticky bits. "data", meant for archives that only carry data, also refuses links to absolute paths or outside the destination and device files, ignores owners and keeps only safe permission bits. Since Python 3.14, "data" is the default when you pass no filter. The example builds a test archive in memory and extracts it member by member:

import io
import tarfile
from pathlib import Path


def add_file(tar, name, text):
    data = text.encode("utf-8")
    info = tarfile.TarInfo(name)
    info.size = len(data)
    tar.addfile(info, io.BytesIO(data))


def add_symlink(tar, name, target):
    info = tarfile.TarInfo(name)
    info.type = tarfile.SYMTYPE
    info.linkname = target
    tar.addfile(info)


# A test archive: one ordinary file and four members whose names or links point outside the
# folder it is unpacked into. It is built in memory, so the example needs no files of its own.
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w:gz") as tar:
    add_file(tar, "report.txt", "quarterly numbers\n")
    add_file(tar, "../outside.txt", "written next to the destination\n")
    add_file(tar, "/abs/path.txt", "an absolute name\n")
    add_symlink(tar, "shared", "/var/data/shared.txt")
    add_symlink(tar, "escape", "../../..")

dest = Path("unpacked")
buffer.seek(0)
with tarfile.open(fileobj=buffer, mode="r:gz") as tar:
    for member in tar.getmembers():
        try:
            tar.extract(member, dest, filter="data")
            print(f"{member.name:<15} extracted")
        except tarfile.FilterError as err:
            print(f"{member.name:<15} refused: {type(err).__name__}")

print("files in unpacked:", sorted(p.relative_to(dest).as_posix() for p in dest.rglob("*")))
print("outside.txt was written outside:", Path("outside.txt").exists())

# Without a filter argument, Python 3.14 uses "data" too: extractall() stops at the first refusal.
buffer.seek(0)
with tarfile.open(fileobj=buffer, mode="r:gz") as tar:
    try:
        tar.extractall("default")
    except tarfile.FilterError as err:
        print(f"extractall() stopped at {err.tarinfo.name!r} with {type(err).__name__}")

report.txt      extracted
../outside.txt  refused: OutsideDestinationError
/abs/path.txt   extracted
shared          refused: AbsoluteLinkError
escape          refused: LinkOutsideDestinationError
files in unpacked: ['abs', 'abs/path.txt', 'report.txt']
outside.txt was written outside: False
extractall() stopped at '../outside.txt' with OutsideDestinationError

Step through what the filter did with each member:

report.txt came out normally, and the absolute name was made relative, so /abs/path.txt landed at unpacked/abs/path.txt. The other three were refused with exceptions that all derive from tarfile.FilterError, and outside.txt was never written. The last line shows the default at work: extractall() without a filter raised at the first refused member. Members extracted before the error stay on disk, so extract into a fresh folder and delete it if extraction fails.

zip rewrites dangerous names, so check names and sizes yourself

zip archives have no filter argument. Instead, ZipFile.extract() and extractall() rewrite dangerous names: they strip leading slashes and drive letters and remove .. parts. The example shows what that means for an upload:

import zipfile
from pathlib import Path, PurePosixPath

# A zip with suspicious member names, built in memory and saved like an upload.
with zipfile.ZipFile("upload.zip", "w") as zf:
    zf.writestr("photos/beach.jpg", b"not really a photo")
    zf.writestr("../../settings.ini", b"[app]\ntheme = dark\n")
    zf.writestr("/var/data/report.txt", b"an absolute name\n")


def suspicious(name):
    """True for names that are absolute or climb out of the destination with '..'."""
    path = PurePosixPath(name.replace("\\", "/"))
    return path.is_absolute() or ".." in path.parts


with zipfile.ZipFile("upload.zip") as zf:
    for name in zf.namelist():
        print(f"{name:<22} suspicious: {suspicious(name)}")
    zf.extractall("out")

# extractall() rewrites dangerous names instead of refusing them, so nothing escapes "out",
# but files still land under names the archive's author chose.
print(sorted(p.relative_to("out").as_posix() for p in Path("out").rglob("*") if p.is_file()))

photos/beach.jpg       suspicious: False
../../settings.ini     suspicious: True
/var/data/report.txt   suspicious: True
['photos/beach.jpg', 'settings.ini', 'var/data/report.txt']

The table follows each member from its name in the archive to the file that extractall() wrote:

Nothing left the out folder, but nothing was refused either: ../../settings.ini arrived as settings.ini, a name the archive’s author picked, which may overwrite a file your program expects there. For uploads, check every name before you extract anything and refuse the whole archive when one looks wrong, as suspicious() does: a name is dangerous when it is absolute or when its parts include .., with forward slashes and backslashes both counted as separators. zipfile.Path, the pathlib-style interface to a zip, rewrites nothing at all, and its documentation leaves validating names to the caller.

Sizes need a check too, and the archive’s own claim is only the first of two. The next example makes a test upload for your own extractor, a megabyte of zeros, and runs both checks on it:

import zipfile
from pathlib import Path

# A test upload for your own extractor: 1 MB of zeros, which DEFLATE stores in very little space.
with zipfile.ZipFile("upload.zip", "w", compression=zipfile.ZIP_DEFLATED) as zf:
    zf.writestr("zeros.bin", bytes(1_000_000))
    zf.writestr("note.txt", b"a short note\n")
with zipfile.ZipFile("normal.zip", "w", compression=zipfile.ZIP_DEFLATED) as zf:
    zf.writestr("note.txt", b"a short note\n")

LIMIT = 100_000  # the most this service unpacks from one upload


def declared_total(zf):
    """The bytes the members say they unpack to: the archive's own claim, read before anything is written."""
    return sum(info.file_size for info in zf.infolist())


def copy_capped(zf, info, path, budget):
    """Write one member to path in chunks, and stop once it has produced more than budget bytes."""
    written = 0
    with zf.open(info) as src, open(path, "wb") as out:
        while chunk := src.read(65536):
            written += len(chunk)
            if written > budget:
                raise ValueError(f"{info.filename} produced more than {budget:,} bytes")
            out.write(chunk)
    return written


size = Path("upload.zip").stat().st_size
with zipfile.ZipFile("upload.zip") as zf:
    total = declared_total(zf)
    print(f"upload.zip: {size:,} bytes, members declare {total:,} bytes ({total // size:,} times as many)")
    print(f"declared-size check: {'refused' if total > LIMIT else 'accepted'} (limit {LIMIT:,} bytes)")
    try:
        copy_capped(zf, zf.getinfo("zeros.bin"), "zeros.bin", LIMIT)
    except ValueError as err:
        print(f"capped copy: stopped, {err}; {Path('zeros.bin').stat().st_size:,} bytes on disk")

with zipfile.ZipFile("normal.zip") as zf:
    total = declared_total(zf)
    print(f"normal.zip: members declare {total:,} bytes: {'refused' if total > LIMIT else 'accepted'}")

upload.zip: 1,208 bytes, members declare 1,000,013 bytes (827 times as many)
declared-size check: refused (limit 100,000 bytes)
capped copy: stopped, zeros.bin produced more than 100,000 bytes; 65,536 bytes on disk
normal.zip: members declare 13 bytes: accepted

The 1,208-byte upload declared 1,000,013 bytes, 827 times as many, so adding up the members’ file_size refused it before anything was written. The declared sizes come from the archive, though, so a careful extractor also counts the bytes it actually writes: copy_capped() reads the member with zf.open(info) in chunks and stopped once the member had produced more than the limit, with 65,536 bytes on disk. The normal upload passed both checks. The exercise below builds the name and size checks into one function.

Practise: back up a folder and unzip uploads safely

The exercise asks for backup(), which zips a folder under relative names with DEFLATE, and safe_unzip(), which refuses an archive with an absolute name, a .. part or a declared total above a limit before it writes anything, and otherwise extracts it. Its sample tests run in your browser:

import zipfile
from pathlib import Path, PurePosixPath


def backup(folder, dest):
    """Zip every file under folder into dest/<folder name>.zip and return the archive's path."""
    raise NotImplementedError("write backup()")


def safe_unzip(zip_path, dest, max_bytes=50_000_000):
    """Extract zip_path into dest after checking every member; raise ValueError for a dangerous archive."""
    raise NotImplementedError("write safe_unzip()")

import tempfile
import zipfile
from pathlib import Path

from archive_tools import backup, safe_unzip


def make_folder():
    root = Path(tempfile.mkdtemp())
    folder = root / "photos"
    (folder / "trips").mkdir(parents=True)
    (folder / "a.txt").write_text("first file\n")
    (folder / "trips" / "goa.txt").write_text("beach day\n" * 50)
    return root, folder


def make_zip(names, size=10):
    path = Path(tempfile.mkdtemp()) / "upload.zip"
    with zipfile.ZipFile(path, "w") as zf:
        for name in names:
            zf.writestr(name, b"x" * size)
    return path


def test_backup_path():
    """backup() writes dest/<folder name>.zip and returns that path"""
    root, folder = make_folder()
    archive = backup(folder, root / "backups")
    assert Path(archive) == root / "backups" / "photos.zip"
    assert Path(archive).is_file()


def test_backup_creates_nested_dest_and_runs_twice():
    """dest may be several folders deep, and a second backup into it works too"""
    root, folder = make_folder()
    dest = root / "backups" / "daily"
    backup(folder, dest)
    archive = backup(folder, dest)
    assert Path(archive) == dest / "photos.zip"


def test_backup_names_and_contents():
    """the archive holds every file under relative names, compressed"""
    root, folder = make_folder()
    with zipfile.ZipFile(backup(folder, root / "out")) as zf:
        assert sorted(zf.namelist()) == ["a.txt", "trips/goa.txt"]
        assert zf.read("trips/goa.txt") == b"beach day\n" * 50
        assert zf.getinfo("trips/goa.txt").compress_type == zipfile.ZIP_DEFLATED


def test_safe_unzip_extracts_normal_archive():
    """a normal archive is extracted and its file names come back sorted"""
    archive = make_zip(["b.txt", "docs/", "docs/a.txt"])
    dest = Path(tempfile.mkdtemp()) / "dest"
    assert safe_unzip(archive, dest) == ["b.txt", "docs/a.txt"]
    assert (dest / "docs" / "a.txt").read_bytes() == b"x" * 10


def test_rejects_parent_parts():
    """a name with .. is refused before anything is written"""
    archive = make_zip(["ok.txt", "../evil.txt"])
    dest = Path(tempfile.mkdtemp()) / "dest"
    try:
        safe_unzip(archive, dest)
    except ValueError:
        assert not (dest / "ok.txt").exists()
        return
    assert False, "safe_unzip accepted ../evil.txt"


def test_rejects_absolute_and_backslash_names():
    """absolute names, drive letters and backslash tricks are refused"""
    for bad in ["/etc/thing.txt", "C:/thing.txt", "..\\up.txt", "a/b/../../../c.txt"]:
        try:
            safe_unzip(make_zip([bad]), Path(tempfile.mkdtemp()))
        except ValueError:
            continue
        assert False, f"safe_unzip accepted {bad!r}"


def test_rejects_too_large():
    """an archive that would unpack to more than max_bytes is refused"""
    archive = make_zip(["big.bin"], size=5000)
    try:
        safe_unzip(archive, Path(tempfile.mkdtemp()), max_bytes=4999)
    except ValueError:
        return
    assert False, "safe_unzip ignored max_bytes"

Formats, zip, tar, both halves of compression and both kinds of safe extraction are in the practice below. Your weakest idea comes up first, a right answer leads to a harder question, and every choice explains itself:

Interview questions

What is the difference between an archive format and a compression format?

An archive format collects many files into one and records their names, sizes and other metadata, such as tar, which also keeps Unix permissions and links. A compression format encodes a stream of bytes in fewer bytes and decodes it back exactly, such as gzip, bz2, xz or Zstandard; it knows nothing about files or names. Formats combine: a .tar.gz is a tar archive whose whole stream is compressed with gzip, while a zip is both an archive and a compressor at once, because it compresses each member separately and lists them in a central directory at the end. The difference shows up in use: a zip lets a program read one member directly, while a .tar.gz must be decompressed from the start, but often comes out smaller because the compressor sees repetition across all the files.

Key takeaways

• An archive bundles files under names and a compressor shrinks bytes: zip does both per member and can read one member directly, while tar only archives and is wrapped by gzip, bz2, xz or zstd as one stream.
• zipfile.ZipFile stores members uncompressed unless you pass compression=zipfile.ZIP_DEFLATED; give write() a relative arcname, and let shutil.make_archive() handle the common formats in one call.
• A plain tar is a sequence of 512-byte records, 10,240 bytes for the lesson’s notes, so it is nearly always compressed; tarfile.open() modes such as "w:gz" and, from Python 3.14, "w:zst" choose the compressor.
• Text shrinks to a quarter of its size or less while compressed media does not shrink at all; choose a compressor by who reads the files and how often: in the lesson’s runs zstd was the fastest, and bz2 and lzma made the smallest files.
• For tar, rely on the "data" filter, the default from Python 3.14, and pass it explicitly on 3.12 and 3.13.
• For zip, check every name and the declared total before extracting, and cap the bytes you actually write.