Units, powers of two and time conversions

Most estimates that go wrong in a design round go wrong in the units, not in the idea: a rate per minute read as a rate per second, gigabits taken for gigabytes, or a drive’s decimal terabytes compared with memory’s binary gibibytes. This lesson settles the units first, so that every later estimate in this module stands on firm ground: the two families of size prefixes, bits and bytes, the powers of two that set real limits, the step from a daily count to a rate per second, and how to round without fooling yourself.

Two families of size prefixes

Kilo, mega, giga, tera and peta are SI prefixes, and SI prefixes are powers of ten: a kilobyte (kB) is 1,000 bytes and a gigabyte (GB) is $10^{9}$ bytes (NIST, SI prefixes). Computer memory is built in powers of two, because its addresses are binary numbers, so a second family of prefixes names exact binary multiples: kibi (Ki, $2^{10}$), mebi (Mi, $2^{20}$), gibi (Gi, $2^{30}$), tebi (Ti) and pebi (Pi). The IEC defined these binary prefixes, and they are not part of the SI (NIST, binary prefixes). Watch the letter case: the decimal kilo is a lowercase k (kB), the binary kibi a capital K (KiB).

# Decimal (SI) prefixes are powers of 10; binary (IEC) prefixes are powers of 2.
# The two families start close together and drift apart with every step.
PREFIXES = [("kilo", "kB", "KiB"), ("mega", "MB", "MiB"), ("giga", "GB", "GiB"), ("tera", "TB", "TiB"), ("peta", "PB", "PiB"), ("exa", "EB", "EiB")]

print(f"{'Prefix':<8}{'Decimal':<13}{'Binary':<13}{'Gap':>6}")
for step, (name, dec, binary) in enumerate(PREFIXES, start=1):
    gap = (2 ** (10 * step) / 10 ** (3 * step) - 1) * 100
    print(f"{name:<8}{dec:<3} = 10^{3 * step:<4}{binary:<3} = 2^{10 * step:<5}{gap:5.1f}%")
print("Gap: how much bigger the binary unit is than the decimal one")

print()
print(f"1 GB  = {10**9:>13,} bytes")
print(f"1 GiB = {2**30:>13,} bytes")
disk = 10**12  # a drive sold as "1 TB"
print(f'A "1 TB" drive holds {disk / 2**30:.1f} GiB, which is {disk / 2**40:.3f} TiB')

Prefix  Decimal      Binary          Gap
kilo    kB  = 10^3   KiB = 2^10     2.4%
mega    MB  = 10^6   MiB = 2^20     4.9%
giga    GB  = 10^9   GiB = 2^30     7.4%
tera    TB  = 10^12  TiB = 2^40    10.0%
peta    PB  = 10^15  PiB = 2^50    12.6%
exa     EB  = 10^18  EiB = 2^60    15.3%
Gap: how much bigger the binary unit is than the decimal one

1 GB  = 1,000,000,000 bytes
1 GiB = 1,073,741,824 bytes
A "1 TB" drive holds 931.3 GiB, which is 0.909 TiB

The two start 2.4% apart and drift further with every step: a tebibyte is 10% bigger than a terabyte, a pebibyte 12.6% bigger than a petabyte. The last line explains a familiar puzzle. A program that divides a byte count by $2^{30}$ but prints the label “GB” shows a 1 TB drive as about 931 GB. No space is missing; the unit changed and the label did not. NIST’s page records the same split in practice: memory makers have used the binary megabyte, and storage makers usually the decimal one.

In an estimate, choose one family and say which. Decimal units are easier to multiply in your head, and for most estimates the gap is smaller than the uncertainty in the assumptions. Switch to binary units when the thing you are sizing is counted in binary: memory, page and block sizes, and any limit that a system documents in KiB or MiB.

A program computes the same thing whichever family you meant, so a mix-up gives no error, only a wrong number:

disk_bytes = 500 * 10**9  # a drive sold as "500 GB" (decimal units)
print(round(disk_bytes / 2**30, 1), "GiB")

465.7 GiB

Bits and bytes

A byte is 8 bits (NIST). Files, disks and memory are counted in bytes, written with a capital B, but links are rated in bits per second, written with a lowercase b: the IEEE 802.3 working group names Ethernet speeds such as 400 Gb/s. Confusing the two puts an estimate out by exactly a factor of eight.

# Links are rated in bits per second; files and disks are measured in bytes.
BITS_PER_BYTE = 8

link_bits_per_s = 1 * 10**9  # a "1 Gbit/s" link
bytes_per_s = link_bits_per_s / BITS_PER_BYTE
print(f"1 Gbit/s carries at most {bytes_per_s / 10**6:.0f} MB/s ({bytes_per_s / 2**20:.1f} MiB/s)")

file_bytes = 1 * 10**12  # copying "1 TB"
seconds = file_bytes / bytes_per_s
print(f"Copying 1 TB over it takes at least {seconds:,.0f} s, about {seconds / 3600:.1f} hours")
print(f"Forgetting the 8 would promise {file_bytes / link_bits_per_s:,.0f} s instead")

1 Gbit/s carries at most 125 MB/s (119.2 MiB/s)
Copying 1 TB over it takes at least 8,000 s, about 2.2 hours
Forgetting the 8 would promise 1,000 s instead

Treat a link’s rating as a ceiling, not a promise: the same link also carries protocol headers, other traffic and retransmissions, so a real copy takes longer than the time computed here. The figure is still useful, because it is a lower bound. If even the best case is too slow, no tuning will rescue the design.

Powers of two worth knowing

Ten doublings make about a thousand, because $2^{10} = 1024$. Twenty doublings make about a million and thirty about a billion. Two more powers turn up in designs because they are the sizes of common integer types: a 32-bit unsigned integer has $2^{32}$ possible values and a 64-bit one $2^{64}$.

# Powers of two worth knowing, and how long an ID space lasts.
for exponent in [10, 16, 20, 30, 32, 40, 64]:
    value = 2**exponent
    print(f"2^{exponent:<3} = {value:>26,}  (about {value:.1e})")

print()
SECONDS_PER_DAY = 86_400
SECONDS_PER_YEAR = 365 * SECONDS_PER_DAY
ids_32 = 2**32
print(f"A 32-bit counter, 1,000 new IDs a second: full after {ids_32 / 1_000 / SECONDS_PER_DAY:.1f} days")
ids_64 = 2**64
print(f"A 64-bit counter, 1 million new IDs a second: full after {ids_64 / 10**6 / SECONDS_PER_YEAR:,.0f} years")
codes = 62**7
print(f"7-character codes from 62 letters and digits: {codes:,} (about {codes:.1e})")

2^10  =                      1,024  (about 1.0e+03)
2^16  =                     65,536  (about 6.6e+04)
2^20  =                  1,048,576  (about 1.0e+06)
2^30  =              1,073,741,824  (about 1.1e+09)
2^32  =              4,294,967,296  (about 4.3e+09)
2^40  =          1,099,511,627,776  (about 1.1e+12)
2^64  = 18,446,744,073,709,551,616  (about 1.8e+19)

A 32-bit counter, 1,000 new IDs a second: full after 49.7 days
A 64-bit counter, 1 million new IDs a second: full after 584,942 years
7-character codes from 62 letters and digits: 3,521,614,606,208 (about 3.5e+12)

These limits decide real designs. A 32-bit counter that hands out 1,000 IDs a second runs out in under two months, while a 64-bit counter at a million a second outlasts any system you will build. Short codes work the same way: seven characters drawn from 62 letters and digits give about 3.5 trillion codes, which is why a link shortener’s links can stay short for a very long time.

From a daily count to a rate per second

Requirements usually arrive as counts per day or per month (“20 million messages a day”), while servers, databases and queues are sized in operations per second. A day has $24 \times 60 \times 60 = 86400$ seconds, and the rest follows from it:

# Seconds in the periods that estimates use (a month is taken as 30 days, a year as 365).
MINUTE = 60
HOUR = 60 * MINUTE
DAY = 24 * HOUR
MONTH = 30 * DAY
YEAR = 365 * DAY

for name, seconds in [("minute", MINUTE), ("hour", HOUR), ("day", DAY), ("30-day month", MONTH), ("365-day year", YEAR)]:
    print(f"1 {name:<13} = {seconds:>10,} s")

print()
print(f"{'Events a day':>16}  {'average per second':>18}")
for exponent in range(6, 11):
    per_day = 10**exponent
    print(f"{per_day:>16,}  {per_day / DAY:>18,.1f}")

print()
monthly = 3 * 10**9  # an example: 3 billion events in a 30-day month
print(f"3 billion a month = {monthly / MONTH:,.0f} per second on average")

1 minute        =         60 s
1 hour          =      3,600 s
1 day           =     86,400 s
1 30-day month  =  2,592,000 s
1 365-day year  = 31,536,000 s

    Events a day  average per second
       1,000,000                11.6
      10,000,000               115.7
     100,000,000             1,157.4
   1,000,000,000            11,574.1
  10,000,000,000           115,740.7

3 billion a month = 1,157 per second on average

Two shortcuts are worth remembering: a million events a day is about 12 a second, and a billion a day about 11,600 a second. For a 30-day month divide by about 2.6 million, and for a year by about 31.5 million. Every one of these is an average over the whole period. Real traffic is never flat; the lesson on estimating traffic turns an average into a peak.

Rounding on purpose

State an estimate with one or two significant figures: “about 17,000 requests a second” is honest, while 17,361 claims a precision that the assumptions behind it do not have. Every rounding pushes the result one way, though, so round on purpose and say which way it went.

import math

DAY = 86_400


def sig(x, digits=2):
    """x rounded to `digits` significant figures."""
    if x == 0:
        return 0
    places = digits - 1 - math.floor(math.log10(abs(x)))
    value = round(x, places)
    return int(value) if places <= 0 else value


per_day = 1.5 * 10**9  # an estimate: 1.5 billion requests a day

exact = per_day / DAY
rounded_day = per_day / 10**5  # treating a day as 100,000 seconds

print(f"Exact:                  {exact:,.0f} per second")
print(f"Day taken as 10^5 s:    {rounded_day:,.0f} per second ({(rounded_day / exact - 1) * 100:+.1f}%)")
print(f"Two significant digits: {sig(exact):,} per second")
print()
# Every rounding pushes an estimate one way or the other: say which.
for label, value in [("Day rounded up to 10^5 s", rounded_day), ("Day rounded down to 8 x 10^4 s", per_day / (8 * 10**4))]:
    direction = "under" if value < exact else "over"
    print(f"{label:<31} {value:>7,.0f}/s  {direction}estimates the load")

Exact:                  17,361 per second
Day taken as 10^5 s:    15,000 per second (-13.6%)
Two significant digits: 17,000 per second

Day rounded up to 10^5 s         15,000/s  underestimates the load
Day rounded down to 8 x 10^4 s   18,750/s  overestimates the load

Dividing by $10^{5}$ instead of 86,400 is tempting because it is easy, but every rate comes out about 14% too low (the true rate is about 16% higher than the shortcut’s). When you size capacity, that is the unsafe direction: you would plan for less load than will arrive. When the question is how much load a system must carry, round so that the load comes out higher; when it is how long a resource will last, round so that the time comes out shorter. If you do take a shortcut, say so: “about 15,000 a second, a little low because I used $10^{5}$ seconds a day”.

LAKH = 10**5
CRORE = 10**7
DAY = 86_400

# Python accepts _ between digits, so Indian digit grouping can be written as it is printed.
monthly = 2_45_089.58 * LAKH  # a monthly volume given in lakh
print(f"2,45,089.58 lakh = {monthly:,.0f} = {monthly:.2e}")
print(f"Spread over a 31-day month: {monthly / (31 * DAY):,.0f} per second")
print(f"1 crore a day = {CRORE / DAY:,.1f} per second")

2,45,089.58 lakh = 24,508,958,000 = 2.45e+10
Spread over a 31-day month: 9,151 per second
1 crore a day = 115.7 per second

Mistakes to check before you say a number

• Bits for bytes. A “Gb” is a gigabit. Divide a link rate by 8 before comparing it with a file size.
• Binary for decimal. A GiB is 7.4% more than a GB, and a TiB 10% more than a TB. Say which one you mean.
• The wrong period. Per minute and per second differ by 60 times. Even month lengths matter a little: a 31-day month has 3.3% more seconds than a 30-day one.
• An average taken for a peak. A daily count divided by 86,400 is the average rate. The busiest hour always brings at least that much, and usually far more.

Key takeaways

• SI prefixes (kB, MB, GB) are powers of ten; IEC prefixes (KiB, MiB, GiB) are powers of two. They differ by 2.4% at kilo and 10% at tera, so pick one family and say which.
• Links are rated in bits per second and data is counted in bytes: divide by 8.
• $2^{10} \approx 10^{3}$, and the sizes of 32-bit and 64-bit integers decide how long an ID space lasts.
• Divide a daily count by 86,400 to get the average rate per second; a million a day is about 12 a second.
• Keep one or two significant figures, and round in the direction that keeps the answer safe.