Nulls inside collections and generics

An electricity meter app uploads seven readings that a field worker typed in: one never arrived, one was left blank and one was mistyped as 10x5. Stored as a List<String?>, the type says that any reading may be missing; stored as a List<String>?, it says something else, that the whole list may be missing.

You will tell those types apart and read each one safely, clean nullable data in one pass with filterNotNull, mapNotNull and firstNotNullOfOrNull, and see how generics carry nullability: a type parameter T can stand for a nullable type, and T & Any is its version without null.

A question mark on the items and one on the list mean different things

Nulls inside collections come in two places, and the type says which. A List<String?> is a list that is always there, whose items may be null; a List<String>? is a list that may be missing as a whole, with items that never are; and a List<String?>? allows both. Three lists from a booking form:

// Three lists from a booking form: where the ? goes decides what can be missing.
fun main() {
    // Items may be missing: a middle name is optional for each passenger.
    val middleNames: List<String?> = listOf("Kumar", null, "Devi")
    // The whole list may be missing: nobody has filled in the co-passengers yet.
    val coPassengers: List<String>? = listOf<List<String>?>(null).first()
    // Both: the nominee list may be missing, and so may each nominee.
    val nominees: List<String?>? = listOf(null, "Ravi")

    println("middle names: ${middleNames.size}, length of the second: ${middleNames[1]?.length}")
    println("co-passengers: ${coPassengers?.size ?: 0}")
    println("first nominee: ${nominees?.get(0)?.length}, second: ${nominees?.get(1)?.length}")
    println("co-passengers, or an empty list: ${coPassengers.orEmpty()}")
}

middle names: 3, length of the second: null
co-passengers: 0
first nominee: null, second: 4
co-passengers, or an empty list: []

Each ? in a type asks for a ?. in the code where it stands: middleNames.size needs none, an item of that list does, and the co-passenger list needs one before size. A nominee’s length needs two, because everything after a safe call is nullable.

orEmpty() turns a missing list into an empty one, which the code then reads like any other list: the last line printed [].

filterNotNull, mapNotNull and firstNotNullOfOrNull clean nullable data

Nulls inside collections usually have to go before the values can be used. The function filterNotNull() drops the items that are null. mapNotNull { … } runs its block on each item and keeps the results that are not null, so it can parse and clean in a single pass. firstNotNullOfOrNull { … } stops at the first item whose block gives a value:

// Meter readings typed in by hand: one never arrived, one is blank and one is mistyped.
fun main() {
    val raw: List<String?> = listOf("1042", null, "1057", "", "10x5", "1063", "1071")

    // Parse each reading and drop every one that is missing or not a number, in one pass.
    val readings: List<Int> = raw.mapNotNull { it?.toIntOrNull() }
    println("kept ${readings.size} of ${raw.size}: $readings")
    println("dropped ${raw.size - readings.size}")

    // filterNotNull drops only the nulls: the blank and the mistyped text stay.
    println("not null: ${raw.filterNotNull()}")

    // The first reading that parses, without parsing the rest.
    println("first number: ${raw.firstNotNullOfOrNull { it?.toIntOrNull() }}")
    println("units used: ${readings.last() - readings.first()}")
}

kept 4 of 7: [1042, 1057, 1063, 1071]
dropped 3
not null: [1042, 1057, , 10x5, 1063, 1071]
first number: 1042
units used: 29

Step through the mapNotNull call one reading at a time:

filterNotNull() kept the blank and “10x5”: they are strings, not nulls. mapNotNull { it?.toIntOrNull() } dropped all three gaps, and its result is a List<Int>, so the rest of the code reads it without a single ?.: the 29 units used came straight from it.

A type parameter can stand for a nullable type

Generics carry nullability through to their callers. A type parameterThe placeholder T in a declaration such as fun <T> firstItem(items: List<T>): T. Each call fills it with a real type, here the type of the list's items. T with no bound can be any type, nullable types included, because its default upper bound is Any?. So firstItem(nominees) returns a String? when the list is a List<String?>:

// A type parameter takes whatever type its caller has, a nullable one included.
fun <T> firstItem(items: List<T>): T = items.first()

// T & Any is "T without null": the result is never null, whatever T is.
fun <T> firstPresent(items: List<T>): T & Any =
    items.firstNotNullOfOrNull { it } ?: throw NoSuchElementException("no item is present")

fun main() {
    val nominees: List<String?> = listOf(null, "Ravi")

    // Here T is String?, so the result is a String? and needs ?. to read.
    val first = firstItem(nominees)
    println("firstItem gave $first, and first?.length gives ${first?.length}")

    // Here the result is a String, so .length needs no ?.
    val present = firstPresent(nominees)
    println("firstPresent gave $present, and present.length gives ${present.length}")
}

firstItem gave null, and first?.length gives null
firstPresent gave Ravi, and present.length gives 4

T & Any is the type T without null, a definitely non-nullable typedefinitely non-nullable type: A type written T & Any for a type parameter T: the type T without null, whatever type a caller gives T. A function can return T & Any even when T itself may be nullable., stable since Kotlin 1.7.0. firstPresent returns one whatever T is, so present.length needs no ?.. A bound, <T : Any>, goes further and refuses nullable types for T altogether. With generics, choose by what the function accepts: T : Any when null has no place in it, and plain T with T & Any in the result when it accepts nulls and removes them.

Missing, null or empty: collections and maps in an API, measured Pro: opens with a Pro, Premium or Ultimate pass

Generic functions that keep null where it belongs, with T & Any Pro: opens with a Pro, Premium or Ultimate pass

Practise: scores from typed-in marks

This program checks a function that turns typed-in marks into scores from 0 to 100 and counts what it rejected. As given, two of its five checks fail:

// Practice: fix parseScores so that all five checks pass. It returns the valid scores, 0 to 100, in their order,
// and the number of entries it rejected: missing, blank, not a number or out of range.
fun parseScores(raw: List<String?>): Pair<List<Int>, Int> {
    val scores = raw.mapNotNull { it?.toIntOrNull() }
    return scores to raw.size - scores.size
}

fun main() {
    val checks = listOf(
        listOf("90", null, "100") to (listOf(90, 100) to 1),
        listOf("", "abc") to (emptyList<Int>() to 2),
        listOf("-5", "7") to (listOf(7) to 1),
        listOf("101", "0") to (listOf(0) to 1),
        emptyList<String?>() to (emptyList<Int>() to 0),
    )
    var passed = 0
    for ((raw, want) in checks) {
        val got = parseScores(raw)
        val ok = got == want
        if (ok) passed++
        println("${if (ok) "ok  " else "FAIL"} parseScores($raw) = $got, want $want")
    }
    println("$passed of ${checks.size} checks pass")
    if (passed < checks.size) kotlin.system.exitProcess(1)
}

ok   parseScores([90, null, 100]) = ([90, 100], 1), want ([90, 100], 1)
ok   parseScores([, abc]) = ([], 2), want ([], 2)
FAIL parseScores([-5, 7]) = ([-5, 7], 0), want ([7], 1)
FAIL parseScores([101, 0]) = ([101, 0], 0), want ([0], 1)
ok   parseScores([]) = ([], 0), want ([], 0)
3 of 5 checks pass

Copy it, fix parseScores, and run it with kotlinc ParseScores.kt -include-runtime -d scores.jar && java -jar scores.jar until all five checks pass. First hint: a number outside 0 to 100 should give null inside the mapNotNull block, so that it is dropped and counted like a blank.

The questions below cover the three ideas of the first three sections, the one you know least first:

Interview questions

Key takeaways

• List<String?> may hold null items, List<String>? may be null as a whole, and List<String?>? may be both; each ? asks for a ?. where it stands.
• filterNotNull() removes only the nulls, mapNotNull { … } parses and cleans in one pass and gives a list without nulls, and firstNotNullOfOrNull { … } stops at the first value its block gives.
• A type parameter T can be a nullable type unless it is bounded with T : Any, and T & Any is T without null.