Iteration with purrr

Five clinics send a fee export each night, and one of them has the letter O typed where a zero should be: map(exports, read_fees) stops at that third export, and the four good tables are lost with it. purrr’s typed maps, its maps over several inputs and its two functions for failures let the same loop keep the four tables, name the one that failed and stack the rest into a single data frame.

On recorded runs of R 4.6.1 with purrr 1.2.2 you will map with functions that promise a type, walk two or more inputs with map2(), imap() and pmap(), catch failures with safely() and possibly(), stack results with list_rbind() and list_c(), and decide when the base functionals of the previous lesson are the better choice. purrr is a package: install it once with install.packages("purrr") and load it with library(purrr).

map() returns a list, and a typed map returns one type or stops

map(admissions, max) returns a list, like lapply(); map_dbl(admissions, mean) returns a vector of doubles. purrr’s reference page describes the typed maps, map_lgl(), map_int(), map_dbl() and map_chr(), as returning an atomic vector of the indicated type, or dying trying: every result must be one value of that type. A typed map is therefore type-stabletype-stable: Said of a function whose result has a type and length you can tell from its inputs alone, not from their values. vapply() and purrr's map_dbl() are type-stable; sapply() is not., as vapply() is, without a template to write:

The run tries each, then two results that break the promise, then the shorthand that extracts one element from each record:

from session import run, version

version()
# R reads map_typed.R as if each line were typed at the console.
run("""
R --vanilla --quiet -f map_typed.R
""")

# R version 4.6.1 -- "Happy Hop"
$ R --vanilla --quiet -f map_typed.R
> library(purrr)
> packageVersion("purrr")
[1] ‘1.2.2’
> source("wards.R")
> # map() always returns a list.
> map(admissions, max)
$ward_a
[1] 18

$ward_b
[1] 10

$ward_c
[1] 25

> # The typed maps return a vector of one type, or stop.
> map_dbl(admissions, mean)
   ward_a    ward_b    ward_c 
14.142857  8.142857 22.000000 
> map_int(admissions, \(d) sum(d > 15))
ward_a ward_b ward_c 
     2      0      7 
> map_lgl(admissions, \(d) any(d > 20))
ward_a ward_b ward_c 
 FALSE  FALSE   TRUE 
> map_chr(admissions, \(d) if (mean(d) > 15) "busy" else "normal")
  ward_a   ward_b   ward_c 
"normal" "normal"   "busy" 
> # A result of the wrong type stops the map, and the error names the element.
> tryCatch(map_dbl(admissions, \(d) if (max(d) > 24) "full" else max(d)), error = \(e) cat(conditionMessage(e), "\n"))
ℹ In index: 3.
ℹ With name: ward_c.
Caused by error:
! Can't coerce from a string to a double. 
> # map_chr() no longer turns numbers into text (purrr 1.2.0): convert them yourself.
> tryCatch(map_chr(admissions, max), error = \(e) cat(conditionMessage(e), "\n"))
ℹ In index: 1.
ℹ With name: ward_a.
Caused by error:
! Can't coerce from a number to a string. 
> map_chr(admissions, \(d) as.character(max(d)))
ward_a ward_b ward_c 
  "18"   "10"   "25" 
> # A name or a position extracts one element from each record.
> patients <- list(list(id = "P1", age = 34), list(id = "P2", age = 51))
> map_chr(patients, "id")
[1] "P1" "P2"
> map_dbl(patients, 2)
[1] 34 51

The errors say where the map stopped: “In index: 3”, “With name: ward_c”, then the cause, “Can’t coerce from a string to a double”. The reference page adds that every error inside a map is wrapped this way, in an error of class purrr_error_indexed. The second error is a change of purrr 1.2.0: map_chr() no longer converts numbers to text, so map_chr(admissions, max) stops, and the fix is to say what you mean, \(d) as.character(max(d)). A string or a position in place of the function extracts an element from each record: map_chr(patients, "id") read each id.

These changes arrived in three releases, so code written for an older purrr may stop on a newer one:

map2(), imap() and pmap() take several inputs at once

map2_int(admissions, targets, ...) counts each ward’s days above its own target, the job mapply() did in the previous lesson, with a typed result. imap() is map2() over each element and its name, or its position when there are no names. pmap() takes any number of inputs as a list, or the columns of a data frame, and calls the function once per row, matching columns to arguments by name. Watch it work through a grid of study sizes and effects:

The run puts all three to work, and gives map2() two inputs whose lengths do not match:

from session import run, version

version()
# R reads several_inputs.R as if each line were typed at the console.
run("""
R --vanilla --quiet -f several_inputs.R
""")

# R version 4.6.1 -- "Happy Hop"
$ R --vanilla --quiet -f several_inputs.R
> library(purrr)
> source("wards.R")
> # map2() walks two inputs in step and returns a typed result.
> map2_int(admissions, targets, \(days, target) sum(days > target))
ward_a ward_b ward_c 
     3      1      4 
> map2_chr(c("x", "y"), 1:2, \(s, n) strrep(s, n))
[1] "x"  "yy"
> # Unlike mapply(), it recycles only an input of length 1, and stops otherwise.
> map2_dbl(1:3, 10, \(a, b) a * b)
[1] 10 20 30
> tryCatch(map2_int(1:4, 1:2, \(a, b) a + b), error = \(e) cat(conditionMessage(e), "\n"))
Can't recycle `.x` (size 4) to match `.y` (size 2). 
> # imap() passes each element with its name, or with its position when there are no names.
> imap_chr(targets, \(target, ward) paste(ward, "aims for", target))
              ward_a               ward_b               ward_c 
"ward_a aims for 14"  "ward_b aims for 9" "ward_c aims for 21" 
> imap_chr(c(70, 82), \(score, i) paste("candidate", i, "scored", score))
[1] "candidate 1 scored 70" "candidate 2 scored 82"
> # pmap() takes a list of inputs, or a data frame, and calls once per row.
> grid <- expand.grid(n = c(20, 40), delta = c(0.5, 1), sd = 1)
> grid$power <- pmap_dbl(grid, \(n, delta, sd) power.t.test(n = n, delta = delta, sd = sd)$power)
> grid$power <- round(grid$power, 3)
> grid
   n delta sd power
1 20   0.5  1 0.338
2 40   0.5  1 0.598
3 20   1.0  1 0.869
4 40   1.0  1 0.993

The grid answers a planning question in one call: with 20 people a group, a two-sample t-test detects an effect of half a standard deviation with a power of 0.338, and with 40 people 0.598, while an effect of a whole standard deviation reaches 0.869 and 0.993. The unequal lengths show a difference from base R that protects you: map2() recycles only an input of length 1, such as the 10 in map2_dbl(1:3, 10, ...), and stops with “Can’t recycle .x (size 4) to match .y (size 2)” where mapply() would have recycled without a word.

safely() and possibly() keep a loop going after a failure

map(exports, read_fees) stops at the third export and returns nothing, although four of the five exports were fine. purrr calls safely() and possibly() adverbsadverb: In purrr, a function that takes a function and returns a changed version of it, such as safely() and possibly(): it modifies a verb, hence the name.: each takes a function and returns a changed version of it. possibly(f, otherwise = NULL) returns the fallback where f would fail. safely(f) always succeeds and returns a list of two, result and error, one of them NULL. Follow the five exports through possibly():

The run shows the plain map stopping, then the two adverbs:

from session import run, version

version()
# R reads read_many.R as if each line were typed at the console.
run("""
R --vanilla --quiet -f read_many.R
""")

# R version 4.6.1 -- "Happy Hop"
$ R --vanilla --quiet -f read_many.R
> library(purrr)
> source("exports.R")
> # One bad export stops map(), and the error says which one.
> tryCatch(map(exports, read_fees), error = \(e) cat(conditionMessage(e), "\n"))
ℹ In index: 3.
ℹ With name: hadapsar.
Caused by error in `scan()`:
! scan() expected 'a real', got '2O0' 
> # possibly() returns a fallback instead of the error, here NULL.
> tables <- map(exports, possibly(read_fees, otherwise = NULL))
> map_lgl(tables, is.null)
   aundh    baner hadapsar  kothrud    wakad 
   FALSE    FALSE     TRUE    FALSE    FALSE 
> # safely() returns each result with its error, so failures can be reported.
> attempts <- map(exports, safely(read_fees))
> str(attempts$hadapsar, max.level = 1)
List of 2
 $ result: NULL
 $ error :List of 2
  ..- attr(*, "class")= chr [1:3] "simpleError" "error" "condition"
> failed <- keep(attempts, \(a) !is.null(a$error))
> map_chr(failed, \(a) conditionMessage(a$error))
                             hadapsar 
"scan() expected 'a real', got '2O0'" 

The plain map’s error named the export, “With name: hadapsar”, and the cause, scan() expecting a number and finding 2O0. With possibly() the map finished and map_lgl(tables, is.null) marked Hadapsar as the one that failed. With safely(), Hadapsar’s attempt held a NULL result and the error, so keep(), purrr’s version of Filter(), picked out the failures and map_chr() turned them into a report of one line per file.

The results are stacked into one table or one vector

list_rbind(tables, names_to = "clinic") turns the four tables into one data frame of 6 rows, with a column that says which clinic each row came from. purrr 1.0.0 added list_c(), list_rbind() and list_cbind() to combine the results of a map, and made the older map_dfr() and map_dfc() supersededsuperseded: A lifecycle stage of a function that still works and is kept but has a better replacement, such as list_rbind(map(...)) for purrr's map_dfr().: they still work, but list_rbind(map(...)) is the form to write. Step through the stacking:

The run stacks the tables, joins the fee vectors with list_c(), and prints one line per clinic with iwalk(), the side-effect form of imap():

from session import run, version

version()
# R reads combine.R as if each line were typed at the console.
run("""
R --vanilla --quiet -f combine.R
""")

# R version 4.6.1 -- "Happy Hop"
$ R --vanilla --quiet -f combine.R
> library(purrr)
> source("exports.R")
> tables <- map(exports, possibly(read_fees, otherwise = NULL))
> # list_rbind() stacks the data frames; names_to adds a column from the list's names.
> fees <- list_rbind(tables, names_to = "clinic")
> fees
   clinic visit fee
1   aundh     1 250
2   aundh     2 400
3   baner     1 300
4 kothrud     1 500
5 kothrud     2 500
6 kothrud     3 200
> # The failed export left NULL, which list_rbind() skipped; Wakad's export had no rows.
> table(fees$clinic)

  aundh   baner kothrud 
      2       1       3 
> # list_c() joins vectors of any lengths into one vector.
> list_c(map(tables, \(t) t$fee))
[1] 250 400 300 500 500 200
> # walk() is map() for side effects, such as printing, and returns its input invisibly.
> iwalk(compact(tables), \(t, clinic) cat(clinic, "visits:", nrow(t), "\n"))
aundh visits: 2 
baner visits: 1 
kothrud visits: 3 
wakad visits: 0 

table(fees$clinic) shows 2 rows from Aundh, 1 from Baner and 3 from Kothrud: Hadapsar’s NULL was skipped, and Wakad’s export had no visits that night. list_c() joined the six fees into one vector. walk() and iwalk() call the function for what it does, here printing, and return their input invisibly, so they suit printing, writing files and plotting; compact() dropped the NULL before the walk.

purrr or base R: the same loops, different promises

Every map in this lesson has a base R twin, and the run confirms that they give identical() results for the wards of the first section. The choice is therefore not about what can be computed, but about what each call promises when the data turn out to be unusual, and what it costs to have:

from session import run, version

version()
# R reads purrr_or_base.R as if each line were typed at the console.
run("""
R --vanilla --quiet -f purrr_or_base.R
""")

# R version 4.6.1 -- "Happy Hop"
$ R --vanilla --quiet -f purrr_or_base.R
> library(purrr)
> source("wards.R")
> # The same results with purrr and with base R.
> identical(map_dbl(admissions, mean), vapply(admissions, mean, numeric(1)))
[1] TRUE
> identical(map(admissions, max), lapply(admissions, max))
[1] TRUE
> identical(map2_int(admissions, targets, \(d, t) sum(d > t)), mapply(\(d, t) sum(d > t), admissions, targets))
[1] TRUE
> # What purrr brings with it: the packages it imports.
> cat(packageDescription("purrr")$Imports, "\n")
cli (>= 3.6.1), lifecycle (>= 1.0.3), magrittr (>= 1.5.0),
rlang (>= 1.1.1), vctrs (>= 0.6.3) 

purrr 1.2.2 imports five packages, cli, lifecycle, magrittr, rlang and vctrs, which a project or a package then depends on as well. In return it gives the same argument order everywhere, .x first, errors that name the element, recycling that refuses to guess, and in_parallel(), which purrr 1.1.0 added to run a map in parallel worker processes through the mirai package; this lesson’s runs do not use it.

Practise: predict, then check

Before you run the four lines, write down the type and the values you expect from each.

from session import run, version

version()
# R reads practice.R as if each line were typed at the console.
run("""
R --vanilla --quiet -f practice.R
""")

# R version 4.6.1 -- "Happy Hop"
$ R --vanilla --quiet -f practice.R
> library(purrr)
> # 1. A typed map:
> map_int(list(1:3, 4:9), length)
[1] 3 6
> # 2. Two inputs:
> map2_dbl(c(2, 3), c(10, 100), \(a, b) a * b)
[1]  20 300
> # 3. A fallback:
> map_dbl(list(4, "x", 16), possibly(sqrt, otherwise = NA_real_))
[1]  2 NA  4
> # 4. Stacking with a source column:
> list_rbind(list(north = data.frame(n = 1:2), south = data.frame(n = 5)), names_to = "region")
  region n
1  north 1
2  north 2
3  south 5

The lengths 3 and 6 came back as integers; the products 20 and 300 as doubles; possibly() turned the failed square root of "x" into NA between 2 and 4; and the stacked table has three rows, two from north and one from south.

When you are ready, answer the questions below. They start where your answers have been weakest, and each choice explains the promise or the rule it relies on:

Interview questions

Warm-up: what does map_dbl() promise that map() does not?

map() returns a list with one element per input, and each element can be anything, so the caller has to check what came back. map_dbl() promises a double vector of the same length as its input, one number per element, with the input’s names. If a result is not a single number, for example text, a vector of two values or NULL, it stops with an error that gives the index and the name of the element, wrapped in the class purrr_error_indexed. That makes it type-stable, like vapply(x, f, numeric(1)). Since purrr 1.2.0 the typed maps also refuse conversions they used to make: map_chr() no longer turns numbers into text, so write as.character() inside the function when that is the intent.

Key takeaways

• map() returns a list, and map_lgl(), map_int(), map_dbl() and map_chr() return a vector of their type or stop with an error that names the element; since purrr 1.2.0 map_chr() no longer turns numbers into text.
• map2() walks two inputs in step and refuses unequal lengths, imap() passes each element with its name or position, and pmap() calls the function once per row of a data frame, matching columns to arguments by name.
• possibly() replaces a failure with a fallback, and safely() returns every result with its error, so one bad input no longer stops the loop and the failure can still be reported.
• list_rbind(), with names_to for the source, and list_c() combine the results at the end, replacing the superseded map_dfr(); walk() and iwalk() are for side effects such as printing.
• purrr and base R give identical results; purrr adds clearer errors and stricter recycling at the cost of five imported packages.