Lists: create, index, slice and modify

A class of 40 students hands in marks, a shopping cart gains and loses items, and a file arrives line by line. Each is an ordered group of values that changes while the program runs, and Python’s list holds such a group. You write one with square brackets, as in cart = ["rice", "tea"], read an item by its indexindex: The position of an item in a sequence such as a string or a list. Python counts from 0 at the left and from -1 at the right, so s[0] is the first item and s[-1] the last. More in “Strings: literals, indexing and slicing”, take a run of items with a sliceslice: A run of items taken from a sequence with s[start:stop:step]. It includes start, stops before stop, and shortens itself instead of raising an error when a position is past the end. More in “Strings: literals, indexing and slicing”, and add, remove or replace items. A list keeps its items in the order you put them, holds values of any type and is mutablemutable: Able to change in place after it is made. Lists, dicts and sets are mutable: append() changes the list itself, and every name bound to that list sees the change.: one list object changes in place.

Below you can step through a cart while each method changes it, and watch the items after an insert or a removal move along. You will also catch the two habits behind many list bugs: expecting a method such as sort() to hand the list back, and changing the front of a long list as if it cost no more than the back.

Square brackets, list() and repetition make lists

Most lists start as square brackets with commas between the items. This program makes lists in five ways and asks a few questions of them:

empty = []
cart = ["rice", "lentils", "tea"]
mixed = [3, "three", 3.0, None, ["a", "list", "inside"]]
letters = list("chai")  # any iterable: here, a string's characters
evens = list(range(0, 10, 2))
zeros = [0] * 4  # repetition

print(len(empty), len(cart), len(mixed))
print(letters)
print(evens)
print(zeros)
print(cart + ["salt"])  # + builds a new list ...
print(cart)  # ... and leaves cart as it was
print("tea" in cart, "milk" in cart)

0 3 5
['c', 'h', 'a', 'i']
[0, 2, 4, 6, 8]
[0, 0, 0, 0]
['rice', 'lentils', 'tea', 'salt']
['rice', 'lentils', 'tea']
True False

[] is an empty list. list() makes a list from any iterableiterable: A value a for loop can go through one item at a time, such as a string, a list, a tuple, a dict, a set or a range. A dict gives its keys. More in “for loops with range, enumerate and zip”: a string gives its characters and a range its numbers. [0] * 4 repeats the items of a list, and + joins two lists into a third one, which is why cart still has three items after cart + ["salt"]. Both build new lists and leave the lists they were given alone. len() counts the items, and in checks whether any item is equal to a value, comparing items one at a time from the front.

The list called mixed holds an integer, a string, a float, None and another list. Python allows that, but most real lists hold one kind of value, because code that loops over a list usually does the same thing to every item. Press Edit, change list("chai") to list("masala") and run the program again: letters now holds six one-letter strings.

Indexes count from both ends, and a slice stops before its stop

Every item has a position, its index, counted from 0. Negative indexes count from the end, so cart[-1] is the last item and cart[-2] the one before it, however long the list is. The diagram shows both numberings for a cart of five items, and one slice:

A slice takes a run of items: cart[start:stop] starts at start and stops just before stop, the same rule as range(). Leave out start to begin at the front and stop to run to the end, and add a third number, the step, to take every second or third item. A negative step walks backwards, which makes cart[::-1] a reversed copy:

cart = ["rice", "lentils", "tea", "salt", "oil"]

print(cart[0], cart[2], cart[-1], cart[-2])
print(cart[1:3])  # positions 1 and 2: the stop is not included
print(cart[:2])  # from the start
print(cart[3:])  # to the end
print(cart[-2:])  # the last two items
print(cart[::2])  # every second item
print(cart[::-1])  # a reversed copy
print(cart[3:99])  # a slice that runs past the end is just shorter
print(cart[9])  # one position past the end is an error

rice tea oil salt
['lentils', 'tea']
['rice', 'lentils']
['salt', 'oil']
['salt', 'oil']
['rice', 'tea', 'oil']
['oil', 'salt', 'tea', 'lentils', 'rice']
['salt', 'oil']

Traceback (most recent call last):
  File "positions.py", line 11, in <module>
    print(cart[9])  # one position past the end is an error
          ~~~~^^^
IndexError: list index out of range

Slices and indexes treat the edges differently. A slice that runs past the end is cut short without complaint, so cart[3:99] simply gives the last two items, and a slice that starts past the end is empty. A single index must point at an item that exists, and cart[9] stops the program with IndexError: list index out of range. A slice is also always a new list: changing it does not change cart. Press Edit, change the last line to print(cart[4]) and run it again: position 4 exists, so the program now ends normally, after printing oil.

append(), insert(), remove() and pop() change the list itself

The methods of a list change that very list instead of making a new one. Step through one cart as seven changes are made to it. The lit line has just run; watch the items after insert() and remove() move along:

cart = ["rice", "lentils"]
cart.append("tea")
cart.extend(["salt", "oil"])
cart.insert(1, "flour")
cart[0] = "brown rice"
cart.remove("lentils")
last = cart.pop()
second = cart.pop(1)
print(cart, last, second)

['brown rice', 'tea', 'salt'] oil flour

insert(1, "flour") put the new item before position 1, and the four items after it each moved one place to the right. remove("lentils") deleted the first item equal to its argument, and the three items after it moved back. Each pop() took an item out and handed it back, which is how last and second got their values. The full set:

• append(x) adds one item at the end, and extend(items) adds every item of an iterable, one by one.
• insert(i, x) puts x before position i, and every later item moves one place to the right.
• cart[i] = x replaces the item at a position.
• remove(x) deletes the first item equal to x; pop() removes the last item and returns it, and pop(i) does the same for position i. del cart[i] deletes by position without returning anything, and clear() empties the whole list.
• index(x) gives the position of the first item equal to x, and count(x) says how many items are equal to it.

The next program makes the same changes, prints the cart after each one, and ends with two classic surprises:

cart = ["rice", "lentils"]
print("start:          ", cart)

cart.append("tea")
print("append('tea'):  ", cart)

cart.extend(["salt", "oil"])
print("extend([...]):  ", cart)

cart.insert(1, "flour")
print("insert(1, ...): ", cart)

cart[0] = "brown rice"
print("cart[0] = ...:  ", cart)

cart.remove("lentils")
print("remove(...):    ", cart)

last = cart.pop()
print("pop():          ", cart, "gave", repr(last))

second = cart.pop(1)
print("pop(1):         ", cart, "gave", repr(second))

print("index('salt'):  ", cart.index("salt"))
print("count('tea'):   ", cart.count("tea"))

cart.append(["soap", "comb"])  # one new item: a whole list
cart.extend("ink")  # three new items: 'i', 'n' and 'k'
print("surprises:      ", cart)
print("length:         ", len(cart))

start:           ['rice', 'lentils']
append('tea'):   ['rice', 'lentils', 'tea']
extend([...]):   ['rice', 'lentils', 'tea', 'salt', 'oil']
insert(1, ...):  ['rice', 'flour', 'lentils', 'tea', 'salt', 'oil']
cart[0] = ...:   ['brown rice', 'flour', 'lentils', 'tea', 'salt', 'oil']
remove(...):     ['brown rice', 'flour', 'tea', 'salt', 'oil']
pop():           ['brown rice', 'flour', 'tea', 'salt'] gave 'oil'
pop(1):          ['brown rice', 'tea', 'salt'] gave 'flour'
index('salt'):   2
count('tea'):    1
surprises:       ['brown rice', 'tea', 'salt', ['soap', 'comb'], 'i', 'n', 'k']
length:          7

append() adds its argument as a single item, so appending a list nests it: ['soap', 'comb'] is one item, and the cart’s length counts it once. extend() loops over its argument, so extending by the string "ink" adds three one-letter strings. cart += "ink" behaves like extend() and adds the three letters too, while cart.append("ink") and cart += ["ink"] add the whole word as one item.

When the item is not there

Two of the methods need the item to be in the list. This program asks for one that is not:

cart = ["rice", "tea"]

if "milk" in cart:  # test first ...
    cart.remove("milk")
print("after the check:", cart)

print(cart.index("milk"))  # ... or index() and remove() raise ValueError

after the check: ['rice', 'tea']

Traceback (most recent call last):
  File "missing_item.py", line 7, in <module>
    print(cart.index("milk"))  # ... or index() and remove() raise ValueError
          ~~~~~~~~~~^^^^^^^^
ValueError: list.index(x): x not in list

index() and remove() raise ValueError when no item is equal to the value, and pop() on an empty list raises IndexError. When an item may be missing, test with in first, as the first lines do. The lesson on exceptions shows the other way: catch the error.

Assigning to a slice replaces a whole range

A slice can also stand on the left of =, which is called slice assignmentslice assignment: An assignment with a slice on its left, as in week[1:3] = ['holiday']: the selected items are replaced by the new ones, which may be more or fewer unless the slice has a step.. The items of the slice are replaced by the items on the right, and the two counts do not have to match, so a list can grow or shrink in one statement:

week = ["mon", "tue", "wed", "thu", "fri"]

week[1:3] = ["holiday"]  # two items replaced by one
print(week)

week[1:1] = ["tue", "wed"]  # an empty slice: insert, replace nothing
print(week)

del week[::2]  # delete positions 0, 2 and 4
print(week)

nums = list(range(10))
nums[::3] = ["x", "x", "x", "x"]  # an extended slice takes exactly as many items
print(nums)

['mon', 'holiday', 'thu', 'fri']
['mon', 'tue', 'wed', 'holiday', 'thu', 'fri']
['tue', 'holiday', 'fri']
['x', 1, 2, 'x', 4, 5, 'x', 7, 8, 'x']

Two weekdays became one holiday, and the list got shorter. The empty slice week[1:1] is the gap before position 1: assigning to it inserts items and replaces none. del with a slice removes a range, and with a step, every second or third item. Press Edit, change the first assignment to week[1:3] = [] and run it: two items go and nothing takes their place, the same as del week[1:3].

Slices with a step other than 1 are stricter when you assign to them: they must get exactly as many items as they select, and nums[::3] = [1, 2] raises ValueError: attempt to assign sequence of size 2 to extended slice of size 4.

Methods that change a list return None, except pop()

sort(), reverse(), append(), extend(), insert(), remove() and clear() all change the list and return None. That is a deliberate convention in Python: a method that changes an object in placein place: Changing an object itself instead of making a new one. list.sort() sorts a list in place and returns None, while sorted() leaves its input as it was and returns a new list. returns None unless it hands back an item, as pop() does, so that nobody mistakes it for one that makes a new object. Forget it, and the list is gone. Step through both halves of this program and watch the rows of queue and names:

queue = ["Ravi", "Asha", "Mei"]
result = queue.sort()
print(result)
print(queue)

names = ["Ravi", "Asha", "Mei"]
names = names.sort()
print(names)
names.append("Kiran")

None
['Asha', 'Mei', 'Ravi']
None
Error AttributeError: 'NoneType' object has no attribute 'append'

names = names.sort() sorts the list and then binds names to the None that sort() returned. Nothing fails on that line. The error comes later, wherever the code next uses the list, and says 'NoneType' object has no attribute 'append', far from its cause. When you want a sorted copy, call sorted(names), which returns a new list; the lesson on sorting covers both in depth. The table pairs each change with the way to get a new list instead:

reversed(xs) is the odd one out: it returns neither None nor a list, but an iterator that reads the list from the end, ready for a for loop, and list(reversed(xs)) turns it into a list.

Changing the front of a list moves every other item

The Python documentation describes how CPython stores a list: as an array of references to the item objects, with some spare room at the end. Adding or removing at the end touches one slot of that array. Inserting or removing at the front moves every other reference one place. Step through five tickets served from both ends, and count what moves:

tickets = [101, 102, 103, 104, 105]
tickets.pop()
tickets.pop(0)
tickets.insert(0, 100)
tickets.append(106)
print(tickets)

[100, 102, 103, 104, 106]

pop() took 105 and nothing else moved. pop(0) took 101, and the three tickets behind it each moved up one place; insert(0, 100) then moved all three one place along again. On five items that costs nothing worth measuring, but the moves grow with the length of the list. This program empties lists of 10,000, 20,000 and 40,000 items from each end and times it:

from collections import deque
from time import perf_counter


def best_ms(empty, make, n):
    """The fastest of three runs that empty a fresh n-item container, in ms."""
    times = []
    for _ in range(3):
        items = make(range(n))
        start = perf_counter()
        empty(items)
        times.append(perf_counter() - start)
    return min(times) * 1000


def from_back(items):
    while items:
        items.pop()


def from_front(items):
    while items:
        items.pop(0)


def from_deque(items):
    while items:
        items.popleft()


for n in (10_000, 20_000, 40_000):
    back = best_ms(from_back, list, n)
    front = best_ms(from_front, list, n)
    left = best_ms(from_deque, deque, n)
    print(f"{n:>6,} items  pop(): {back:5.1f} ms  pop(0): {front:6.1f} ms  deque.popleft(): {left:4.1f} ms")

10,000 items  pop():   0.2 ms  pop(0):    7.0 ms  deque.popleft():  0.2 ms
20,000 items  pop():   0.4 ms  pop(0):   34.1 ms  deque.popleft():  0.3 ms
40,000 items  pop():   0.7 ms  pop(0):  148.0 ms  deque.popleft():  0.7 ms

pop() emptied each list in about a millisecond or less. pop(0) needed dozens of times longer for 10,000 items and hundreds of times longer for 40,000, and each doubling of the list made it four to five times slower. The reason is the moving: emptying a list of n items from the front moves about n × n ÷ 2 references, some 800 million for 40,000 items, while pop() moves none. insert(0, x) has the same problem in the other direction.

So a list is the right tool for a stackA collection where the last item added is the first one taken out, like a pile of plates. A list makes a good one: append() adds at the end and pop() takes from the same end., where you add and remove at the end, and the wrong one for a queueA collection served in arrival order: the first item added is the first one taken out, like people waiting at a counter. served from the front. For a queue, use collections.deque, which adds and removes at both ends in constant time; the lesson on the collections module shows it in use.

Practise: rotate a list and drop repeats

The exercise asks for two functions that return new lists and leave the list they are given unchanged: rotate() moves the items round with two slices, and dedupe_keep_order() keeps the first copy of each item. Its sample tests run in your browser, and its first hint shows where to start:

def rotate(xs, k):
    """Return a new list: xs rotated k places to the right (negative k: to the left)."""
    return xs


def dedupe_keep_order(xs):
    """Return a new list with the first copy of each item, in their original order."""
    return xs

from listtools import dedupe_keep_order, rotate


def test_rotate_right():
    """rotates to the right"""
    assert rotate([1, 2, 3, 4, 5], 2) == [4, 5, 1, 2, 3]
    assert rotate(["a", "b", "c"], 1) == ["c", "a", "b"]


def test_rotate_left_and_large():
    """rotates to the left for a negative k, and wraps a k larger than the list"""
    assert rotate([1, 2, 3, 4, 5], -1) == [2, 3, 4, 5, 1]
    assert rotate([1, 2, 3, 4, 5], 7) == [4, 5, 1, 2, 3]
    assert rotate([1, 2, 3, 4, 5], -12) == [3, 4, 5, 1, 2]


def test_rotate_edges():
    """returns a new list for k = 0 and an empty list for an empty list"""
    xs = [1, 2, 3]
    same = rotate(xs, 0)
    assert same == [1, 2, 3]
    assert same is not xs
    assert rotate([], 3) == []
    assert rotate([9], 4) == [9]


def test_rotate_keeps_input():
    """leaves the list it is given unchanged"""
    xs = [1, 2, 3, 4]
    rotate(xs, 1)
    assert xs == [1, 2, 3, 4]


def test_dedupe():
    """keeps the first copy of each item, in order"""
    assert dedupe_keep_order([3, 1, 3, 2, 1]) == [3, 1, 2]
    assert dedupe_keep_order(["tea", "rice", "tea"]) == ["tea", "rice"]
    assert dedupe_keep_order([7, 7, 7]) == [7]


def test_dedupe_edges():
    """returns a new list, also for an empty list or a list without repeats"""
    xs = [1, 2, 3]
    result = dedupe_keep_order(xs)
    assert result == [1, 2, 3]
    assert result is not xs
    assert dedupe_keep_order([]) == []


def test_dedupe_keeps_input():
    """leaves the list it is given unchanged"""
    xs = [2, 2, 1]
    dedupe_keep_order(xs)
    assert xs == [2, 2, 1]

Then work through the lesson’s self-check. It begins with the idea your answers show you know least, and each choice explains itself the moment you check it:

Interview questions

Fresher · What is the difference between append() and extend() on a Python list?

append(x) adds x as one new item at the end, whatever x is, so appending a list of three items makes the list one item longer, with the inner list nested inside it. extend(items) loops over its argument and adds each item on its own, so extending by a list of three items makes the list three items longer. Strings show the difference best: cart.extend("tea") adds the letters ‘t’, ‘e’ and ‘a’, because a string is a sequence of characters, while cart.append("tea") adds one string. Both change the list in place and return None, and += on a list works like extend(). To combine two lists without changing either, write a + b, which builds a new list.

Key takeaways

• A list is ordered and mutable and can hold any values; brackets, list(iterable), [x] * n and a + b make one, and the last two build new lists from old ones.
• Index 0 is the first item and -1 the last; a[start:stop:step] stops before stop, never raises IndexError and always returns a new list, while a single index past the end raises it.
• append() adds one item and extend() each item of an iterable, insert() and remove() move the items after them, and pop() returns what it removes, while index() and remove() raise ValueError for a missing value.
• Assigning to a slice replaces a whole range, and the sizes may differ unless the slice has a step other than 1.
• Methods that change a list return None, pop() aside, so x = x.sort() throws the list away; sorted(x) returns a new list.
• Changing the front of a list moves every other item, so a list makes a good stack and collections.deque a good queue.