Attribute lookup and the data model

A program writes 99 into an object’s own dictionary under the name value, and r.value still reads 21.5; it writes 'shadowed' there under the name of a method, and the method is gone. Both results come from attribute lookup, the search Python runs for every obj.name. The default __getattribute__, inherited from object, looks first for a data descriptordata descriptor: A descriptor that defines __set__ or __delete__, such as a property or a slot. Attribute lookup checks it before the instance's dictionary, so an instance attribute of the same name can never hide it. More in “Descriptors: how properties and methods work” of that name on the class, such as a property, then in the instance’s own dictionary, then for anything else of that name on the class, such as a method or a class attribute. Only when all three come up empty does it raise AttributeError, and only then does Python call __getattr__. These rules belong to Python’s data model, the part of the language reference that says which special method each piece of syntax calls.

Below you will step through that search on one object, watch assignment and del take a shorter path, build lazy settings and a logging proxy, and reproduce the recursion that catches most first attempts at __getattribute__.

A read looks in four places, always in the same order

lookup_order.py gives a class one name of each kind: a property, a method, a plain class attribute, and a name that only __getattr__ can answer. It reads all four, writes four keys straight into vars(r), past the property, and reads them again:

class Reading:
    unit = "celsius"                 # a plain class attribute

    def __init__(self, value):
        self._value = value

    @property
    def value(self):                 # a property: a data descriptor
        return self._value

    def label(self):                 # a function: a non-data descriptor
        return f"{self.value} {self.unit}"

    def __getattr__(self, name):     # runs only when every other step fails
        return f"<no {name}>"


r = Reading(21.5)
print("before:", r.value, "|", r.label(), "|", r.unit, "|", r.colour)

# Write straight into the instance's own dictionary, past any property:
vars(r)["value"] = 99
vars(r)["label"] = "shadowed"
vars(r)["unit"] = "kelvin"
vars(r)["colour"] = "red"
print("after: ", r.value, "|", r.label, "|", r.unit, "|", r.colour)
print("vars:  ", vars(r))

before: 21.5 | 21.5 celsius | celsius | <no colour>
after:  21.5 | shadowed | kelvin | red
vars:   {'_value': 21.5, 'value': 99, 'label': 'shadowed', 'unit': 'kelvin', 'colour': 'red'}

• Before the instance’s dictionary is filled, every name is found where you would expect: value through the property, label and unit in the class, and colour by __getattr__, because no other step finds it.
• Afterwards r.value still gives 21.5. A property is a data descriptor, an object on the class with a __set__ method, and data descriptors are checked before the instance’s dictionary.
• r.label and r.unit now give the instance’s own values. A function in the class is a non-data descriptornon-data descriptor: A descriptor that defines only __get__, such as a function or functools.cached_property. An entry of the same name in the instance's dictionary hides it, which is how an instance attribute can shadow a method. More in “Descriptors: how properties and methods work”: its __get__ method turns it into a bound methodbound method: What reading a method through an instance gives: the function together with that instance, so calling it later passes the instance in as self. c.step is a bound method; c.step(5) calls it. More in “Classes, objects and __init__”, but it has no __set__, so an instance attribute of the same name hides it. A plain class attribute is hidden in the same way.
• r.colour no longer reaches __getattr__. The name is found in the second step, and __getattr__ runs only after every step has failed.

The whole search fits in one column of questions, each answered on the class or on the instance:

Now follow the eight reads of the run through it. The lit stage is the one that answers each read: watch the property answer before the instance’s dictionary is consulted, and __getattr__ drop out once colour has a value of its own.

The first and third steps search the class and its bases in method resolution ordermethod resolution order (MRO): The order in which Python searches a class and its bases for an attribute, and the order super() follows. Class.__mro__ lists it, from the class itself to object. More in “Multiple inheritance, MRO and mixins”, the order that super() follows, and the Descriptor Guide in the Python documentation writes the whole search out in about twenty lines of pure Python. What a descriptor is, and how properties and methods use one, is the subject of the next lesson. For lookup, one question decides: does the object found on the class define __set__ or __delete__? If it does, it wins over the instance; if it does not, the instance wins.

Writes and deletes take a shorter path

Assignment does not search the way a read does. It asks one question, whether the class has a data descriptor of that name, and otherwise stores the value in the instance. Step through it on a Reading like the one above. own is vars(r), so every store and every delete shows there:

class Reading:
    unit = "celsius"

    def __init__(self, value):
        self._value = value

    @property
    def value(self):
        return self._value


r = Reading(21.5)
own = vars(r)
r.unit = "kelvin"
mine = r.unit
theirs = Reading.unit
del r.unit
back = r.unit
r.value = 30

Error AttributeError: property 'value' of 'Reading' object has no setter

r.unit = "kelvin" landed in own, and Reading.unit kept 'celsius'. del r.unit removed only r’s entry, so the class value showed through again. The last line stopped the program: a property takes over assignment for its name, and this one has no setter. The longer example below repeats those steps and adds a class whose __setattr__ reports every store:

class Reading:
    unit = "celsius"

    def __init__(self, value):
        self._value = value

    @property
    def value(self):
        return self._value


r = Reading(21.5)
r.unit = "kelvin"                  # stored in r's own dictionary
print(r.unit, Reading.unit)        # the class keeps its value
del r.unit                         # removes r's own entry, nothing else
print(r.unit)                      # the class attribute shows through again

try:
    r.value = 30                   # the property has no setter
except AttributeError as e:
    print("r.value = 30:", e)
try:
    del r.unit                     # r has no unit of its own any more
except AttributeError as e:
    print("del r.unit:", e)


class Audited:
    """Every assignment, those in __init__ included, passes through __setattr__."""

    def __init__(self, owner):
        self.owner = owner
        self.balance = 0

    def __setattr__(self, name, value):
        print(f"  set {name} = {value!r}")
        super().__setattr__(name, value)   # object.__setattr__ does the real store


a = Audited("Asha")
a.balance += 500
print(vars(a))

kelvin celsius
celsius
r.value = 30: property 'value' of 'Reading' object has no setter
del r.unit: 'Reading' object has no attribute 'unit'
  set owner = 'Asha'
  set balance = 0
  set balance = 500
{'owner': 'Asha', 'balance': 500}

• The property’s refusal says why it refused: property 'value' of 'Reading' object has no setter.
• del through an instance never deletes from the class: the second del r.unit fails although Reading.unit exists.
• A class can replace the store with __setattr__, which runs for every assignment to its instances, those in __init__ included. Augmented assignment calls it too: a.balance += 500 reads the old value with a normal lookup and then calls __setattr__ with the sum.

__delattr__ works the same way for del: inside it, the real delete is super().__delattr__(name).

The fallback for missing names runs only after the lookup fails

__getattr__(self, name) is called only after the normal lookup has failed. It costs nothing for attributes that exist and cannot hide them, which makes it the safe hook for names an object does not have yet, such as settings loaded on first use. Step through a small version and watch own, the instance’s dictionary, and the count of loads:

class Settings:
    def __init__(self, loaders):
        self._load = loaders
        self.loads = 0

    def __getattr__(self, name):
        if name not in self._load:
            raise AttributeError(name)
        self.loads += 1
        value = self._load[name]()
        setattr(self, name, value)
        return value


s = Settings({"tax": lambda: 18})
own = vars(s)
first = s.tax
second = s.tax
loads = s.loads
smtp = hasattr(s, "smtp")

The first read of s.tax missed, so __getattr__ loaded the value and stored it with setattr(); the second read found it in own, and loads stayed 1. The fuller version loads two sections and shows what goes wrong when the fallback raises the wrong exception:

LOADERS = {
    "database": lambda: {"host": "db.internal", "port": 5432},
    "payments": lambda: {"currency": "INR", "retries": 3},
}


class Settings:
    """Loads each section the first time it is read, then keeps it."""

    def __init__(self, loaders):
        self._loaders = loaders
        self.loads = 0

    def __getattr__(self, name):      # called only when the normal lookup fails
        try:
            loader = self._loaders[name]
        except KeyError:
            raise AttributeError(f"no settings section {name!r}", name=name, obj=self) from None
        self.loads += 1
        value = loader()
        setattr(self, name, value)    # cached: the next read finds it in the instance
        return value


s = Settings(LOADERS)
print(s.payments["currency"], s.payments["retries"], s.database["port"])
print("loads:", s.loads, "| own attributes:", sorted(vars(s)))
print(hasattr(s, "smtp"), getattr(s, "smtp", "not configured"))


class LeakySettings(Settings):
    """The same, except that an unknown name lets KeyError escape."""

    def __getattr__(self, name):
        value = self._loaders[name]()
        setattr(self, name, value)
        return value


leaky = LeakySettings(LOADERS)
try:
    hasattr(leaky, "smtp")
except KeyError as e:
    print("hasattr() raised KeyError:", e)
try:
    getattr(leaky, "smtp", "not configured")
except KeyError as e:
    print("getattr() with a default raised KeyError:", e)

INR 3 5432
loads: 2 | own attributes: ['_loaders', 'database', 'loads', 'payments']
False not configured
hasattr() raised KeyError: 'smtp'
getattr() with a default raised KeyError: 'smtp'

• Three reads cost two loads: the second read of s.payments found the stored value in the instance’s dictionary and never reached __getattr__.
• For a name it cannot provide, __getattr__ must raise AttributeError, because hasattr() and getattr() with a default treat exactly that exception as “no such attribute”. Settings therefore answers False and “not configured”. Passing name= and obj= to AttributeError, which Python accepts since version 3.10, gives tracebacks what they need to suggest a similar name.
• LeakySettings lets the dictionary’s KeyError escape. Now hasattr() and getattr() with a default raise instead of answering, and any code that asks “is smtp configured?” crashes.

A failed lookup also includes a property whose getter raises AttributeError: Python then calls __getattr__ with the property’s name. Keep that in mind when a class has both and an error names an attribute that plainly exists.

An override of every read sees its own reads too

__getattribute__ replaces the lookup itself. It runs for every attribute read on the instance, found or not, methods included, which makes it the tool for counting or controlling every access, and the easiest special method to get wrong. The first class below counts reads the obvious way; the second counts them correctly:

class Broken:
    """Counts reads, but reads self.reads to do it."""

    def __init__(self):
        self.reads = 0

    def __getattribute__(self, name):
        self.reads += 1                  # this read calls __getattribute__ again
        return object.__getattribute__(self, name)


try:
    Broken().reads
except RecursionError as e:
    print("Broken:", type(e).__name__)


class Counted:
    """Counts every attribute read that goes through __getattribute__."""

    def __init__(self, price, qty):
        self.price = price
        self.qty = qty
        self.reads = {}

    def __getattribute__(self, name):
        reads = object.__getattribute__(self, "reads")   # the default lookup: no recursion
        reads[name] = reads.get(name, 0) + 1
        return object.__getattribute__(self, name)

    def total(self):
        return self.price * self.qty

    def __repr__(self):
        return f"Counted({self.price}, {self.qty})"


c = Counted(250, 4)
print(c.total(), c.total())
print(c)                                   # repr() finds __repr__ on the class itself
print(object.__getattribute__(c, "reads"))

Broken: RecursionError
1000 1000
Counted(250, 4)
{'total': 2, 'price': 3, 'qty': 3}

Follow the calls behind both results. The arrow that loops back is the override calling itself, and the third column is the default lookup, which a correct override uses for every real read:

• Broken.__getattribute__ reads self.reads, and that read is itself an attribute read of the instance, so it calls __getattribute__ again, which reads self.reads again, until Python gives up with RecursionError.
• Counted reaches attributes through object.__getattribute__(self, name), the default lookup, which never calls the override. Every read inside its methods counts: two calls of total() read total twice and price and qty twice each, and the repr read each of them once more.
• __repr__ itself is missing from the counts. print(c) finds it on the class, as every implicit use of a special method does, so the override never sees that lookup. The last line reads reads through object.__getattribute__ as well, so printing the counts does not change them.

When a class defines both hooks, __getattribute__ runs first and __getattr__ only after it raises AttributeError. Reach for __getattribute__ rarely: it adds a Python function call to every attribute read on its instances, self.__dict__ and the class’s own helpers included, while __getattr__ covers defaults, lazy values and delegation without touching the names that exist. Side by side, the two hooks differ in when they run, what they cost and how they go wrong:

A proxy forwards names, but len() looks on the class

A proxy stands in for another object and forwards what it does not handle itself. Wrappers that add logging, access checks or remote calls are proxies, and __getattr__ makes the forwarding a few lines long:

class LoggingProxy:
    """Wraps any object, forwards attribute access to it and logs each name used."""

    def __init__(self, target):
        # object.__setattr__ stores on the proxy itself, past our own __setattr__.
        object.__setattr__(self, "_target", target)
        object.__setattr__(self, "log", [])

    def __getattr__(self, name):           # names the proxy itself does not have
        self.log.append(f"get {name}")
        return getattr(self._target, name)

    def __setattr__(self, name, value):    # every assignment goes to the target
        self.log.append(f"set {name}")
        setattr(self._target, name, value)

    def __dir__(self):                      # what dir() and tab completion show
        return dir(self._target)


queue = LoggingProxy([30, 10, 20])
queue.append(5)
queue.sort()
print(queue._target, queue.count(10))
print(queue.log)

try:
    len(queue)
except TypeError as e:
    print("len(queue): TypeError:", e)
print("queue.__len__():", queue.__len__())
print("'sort' in dir(queue):", "sort" in dir(queue))


class ListProxy(LoggingProxy):
    """Special methods must be defined on the class, because Python looks them up there."""

    def __len__(self):
        return len(self._target)

    def __iter__(self):
        return iter(self._target)

    def __getitem__(self, index):
        return self._target[index]


q = ListProxy([3, 1, 2])
print(len(q), list(q), q[-1], 2 in q, q.log)

[5, 10, 20, 30] 1
['get append', 'get sort', 'get count']
len(queue): TypeError: object of type 'LoggingProxy' has no len()
queue.__len__(): 4
'sort' in dir(queue): True
3 [3, 1, 2] 2 True []

The run took two different routes to the list. Step through both: an attribute read that reaches the list through __getattr__, and len(), which goes straight to the class:

• The proxy keeps _target and log in its own dictionary, stored with object.__setattr__ because its own __setattr__ sends assignments to the target. Reads of those two names succeed in the normal lookup and are never forwarded.
• append, sort and count are not attributes of the proxy, so each read reached __getattr__, was logged and came back as the list’s own bound method.
• len(queue) failed although the list has a length. Implicit uses of special methods, by len(), iter(), in, indexing, operators, repr() and with, look the method up on the type, skipping the instance, __getattribute__ and __getattr__. Written out as an attribute, queue.__len__() is an ordinary read, so it was forwarded and worked.
• ListProxy defines the special methods it needs on the class, and len(), list(), indexing and in work. in fell back to __iter__, because the class has no __contains__, and the log stayed empty: none of these went through __getattr__.
• __dir__ makes dir(queue) list the target’s names, so tab completion in a shell or an editor shows what the proxy can really do.

The rule is deliberate. Methods such as __hash__ and __repr__ exist on every object, classes included, so looking them up through the instance would find the wrong one whenever the object is itself a class; the language reference’s section on special method lookup shows the effect. For a proxy it means writing out, once, the special methods it promises to support.

A module can keep an old name working, with a warning

A module can define __getattr__ and __dir__ as plain functions. Python calls the module’s __getattr__(name) when a name is not found in it, which is the usual way to keep an old name working while warning about it:

import warnings

import geometry

with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    print(round(geometry.area_of_circle(2), 2))   # the old name still works ...
print([str(w.message) for w in caught])         # ... with a warning

print(round(geometry.circle_area(2), 2), dir(geometry))
try:
    geometry.volume
except AttributeError as e:
    print("AttributeError:", e)

12.57
['area_of_circle() is deprecated: use circle_area()']
12.57 ['circle_area']
AttributeError: module 'geometry' has no attribute 'volume'

"""Shapes. Version 2 renamed area_of_circle() to circle_area()."""
import math
import warnings

_RENAMED = {"area_of_circle": "circle_area"}


def circle_area(radius):
    return math.pi * radius**2


def __getattr__(name):                  # called only for names this module does not define
    if name in _RENAMED:
        new = _RENAMED[name]
        warnings.warn(f"{name}() is deprecated: use {new}()", DeprecationWarning, stacklevel=2)
        return globals()[new]
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")


def __dir__():                          # what dir(geometry) lists
    return ["circle_area"]

geometry.py defines only circle_area(). Reading geometry.area_of_circle fails the normal lookup, so the module’s __getattr__ issues a DeprecationWarning and returns the new function, and stacklevel=2 makes the warning point at the caller’s line instead of the module’s. An unknown name still raises AttributeError, and dir(geometry) lists only the current name. The hook also serves from geometry import area_of_circle, but not code inside geometry.py itself, which reads its own names directly.

Practise: a read-only proxy, then check yourself

The exercise asks for a proxy that forwards every read and refuses every change, with the special methods it needs written on its class. Its sample tests run in your browser.

class ReadOnlyProxy:
    """Wraps target: reads are forwarded to it; assignments and deletions raise AttributeError."""

    def __init__(self, target):
        self._target = target

from readonly import ReadOnlyProxy


class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def moved(self, dx):
        return Point(self.x + dx, self.y)


def refuses(action):
    try:
        action()
    except AttributeError:
        return True
    return False


def test_reads_are_forwarded():
    """attributes and methods of the target are read through the proxy"""
    p = ReadOnlyProxy(Point(2, 3))
    assert (p.x, p.y) == (2, 3)
    assert p.moved(5).x == 7
    assert ReadOnlyProxy("asha").upper() == "ASHA"


def test_writes_and_deletes_are_refused():
    """setting, adding or deleting an attribute raises AttributeError and changes nothing"""
    target = Point(2, 3)
    p = ReadOnlyProxy(target)
    assert refuses(lambda: setattr(p, "x", 10)), "p.x = 10 should raise AttributeError"
    assert refuses(lambda: setattr(p, "z", 1)), "p.z = 1 should raise AttributeError"
    assert refuses(lambda: delattr(p, "y")), "del p.y should raise AttributeError"
    assert refuses(lambda: delattr(p, "_target")), "the proxy's own attributes cannot be deleted either"
    assert (target.x, target.y) == (2, 3)
    assert not hasattr(target, "z")
    assert p.x == 2, "the proxy still works after the refused changes"


def test_missing_names():
    """a name the target lacks raises AttributeError, so hasattr() and getattr() with a default work"""
    p = ReadOnlyProxy(Point(2, 3))
    assert not hasattr(p, "colour")
    assert getattr(p, "colour", "none") == "none"


def test_dir_lists_the_target():
    """dir() of the proxy lists the target's attributes"""
    names = dir(ReadOnlyProxy(Point(2, 3)))
    assert "x" in names and "moved" in names


def test_special_methods():
    """len(), iteration, in, indexing and repr() work through the proxy"""
    p = ReadOnlyProxy([10, 20, 30])
    assert len(p) == 3
    assert list(p) == [10, 20, 30]
    assert 20 in p and 40 not in p
    assert p[-1] == 30 and p[0:2] == [10, 20]
    assert repr(p) == "ReadOnlyProxy([10, 20, 30])"

Then test all six ideas of the lesson. Practice starts where your answers have been weakest and explains every answer the moment you give it:

Interview questions

Fresher to 2–5 years · What is the difference between __getattr__ and __getattribute__?

__getattribute__ runs for every attribute read on an instance, whether the attribute exists or not; the default one, object.__getattribute__, performs the normal search through data descriptors, the instance’s dictionary and the class. __getattr__ is a fallback: Python calls it only after that search has raised AttributeError, so it never sees names that exist. Use __getattr__ for defaults, lazy loading and forwarding to another object, where only the missing names matter. Override __getattribute__ only to observe or control every read, and reach the real attributes through object.__getattribute__(self, name) inside it, or each read calls the method again until RecursionError. Neither hook sees the implicit special-method lookups of len(), + or repr(), which go straight to the class.

Key takeaways

• obj.name checks for a data descriptor on the class, then vars(obj), then the rest of the class, and calls __getattr__ only when all three fail.
• A property beats an instance attribute of the same name; a method or a class attribute does not.
• Assignment and del change the instance’s dictionary unless a data descriptor on the class takes over; through an instance they never change the class.
• __getattr__ must raise AttributeError for unknown names, or hasattr() and getattr() with a default break.
• Inside __getattribute__ and __setattr__, reach the real attributes through object.__getattribute__ and object.__setattr__.
• len(), in, indexing and other syntax look special methods up on the class, so a proxy defines the ones it supports.