Python Descriptors

Introduction

Python descriptors are a powerful feature that allows you to customize how attributes on objects are accessed, set, or deleted. They provide a way to create reusable properties that can be applied across multiple classes. If you've used properties in Python before, you're already familiar with a specific implementation of descriptors.

Descriptors are a fundamental part of Python's object-oriented programming model and are used extensively in the language itself. Understanding descriptors will deepen your knowledge of Python and enable you to write more elegant and maintainable code.

What Are Descriptors?

A descriptor is any object that implements at least one of the following methods from the descriptor protocol:

• __get__(self, obj, objtype=None) - Called when the attribute is accessed
• __set__(self, obj, value) - Called when the attribute is set
• __delete__(self, obj) - Called when the attribute is deleted

Descriptors that implement both __get__ and __set__ are called "data descriptors." Those that implement only __get__ are called "non-data descriptors."

Basic Descriptor Example

Let's start with a simple descriptor that logs when an attribute is accessed or modified:

python

class LoggingDescriptor:
    def __init__(self, name):
        self.name = name
        
    def __get__(self, instance, owner):
        print(f"Accessing {self.name}")
        if instance is None:
            return self
        return instance.__dict__.get(self.name)
        
    def __set__(self, instance, value):
        print(f"Setting {self.name} to {value}")
        instance.__dict__[self.name] = value
        
class Person:
    name = LoggingDescriptor('name')
    age = LoggingDescriptor('age')
    
    def __init__(self, name, age):
        self.name = name
        self.age = age

# Let's use our descriptor
person = Person("Alice", 30)
# Output: Setting name to Alice
# Output: Setting age to 30

print(person.name)
# Output: Accessing name
# Output: Alice

person.age = 31
# Output: Setting age to 31

In this example, the LoggingDescriptor logs when attributes are accessed or set. When we create a new Person instance, set, or access its attributes, the descriptor methods are automatically called.

How Descriptors Work

When you access an attribute on an object, Python follows a specific lookup sequence:

1. Check if it's a data descriptor
2. Check if it exists in the instance's __dict__
3. Check if it's a non-data descriptor
4. Look in the class's __dict__
5. Look in parent classes
6. Call __getattr__ if defined

This explains why descriptors, which are defined at the class level, can intercept attribute access at the instance level.

Property vs Descriptors

Python's built-in property decorator is actually a descriptor implementation. Here's how they compare:

python

# Using property
class PersonWithProperty:
    def __init__(self, name):
        self._name = name
        
    @property
    def name(self):
        print("Accessing name")
        return self._name
    
    @name.setter
    def name(self, value):
        print(f"Setting name to {value}")
        self._name = value

# Using descriptors
class NameDescriptor:
    def __get__(self, instance, owner):
        print("Accessing name")
        if instance is None:
            return self
        return instance._name
    
    def __set__(self, instance, value):
        print(f"Setting name to {value}")
        instance._name = value

class PersonWithDescriptor:
    name = NameDescriptor()
    
    def __init__(self, name):
        self.name = name

While property is convenient for simple cases, descriptors are more powerful because they:

• Are reusable across multiple classes
• Can maintain their own state
• Provide greater customization options

Practical Applications of Descriptors

Type Validation

One common use of descriptors is for field validation:

python

class Typed:
    def __init__(self, name, expected_type):
        self.name = name
        self.expected_type = expected_type
        
    def __get__(self, instance, owner):
        if instance is None:
            return self
        return instance.__dict__.get(self.name)
        
    def __set__(self, instance, value):
        if not isinstance(value, self.expected_type):
            raise TypeError(f"Expected {self.expected_type}")
        instance.__dict__[self.name] = value

class Integer(Typed):
    def __init__(self, name):
        super().__init__(name, int)

class String(Typed):
    def __init__(self, name):
        super().__init__(name, str)

class Product:
    name = String('name')
    price = Integer('price')
    
    def __init__(self, name, price):
        self.name = name
        self.price = price

# Works fine
product = Product("Laptop", 999)
print(product.name)  # Output: Laptop
print(product.price)  # Output: 999

try:
    # Will raise a TypeError
    product.price = "expensive"
except TypeError as e:
    print(e)  # Output: Expected <class 'int'>

Lazy Properties

Descriptors can be used to create properties that calculate their value only when needed and cache the result:

python

class LazyProperty:
    def __init__(self, function):
        self.function = function
        self.name = function.__name__
        
    def __get__(self, instance, owner):
        if instance is None:
            return self
            
        value = self.function(instance)
        # Cache the value in the instance's __dict__
        instance.__dict__[self.name] = value
        return value

class DataProcessor:
    def __init__(self, data):
        self.data = data
        
    @LazyProperty
    def processed_data(self):
        print("Processing data...")
        # Simulate expensive operation
        import time
        time.sleep(1)
        return [x * 2 for x in self.data]

processor = DataProcessor([1, 2, 3, 4, 5])
print("Processor created")  # Output: Processor created

print("First access:")
print(processor.processed_data)  # Output: Processing data... [2, 4, 6, 8, 10]

print("Second access:")
print(processor.processed_data)  # Output: [2, 4, 6, 8, 10] (no processing message)

In this example, the expensive processing happens only on the first access, and subsequent accesses retrieve the cached value.

Unit Conversions

Descriptors can handle unit conversions automatically:

python

class Celsius:
    def __get__(self, instance, owner):
        if instance is None:
            return self
        return instance._temperature
        
    def __set__(self, instance, value):
        instance._temperature = value
        
class Fahrenheit:
    def __get__(self, instance, owner):
        if instance is None:
            return self
        return instance._temperature * 9/5 + 32
        
    def __set__(self, instance, value):
        instance._temperature = (value - 32) * 5/9

class Temperature:
    celsius = Celsius()
    fahrenheit = Fahrenheit()
    
    def __init__(self, temp_celsius):
        self._temperature = temp_celsius

temp = Temperature(25)
print(f"{temp.celsius}°C")  # Output: 25.0°C
print(f"{temp.fahrenheit}°F")  # Output: 77.0°F

temp.fahrenheit = 68
print(f"{temp.celsius}°C")  # Output: 20.0°C

Advanced Descriptor Concepts

Descriptor Priority

Data descriptors (with both __get__ and __set__) take precedence over instance variables, while non-data descriptors (with only __get__) don't. This explains why properties can override instance attributes:

python

class NonDataDescriptor:
    def __get__(self, instance, owner):
        return "Non-data descriptor"

class DataDescriptor:
    def __get__(self, instance, owner):
        return "Data descriptor"
    
    def __set__(self, instance, value):
        pass

class Test:
    non_data = NonDataDescriptor()
    data = DataDescriptor()
    
    def __init__(self):
        self.non_data = "Instance attribute"  # Will override descriptor
        self.data = "Instance attribute"  # Won't override descriptor

test = Test()
print(test.non_data)  # Output: Instance attribute
print(test.data)  # Output: Data descriptor

__set_name__ Method

Since Python 3.6, descriptors can implement a __set_name__ method that gets called at class creation time, simplifying descriptor initialization:

python

class FieldValidator:
    def __init__(self, expected_type):
        self.expected_type = expected_type
        self.name = None  # Will be set by __set_name__
        
    def __set_name__(self, owner, name):
        self.name = name
        
    def __get__(self, instance, owner):
        if instance is None:
            return self
        return instance.__dict__.get(self.name)
        
    def __set__(self, instance, value):
        if not isinstance(value, self.expected_type):
            raise TypeError(f"{self.name} must be of type {self.expected_type.__name__}")
        instance.__dict__[self.name] = value

class Person:
    # No need to pass field names manually
    name = FieldValidator(str)
    age = FieldValidator(int)
    
    def __init__(self, name, age):
        self.name = name
        self.age = age

person = Person("Alice", 30)
try:
    person.age = "thirty"  # Raises TypeError
except TypeError as e:
    print(e)  # Output: age must be of type int

Summary

Python descriptors provide a powerful way to customize attribute access in your classes. They are the foundation of many Python features, including properties, class methods, and static methods. By understanding descriptors, you can:

• Create reusable attribute access patterns
• Implement data validation
• Create lazy-evaluated properties
• Add logging or additional behavior to attribute access
• Build more maintainable and robust code

While descriptors may seem advanced, they allow you to write cleaner, more maintainable code by encapsulating common attribute-related behaviors.

Additional Resources

• Python Documentation on Descriptors
• Python Data Model - Descriptor Guide
• Raymond Hettinger's talk, "Descriptor Protocol" from PyCon

Exercises

1. Create a descriptor that ensures a number stays within a specified range.
2. Implement a descriptor for a read-only attribute that can only be set once.
3. Build a descriptor that automatically converts strings to a specific format (like lowercase or uppercase).
4. Create a descriptor that tracks the history of values assigned to an attribute.
5. Implement a descriptor-based cache decorator similar to functools.lru_cache.

By working through these exercises, you'll gain a deeper understanding of descriptors and their applications in real-world Python programming.