Skip to main content

Python Higher Order Functions

In the world of functional programming, higher order functions are a powerful concept that can make your code more elegant, reusable, and easier to understand. But what exactly are they, and how can we use them in Python?

What Are Higher Order Functions?​

A higher order function is a function that does at least one of the following:

  • Takes one or more functions as arguments
  • Returns a function as its result

This might sound complex, but Python makes it quite straightforward. Higher order functions allow you to abstract over actions, not just values, enabling more flexible code.

Why Use Higher Order Functions?​

Before diving into examples, let's understand why higher order functions are valuable:

  1. Code Reusability: They promote the DRY (Don't Repeat Yourself) principle
  2. Abstraction: They let you focus on what to do rather than how to do it
  3. Expressiveness: They can make your code more concise and readable
  4. Composability: They allow you to build complex operations from simpler ones

Common Higher Order Functions in Python​

Python comes with several built-in higher order functions. Let's explore the most common ones.

The map() Function​

The map() function applies a given function to each item in an iterable and returns a map object (which is an iterator).

Syntax:​

python
map(function, iterable, ...)

Example 1: Converting temperatures from Celsius to Fahrenheit​

python
# Function to convert Celsius to Fahrenheit
def celsius_to_fahrenheit(celsius):
    return (celsius * 9/5) + 32

# List of temperatures in Celsius
temperatures_in_celsius = [0, 10, 20, 30, 40]

# Using map to convert all temperatures
temperatures_in_fahrenheit = list(map(celsius_to_fahrenheit, temperatures_in_celsius))

print("Celsius temperatures:", temperatures_in_celsius)
print("Fahrenheit temperatures:", temperatures_in_fahrenheit)

Output:

Celsius temperatures: [0, 10, 20, 30, 40]
Fahrenheit temperatures: [32.0, 50.0, 68.0, 86.0, 104.0]

Example 2: Using map with lambda functions​

Lambda functions (anonymous functions) are often used with higher order functions:

python
# Square each number in a list
numbers = [1, 2, 3, 4, 5]
squared = list(map(lambda x: x**2, numbers))

print("Original numbers:", numbers)
print("Squared numbers:", squared)

Output:

Original numbers: [1, 2, 3, 4, 5]
Squared numbers: [1, 4, 9, 16, 25]

The filter() Function​

The filter() function constructs an iterator from elements of an iterable for which a function returns true.

Syntax:​

python
filter(function, iterable)

Example: Filtering even numbers​

python
# Function to check if a number is even
def is_even(number):
    return number % 2 == 0

# List of numbers
numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# Using filter to get even numbers
even_numbers = list(filter(is_even, numbers))

print("All numbers:", numbers)
print("Even numbers:", even_numbers)

Output:

All numbers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Even numbers: [2, 4, 6, 8, 10]

With a lambda function:

python
# Filter out negative numbers
numbers = [-3, -2, -1, 0, 1, 2, 3]
positive_numbers = list(filter(lambda x: x > 0, numbers))

print("All numbers:", numbers)
print("Positive numbers:", positive_numbers)

Output:

All numbers: [-3, -2, -1, 0, 1, 2, 3]
Positive numbers: [1, 2, 3]

The reduce() Function​

The reduce() function applies a function of two arguments cumulatively to the items of an iterable, reducing them to a single value.

Note: In Python 3, reduce() is not a built-in function anymore. It has been moved to the functools module.

Syntax:​

python
functools.reduce(function, iterable[, initializer])

Example: Computing the product of a list of numbers​

python
from functools import reduce

# Function to multiply two numbers
def multiply(x, y):
    return x * y

# List of numbers
numbers = [1, 2, 3, 4, 5]

# Using reduce to compute the product
product = reduce(multiply, numbers)

print("Numbers:", numbers)
print("Product:", product)

Output:

Numbers: [1, 2, 3, 4, 5]
Product: 120

With a lambda function:

python
from functools import reduce

# Calculate the sum of all numbers
numbers = [1, 2, 3, 4, 5]
sum_of_numbers = reduce(lambda x, y: x + y, numbers)

print("Numbers:", numbers)
print("Sum:", sum_of_numbers)

Output:

Numbers: [1, 2, 3, 4, 5]
Sum: 15

Creating Your Own Higher Order Functions​

Now that we understand built-in higher order functions, let's create our own.

Example: A function that returns another function​

python
def create_multiplier(factor):
    """Returns a function that multiplies its argument by the given factor"""
    def multiplier(x):
        return x * factor
    return multiplier

# Create specific multiplier functions
double = create_multiplier(2)
triple = create_multiplier(3)

# Use these functions
print(double(5))  # 10
print(triple(5))  # 15

Output:

10
15

This is a simple example of a function factory - a higher order function that creates and returns custom functions.

Example: A function that takes a function as an argument​

python
def apply_operation(func, x, y):
    """Apply the given function to x and y"""
    return func(x, y)

# Define some operations
def add(a, b):
    return a + b

def subtract(a, b):
    return a - b

# Use our higher order function
print(apply_operation(add, 5, 3))      # 8
print(apply_operation(subtract, 5, 3)) # 2

Output:

8
2

Real-World Applications​

Higher order functions are not just theoretical concepts; they have practical applications:

Example: Data Processing Pipeline​

python
def process_data(data, transformations):
    """
    Process data through a series of transformation functions
    
    Args:
        data: The initial data to process
        transformations: A list of functions to apply in sequence
    
    Returns:
        The processed data
    """
    result = data
    for transform in transformations:
        result = transform(result)
    return result

# Define some transformation functions
def clean_data(data):
    return [item for item in data if item is not None]

def convert_to_integers(data):
    return [int(item) if isinstance(item, str) else item for item in data]

def square_values(data):
    return [item ** 2 for item in data]

# Sample data with some issues
raw_data = [None, '1', 2, None, '3', 4, '5']

# Create a processing pipeline
transformations = [clean_data, convert_to_integers, square_values]

# Process the data
processed_data = process_data(raw_data, transformations)

print("Raw data:", raw_data)
print("Processed data:", processed_data)

Output:

Raw data: [None, '1', 2, None, '3', 4, '5']
Processed data: [1, 4, 9, 16, 25]

Example: Event Handler System​

python
class EventSystem:
    def __init__(self):
        self.handlers = {}
    
    def register(self, event_name, handler):
        if event_name not in self.handlers:
            self.handlers[event_name] = []
        self.handlers[event_name].append(handler)
    
    def emit(self, event_name, *args, **kwargs):
        if event_name in self.handlers:
            for handler in self.handlers[event_name]:
                handler(*args, **kwargs)

# Create an event system
events = EventSystem()

# Define some handlers
def log_user_login(username):
    print(f"USER LOGIN: {username} logged in")

def send_welcome_email(username):
    print(f"Sending welcome email to {username}")

# Register handlers for the 'user_login' event
events.register('user_login', log_user_login)
events.register('user_login', send_welcome_email)

# Emit the event
events.emit('user_login', 'john_doe')

Output:

USER LOGIN: john_doe logged in
Sending welcome email to john_doe

Decorators: Higher Order Functions in Disguise​

Python decorators are a special case of higher order functions. They take a function and return a new function with enhanced behavior.

python
def timer_decorator(func):
    import time
    
    def wrapper(*args, **kwargs):
        start_time = time.time()
        result = func(*args, **kwargs)
        end_time = time.time()
        print(f"Function {func.__name__} took {end_time - start_time:.4f} seconds to run")
        return result
    
    return wrapper

# Apply the decorator
@timer_decorator
def slow_function():
    import time
    time.sleep(1)
    print("Function executed")

# Run the decorated function
slow_function()

Output:

Function executed
Function slow_function took 1.0012 seconds to run

Summary​

Higher order functions are a powerful feature in Python that allows us to:

  • Pass functions as arguments to other functions
  • Return functions from other functions
  • Create more abstract, reusable, and composable code

We've explored:

  1. Built-in higher order functions like map(), filter(), and reduce()
  2. How to create our own higher order functions
  3. Real-world applications like data processing pipelines and event systems
  4. Decorators as a special case of higher order functions

By understanding and using higher order functions, you can write more elegant, functional, and concise Python code.

Exercises​

  1. Create a higher order function that takes a list of strings and a transformation function, and applies the transformation to each string.
  2. Implement a simple memoization decorator that caches the results of expensive function calls.
  3. Write a function that takes multiple functions and returns a new function that is the composition of those functions.
  4. Create a data validation pipeline using higher order functions.
  5. Implement your own version of the map() function that works with multiple iterables.

Additional Resources​

Happy coding with higher order functions!


If you spot any mistakes on this website, please let me know at feedback@compilenrun.com. I’d greatly appreciate your feedback! :)