Python Debugging Techniques

Debugging is an essential skill for any programmer. As you write more complex Python programs, you'll inevitably encounter bugs and errors that need fixing. This guide will walk you through various debugging techniques in Python, from simple print statements to sophisticated debugging tools.

Introduction to Debugging

Debugging is the process of finding and resolving defects or problems within a program that prevent correct operation. When your Python code doesn't work as expected, debugging helps you:

Identify where the problem is occurring
Understand why it's happening
Fix the issue effectively

Even experienced programmers spend a significant amount of time debugging. Learning effective debugging techniques will make you a more efficient and confident Python developer.

Basic Debugging Techniques

1. Using Print Statements

The simplest debugging technique is to add print() statements to your code to inspect variable values and program flow.

python
def calculate_average(numbers):
    print(f"Input received: {numbers}")  # Debug print
    
    total = sum(numbers)
    print(f"Total: {total}")  # Debug print
    
    average = total / len(numbers)
    print(f"Average: {average}")  # Debug print
    
    return average

# Example usage
result = calculate_average([10, 20, 30, 40])
print(f"Result: {result}")

Output:

Input received: [10, 20, 30, 40]
Total: 100
Average: 25.0
Result: 25.0

While this approach is simple, it can clutter your code. It's also inefficient for complex debugging scenarios.

2. Using Assertions

Assertions help validate assumptions in your code. If an assertion fails, it raises an AssertionError.

python
def divide(a, b):
    # Verify b is not zero before division
    assert b != 0, "Division by zero is not allowed"
    return a / b

# This works fine
print(divide(10, 2))

# This will raise an AssertionError
try:
    print(divide(10, 0))
except AssertionError as e:
    print(f"Error caught: {e}")

Output:

5.0
Error caught: Division by zero is not allowed

Assertions are useful during development but should not be used for handling runtime errors in production code.

Intermediate Debugging Techniques

1. Using the `logging` Module

The logging module provides a more sophisticated way to output debug information than using print().

python
import logging

# Configure logging
logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(levelname)s - %(message)s'
)

def process_data(data):
    logging.debug(f"Processing data: {data}")
    
    if not data:
        logging.warning("Empty data received")
        return None
    
    try:
        result = [x * 2 for x in data]
        logging.info(f"Data processing complete")
        return result
    except Exception as e:
        logging.error(f"Error processing data: {e}")
        raise

# Example usage
process_data([1, 2, 3])
process_data([])
try:
    process_data("not a list")
except Exception as e:
    pass  # We already logged the error

Output:

2023-10-26 14:23:45,123 - DEBUG - Processing data: [1, 2, 3]
2023-10-26 14:23:45,125 - INFO - Data processing complete
2023-10-26 14:23:45,126 - DEBUG - Processing data: []
2023-10-26 14:23:45,127 - WARNING - Empty data received
2023-10-26 14:23:45,128 - DEBUG - Processing data: not a list
2023-10-26 14:23:45,129 - ERROR - Error processing data: 'str' object is not iterable

Advantages of using logging over print():

Different severity levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)
Includes timestamps and other contextual information
Can be configured to output to files, network, email, etc.
Can be easily enabled/disabled without removing code

2. Using the Python Debugger (pdb)

Python's built-in debugger pdb allows you to:

Execute code step by step
Inspect variables at each step
Set breakpoints
Evaluate expressions in the current context

python
def complex_function(x, y):
    result = x + 2 * y
    intermediate = result * 3
    final = intermediate - x
    return final

# To debug this function, add these lines
import pdb

x = 5
y = 10
pdb.set_trace()  # The debugger will start here
result = complex_function(x, y)
print(f"Final result: {result}")

When this code runs, the debugger will start at the set_trace() line. Here are some common pdb commands:

n (next): Execute the current line and move to the next one
s (step): Step into a function call
c (continue): Continue execution until the next breakpoint
p variable_name: Print the value of a variable
q (quit): Quit the debugger
h (help): Get help on debugger commands

In Python 3.7+, you can use the simpler breakpoint() function instead of import pdb; pdb.set_trace().

Advanced Debugging Techniques

1. Using Interactive Debuggers in IDEs

Most Python IDEs (like PyCharm, VS Code, Spyder) include powerful graphical debuggers. These offer:

Breakpoints with conditions
Variable inspection and modification
Call stack visualization
Step into/over/out controls
Expression evaluation

For example, in VS Code, you can set a breakpoint by clicking in the margin next to the line number, then start debugging with F5.

2. Using `try`-`except` with Traceback

You can capture and analyze the complete traceback of exceptions:

python
import traceback

def function_c():
    return 1 / 0  # Will raise ZeroDivisionError

def function_b():
    return function_c()

def function_a():
    try:
        return function_b()
    except Exception as e:
        print(f"Error: {e}")
        print("\nTraceback:")
        traceback.print_exc()
        return None

# Try to run the function
result = function_a()
print(f"Result: {result}")

Output:

Error: division by zero

Traceback:
Traceback (most recent call last):
  File "example.py", line 10, in function_a
    return function_b()
  File "example.py", line 7, in function_b
    return function_c()
  File "example.py", line 4, in function_c
    return 1 / 0  # Will raise ZeroDivisionError
ZeroDivisionError: division by zero
Result: None

3. Using `pdb` Post-Mortem Debugging

When your program crashes, you can analyze it after the fact:

python
import pdb

def problematic_function(data):
    result = []
    for i, value in enumerate(data):
        result.append(value / (i - 1))  # Will cause division by zero when i=1
    return result

try:
    problematic_function([10, 20, 30])
except:
    pdb.post_mortem()  # Start debugging at the point of exception

This will open the debugger at the exact point where the exception occurred.

Real-World Debugging Example

Let's walk through a practical example of debugging a more complex function:

python
def analyze_student_scores(student_data):
    """
    Calculates statistics for student exam scores.
    
    Args:
        student_data: A dictionary mapping student names to their score lists
        
    Returns:
        Dictionary with statistics (average, highest performer, etc.)
    """
    try:
        logging.debug(f"Processing data for {len(student_data)} students")
        
        # Calculate average for each student
        averages = {}
        for student, scores in student_data.items():
            if not scores:  # Check for empty score list
                logging.warning(f"No scores found for student: {student}")
                averages[student] = 0
            else:
                averages[student] = sum(scores) / len(scores)
        
        logging.debug(f"Calculated averages: {averages}")
        
        # Find highest performer
        if averages:
            highest_performer = max(averages, key=averages.get)
            highest_score = averages[highest_performer]
        else:
            logging.warning("No student data to determine highest performer")
            highest_performer = None
            highest_score = 0
            
        # Calculate overall class average
        all_scores = [score for scores in student_data.values() for score in scores]
        class_average = sum(all_scores) / len(all_scores) if all_scores else 0
        
        return {
            "student_averages": averages,
            "highest_performer": highest_performer,
            "highest_score": highest_score,
            "class_average": class_average,
            "number_of_students": len(student_data)
        }
        
    except Exception as e:
        logging.error(f"Error analyzing student data: {e}")
        # For debugging complex issues, we can use traceback
        import traceback
        traceback.print_exc()
        raise

To test and debug this function:

python
import logging
logging.basicConfig(level=logging.DEBUG, format='%(levelname)s: %(message)s')

# Test case 1: Normal data
test_data_1 = {
    "Alice": [85, 90, 92],
    "Bob": [70, 75, 80],
    "Charlie": [95, 85, 90]
}

# Test case 2: Problem data (includes a non-numeric value)
test_data_2 = {
    "Alice": [85, 90, 92],
    "Bob": [70, "75", 80],  # String instead of integer
    "Charlie": [95, 85, 90]
}

# Test with normal data
try:
    result_1 = analyze_student_scores(test_data_1)
    print("Result 1:", result_1)
except Exception as e:
    print(f"Test 1 failed: {e}")

# Test with problem data
try:
    result_2 = analyze_student_scores(test_data_2)
    print("Result 2:", result_2)
except Exception as e:
    print(f"Test 2 failed: {e}")

When we run this, the second test will fail due to trying to add a string to numbers. We can modify our function to handle this:

python
def analyze_student_scores(student_data):
    """Improved version with better error handling"""
    try:
        logging.debug(f"Processing data for {len(student_data)} students")
        
        # Calculate average for each student
        averages = {}
        for student, scores in student_data.items():
            if not scores:
                logging.warning(f"No scores found for student: {student}")
                averages[student] = 0
                continue
                
            # Validate all scores are numeric
            valid_scores = []
            for score in scores:
                try:
                    valid_scores.append(float(score))  # Convert to float
                except (ValueError, TypeError):
                    logging.warning(f"Invalid score '{score}' for {student} - ignoring")
            
            if valid_scores:
                averages[student] = sum(valid_scores) / len(valid_scores)
            else:
                logging.warning(f"No valid scores for student: {student}")
                averages[student] = 0
        
        # Rest of the function remains the same...
        

This improved version will handle the non-numeric score gracefully.

Summary and Best Practices

Effective debugging is a critical skill that will save you countless hours of frustration. Here's a summary of the techniques we've covered:

Start simple: Use print statements to understand program flow
Use logging: Replace print statements with proper logging for better control
Use assertions: Verify assumptions in your code during development
Use a debugger: Step through code execution to identify issues
Exception handling: Use try-except blocks to capture and analyze errors
Leverage tools: Take advantage of IDE debugging features

Remember these debugging best practices:

Be methodical: Develop a systematic approach to isolating and fixing bugs
Test one change at a time: Make a single change and test if it fixed the issue
Read error messages carefully: Python's error messages contain valuable information
Use version control: This allows you to revert changes if your fixes create more problems
Document your fixes: Note what you changed and why, especially for complex bugs

Additional Resources and Exercises

Resources

Exercises

Debug a Recursive Function Create a function to calculate Fibonacci numbers recursively, and use a debugger to trace through its execution for n=5.
Logging Challenge Add proper logging to an existing program with at least 3 different severity levels.

Error Detective Debug the following function:

python
def process_names(names_list):
    processed = []
    for i in range(len(names_list)):
        if names_list[i]:
            processed.append(names_list[i].strip().title())
    
    for i in range(len(processed)):
        print(f"Name {i+1}: {processed[i]}")
        
    return len(processed)

# This contains several bugs, try to find and fix them
result = process_names(["alice  ", "BOB", "", "Charlie", None, "  david"])

Post-Mortem Analysis Write a program that will fail with an exception, and practice using pdb.post_mortem() to examine the state at the point of failure.

By mastering these debugging techniques, you'll be well-equipped to handle any issues that arise in your Python code, saving time and reducing frustration. Debugging is not just about fixing errors—it's about understanding your code more deeply.

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)

Introduction to Debugging​

Basic Debugging Techniques​

1. Using Print Statements​

2. Using Assertions​

Intermediate Debugging Techniques​

1. Using the logging Module​

2. Using the Python Debugger (pdb)​

Advanced Debugging Techniques​

1. Using Interactive Debuggers in IDEs​

2. Using try-except with Traceback​

3. Using pdb Post-Mortem Debugging​

Real-World Debugging Example​

Summary and Best Practices​

Additional Resources and Exercises​

Resources​

Exercises​

Introduction to Debugging

Basic Debugging Techniques

1. Using Print Statements

2. Using Assertions

Intermediate Debugging Techniques

1. Using the `logging` Module

2. Using the Python Debugger (pdb)

Advanced Debugging Techniques

1. Using Interactive Debuggers in IDEs

2. Using `try`-`except` with Traceback

3. Using `pdb` Post-Mortem Debugging

Real-World Debugging Example

Summary and Best Practices

Additional Resources and Exercises

Resources

Exercises