FastAPI JSON Encoders

When building APIs with FastAPI, you'll often need to convert Python objects to JSON-compatible formats for your responses. While FastAPI handles many conversions automatically, there are scenarios where you need more control over how your data is serialized. This is where JSON encoders come in.

Introduction to JSON Encoders

JSON (JavaScript Object Notation) is a common data interchange format used in web APIs. However, not all Python data types can be directly converted to JSON. For example, Python's datetime objects, custom classes, or database models aren't natively JSON serializable.

FastAPI provides powerful JSON encoding tools to help solve this problem and make the serialization process smoother.

The Problem: Non-Serializable Types

Let's explore what happens when we try to return objects that aren't JSON-serializable:

python

from datetime import datetime
from fastapi import FastAPI

app = FastAPI()

@app.get("/current-time")
def get_current_time():
    return {"current_time": datetime.now()}  # This will cause issues

If you run this code and visit /current-time, you'll encounter an error because datetime.now() isn't directly serializable to JSON.

Using FastAPI's jsonable_encoder

FastAPI provides a utility function called jsonable_encoder that converts Python objects to JSON-compatible types.

Basic Usage

python

from datetime import datetime
from fastapi import FastAPI
from fastapi.encoders import jsonable_encoder

app = FastAPI()

@app.get("/current-time")
def get_current_time():
    current_time = datetime.now()
    json_compatible_time = jsonable_encoder(current_time)
    return {"current_time": json_compatible_time}  # Now returns an ISO formatted string

Output:

json

{
    "current_time": "2023-04-15T14:30:45.123456"
}

The jsonable_encoder automatically converts the datetime object to an ISO formatted string, which is JSON-compatible.

Converting Custom Classes and Models

Pydantic Models

Pydantic models work seamlessly with jsonable_encoder:

python

from datetime import datetime
from fastapi import FastAPI
from fastapi.encoders import jsonable_encoder
from pydantic import BaseModel

app = FastAPI()

class User(BaseModel):
    id: int
    name: str
    created_at: datetime
    active: bool

@app.get("/user")
def get_user():
    user = User(
        id=123,
        name="John Doe",
        created_at=datetime.now(),
        active=True
    )
    return jsonable_encoder(user)

Output:

json

{
    "id": 123,
    "name": "John Doe",
    "created_at": "2023-04-15T14:35:23.123456",
    "active": true
}

Custom Classes

For custom classes, we need to define how they should be converted:

python

from datetime import datetime
from fastapi import FastAPI
from fastapi.encoders import jsonable_encoder

app = FastAPI()

class Product:
    def __init__(self, name: str, price: float, created_at: datetime):
        self.name = name
        self.price = price
        self.created_at = created_at

@app.get("/product")
def get_product():
    product = Product(
        name="Laptop",
        price=999.99,
        created_at=datetime.now()
    )
    # Convert to dict first, then use jsonable_encoder
    product_dict = {
        "name": product.name,
        "price": product.price,
        "created_at": product.created_at
    }
    return jsonable_encoder(product_dict)

Output:

json

{
    "name": "Laptop",
    "price": 999.99,
    "created_at": "2023-04-15T14:40:12.123456"
}

Advanced Usage: Custom Encoders

Include and Exclude Fields

You can customize which fields are included or excluded:

python

from datetime import datetime
from fastapi import FastAPI
from fastapi.encoders import jsonable_encoder
from pydantic import BaseModel

app = FastAPI()

class Item(BaseModel):
    id: int
    name: str
    description: str = None
    price: float
    inventory: int
    created_at: datetime

@app.get("/item")
def get_item():
    item = Item(
        id=1,
        name="Smartphone",
        description="Latest model",
        price=799.99,
        inventory=50,
        created_at=datetime.now()
    )
    
    # Include only specific fields
    json_item = jsonable_encoder(
        item,
        include={"id", "name", "price"}
    )
    
    return json_item

Output:

json

{
    "id": 1,
    "name": "Smartphone",
    "price": 799.99
}

To exclude fields instead:

python

json_item = jsonable_encoder(
    item,
    exclude={"created_at", "inventory"}
)

Custom Encoding Rules

For more complex scenarios, you can create custom encoding functions:

python

from datetime import datetime
from fastapi import FastAPI
from fastapi.encoders import jsonable_encoder
from typing import Any

app = FastAPI()

def custom_encoder(obj: Any) -> Any:
    if isinstance(obj, datetime):
        return obj.strftime("%Y-%m-%d %H:%M:%S")
    return obj

@app.get("/custom-datetime")
def get_custom_datetime():
    now = datetime.now()
    return {"formatted_time": custom_encoder(now)}

Output:

json

{
    "formatted_time": "2023-04-15 14:45:30"
}

Real-World Application: API with Database Models

Here's a more practical example using SQLAlchemy ORM:

python

from datetime import datetime
from typing import List
from fastapi import FastAPI, Depends
from fastapi.encoders import jsonable_encoder
from sqlalchemy import Column, Integer, String, DateTime, create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, Session

app = FastAPI()

# Database setup (simplified)
SQLALCHEMY_DATABASE_URL = "sqlite:///./test.db"
engine = create_engine(SQLALCHEMY_DATABASE_URL)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
Base = declarative_base()

# Database model
class ArticleModel(Base):
    __tablename__ = "articles"
    id = Column(Integer, primary_key=True, index=True)
    title = Column(String, index=True)
    content = Column(String)
    published = Column(DateTime, default=datetime.now)

Base.metadata.create_all(bind=engine)

# Dependency to get DB session
def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()

# Create a sample article
@app.post("/articles/")
def create_article(title: str, content: str, db: Session = Depends(get_db)):
    article = ArticleModel(title=title, content=content)
    db.add(article)
    db.commit()
    db.refresh(article)
    return jsonable_encoder(article)

# Get all articles
@app.get("/articles/")
def get_articles(db: Session = Depends(get_db)):
    articles = db.query(ArticleModel).all()
    return jsonable_encoder(articles)

This example demonstrates how you can seamlessly convert SQLAlchemy database model instances to JSON-compatible dictionaries for API responses.

Common Pitfalls and Solutions

Circular References

If you have circular references in your data structures, you might encounter recursion errors:

python

class Node:
    def __init__(self, name: str):
        self.name = name
        self.connections = []  # Other Node objects

# This can cause issues with circular references

Solution: Break circular references before encoding or implement custom logic:

python

def encode_node(node, visited=None):
    if visited is None:
        visited = set()
    
    if id(node) in visited:
        return {"name": node.name, "connections": "...circular reference..."}
    
    visited.add(id(node))
    return {
        "name": node.name,
        "connections": [encode_node(conn, visited.copy()) for conn in node.connections]
    }

Large Objects

For very large objects, encoding can be slow and memory-intensive:

Solution: Use pagination or limit the data returned:

python

@app.get("/large-data")
def get_large_data(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
    data = db.query(LargeModel).offset(skip).limit(limit).all()
    return jsonable_encoder(data)

Summary

FastAPI's JSON encoders are powerful tools for converting Python objects to JSON-compatible types:

1. jsonable_encoder converts Python objects (including Pydantic models) to JSON-compatible types
2. You can include/exclude fields to control what gets serialized
3. Custom encoders can be implemented for special cases
4. JSON encoders help you avoid common serialization issues with dates, custom classes, and database models

By mastering JSON encoders, you'll be able to build more robust APIs that can handle complex data structures and provide consistent, well-formatted responses.

Additional Resources

• FastAPI Official Documentation on JSON Compatible Encoder
• Python's JSON module documentation
• Pydantic documentation on serialization

Exercises

1. Create a FastAPI endpoint that returns a mix of simple types, datetimes, and custom objects using jsonable_encoder
2. Implement a custom encoder for a specific data type (e.g., convert decimal numbers to strings with exactly 2 decimal places)
3. Build an API with nested models (models containing other models) and ensure they serialize correctly
4. Implement pagination for a database query result and return properly serialized JSON

By practicing these exercises, you'll become confident in handling various serialization scenarios in your FastAPI applications.