Skip to main content

Python JSON Files

Introduction​

JSON (JavaScript Object Notation) is a lightweight data format that has become extremely popular for data exchange between applications. Despite its origins in JavaScript, JSON is language-independent and Python provides excellent support for working with JSON data through its built-in json module.

In this tutorial, you'll learn:

  • What JSON is and why it's important
  • How to convert Python objects to JSON (serialization)
  • How to convert JSON back to Python objects (deserialization)
  • How to read and write JSON files
  • How to handle complex data and formatting

What is JSON?​

JSON is a text-based data format that's easy for humans to read and write, and easy for machines to parse and generate. It's built on two structures:

  • A collection of key-value pairs (similar to Python dictionaries)
  • An ordered list of values (similar to Python lists)

Here's a simple JSON example:

json
{
  "name": "John",
  "age": 30,
  "city": "New York",
  "languages": ["Python", "JavaScript", "SQL"],
  "is_programmer": true,
  "hobbies": null
}

JSON supports these data types:

  • Strings: "Hello, World!"
  • Numbers: 42, 3.14
  • Booleans: true, false
  • Arrays: [1, 2, 3]
  • Objects: {"name": "John"}
  • Null: null

The Python json Module​

Python's built-in json module provides functionality to encode Python objects as JSON strings and decode JSON strings back into Python objects.

Let's first import the module:

python
import json

Converting Python Objects to JSON (Serialization)​

The process of converting Python objects to JSON format is called serialization (or encoding). We use the json.dumps() function for this.

python
# Python dictionary
person = {
    "name": "Alice",
    "age": 25,
    "city": "Boston",
    "languages": ["Python", "Java"],
    "is_student": True,
    "grades": None
}

# Convert to JSON string
json_string = json.dumps(person)
print(json_string)

Output:

{"name": "Alice", "age": 25, "city": "Boston", "languages": ["Python", "Java"], "is_student": true, "grades": null}

Notice how Python's True became true and None became null in the JSON output.

Making JSON Output More Readable​

You can make the JSON output more readable by adding indentation:

python
# Pretty printing with indentation
formatted_json = json.dumps(person, indent=2)
print(formatted_json)

Output:

{
  "name": "Alice",
  "age": 25,
  "city": "Boston",
  "languages": [
    "Python",
    "Java"
  ],
  "is_student": true,
  "grades": null
}

Additional Formatting Options​

The json.dumps() function provides several parameters to customize the output:

python
# Sort keys and add indentation
json_string = json.dumps(person, indent=4, sort_keys=True)
print(json_string)

Output:

{
    "age": 25,
    "city": "Boston",
    "grades": null,
    "is_student": true,
    "languages": [
        "Python",
        "Java"
    ],
    "name": "Alice"
}

Converting JSON to Python Objects (Deserialization)​

To convert JSON strings back to Python objects, use the json.loads() function:

python
# JSON string
json_data = '{"name": "Bob", "age": 30, "city": "New York"}'

# Parse JSON string to Python dictionary
parsed_data = json.loads(json_data)
print(parsed_data)
print(type(parsed_data))
print(f"Name: {parsed_data['name']}")

Output:

{'name': 'Bob', 'age': 30, 'city': 'New York'}
<class 'dict'>
Name: Bob

Reading and Writing JSON Files​

Writing JSON to a File​

To save JSON data to a file, use the json.dump() function:

python
# Python dictionary to save
user_data = {
    "users": [
        {"id": 1, "name": "John", "active": True},
        {"id": 2, "name": "Sara", "active": False},
        {"id": 3, "name": "Mike", "active": True}
    ],
    "total": 3,
    "page": 1
}

# Writing to a JSON file
with open('users.json', 'w') as file:
    json.dump(user_data, file, indent=4)
    
print("Data successfully written to users.json")

Output:

Data successfully written to users.json

This creates a file called users.json in your current directory with the formatted JSON data.

Reading JSON from a File​

To read JSON data from a file, use the json.load() function:

python
# Reading from a JSON file
with open('users.json', 'r') as file:
    loaded_data = json.load(file)
    
print("Data loaded from file:")
print(f"Total users: {loaded_data['total']}")
print("User names:")
for user in loaded_data['users']:
    print(f"- {user['name']} (Active: {user['active']})")

Output:

Data loaded from file:
Total users: 3
User names:
- John (Active: True)
- Sara (Active: False)
- Mike (Active: True)

Working with Complex Data Types​

Handling Dates and Custom Objects​

JSON doesn't natively support dates or custom Python objects. To handle these, we need custom encoding/decoding.

python
from datetime import datetime

# Dictionary with a datetime object
event = {
    "name": "Python Workshop",
    "date": datetime.now(),
    "attendees": 150
}

# This will raise a TypeError
try:
    json_string = json.dumps(event)
except TypeError as e:
    print(f"Error: {e}")

# Solution: Create a custom encoder
def datetime_to_str(obj):
    if isinstance(obj, datetime):
        return obj.isoformat()
    raise TypeError("Type not serializable")

# Now it works
json_string = json.dumps(event, default=datetime_to_str, indent=2)
print(json_string)

Output:

Error: Object of type datetime is not JSON serializable
{
  "name": "Python Workshop",
  "date": "2023-07-21T14:30:45.123456",
  "attendees": 150
}

Custom JSON Encoder​

For more complex serialization needs, you can create a custom JSON encoder by subclassing json.JSONEncoder:

python
class CustomEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return obj.isoformat()
        elif hasattr(obj, 'to_json'):
            return obj.to_json()
        return super().default(obj)

# Example class with to_json method
class Product:
    def __init__(self, id, name, price):
        self.id = id
        self.name = name
        self.price = price
    
    def to_json(self):
        return {
            "id": self.id,
            "name": self.name,
            "price": self.price
        }

# Complex data with custom objects and dates
data = {
    "product": Product(1, "Laptop", 999.99),
    "created_at": datetime.now()
}

# Use the custom encoder
json_string = json.dumps(data, cls=CustomEncoder, indent=2)
print(json_string)

Output:

{
  "product": {
    "id": 1,
    "name": "Laptop",
    "price": 999.99
  },
  "created_at": "2023-07-21T14:32:10.987654"
}

Real-World Application: Weather Data​

Let's create a practical example where we fetch weather data from a file and analyze it:

python
# Sample weather data
weather_data = {
    "city": "San Francisco",
    "days": [
        {"date": "2023-07-15", "temperature": 18, "humidity": 85, "conditions": "Cloudy"},
        {"date": "2023-07-16", "temperature": 22, "humidity": 70, "conditions": "Sunny"},
        {"date": "2023-07-17", "temperature": 20, "humidity": 75, "conditions": "Partly Cloudy"},
        {"date": "2023-07-18", "temperature": 21, "humidity": 80, "conditions": "Foggy"},
        {"date": "2023-07-19", "temperature": 23, "humidity": 65, "conditions": "Sunny"}
    ]
}

# Save to file
with open('weather.json', 'w') as file:
    json.dump(weather_data, file, indent=2)
print("Weather data saved to weather.json")

# Now let's read and analyze the data
with open('weather.json', 'r') as file:
    weather = json.load(file)

# Calculate average temperature
temps = [day["temperature"] for day in weather["days"]]
avg_temp = sum(temps) / len(temps)

# Find the hottest day
hottest_day = max(weather["days"], key=lambda day: day["temperature"])

print(f"Weather analysis for {weather['city']}:")
print(f"Average temperature: {avg_temp:.1f}°C")
print(f"Hottest day: {hottest_day['date']} with {hottest_day['temperature']}°C ({hottest_day['conditions']})")

Output:

Weather data saved to weather.json
Weather analysis for San Francisco:
Average temperature: 20.8°C
Hottest day: 2023-07-19 with 23°C (Sunny)

Common Errors and How to Handle Them​

1. Invalid JSON Format​

python
try:
    # Invalid JSON (missing quotation marks around keys)
    bad_json = '{name: "John", age: 30}'
    parsed = json.loads(bad_json)
except json.JSONDecodeError as e:
    print(f"JSON parsing error: {e}")

Output:

JSON parsing error: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

2. File Not Found​

python
try:
    with open('nonexistent_file.json', 'r') as file:
        data = json.load(file)
except FileNotFoundError:
    print("Error: The specified file was not found.")

Output:

Error: The specified file was not found.

Summary​

In this tutorial, you've learned:

  1. What JSON is: A lightweight, text-based data format used for data exchange
  2. Serialization: Converting Python objects to JSON using json.dumps() and json.dump()
  3. Deserialization: Converting JSON to Python objects using json.loads() and json.load()
  4. File Operations: Reading and writing JSON files
  5. Handling Complex Types: Using custom encoders for datetime and custom objects
  6. Real-world Applications: Working with structured data like weather information
  7. Error Handling: Dealing with common JSON-related errors

JSON is an essential skill for any Python developer, especially if you're working with APIs, configuration files, or data exchange between systems.

Additional Exercises​

  1. Create a program that reads a JSON file containing a list of books, and allows users to add new books, search for books by title or author, and save changes back to the file.

  2. Write a script that converts a CSV file to JSON format, handling different data types appropriately.

  3. Create a simple address book application that stores contacts in a JSON file, with functions to add, edit, delete, and search for contacts.

  4. Modify the weather data example to analyze more aspects, such as the most common weather condition or the day with the highest humidity.

Additional Resources​

Happy coding with JSON in Python!


If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)