Django Database Migrations
Introduction
Database migrations are a way to propagate changes you make to your models (adding a field, deleting a model, etc.) into your database schema. They're designed to be mostly automatic, but you'll need to know when to make migrations, when to apply them, and how to troubleshoot when things go wrong.
In Django, migrations are files that record changes to your models and transform those changes into database operations. These operations can create, modify, or delete database tables and columns to match your Django models. This allows you to evolve your database schema over time, without having to drop and recreate your database every time you make a change.
Why Migrations Are Important
Before diving into the technical details, let's understand why migrations are crucial:
- Version control for database schema: Migrations serve as a version history for your database structure.
- Team collaboration: They allow multiple developers to make and share database changes.
- Easy deployment: You can apply the same changes to different environments (development, staging, production).
- Safe schema evolution: Migrations help you modify your database without losing data.
Migration Commands
Django provides several commands for working with migrations:
Creating Migrations
When you make changes to your models, you need to create a migration file:
python manage.py makemigrations
This creates a migration file in the migrations directory of your app. If you have multiple apps and want to create migrations for a specific app:
python manage.py makemigrations myapp
Example:
Let's say we have a Book model in our books app:
# books/models.py
from django.db import models
class Book(models.Model):
title = models.CharField(max_length=200)
author = models.CharField(max_length=100)
published_date = models.DateField()
After creating this model, we run:
python manage.py makemigrations books
Output:
Migrations for 'books':
books/migrations/0001_initial.py
- Create model Book
Applying Migrations
To apply migrations and actually change your database schema:
python manage.py migrate
This will apply all pending migrations. To apply migrations for a specific app:
python manage.py migrate books
You can also migrate to a specific migration:
python manage.py migrate books 0002
Example:
python manage.py migrate books
Output:
Operations to perform:
Apply all migrations: books
Running migrations:
Applying books.0001_initial... OK
Viewing Migration Status
To see which migrations have been applied and which are pending:
python manage.py showmigrations
Output might look like:
admin
[X] 0001_initial
[X] 0002_logentry_remove_auto_add
books
[X] 0001_initial
[ ] 0002_book_publisher
Viewing SQL for Migrations
If you want to see what SQL Django will execute for migrations without actually running them:
python manage.py sqlmigrate books 0001
This shows the SQL statements that would be executed for that migration.
Migration Workflow
Now let's walk through a complete workflow with migrations:
1. Initial Model Creation
# books/models.py
from django.db import models
class Book(models.Model):
title = models.CharField(max_length=200)
author = models.CharField(max_length=100)
published_date = models.DateField()
2. Creating Initial Migration
python manage.py makemigrations books
This creates books/migrations/0001_initial.py.
3. Applying Initial Migration
python manage.py migrate books
4. Modifying the Model
Let's add a new field to our model:
# books/models.py
from django.db import models
class Book(models.Model):
title = models.CharField(max_length=200)
author = models.CharField(max_length=100)
published_date = models.DateField()
publisher = models.CharField(max_length=200, default='Unknown') # New field
5. Creating Migration for the Change
python manage.py makemigrations books
Output:
Migrations for 'books':
books/migrations/0002_book_publisher.py
- Add field publisher to book
6. Applying the New Migration
python manage.py migrate books
Output:
Operations to perform:
Apply all migrations: books
Running migrations:
Applying books.0002_book_publisher... OK
Advanced Migration Concepts
Data Migrations
Sometimes you need to not only change the schema but also transform the data. For this, you can create a data migration:
python manage.py makemigrations --empty books --name=populate_publisher
Then edit the created migration file:
# books/migrations/0003_populate_publisher.py
from django.db import migrations
def set_default_publisher(apps, schema_editor):
Book = apps.get_model('books', 'Book')
for book in Book.objects.all():
if book.publisher == 'Unknown':
book.publisher = 'Default Publisher'
book.save()
class Migration(migrations.Migration):
dependencies = [
('books', '0002_book_publisher'),
]
operations = [
migrations.RunPython(set_default_publisher),
]
Handling Migration Conflicts
When working in a team, conflicts can occur when multiple developers create migrations. To resolve these:
- Pull the latest code with the other developer's migrations
- Run
python manage.py migrateto apply their migrations - Make your model changes
- Run
python manage.py makemigrations
Django will detect that there's already a newer migration and create a new one that depends on it.
Squashing Migrations
As your project grows, you may accumulate many migration files. You can squash them into fewer files:
python manage.py squashmigrations books 0001 0004
This creates a new migration that combines migrations 0001 through 0004.
Real-world Example: Building a Blog
Let's walk through a more complex example of evolving a blog application:
Initial Blog Post Model
# blog/models.py
from django.db import models
class Post(models.Model):
title = models.CharField(max_length=200)
content = models.TextField()
created_date = models.DateTimeField(auto_now_add=True)
Create and apply migrations:
python manage.py makemigrations blog
python manage.py migrate blog
Adding Categories
Now, let's add categories:
# blog/models.py
from django.db import models
class Category(models.Model):
name = models.CharField(max_length=100)
def __str__(self):
return self.name
class Post(models.Model):
title = models.CharField(max_length=200)
content = models.TextField()
created_date = models.DateTimeField(auto_now_add=True)
category = models.ForeignKey(Category, on_delete=models.CASCADE, null=True)
Create migrations:
python manage.py makemigrations blog
Django will prompt you:
You are trying to add a non-nullable field 'category' to post without a default;
we can't do that (the database needs something to populate existing rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
2) Quit, and let me add a default in models.py
Select an option:
Choose option 2, modify the model to add null=True, then create migrations again:
python manage.py makemigrations blog
python manage.py migrate blog
Creating Data Migration for Categories
Now we want to assign a default category to existing posts:
python manage.py makemigrations --empty blog --name=populate_default_category
Edit the created migration file:
# blog/migrations/xxxx_populate_default_category.py
from django.db import migrations
def create_default_category_and_assign(apps, schema_editor):
Category = apps.get_model('blog', 'Category')
Post = apps.get_model('blog', 'Post')
# Create default category
default_category = Category.objects.create(name='General')
# Assign to all posts without a category
for post in Post.objects.filter(category__isnull=True):
post.category = default_category
post.save()
class Migration(migrations.Migration):
dependencies = [
('blog', '0002_category_post_category'),
]
operations = [
migrations.RunPython(create_default_category_and_assign),
]
Apply the migration:
python manage.py migrate blog
Best Practices for Migrations
- Run migrations frequently: Don't let unapplied migrations pile up.
- Include migrations in version control: They're part of your application code.
- Test migrations: Especially before deploying to production.
- Back up your database before migrations: Particularly for critical production systems.
- Use data migrations wisely: For complex data transformations.
- Consider database constraints: Be careful with dropping columns that have foreign key relationships.
- Be aware of database-specific behaviors: Some operations work differently across database backends.
Common Migration Problems and Solutions
1. Migration dependency conflicts
Problem: "Conflicting migrations detected"
Solution:
- Merge the migrations using
python manage.py makemigrations --merge - If that doesn't work, you may need to manually edit migration dependencies
2. Unapplied migrations in production
Problem: Deploying code that depends on database changes that haven't been applied
Solution:
- Always run migrations as part of your deployment process
- Consider using a deployment script that includes migration commands
3. Migrations that take too long
Problem: Large tables that take a long time to migrate
Solution:
- Consider alternative approaches like database-native operations
- Split large migrations into smaller steps
- Schedule migrations during off-peak hours
Summary
Django migrations are a powerful tool for evolving your database schema over time. They allow you to:
- Create new models and fields
- Modify existing fields
- Remove models and fields
- Transform data during schema changes
- Collaborate with a team on database changes
By understanding how to create, apply, and manage migrations, you can maintain your Django application's database schema efficiently.
Additional Resources
- Django's Official Migration Documentation
- Django Migration Operations Reference
- Django Schema Editor Documentation
Exercises
- Create a simple Django app with a
Customermodel with fields for name and email. - Generate and apply initial migrations.
- Add a
phone_numberfield to the model and create and apply a new migration. - Create a data migration that sets a default area code for all existing phone numbers.
- Try to squash migrations into a single file.
- Add a
purchase_historymodel with a foreign key toCustomerand migrate.
If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)