Technology for ponies

Jun 12, 2015

Convert a Foreign Key to Many to Many using Django 1.7 buit-in migrations

Django 1.7 comes with a built-in migration system for your data model, replacing the need of a third party module, usually South. This system is pretty good at doing a lot of stuff automatically, but there are some situations where you need to get your hands dirty, especially when there is data migration involved.

This guide will show you, step by step, how to convert a Foreign Key field into a Many to Many, while preserving existing relations.

Initial set up

Let's say you are starting a pizzeria and want your first pizzas to be very simple, they'll have only one topping:

from django.db import models

class Topping(models.Model):
    name = models.CharField(max_length=20)

class Pizza(models.Model):
    name = models.CharField(max_length=20)
    topping = models.ForeignKey(Topping)

Create the initial migration file (assuming our app is called pizza) and run it:

$ ./manage.py makemigrations pizza
$ ./manage.py migrate

Migrations for 'pizza':
    - Create model Pizza
    - Create model Topping
    - Add field topping to pizza

And let's create a few pizzas to test our example:

$ ./manage.py shell
from pizza.models import Topping, Pizza
for topping_name, pizza_name in [('mozzarella','Margherita'), ('mushrooms', 'Capricciosa'), ('artichoke', 'Vegetariana'), ('pineapple', 'Hawaii')]:
    topping = Topping.objects.create(name=topping_name)
    pizza = Pizza.objects.create(name=pizza_name, topping=topping)

Here are our data:

sqlite> select * from pizza_topping;

sqlite> select * from pizza_pizza;

Convert to Many to Many

After making a few pizzas you soon realize only one topping per pizza is not tasty enough. Let's tweak our model and add a field which have many toppings:

class Pizza(models.Model):
    name = models.CharField(max_length=20)
    topping = models.ForeignKey(Topping, related_name='old_topping')
    toppings = models.ManyToManyField(Topping)

We must add a related_name attribute on one of the 2 fields to have the reverse accessor (from Topping to Pizza) to work. We now run an automatic migration to create the intermediary table that will hold the new relation:

$ ./manage.py makemigrations pizza

Migrations for 'pizza':
    - Add field toppings to pizza
    - Alter field topping on pizza

Great. If we run migrate we'll now have an empty table pizza_pizza_toppings that is able to store many toppings per pizza. But we want to keep toppings we already have on our pizzas. So let's create a data migration:

$ ./manage.py makemigrations pizza --empty

Migrations for 'pizza':

Edit the newly created file pizza/migrations/0003_auto_xxx.py. We'll create a function that for each pizza, copy data from the field topping to the new field toppings:

from django.db import models, migrations

def forward(apps, schema_editor):
    Pizza = apps.get_model("pizza", "Pizza")
    for pizza in Pizza.objects.all():

class Migration(migrations.Migration):

    dependencies = [
        ('pizza', '0002_auto_20150612_0903'),

    operations = [

After running migrate, we can see toppings relationships had been copied to the new table:

sqlite> select * from pizza_pizza_toppings;

Last step is to clean up our model and remove the old topping field:

class Pizza(models.Model):
    name = models.CharField(max_length=20)
    toppings = models.ManyToManyField(Topping)
$ ./manage.py makemigrations pizza

Migrations for 'pizza':
    - Remove field topping from pizza

Voila! You can now start making delicious pizza!