Django 1.10 removed the very helpful --list (or -l) option from the manage migrate command. This option was deprecated since 1.8. But don't worry, you can have the same output using:
./manage.py showmigrations
You can also see in which order your migrations will be applied using the --plan option:
./manage.py showmigrations --plan
Django 1.7 comes with a built-in migration system for your data model, replacing the need of a third party module, usually South. This system is pretty good at doing a lot of stuff automatically, but there are some situations where you need to get your hands dirty, especially when there is data migration involved.
This guide will show you, step by step, how to convert a Foreign Key field into a Many to Many, while preserving existing relations.
Initial set up
Let's say you are starting a pizzeria and want your first pizzas to be very simple, they'll have only one topping:
from django.db import models
class Topping(models.Model):
name = models.CharField(max_length=20)
class Pizza(models.Model):
name = models.CharField(max_length=20)
topping = models.ForeignKey(Topping)
Create the initial migration file (assuming our app is called pizza) and run it:
$ ./manage.py makemigrations pizza
$ ./manage.py migrate
Migrations for 'pizza':
0001_initial.py:
- Create model Pizza
- Create model Topping
- Add field topping to pizza
And let's create a few pizzas to test our example:
$ ./manage.py shell
from pizza.models import Topping, Pizza
for topping_name, pizza_name in [('mozzarella','Margherita'), ('mushrooms', 'Capricciosa'), ('artichoke', 'Vegetariana'), ('pineapple', 'Hawaii')]:
topping = Topping.objects.create(name=topping_name)
pizza = Pizza.objects.create(name=pizza_name, topping=topping)
Here are our data:
sqlite> select * from pizza_topping;
1|mozzarella
2|mushrooms
3|artichoke
4|pineapple
sqlite> select * from pizza_pizza;
1|Margherita|1
2|Capricciosa|2
3|Vegetariana|3
4|Hawaii|4
Convert to Many to Many
After making a few pizzas you soon realize only one topping per pizza is not tasty enough. Let's tweak our model and add a field which have many toppings:
class Pizza(models.Model):
name = models.CharField(max_length=20)
topping = models.ForeignKey(Topping, related_name='old_topping')
toppings = models.ManyToManyField(Topping)
We must add a related_name attribute on one of the 2 fields to have the reverse accessor (from Topping to Pizza) to work.
We now run an automatic migration to create the intermediary table that will hold the new relation:
$ ./manage.py makemigrations pizza
Migrations for 'pizza':
0002_auto_20150612_0903.py:
- Add field toppings to pizza
- Alter field topping on pizza
Great. If we run migrate we'll now have an empty table pizza_pizza_toppings that is able to store many toppings per pizza. But we want to keep toppings we already have on our pizzas. So let's create a data migration:
$ ./manage.py makemigrations pizza --empty
Migrations for 'pizza':
0003_auto_20150612_0920.py:
Edit the newly created file pizza/migrations/0003_auto_xxx.py. We'll create a function that for each pizza, copy data from the field topping to the new field toppings:
from django.db import models, migrations
def forward(apps, schema_editor):
Pizza = apps.get_model("pizza", "Pizza")
for pizza in Pizza.objects.all():
pizza.toppings.add(pizza.topping)
class Migration(migrations.Migration):
dependencies = [
('pizza', '0002_auto_20150612_0903'),
]
operations = [
migrations.RunPython(forward)
]
After running migrate, we can see toppings relationships had been copied to the new table:
sqlite> select * from pizza_pizza_toppings;
1|1|1
2|2|2
3|3|3
4|4|4
Last step is to clean up our model and remove the old topping field:
class Pizza(models.Model):
name = models.CharField(max_length=20)
toppings = models.ManyToManyField(Topping)
$ ./manage.py makemigrations pizza
Migrations for 'pizza':
0004_remove_pizza_topping.py:
- Remove field topping from pizza
Voila! You can now start making delicious pizza!
We are currently working on a project that loads and dumps very big JSON objects. We recently had some slowness issues and though giving a try to some C compiled JSON modules could give a boost to our API.
After some googling we found out a few exsting modules and decided to setup a benchmark to decide which one to use. The selected modules are:
We wrote a little code snippet to automate our benchmark. It is using the timeit standard module.
from timeit import timeit
modules = ['json', 'cjson', 'jsonlib', 'simplejson', 'ujson', 'yajl']
NUMBER = 20
for module in modules:
loads = "loads"
dumps = "dumps"
if module == "cjson":
loads = "decode"
dumps = "encode"
print "[%s]" % module
load_time = timeit("obj = json.%s(data)" % loads, setup="import %s as json; data = open('file.json').read()" % module, number=NUMBER)
dump_time = timeit("json.%s(obj)" % dumps, setup="import %s as json; obj = json.%s(open('file.json').read())" % (module, loads), number=NUMBER)
print "Load time: ", load_time
print "Dump time: ", dump_time
Notes:
- The file.json is 15Mb.
- cjson is not a drop-in replacement of the standard json module. We have to patch the loads and dumps functions.
- Loading and dumping are done 20 times for a better accuracy
And the winners are:
[json]
Load time: 12.7440698147
Dump time: 6.37574481964
[cjson]
Load time: 6.24644708633
Dump time: 11.6356570721
[jsonlib]
Load time: 13.1087989807
Dump time: 15.4686369896
[simplejson]
Load time: 4.67061305046
Dump time: 6.65470814705
[ujson]
Load time: 5.84970593452
Dump time: 5.00060105324
[yajl]
Load time: 6.7509560585
Dump time: 16.4374480247
simplejson for loading and ujson for dumping.
Choose one or the other depending if your program is more loading or dumping data.