Django's N+1 Trap
Every Django project must start off with a warning about N+1 queries.
The Django ORM makes it very easy to manage and query tables via models and its migration management is a godsend.
For example, querying active users is as simple as
# models.py
class User(models.Model):
active = models.BooleanField()
country = models.ForeignKey(Country, on_delete=models.CASCADE)
# run.py
users = User.objects.filter(active=True)
Which converts to
SELECT *
FROM user
WHERE active = TRUE;
However, this ease is a trap. Since it is so close to the Python code, developers often do not realise the extra queries that go into constructing the objects. Accessing a foreign key attribute looks identical to accessing a normal field, but under the hood Django performs an additional query the first time that relationship is accessed.
# models.py
class Country(models.Model):
name = models.CharField(max_length=100)
class User(models.Model):
active = models.BooleanField()
country = models.ForeignKey(Country, on_delete=models.CASCADE)
# run.py
users = User.objects.filter(active=True)
for user in users:
print(user.country)
Now, the code will have to make one query per user to fetch the country. If there are 500 users, it will make 1 query to get the user, and 500 to fetch the country for each. If one had to write SQL code for it, the extra queries and the more efficient way of doing it via a JOIN are obvious.
SELECT c.name
FROM user u
JOIN country c ON u.country_id = c.id
WHERE u.active = TRUE;
In this example, the issue is fairly easy to catch, but it becomes harder to debug as the code complexity increases. For example, when the iterative query is within a function.
# run.py
users = User.objects.filter(active=True)
for user in users:
print_country(user)
print_parent(user)
# utils.py
def print_country(user)
print(user.country)
def print_parent(user)
print(user.parent)
Now the person writing the loop needs to be aware of all the attributes being accessed in the called functions.
The issue is especially deceptive since it might be hidden at a smaller scale in development, but start causing performance degradation when it hits production scale.
Django provides a way to do joins with the select_related and prefetch_related methods. The above Django code can be written as-
users = User.objects.filter(active=True).select_related(“country”)
for user in users:
print(user.country)
Some ways to avoid this problem -
- Write query regression tests that check the number of queries a block of code can execute.
- Mention the N+1 problem in CLAUDE.md / AGENTS.md and in the AI code review instructions.
- In a manual review, look out for loops or iterators that access model attributes.
The N+1 problem is often cited as one of the reasons people avoid ORMs in the age-old ORMs vs SQL debate. I would personally still prefer using an ORM simply for the speed it offers in prototyping and development, but I will definitely be more careful since I have been burnt by this problem multiple times over the years.
