How to Review AI Generated Django Pull Requests

Six deliberate faults that passed routine database checks

The billing worker returned the same receipt ID on a retry. The database still held one receipt. In Uvik Software’s local test, the customer balance nevertheless increased by 4,800 cents on a 2,400-cent event. The uniqueness constraint protected the receipt. The business effect still happened twice.

An AI-assisted Django change can make this error in a single return path. A person can make it too. A reviewer needs an assertion that would fail on the defect. The same review principle applies when a request supplies another customer’s record ID or a migration must preserve settings written by the old application.

Quick answer

Uvik Software recommends testing the policy that a Django pull request could break. For tenant access, use a second tenant and try a forged selector. For a backfill, rerun after a user sets an explicit value. For a retried write, check the durable business state as well as the receipt. In Uvik Software’s SQLite fixture, all six deliberately faulted implementations passed three routine checks. Six targeted assertions detected every selected fault, and the reference passed all nine checks.

Disclosure. Uvik Software provides Django development and embedded engineering services. The article’s experiments use SQLite, including a synchronized two-worker case. The Django and PostgreSQL examples are implementation and rehearsal patterns. The fixture was designed to demonstrate possible failures and makes no estimate of production fault rates.

Why review evidence needs a negative control

Uvik Software ran a reference implementation and six modified implementations. Each modified implementation makes one deliberate mistake. The routine suite reads an owned invoice, fills a missing preference, and processes the first delivery of an event. The targeted suite tests boundaries that those three inputs leave open. Each check starts with a fresh database. The CSV records results per implementation and per assertion.

Figure 1. Uvik Software’s local SQLite mutation fixture. Three routine checks passed on each of seven implementations. The targeted assertions detected all six deliberately selected changes. Python 3.12.14 and SQLite 3.53.1.

The invoice check with the owned record executed both lines of the two-line lookup in the reference, the unscoped version, and the version that trusted the requested tenant. Uvik Software counted distinct executable source lines using the Python bytecode line table and recorded execution with Python’s line tracer. All three lookups reached 2 of 2 lines in that test. The caller asking for a foreign invoice and the caller changing the tenant selector expose different policy failures.

The replay check shows why a second assertion can matter even when the test hits the right path. Both the reference and the double-credit version return the same receipt ID twice and persist one row. The reference credits 2,400 cents. The faulty version credits 4,800. Check the balance, outbox event or ledger entry that the customer actually depends on.

Use this as a review procedure. Change one behavior the author says is protected, such as removing the tenant filter. The regression test should fail for the expected reason. Restore the implementation and confirm the same test passes. Google researchers described using this kind of mutation testing on changed code during review in a study covering more than 24,000 developers and 1,000 projects. The Uvik Software fixture applies the idea manually to six specified database changes. It measures how these assertions discriminate between selected implementations. 23

What the original database checks found

A separate paired reproduction compares the original and revised SQL logic. Each case has one routine check and one boundary check.

Uvik Software SQLite reproduction. Both versions passed three of three routine checks. The original logic passed zero of three targeted checks and the revised logic passed three of three.

Figure 2. Uvik Software SQLite reproduction comparing original and revised database logic. Targeted checks improved from 0/3 to 3/3, while both versions passed all routine checks.

What the coverage research measured

In a July 2026 preprint, researchers studied 4,882 agent-generated pull requests from AIDev, including 4,350 Python PRs and 532 Java PRs. This sample supported their analysis of test changes. For line coverage, they used a smaller set of merged PRs from repositories they could build and instrument. Existing tests exercised zero changed executable lines in 1,079 of the 1,664 Python PRs in that coverage subset, or 64.8%. 1

Test Coverage Analysis of Agentic Pull Requests, sections II through IV. PR-level counts and executable-line coverage answer different questions. 1

The median Python PR had zero changed-line coverage from existing tests. The median Java PR had 71.1%, and 73 of the 213 Java PRs had complete changed-line coverage from the existing suite. The Python results came from 34 instrumented repositories and the Java results from 10. Buildability and merge status affect which PRs enter those samples. The study measured execution of added lines, with a language-wide sample that extends well beyond Django. It cannot quantify the rate of permission or data faults in Django applications. 1

Figure 3. Uvik Software visualization of selected Python statement categories from the 2026 Test Coverage Analysis of Agentic Pull Requests study, showing the percentage of added executable lines missed by existing tests.

The researchers also counted missed lines by statement type. Tests missed 82.3% of 536 Python raise statements and 81.0% of 1,356 lines classified in try and except blocks. In a Django change, prioritize the actual rejection branch, recovery action, or duplicate-event decision. A test can execute the relevant line and still miss a wrong result unless it asserts the behavior users depend on. 1

When agent written tests help

The same preprint shows measurable coverage gains. Among the 605 instrumented Python PRs that changed production code and tests, adding the agent’s tests increased pooled changed-line coverage from 24.8% to 34.5%. Only 136 PRs improved individually. The line-weighted gain and the share of PRs that improved answer different questions. A review process can use diff coverage to ask which branch or error handler still needs a test, and use an assertion against a known fault to judge what that test proves. 1

Meta’s TestGen-LLM filtered candidates for build, reliable execution, and added line coverage before suggesting improvements. Section 4 of the 2024 FSE industry paper reports improvement in 196 of 1,979 Kotlin test classes across three deployments. At a Facebook app event in December 2023, engineers accepted 144 of 280 generated diffs. Sixty-one received no review, eleven were withdrawn, and sixty-four were rejected or abandoned. At an earlier event, engineers rejected a candidate that lacked an assertion even though the filter measured coverage. Use the generator to propose cases and the review to verify the outcome the case should enforce. 22

Scope the invoice lookup to the authenticated tenant

The expected behavior is that a customer can retrieve their own invoice and receives the application’s normal denial response for another tenant’s invoice. The first version looks up an invoice using only its primary key. The routine check uses an invoice owned by the caller, so both versions return one row.

Use the tenant resolved by the application’s authenticated access context to narrow the queryset before the lookup. The example assumes the route already verifies that the caller is a member of that tenant. Django’s get_object_or_404 accepts a queryset. Object-level authorization belongs on endpoints that receive a record identifier. 2 3

- invoice = get_object_or_404(Invoice, pk=invoice_id)
+ visible = Invoice.objects.filter(tenant_id=tenant.id)
+ invoice = get_object_or_404(visible, pk=invoice_id)

Create two tenants and one invoice for each. Authenticate as a member of the first tenant and request both records. Assert that the owned invoice can be read and the other request returns the expected denial, such as HTTP 404. The database reproduction checks the query directly. It returned one row with the unscoped query and zero with the scoped query for the foreign invoice.

Exercise the same lookup through HTTP

The SQL example isolates the lookup. At the endpoint, add a test that also exercises the application’s tenant resolution and membership policy. Use one authenticated member, two tenant records, and two real invoice identifiers. The example uses project factories and a route that hides a foreign invoice with a 404. Adjust the factory fields and route name to your codebase. Django’s force_login is appropriate when the review is about object access and the mechanics of login are covered elsewhere. 12

from django.test import TestCase
from django.urls import reverse


class InvoiceAccessTests(TestCase):
    @classmethod
    def setUpTestData(cls):
        tenant_a = TenantFactory()
        cls.member = MemberFactory(tenant=tenant_a)
        cls.owned_invoice = InvoiceFactory(tenant=tenant_a)
        cls.foreign_invoice = InvoiceFactory(tenant=TenantFactory())

    def test_member_sees_only_own_tenant_invoice(self):
        self.client.force_login(self.member)
        own_url = reverse("invoice-detail", args=[self.owned_invoice.pk])
        foreign_url = reverse("invoice-detail", args=[self.foreign_invoice.pk])

        self.assertEqual(self.client.get(own_url).status_code, 200)
        self.assertEqual(self.client.get(foreign_url).status_code, 404)

For a JSON endpoint, assert the owned invoice’s identifier or amount as well as the status. If the application scopes tenants by host or URL path, pass the actual tenant context in both requests. Use a second test with a forged tenant selector if the PR touches that part of the route. Check related endpoints that edit, download, or export the same record. A denied write should leave the row and any audit or outbox tables in the state the application promises. 3

When reviewing an existing regression test, try it against the unscoped version. The owned invoice should still return 200. The foreign request should now make the authorization assertion fail. Record the authenticated actor, selected tenant, resource, action, and expected outcome so another reviewer can see which part of the policy the test proves.

Consider two requests against the same foreign invoice. In the first, the authenticated tenant stays fixed and only the invoice ID changes. In the second, the caller also supplies the foreign tenant as a query or path value. The first request catches the missing filter. The second catches a filter populated from an untrusted selector. If tenant switching is supported, use your actual membership check and add a positive case for a caller authorized for both tenants. A denied request and an allowed switch have different evidence requirements. 3

Backfill only values that are missing

The expected behavior is that a missing report page size receives the default of 25, and a value chosen by a user is preserved. The first migration function updates every row. A fixture with a single null value passes, because it has no existing choice to protect.

In a Django data migration, get the historical model from the migration state and use the active migration database alias. Apply the update to rows where page size is null. The bulk update runs at the SQL level. If the application relies on save hooks or audit events for these values, account for those separately in the migration design. 4 5

def backfill_page_size(apps, schema_editor):
    Preference = apps.get_model("reports", "ReportPreference")
    using = schema_editor.connection.alias
    preferences = Preference.objects.using(using)
    preferences.filter(page_size__isnull=True).update(page_size=25)

The reproduction seeds a null preference and another value, runs the update, and then stores a new explicit page size of 75. It re-executes the backfill logic to model replay. The unconditional update overwrites the user’s value with 25. The revised update keeps 75. A Django test can start from the schema before the migration and exercise its forward function. Django records applied migrations, so re-running the function in the fixture is a deliberate test of the operation.

Test a partial backfill and a rerun

Build a migration test from the schema immediately before the data change. Seed one row with a null page size and a second with 50. After the forward operation, read the rows from the database and check for 25 and 50. Change the second row to 75, rerun the forward function, and check that it still reads 75. For a batched migration, also seed the state left by a completed batch and verify that restarting reaches the same final values as a complete run. Keep a test that applies the actual migration sequence so Django’s historical state and dependency order are exercised. 4 6

For a large table, a separate non-atomic data migration can commit one batch at a time. Use the migration connection’s alias for the query and transaction. Repeat the null condition on the update so a row selected earlier is still eligible when it is written. On PostgreSQL under Read Committed, an update waiting for a concurrent updater rechecks its condition against the updated row. This makes the second predicate relevant when a user sets an explicit value between selection and update. 13 19

from django.db import transaction


def backfill_page_size_in_batches(apps, schema_editor):
    Preference = apps.get_model("reports", "ReportPreference")
    using = schema_editor.connection.alias
    preferences = Preference.objects.using(using)
    missing = preferences.filter(page_size__isnull=True)

    while True:
        ids = list(missing.order_by("pk").values_list("pk", flat=True)[:1000])
        if not ids:
            return
        with transaction.atomic(using=using):
            missing.filter(pk__in=ids).update(page_size=25)

The example uses batches of 1,000 as a starting point for a rehearsal. Size each batch against measured lock time and database load. Make new writers supply a page size before depending on the null count reaching zero. For a deployment with mixed app versions, check that the old code can keep serving while the new version starts writing the populated field. Agree whether rollback leaves those data values in place. A reverse update that clears every row set to 25 could erase an explicit choice.

Rehearse the PostgreSQL rollout

A few SQLite rows can prove the backfill predicate. For a production rollout on PostgreSQL, use the deployed major version, the same migration boundaries, and representative data volume and traffic. Look at the SQL produced by django-admin sqlmigrate for the schema operations and read the RunPython function separately. PostgreSQL takes an ACCESS EXCLUSIVE lock for many ALTER TABLE forms, with documented exceptions. A short statement can still wait behind a long transaction, so record time waiting to acquire a lock as well as time spent executing. 14 16

Use a scoped lock_timeout in the migration connection for the rehearsal and decide what happens if it expires. PostgreSQL applies lock_timeout to waits for each lock acquisition. A statement_timeout covers the whole statement. Check wait events in pg_stat_activity and blocker IDs from pg_blocking_pids while a known conflicting transaction is open. Record whether the migration exited cleanly, how it can restart, and what state remains committed. 15 17 18

For a compatible CHECK constraint, separate adding it as NOT VALID from validating existing rows. The first step skips the existing-row scan while enforcing the constraint for subsequent writes. PostgreSQL documents a SHARE UPDATE EXCLUSIVE lock for the validation step on the table being altered. Django provides AddConstraintNotValid and ValidateConstraint for this sequence. Keep the steps in separate migrations and rehearse them on the target version. 14 20

Store one receipt for each event

A worker receives a billing event, stores a receipt, and reports completion. Delivery can repeat after a timeout or lost response. The expected behavior is one receipt for the same event in the same tenant. The original function inserts a receipt each time, so a test that delivers the event once passes.

Make the event identity part of a database uniqueness constraint. In this example, the source issues stable IDs within a tenant. Include the source or provider in the lookup and constraint if providers have separate ID spaces. Django documents the atomicity of get_or_create when the database enforces uniqueness for the lookup fields. 7 8

# In Receipt.Meta
constraints = [
    models.UniqueConstraint(
        fields=["tenant", "event_id"],
        name="receipt_tenant_event_unique",
    )
]

# In the event handler
receipt, created = Receipt.objects.get_or_create(
    tenant=tenant,
    event_id=event_id,
    defaults={"amount_cents": amount_cents},
)
if receipt.amount_cents != amount_cents:
    raise ValueError("Conflicting amount for the same event")

Send the same event twice and check the committed row count. The SQLite reproduction delivered the event in sequence. The original logic stored two rows, and the revised logic stored one. In the Django application, also verify that the response returns the same receipt identifier. Add a concurrent replay test on the production database and verify a conflict when the same event ID is paired with different meaningful data.

Force two workers through the same decision

A guard that looks for an existing receipt before inserting can pass a sequential replay test. Each worker can still read an empty result before either inserts. Uvik Software reproduced that schedule with two independent autocommit connections to a fresh file-backed SQLite database. A barrier held each original worker just after its lookup returned no row. In the revised version, both workers started at the same gate before an insert protected by a unique index on tenant and event. Both then looked up the stored receipt.

Uvik Software two-worker fixture on Python 3.12.14 and SQLite 3.53.1. The 60 runs use a fresh database per trial and a deliberately synchronized schedule.

In a separate check, the revised path rejected a replay that changed the amount from 2,400 cents to 3,700 cents. The original row and amount were retained. The tests exercise a constructed SQLite schedule. Use PostgreSQL and independent database sessions to verify locking, transaction handling, and the real worker or HTTP response before release.

To reproduce the schedule, make a receipt table with an integer primary key and non-null tenant, event, and amount fields. In the first version, have each connection read the receipt identity and wait at a shared barrier before a plain insert when neither finds it. In the revised version, add a unique index on the identity and use the following insert. After both calls finish, read the count and receipt IDs using a new connection. Recreate the database between trials. 11

CREATE UNIQUE INDEX receipt_identity
ON receipt (tenant_id, event_id);

INSERT INTO receipt (tenant_id, event_id, amount_cents)
VALUES (7, 'evt-123', 2400)
ON CONFLICT (tenant_id, event_id) DO NOTHING;

For a Django concurrency test that checks row locking, use TransactionTestCase and the deployed database engine. Django’s normal TestCase wraps each test in a transaction, which can hide a missing atomic block around select_for_update. SQLite does not implement the row locks Django requests through select_for_update. Use separate connections for the workers and assert final state after both have committed. The PR should show whether the second request gets the existing ID, is rejected for a different payload, or is retried under the application’s transient-error policy. 21

If the handler also changes a balance or creates a downstream job, define which work belongs in the same database transaction. Check the balance or the queued action as well as the receipt. An external payment call or email has its own delivery and retry behavior. A durable outbox can record the intent to send it within the database transaction and give a separate consumer a retryable task. Exercise consumer redelivery independently to verify the external effect. 9 10

Confirm the identity and the side effect

The mutation fixture includes separate providers because each can legitimately use the same event ID. With identity defined as tenant, provider and event ID, two such deliveries should produce two receipts and two business effects. An implementation that omits the provider in its read can silently collapse them even with the correct uniqueness constraint on its writes. Test identity scope by varying each dimension that your provider contract permits to repeat.

For one identity, verify three checkpoints. First, after a successful delivery, read the receipt and business state from the committed database. Second, replay the same meaningful payload and check the same ID and unchanged business state. Third, send the same identity with a different amount and check that the conflict leaves the first state intact. In a concurrent test, run independent sessions and verify those outcomes once all workers have completed. For an external action, collect the receiver’s durable result or its idempotency record before marking the exercise passed. 7 10

What a reviewer should ask the author to show

Start with the behavior the author is changing. Ask for a test that sets up the boundary, makes the request or runs the operation, and asserts the final state. The test has more value if it fails on the original patch and passes after the revision. Keep a regular success-path test so the fix preserves intended use.

Use the existing approval policy. Attach the regression result to the revision being reviewed and name who owns any test that must run outside unit CI. Authorization changes need an endpoint-level test with the application’s real membership rules. Backfills need representative pre-existing data. Retryable handlers need tests at the database and worker boundaries.

Ask the author to include the commit being reviewed, the specific assertion that fails on the original patch, and the same assertion passing on the revised patch. Include the database engine and version when the change depends on transactions or locks. For any check scheduled outside CI, record the person responsible, the environment, and the evidence required before deployment. On a retry-sensitive handler, the useful record includes both responses and the final row or action count.

A team adding a senior engineer to an established Django codebase can use one recent PR as a practical review exercise. Ask the engineer to explain the protected behavior, write the targeted test, and document the rollout evidence. Uvik Software’s Python project takeover guide extends the same evidence-first approach to inherited applications. Its coding agent security benchmark collects published security results for a separate question about coding tools.

Reproduce the database results

Run the accompanying files with Python 3.12.14 and SQLite 3.53.1. The scripts use fresh local databases and Python’s standard library. Keep the SQL file beside its runner.

• python3 check_review.py prints the six paired original and revised results from django_review_checks.sql.
• python3 mutation_fixture.py runs the six selected faults and the reference. It regenerates mutation_results.csv and replay_observations.csv.
• python3 django_receipt_race.py runs the separate 60-trial experiment with independent SQLite connections and confirms the changed-payload rejection.

Use the results to design tests for the application and its production database. The duplicate-delivery fixtures use SQLite’s ON CONFLICT syntax. 11

Download the Django PR review evidence package (ZIP) to reproduce Uvik Software’s SQLite experiments. The archive includes Python scripts, SQL fixtures, CSV results, and a README with instructions for running the checks. Use these files to verify tenant isolation, migration backfills, duplicate-event handling, and the targeted tests that detected six deliberately introduced defects.

Review an existing Django change with Uvik Software

If your team is adding AI-assisted delivery to an existing Django product, bring a recent PR that touches a permission, a migration, or a retryable write. Uvik Software can help scope the regression tests and the first work an embedded senior engineer can own. Talk to Uvik Software’s Django development team.

How to cite this article

Uvik Software. How to Review AI Generated Django Pull Requests. 2026. https://uvik.net/blog/review-ai-generated-django-pull-requests/

For the six deliberate faults, cite Uvik Software’s SQLite mutation fixture and specify the routine and targeted suites. The sequential and two-worker reproductions test separate scenarios. Cite the July 2026 preprint for the coverage findings and the Django and PostgreSQL documentation for framework and database behavior. Send fixture corrections or reproducible examples through the Uvik Software contact form.

Sources

1. Test Coverage Analysis of Agentic Pull Requests, 20 July 2026 arXiv preprint. Dataset, coverage method, and findings in Tables I and II.
2. Django 5.2 documentation, get object or 404. Scoped querysets and lookup behavior.
3. OWASP API Security, Broken Object Level Authorization. Authorization checks on object access.
4. Django 5.2 documentation, RunPython. Historical models and database alias.
5. Django 5.2 documentation, QuerySet update. Bulk update behavior.
6. Django 5.2 documentation, Migrations. Migration state and backend behavior.
7. Django 5.2 documentation, get or create. Database uniqueness and atomicity.
8. Django 5.2 documentation, UniqueConstraint. Composite database uniqueness.
9. Django 5.2 documentation, Database transactions. Transaction behavior and post-commit actions.
10. AWS Prescriptive Guidance, Transactional outbox. Atomic recording of change and delivery intent.
11. SQLite documentation, UPSERT. Handling a uniqueness conflict.
12. Django 5.2 documentation, Test Client force_login. Simulating a logged-in user for an endpoint test.
13. Django 5.2 documentation, Non-atomic migrations. Separate transactions for batches in a data migration.
14. PostgreSQL 18 documentation, ALTER TABLE. Operation lock levels and constraint validation.
15. PostgreSQL 18 documentation, lock_timeout. Per-lock waiting limit and interaction with statement timeouts.
16. Django 5.2 documentation, sqlmigrate. Generated SQL for a migration.
17. PostgreSQL 18 documentation, pg_stat_activity. Server session activity and wait events.
18. PostgreSQL 18 documentation, System Information Functions. Identifying sessions that block a lock request.
19. PostgreSQL 18 documentation, Read Committed isolation. Row predicate recheck after a concurrent update.
20. Django 5.2 documentation, PostgreSQL migration operations. Adding and separately validating a check constraint.
21. Django 5.2 documentation, select_for_update. Backend support and the choice of TransactionTestCase.
22. Alshahwan and colleagues, Automated Unit Test Improvement using Large Language Models at Meta. FSE industry paper, 2024. Filtered test generation and developer review, sections 3.4 and 4.
23. Petrović and colleagues, Practical Mutation Testing at Scale. IEEE Transactions on Software Engineering, 2021. Changed-code mutation testing in code review.