Resetting Django Migrations

The Django migration system was designed to deal with huge number of migrations. Typically you shouldn’t mind to keep a significant number of models migrations in your code base. However, you should keep in mind that it may cause certain unintended consequences, such taking considerable time when working with tests. But in circumstances like these you can simply deactivate the migrations.

If you like to make a clean-up, I will provide a few options in this guide.

When you need to modify your database structure but don't want to or can't remove a database table and recreate it, migrations may be quite useful. They are especially handy when you have a production database with tables that contain thousands of rows.

It's not uncommon for a developer to be faced with the need to add, delete, or rename fields in an existing table. It's possible to remove and re-create a table in certain instances, however consider a scenario where your application is already in production with thousands of database rows and you don't have the option of deleting your old tables. Migrations are here to provide you with a more acceptable and competent solution since you can't even bear to think about it.

Why using database migrations

In the case of a production database with tables containing huge amounts of rows, migrations are particularly useful because they allow you to propagate model changes to your database schema. They are particularly useful in the case of a need to change your database structure but you do not want or are unable to drop a database table and recreate it. Anyone who works with databases has come into instances in which he or she has had to make changes to the structure of an existing table by adding, removing, or renaming fields. Although it is possible to solve some problems by dropping the table and re-creating it, this will often result in the developer experiencing additional headaches as a result of the process. However, if your application is already in production and contains millions of database rows, dropping your old tables is not an option. As a result, migrations is here to give you a solution that is more acceptable and professional than what you are now contemplating.

Simply said, migrations allow you to modify the schema of your database while maintaining your data.

When dealing with databases, we often find ourselves having to reset a database because of various reasons such as - We have filled the databse with meaningless information. - We created a database on the basis of a database schema that is prone to errors. - We changed the business logic, which requires us to re-design the whole database structure and so on.

The good news is that these types of situations are rather common in the field of backend web development, and various valuable tools and techniques have been created to help developers deal with them effectively.

Because Django is a framework for building backend web applications, these challenges are typical for django developers, in which we need to reset the migrations and the database. When it comes to resetting migrations, we have many options:

• Return the whole database to its default settings.
• Rollback a Django application to a prior set of migrations

Getting started with migrations

It is simple to get started with migrations, particularly with the most recent versions of Django, which begin with Django 1.7 and beyond. Due to the fact that migrations are incorporated into your Django web development process starting with version 1.7, they are now considered mandatory.

To be able to deal with migrations Django provides you with two important commands:

python manage.py makemigrations  # create migrations files
python manage.py migrate # apply migrations and create actual database tables

You can also show your migrations using the following command:

python manage.py showmigrations #show migrations

How to reset django migrations

Although the Django migrations system is extremely powerful, flexible, and designed to handle large numbers of migrations, having a large number of model migrations causes problems when frequently changing your database structure (which is most often the case during the development phase in most cases) to most django developers, both beginners and even experienced. If you find yourself in this circumstance, there is no other option except to clean up and reset the migrations. But how do you go about resetting Django migrations when things become truly out of hand?

The answer is straightforward, but it is dependent on whether you are still in the development phase or if you are in production, so let us go through the procedure step by step:

Reset Django migrations in development

So you're still in the development phase of the application and it hasn't gone into production. In this particular instance, removing the whole database is not a huge concern. Migrations may be reset in precisely three stages, which are as follows:

1. If you are using a database system such as Posgres or MySQL and you have a database with the name mydb for example, you should begin by deleting it using the SQL language as follows:

delete mydb;

For those of you who are using sqlite, all you have to do is remove the sqlite3 file, which is usually titled db.sqlite3 when working with django.

1. Following that, you must go through all of the migrations folders for each app that is part of your project and remove all of the migrations files with the exception of the __init .py files.
   

This tiresome task may be easily automated if you are working from a command-line interface. As an example:

find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc"  -delete

1. Finally, all that is required is that you run the two migrations commands that you would typically do when synchronizing your models to a new database using Django.

python manage.py makemigrations
python manage.py migrate

Make sure you have created a new database after deleting the old one if you are using any database system other than sqlite.

Deleting the database

Depending on the database system that is being utilized, you may be able to remove the actual database.

Sqlite requires just the deletion of the sqlite3 file, which resides in your current folder and has the extension .sqlite3, in order to be operational.

If you are working with MySQL, you must first establish a connection to your MySQL server using your command line:

mysql -u USERNAME -p PASSWORD 

You can also run the following command:

mysql -u root -p 

Then enter your password when MySQL prompts your for a password.

Next drop the MySQL database and create it again:

drop database YOUR_DATABASE_NAME;
create database YOUR_DATABASE_NAME;

If you want to confirm if the database has been deleted before re-creating it again use the show command

show databases;

For PostgreSQL you need to use psql utility .So first connect to your PostgreSQL database using

psql DBNAME USERNAME       

psql will prompt you for a password, enter it and type Enter.

Now you can drop your database with

DROP DATABASE dbname;

Then create it again

CREATE DATABASE dbname;

Reset Django migrations in production

That's all there is to it for the first scenario, which is while you're still developing your app and aren't yet in production. Because you can't dump your database while your app is in production, the process becomes more complicated. How do you reset migrations in this case?

In a nutshell, we need to maintain the database while removing the migration history from the system. Here is a step-by-step guide on how to do it:

1. The following command should be used to run through each app and remove the migration history for each one.

python manage.py migrate --fake myApp zero

Django will remove any prior migrations that were applied to the specified app (myApp).

1. Next, you'll need to remove the migrations file itself, which means you'll need to go into each app migrations folder and delete everything except for __init .py. This will need you to go through each app migrations folder again. Simply run the following script on any unix-based operating system to get started. Using the power command line in Windows is a viable option:

find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc"  -delete

It searches for Python files (migration files) in the migrations folder of each project app, with the exception of init.py, and deletes them.

The second line searches for Python files in the migrations folder of each project app except for init.py and deletes them.

You may quickly and easily check your migrations by utilizing the following command:

python manage.py showmigrations

1. After that, we'll need to recreate the migrations, so just run the command:

python manage.py makemigrations

Don't forget that the database still contains tables from the first migrations, so what we need to do is migrate our database while also mimicking the initial migrations, which can be accomplished by simply running the following command:

python manage.py migrate --fake-initial     

Using a Django app for resetting migrations

The following Django app, which was recently developed, allows you to reset migrations in a matter of minutes rather than going through all of the procedures above.

The app does all necessary actions, such as removing the migration files, clearing the django migrations database, and then recreating migrations and mimicking old migrations, as well as other tasks.

First, you need to install the app via pip:

pip install django-reset-migrations

Then, add the app in your INSTALLED_APPS under the settings.py file as follows:

    INSTALLED_APPS = (
        # ...
        'reset_migrations',
        # ...
    )

Next, you can simply start using it from the CLI as follows:

python manage.py reset_migration myApp

You can also specify multiple apps for reset as follows:

python manage.py reset_migration myApp myOtherApp

In case you don't want to delete migrations files of your app(s) just add the --cached option:

python manage.py reset_migration myApp myOtherApp --cached  

Manually modifying the migrations

Given that migrations files are just Python scripts, you may want to consider manually updating your migrations to eliminate any potential problems in the future. Don't be frightened of migrations; once you understand how they operate, they are really rather simple to handle. The Django documentation provides excellent explanations on how to write your own migrations.

Merging the migrations

There are occasions when just merging two or more migrations will resolve a problem; in this case, Django will prompt you to do so; however, this method is only effective for basic model updates.

When you appling your migrations, you should consider the following:

python manage.py migrate    

When Django detects that there are conflicts, it will prompt you to run the python manage.py makemigrations –merge command. In order to perform migrations simply follow the directions and run the command as follows:

python manage.py makemigrations –merge

Then, just migrate the database using the following command:

python manage.py migrate

Unfortunately, this only works for simple modifications to your models.

Case 2 example: Dropping the database is not an option

Those are the only steps you need to do in cases when dumping the whole database is not a concern, which is only an option if you are just getting started on your project.

However, this is not always the case; in certain cases, you may not be able to simply delete a database because you want data for testing reasons throughout the development period or when in a production environment. What is the best way to reset your migrations without having them effect your real database?

When we wish to restart migrations, we don't want to drop the database, therefore we use a workaround.

Clear or roll back database migrations

First start by clearing and rolling back migrations without touching the actual migration tables:

python manage.py migrate --fake appName zero

You need to replace appName with the actual app name and to do that for ach app you have in your project.

This command will un-apply all migration files for the specified app.

You can use showmigrations command to see all available migrations and to keep track of what you have cleared.

python manage.py showmigrations

Remove migration files

After faking the migrations for all apps we need to delete the migrations files inside migrations folder in each app.

You can use the previous bash script to automate this process in Unix bases OSs.

find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc"  -delete

This will delete Python source files and also compiled Python files for migrations except for the special Python file init.py

Make migrations again

Now you need to re-create the initial database migrations with the usual commands

python manage.py makemigrations

Next you need to migrate your database again but since your database already exist, we didn’t delete them, remember?! with just need to fake the initial migrations using

python manage.py migrate --fake-initial

If you run migrate without –fake or –fake-initial you are going to get this error:

  django.db.utils.OperationalError: (1050, "Table TABLE_NAME already exists") 

So make sure to add the –fake-initial command switch.

Adding –fake switch marks migrations as run without actually running them.

PS:

Before following any process described above it’s advisable that you make sure you have no un-applied migrations before your proceed by running

python manage.py makemigrations 

You should get a message like this

No changes detected   

Conclusion

So that brings us to the conclusion of this little tutorial; remember, migrations are nothing to be scared of. They are really useful, despite the fact that they may cause you some headaches. However, this is something that you will learn to handle as you get more Django expertise, as well as by understanding how migrations work so that you can avoid causing any conflicts while altering your models.

It's possible that you'll need to reset your Django migrations or just do a clean up at some point. This procedure may be completed quickly and simply in many circumstances, but it can get complicated if you have a large number of migration files and database tables to deal with. The options you learned in this article will allow you to essentially delete or reset your Django database migrations without having to worry about your project's functionality being affected.

• Date: 22 Apr 2022