Create Django Models from .csv File

 

Hello,

I have mentioned in my previous Tutorial how we can Export a Django Model into a .csv File, in this Tutorial we will do the opposite and we will create a Django Model from .csv file, so let's Get Started!

Note: if you know what to do, just clone the project here.

 Install Django, then create a new project:

django-admin startproject Import_csv

navigate to the project folder:

cd Import_csv

create a new app:

django-admin startapp model_to_csv

Now open the project folder in any IDE and you should have this structure (in my case im using ATOM a free opensource IDE)

 Now before we start writing code we must check if the project works, so let's run it by:

py manage.py runserver

you should get a similar out put saying, Starting developement server at http://127.0.0.1:8000

 Now go to your web browser and open http://127.0.0.1:8000

If you get the default page:

 

Like we saw in the warning message in the console we have 17 unapplied migration(s), this means that our current version of code is not synchronized with our migration files, to fix this we must run the command (after you hit Ctrl+c to stop the server):

py manage.py makemigrations 

then 

py manage.py migrate

you should see a similar output in your console:

 

Now let's run the server again and check for the warning:

 

 and as you can see, it's gone!

now stop the developement server and run this command to create a superuser, a superuser is capable to use all the feautures presented by the Django Administration Panel, so let's create one:

py manage.py createsuperuser

Type the superuser usernamen let's say: admin

  Hit enter, provide an email address or hit enter again to leave it blank, write the admin password twice and hit 'y' if you used a weakpassword and you should see a similar output:

 

 Now let's run the server again and go to 127.0.0.1:8000/admin

type the superuser username and password and click on login

 

if everything went okay you should see this page.

Now that we got a nice Administration Panel, let's create our Model that we're goning to use later in the import_csv view!

before we write our Model we must frist register our app in the INSTALLED_APPS list in settings.py file and we do that by adding to the end of the list:

'model_to_csv',

go to the app model_to_csv and open the file models.py (more on models in the Django official documentation) and copy/past this code:

from django.db import models
class Person(models.Model):
    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=30)

as you can see we have just created a new model, that means we have changed our database structure, everytime we do that we need to run the two commands:

py manage.py makemigrations 

then 

py manage.py migrate

you should see and output like this telling you that we have created a Model called Person:

 

after that you should see the migrate process:

 Now to interact with our model in the Administration Panel we need to register it there, go to model_to_csv/admin.py and add the following code:

from django.contrib import admin
from .models import *
@admin.register(Person)
class PersonAdmin(admin.ModelAdmin):
    pass

now refresh the page in your browser and you should see the Person model added under our app section:

 

 click on Add:

 type whatever you want then save and you should get a success message:

Now that we got everything ready let's prepare a csv file, copy/past this and save it (persons.csv you can save the file anywhere you like but in my case i will save it under the model_to_csv app folder):

first_name,last_name
Mark,Gold
Dave,Silver
Micah,Bronze

Before we create our view we need to first create a folder for all our templates, go to the root directory of the project and create a folder names "templates", under it create a file called persons.html

go to settings.py and modify the TEMPLATES list and replace it by:

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR,'templates')],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

Now let's create a simple view to see a list of our existing Persons Objects, go to views.py under model_to_csv app and copy/past the code:

from django.shortcuts import render
from .models import *
def Persons_View(request):
    persons=Person.objects.all()
    context={'persons':persons}
    return render(request,'persons.html',context)

Now edit persons.html:

<!DOCTYPE html>
<html lang="en" dir="ltr">
  <head>
    <meta charset="utf-8">
    <title></title>
  </head>
  <body>
    <ul>

    </ul>
    {%for person in persons%}
    <li>{{person.first_name}} | {{person.last_name}}</li>
    {%endfor%}
  </body>
</html>

and now we just need to modify the urls.py under the Import_csv directory and copy/past this code:

from django.contrib import admin
from django.urls import path
from model_to_csv import views
urlpatterns = [
    path('admin/', admin.site.urls),
    path('', views.Persons_View),
]

this means when we first visit the 127.0.0.1:8000 address we will see our template file "persons.html":

 Now let's create a simple form to add Persons then we will add an upload filed option, go udner the folder app model_to_csv and create a file named forms.py and add the following code to it:

from .models import *
from django import forms
class PersonForm(forms.ModelForm):
    class Meta:
        model = Person
        fields = ['first_name','last_name']

now edit views.py like this:

from django.shortcuts import render
from .models import *
from .forms import *
def Persons_View(request):
    persons=Person.objects.all()
    form=PersonForm()
    context={'persons':persons,'form':form}
    return render(request,'persons.html',context)

 now edit persons.html like this:

<!DOCTYPE html>
<html lang="en" dir="ltr">
  <head>
    <meta charset="utf-8">
    <title></title>
  </head>
  <body>
    <h5>Add Person</h5>
    <form class="" action="" method="POST">
      {% csrf_token %}
      {{form}}
      <input type="submit" name="" value="Add">
    </form>
    <ul>
    {%for person in persons%}
    <li>{{person.first_name}} | {{person.last_name}}</li>
    {%endfor%}
      </ul>
  </body>
</html>

if you get a similar view that means everything is fine:

but until now we didn't tell our views.py (logic tier) to save anything sent from the form so we will do it now, edit views.py as follows:

from django.shortcuts import render
from .models import *
from .forms import *
def Persons_View(request):
    if request.method == 'GET':
        persons=Person.objects.all()
        form=PersonForm()
        context={'persons':persons,'form':form}
    if request.method == 'POST':
        form=PersonForm(request.POST)
        if form.is_valid():
            form.save()
        context={'persons':persons,'form':form}
    return render(request,'persons.html',context)

This is not the best way of doing things but to keep it simple if we receive a GET request we will send a form with a list of persons object to our template, but if we get a POST request we will assign the sent data to a form, then check if all fields are valid and finally save it if its valid else we will return the form data to the user.

type whatever you want into the two fileds and click send, you should see that the page is refreshed and you get the new Person in the below list:

Now that we got a working form, we will modify the code to add the import from a .csv file function, edit views.py as follows (we will replace the old POST block with a new one):

FINAL VERSION:

from django.shortcuts import render, redirect
from .models import *
from .forms import *
import io,csv
from django.http import HttpResponse
def Persons_View(request):
    persons=Person.objects.all()
    form=PersonForm()
    if request.method == 'POST':
        form = PersonForm(request.POST)
        csv_file = request.FILES['file']
        if not csv_file.name.endswith('.csv'):
            messages.error(request, 'THIS IS NOT A CSV FILE')
        data_set = csv_file.read().decode('UTF-8')
        io_string = io.StringIO(data_set)
        next(io_string)
        for column in csv.reader(io_string, delimiter=',', quotechar="|"):
            _, created = Person.objects.update_or_create(
                first_name=column[0],
                last_name=column[1]
            )
        if form.is_valid():
            form.save()
    context={'persons':persons,'form':form}
    return render(request,'persons.html',context)

Now edit the HTML file to add an upload file field: FINAL VERSION

<!DOCTYPE html>
<html lang="en" dir="ltr">
  <head>
    <meta charset="utf-8">
    <title></title>
  </head>
  <body>
    <h5>Add Person</h5>
    <form class="" action="" method="POST" enctype="multipart/form-data">
      {% csrf_token %}
      <label for="file1"> Upload a file</label>
      <input type="file" id="file1" name="file">
      {{form}}
      <input type="submit" name="" value="Add">
    </form>
    <ul>
    {%for person in persons%}
    <li>{{person.first_name}} | {{person.last_name}}</li>
    {%endfor%}
    </ul>
  </body>
</html>

Navigate to 127.0.0.1:8000:

 

Now click on Add and you should see that the content of the csv file is added to the list:

 

 

 And that's it! we did it! if you want to clone the full version of the code check it out on Github, gave it a Star if it was helpful!

Project Link: Here

My Github Profile: ahmedsmaali5

if you need any help with the code just post a comment here!

Have a nice day!