Python Diary: Creating a transparently encrypted field in Django

This is officially PythonDiary's first Python 3 article! Python 2 is now officially dead, so there's less reasons to make that a major focus going forward.

In some rare situations you may wish to have data which may otherwise be visible on the Django site, or through the Django admin, but may wish to have this data transparently encrypted into the database. This could be very useful, if for example, you use an untrusted database where it is not managed by you, and some database administrator can indeed either dump the data, or otherwise view the stored schemas. This is common with managed databases, which are either maintained by a hosting provider, or is shared with other tenants. In this current day and age with many database breaches appearing in the news from large vendors, you can never be 100% sure that the data you save into your database will never be leaked.

Django supports custom fields on your database models, and the various CRUD and model services Django provides will use these fields with ease, making the creation of a globally transparently encrypted field possible. First lets start with the creation of the custom Django field to explain how that works first.

from django.db.models.fields import CharField
import cipher

class EnField(CharField):
    def from_db_value(self, value, expression, connection):
        """ Decrypt the data for display in Django as normal. """
        return cipher.decrypt(value)
    def get_prep_value(self, value):
        """ Encrypt the data when saving it into the database. """
        return cipher.encrypt(value)

As you can see here, it is really straightforward to extend an existing field, such as CharField. I choose CharField in this example, as it tends to render easily everywhere in the Django framework with ease, so it is the most straightforward to test this concept with. You may also wish to use the binhex module's base64 encoding, but most databases should allow binary data to be stored into a VARCHAR. You may also opt to use a binary field as well. It is also the simplest when it comes to playing with a model in the Python shell. Next, let's see how all the magic works in the cipher module.

from Crypto.Cipher import AES
from django.conf import settings
import hashlib, random

def __random5():
    """ Generate a random sequence of 5 bytes for use in a SHA512 hash. """
    return bytes(''.join(map(chr,random.sample(range(255),5))), 'utf-8')

def __fill():
    """ This is used to generate filler data to pad our plain text before encryption. """
    return hashlib.sha512(__random5()).digest()

def __cipher():
    """ A simple constructor we can call from both our encrypt and decrypt functions. """
    key=hashlib.sha256(bytes(settings.SECRET_KEY, 'utf-8')).digest() # Key is generated by our SECRET_KEY in Django.
    return AES.new(key) # Here you should perhaps use MODE_CBC, and add an initialization vector for additional security.  ECB is the default, and isn't very secure.

def encrypt(data):
    """ The entrypoint for encrypting our field. """
    FILL=__fill()+__fill()+__fill() # This is used to generate filler so we can satisfy the block size of AES.  It is best to pad with random data, than to pad with say nulls.
    return __cipher().encrypt(bytes(data, 'utf-8')+b'|'+FILL[len(data)+1:])

def decrypt(data):
    """ Entrypoint for decryption """
    return __cipher().decrypt(data).split(b'|')[0].decode('utf-8')

Pretty neat, huh? Feel free to change how the filler is generated, the cipher being used, and of course the options passed to the cipher to further customize this solution. I have tested this using Python 3.5, and Django 2.2.9. Although, it should work on future versions of both Python and Django. For obvious reasons, you should be using caching on your Django site if you plan on displaying these fields on your front-end. The best part about doing this as a field, rather than placing this code in your model, through a signal, or in a form, is that it 100% works in the Django admin, and any other place you may reference this field within your Django codebase. This is an interesting example, of when to use a custom model field in Django, rather than adding the logic to the model or the form.

from Planet Python
via read more