Stack Abuse: Integrating MongoDB with Flask Using Flask-PyMongo

Introduction

Building a web app almost always means dealing with data from a database. There are various databases to choose from, depending on your preference.

In this article, we shall be taking a look at how to integrate one of the most popular NoSQL databases - MongoDB - with the Flask micro-framework.

They are several Flask extensions for integrating MongoDB, here we'll be using the Flask-PyMongo extension.

We will also be working on a simple Todo-List API to explore the CRUD capabilities of MongoDB.

Setup and Configuration

To follow along with this tutorial, you will need access to a MongoDB instance, You can get one from MongoDB Atlas or you could use a local instance. We will be using a local instance on our own personal machine.

To install a local instance of MongoDB, head over to their official documentation website for instructions on how to download and install it.

You will also need to have Flask installed, and if you don't, you can do so with the following command:

$ pip install flask

Next we need to set up Flask-PyMongo, which is a wrapper around the PyMongo python package.

PyMongo is a low-level wrapper around MongoDB, it uses commands similar to MongoDB CLI commands for:

1. Creating data
2. Accessing data
3. Modifying data

It doesn't use any predefined schema so it can make full use of the schemaless nature of MongoDB.

To begin using Flask-PyMongo, we need to install it with the following command.

$ pip install Flask-PyMongo

Now that we are all set, let us get started integrating MongoDB into our Flask app.

Connecting to a MongoDB Database Instance with Flask

Before we actually perform any work, we want to connect our MongoDB instance to the Flask application. We'll start off' by importing Flask and Flask-PyMongo into our app:

from flask_pymongo import PyMongo
import flask

Next we'll create a Flask app object:

app = flask.Flask(__name__)

Which we'll then use to initialize our MongoDB client. The PyMongo Constructor (imported from flask_pymongo) accepts our Flsk app object, and a database URI string.

This ties our application to the MongoDB Instance:

mongodb_client = PyMongo(app, uri="mongodb://localhost:27017/todo_db")
db = mongodb_client.db

The URI string could also be assigned to the key MONGO_URI in app.config

app.config["MONGO_URI"] = "mongodb://localhost:27017/todo_db"
mongodb_client = PyMongo(app)
db = mongodb_client.db

Once the application has a connection to the instance, we can start implementing the CRUD functionality of the application.

Create Documents - Adding New Items to the Database

MongoDB works with collections, which are analogous to the regular SQL table. Since we're making a TODO list app, we'll have a todos collection. To reference it, we use the db object. Each entity is a document, and a collection is really, a collection of documents.

To insert a new entry into our todos collection, we use the db.colection.insert_one() method. MongoDB works naturally with Python given its syntax for insertion, querying and deletion.

When inserting a document into a MongoDB collection, you'd specify a dictionary with <field>s and <value>s. To insert a document into a MongoDB collection using Python as the middleman, you'll pass in dictionaries that are built-in into Python.

Thus, to insert a new entity, we'll do something along the lines of:

@app.route("/add_one")
def add_one():
    db.todos.insert_one({'title': "todo title", 'body': "todo body"})
    return flask.jsonify(message="success")

We could also add multiple entries at once using the db.colection.insert_many() method. The insert_many() method take a list of dictionaries and adds them to the collection:

@app.route("/add_many")
def add_many():
    db.todos.insert_many([
        {'_id': 1, 'title': "todo title one ", 'body': "todo body one "},
        {'_id': 2, 'title': "todo title two", 'body': "todo body two"},
        {'_id': 3, 'title': "todo title three", 'body': "todo body three"},
        {'_id': 4, 'title': "todo title four", 'body': "todo body four"},
        {'_id': 5, 'title': "todo title five", 'body': "todo body five"},
        {'_id': 1, 'title': "todo title six", 'body': "todo body six"},
        ])
    return flask.jsonify(message="success")

If we try and add a duplicate record, a BulkWriteError will be thrown, meaning that only records up to said duplicate will be inserted, and everything after the duplicate will be lost, so keep this in mind when trying to insert many documents.

If we want to insert only valid and unique records in our list, we will have to set the ordered parameter of the insert_many() method to false and then catch the BulkWriteError exception:

from pymongo.errors import BulkWriteError

@app.route("/add_many")
def add_many():
    try:
        todo_many = db.todos.insert_many([
            {'_id': 1, 'title': "todo title one ", 'body': "todo body one "},
            {'_id': 8, 'title': "todo title two", 'body': "todo body two"},
            {'_id': 2, 'title': "todo title three", 'body': "todo body three"},
            {'_id': 9, 'title': "todo title four", 'body': "todo body four"},
            {'_id': 10, 'title': "todo title five", 'body': "todo body five"},
            {'_id': 5, 'title': "todo title six", 'body': "todo body six"},
        ], ordered=False)
    except BulkWriteError as e:
        return flask.jsonify(message="duplicates encountered and ignored",
                             details=e.details,
                             inserted=e.details['nInserted'],
                             duplicates=[x['op'] for x in e.details['writeErrors']])

    return flask.jsonify(message="success", insertedIds=todo_many.inserted_ids)

This approach will insert all of the valid documents into the MongoDB collection. Additionally, it'll log the details of the failed additions and print it back to the user, as a JSON message.

We've done this via Flasks' jsonify() method, which accepts a message we'd wish to return, as well as additional parameters that let us customize it for logging purposes.

Finally, we return the successful inserts, in much the same way.

Read Documents - Retrieving Data From the Database

Flask-PyMongo provides several methods (extended from PyMongo) and some helper methods for retrieving data from the database.

To retrieve all the documents from the todos collection, we'll use the db.collection.find() method.

This method will return a list of all the todos in our database. Similar to find(), the find_one() method returns one document, given its ID.

Let's start out with find():

@app.route("/")
def home():
    todos = db.todos.find()
    return flask.jsonify([todo for todo in todos])

The find() method can also take an optional filter parameter. This filter parameter is represented with a dictionary which specifies the properties we're looking for. If you've worked with MongoDB before, you'll probably be familiar with how their queries and comparators look like.

If not, here's how we can use Python's dictionary to accomodate the MongoDB query format:

# Query document where the `id` field is `3`
{"id":3}

# Query document where both `id` is `3` and `title` is `Special todo`
{"id":3, "title":"Special todo"}

# Query using special operator - Greater than Or Equal To, denoted with
# the dollar sign and name ($gte)
{"id" : {$gte : 5}}

Some other special operators include the $eq, $ne, $gt, $lt, $lte and $nin operators.

If you're unfamiliar with these, a great place to learn more about them is the official documentation.

Now that we've covered specifying MongoDB queries for filtering the find() method, let's take a look at how to retrieve one document, given its _id:

@app.route("/get_todo/<int:todoId>")
def insert_one(todoId):
    todo = db.todos.find_one({"_id": todoId})
    return todo

So if we we were to send a GET request to http://localhost:5000/get_todo/5, we'd get the following result:

{
    "_id": 5,
    "body": "todo body six",
    "title": "todo title six"
}

Note that 5000 is the default Flask server port, but it can be easily changed while creating a Flask app object

Most times we would want to get an item or return a 404 error if the item was not found.

Flask-PyMongo provides a helper function for this, the find_one_or_404() method which will raise a 404 error if the requested resource was not found.

Update and Replace Documents

To update entries in our database, we may use the update_one() or the replace_one() method to change the value of an existing entity.

replace_one() has the following arguments:

1. filter - A query which defines which entries will be replaced.
2. replacement - Entries that will be put in their place when replaced.
3. {} - A configuration object which has a few options, of which well be focusing on - upsert.

upsert, when set to true will insert replacement as a new document if there are no filter matches in the database. And if there are matches, then it puts replacement in its stead. If upsert if false and you try updating a document that doesn't exist, nothing will happen.

Let's take a look at how we can update documents:

@app.route("/replace_todo/<int:todoId>")
def replace_one(todoId):
    result = db.todos.replace_one({'_id': todoId}, {'title': "modified title"})
    return {'id': result.raw_result}

@app.route("/update_todo/<int:todoId>")
def update_one(todoId):
    result = db.todos.update_one({'_id': todoId}, {"$set": {'title': "updated title"}})
    return result.raw_result

So if we were to send a request to http://localhost:5000/update_todo/5, we'd get the following result:

{
    "id": {
        "n": 1,
        "nModified": 1,
        "ok": 1.0,
        "updatedExisting": true
    }
}

Similarly, if we were too send a request to http://localhost:5000/replace_todo/5, we'd get the following result:

{
    "id": {
        "n": 1,
        "nModified": 1,
        "ok": 1.0,
        "updatedExisting": true
    }
}

The code block will return an UpdatedResult object, which can be a tad tedious to work with. Which is why Flask-PyMongo provides more convenient methods such as find_one_and_update() and find_one_and_replace() - that will update an entry and return that entry:

@app.route("/replace_todo/<int:todoId>")
def replace_one(todoId):
    todo = db.todos.find_one_and_replace({'_id': todoId}, {'title': "modified title"})
    return todo

@app.route("/update_todo/<int:todoId>")
def update_one(todoId):
    result = db.todos.find_one_and_update({'_id': todoId}, {"$set": {'title': "updated title"}})
    return result

So now, if we were to send a request to http://localhost:5000/update_todo/5, we'd get the following result:

{
    "_id": 5,
    "title": "updated title"
}

Similarly, if we were too send a request to http://localhost:5000/replace_todo/5, we'd get the following result:

{
    "_id": 5,
    "title": "modified title"
}

Flask-PyMongo also allows bulk updates with the update_many() method:

@app.route('/update_many')
def update_many():
    todo = db.todos.update_many({'title' : 'todo title two'}, {"$set": {'body' : 'updated body'}})
    return todo.raw_result

The above code block will find and update all entries with the title "todo title two" and results in:

Sending a request to our newly-made enpoints returns the following result:

{
    "n": 1,
    "nModified": 1,
    "ok": 1.0,
    "updatedExisting": true
}

Deleting Documents

As with the others, Flask-PyMongo provides methods for deleting a single or a collection of entries using the delete_one() and the delete_many() methods respectively.

This method's arguments are the same as with the other methods. Let's take a look at an example:

@app.route("/delete_todo/<int:todoId>", methods=['DELETE'])
def delete_todo(todoId):
    todo = db.todos.delete_one({'_id': todoId})
    return todo.raw_result

This will search for and delete the entry with the provided ID. If we sent a DELETE request like so http://localhost:5000/delete_todo/5 to this endpoint, we would get the following result:

{
    "n": 1,
    "ok": 1.0
}

You can alternatively use the find_one_and_delete() method that deletes and returns the deleted item, to avoid using the unhandy result object:

@app.route("/delete_todo/<int:todoId>", methods=['DELETE'])
def delete_todo(todoId):
    todo = db.todos.find_one_and_delete({'_id': todoId})
    if todo is not None:
        return todo.raw_result
    return "ID does not exist"

Sending http://localhost:5000/delete_todo/8 to our server now results in:

{
    "_id": 8,
    "body": "todo body two",
    "title": "todo title two"
}

Finally, you can delete in bulk, using the delete_many() method:

@app.route('/delete_many', methods=['DELETE'])
def delete_many():
    todo = db.todos.delete_many({'title': 'todo title two'})
    return todo.raw_result

Sending http://localhost:5000/delete_many to our server will result in something similar to:

{
    "n": 1,
    "ok": 1.0
}

Saving and Retrieving Files

MongoDB allows us to save binary data to its database using the GridFS specification.

Flask-PyMongo provides the save_file() method for saving a file to GridFS and the send_file() method for retrieving files from GridFS.

Let us begin with a route to upload a file to GridFS:

@app.route("/save_file", methods=['POST', 'GET'])
def save_file():
    upload_form = """<h1>Save file</h1>
                     <form method="POST" enctype="multipart/form-data">
                     <input type="file" name="file" id="file">
                     <br><br>
                     <input type="submit">
                     </form>"""
                     
    if request.method=='POST':
        if 'file' in request.files:
            file = request.files['file']
            mongodb_client.save_file(file.filename, file)
            return {"file name": file.filename}
    return upload_form

In the above code block, we created a form to handle uploads and return the file name of the uploaded document.

Next let's see how to retrieve the file we just uploaded:

@app.route("/get_file/<filename>")
def get_file(filename):
    return mongodb_client.send_file(filename)

This code block will return the file with the given filename or raise a 404 error if the file was not found.

Conclusion

The Flask-PyMongo extension provides a low-level API (very similar to the official MongoDB language) for communicating with our MongoDB instance.

The extension also provides several helper methods so we can avoid having to write too much boilerplate code.

In this article, we have seen how to integrate MongoDB with our Flask app, we have also performed some CRUD operations, and seen how to work with files with MongoDB using GridFS.

I have tried to cover a much as I can but if you have any questions and/or contributions please do leave a comment below.

from Planet Python
via read more