Serverless Framework: Developing an API with PyMongo & Mongo Atlas

Garrett Sweeney
Oct 26, 2019 · 7 min read

If you haven’t checked out Serverless Framework, I encourage you to take a look! To reference the documentation,

The Serverless Framework consists of an open source CLI that makes it easy to develop, deploy and test serverless apps across different cloud providers…

This tool has been incredibly easy to work with to deploy some of my python APIs to AWS.

Great, what does the framework need from me?

Before we get into the details of how to create a Serverless API, let’s describe what our code is going to contain:

A serverless.yml configuration file with the following items:

  • A provider — which is just the desired cloud (AWS), runtime (Python 3.7), environment variables, and policies (to write to Cloudwatch)
  • Your functions — the get , list , create , and delete functions.

And a file for each of your functions that contains the actual python and logic.

Let’s create a Serverless API

Start with a new repository and a serverless.yml file

Let’s create a repo to store this new project code.

cd ~/git
mkdir aws-python-rest-api-with-pymongo

Now create a new serverless.yml file:

vi serverless.yml

Step 1 is to define our server name and version.

Append the following to your serverless.yml file:

service: serverless-pymongo-item-apiframeworkVersion: ">=1.1.0 <2.0.0"

Because we’re building a python API that will require dependencies that we normally store in a requirements.txt file, adding the plugin serverless-python-requirements will allow us to create a requirements.txt file like we normally do and the framework will handle the install for us.

Append the following to your serverless.yml file:

plugins:
- serverless-python-requirements

Next, we’ll define the provider block.

The provider block will tell the framework:

  • That we want to deploy to AWS
  • That our runtime is python3.7
  • That we need an IAM Managed policy so our lambda can write logs to CloudWatch
  • That we need to read in our environment variables to use later — for now, just know those are MONGO_DB_USER , MONGO_DB_PASS, MONGO_DB_URL, MONGO_DB_NAME, MONGO_COLLECTION_NAME.

Append the following to your serverless.yml file:

provider:
name: aws
runtime: python3.7
environment:
MONGO_DB_USER: ${env:MONGO_DB_USER}
MONGO_DB_PASS: ${env:MONGO_DB_PASS}
MONGO_DB_NAME: ${env:MONGO_DB_NAME}
MONGO_DB_URL: ${env:MONGO_DB_URL}
MONGO_COLLECTION_NAME: ${env:MONGO_COLLECTION_NAME}
iamManagedPolicies:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"

Finally, we’ll define our functions. In this example, I am defining a create, get , list, and delete function.

The get and delete will have path variables of id so that we can get and delete those items based on id.

The final thing to note is that these functions will be defined in the item folder at the root level of our project. Later, we’ll define these files — but for now you can reference where they are going to be with the handler attribute of each function.

Append the following to your serverless.yml file:

functions:
create:
handler: item/create.create
events:
- http:
path: item
method: post
cors: true
list:
handler: item/list.list
events:
- http:
path: item
method: get
cors: true
get:
handler: item/get.get
events:
- http:
path: item/{id}
method: get
cors: true
delete:
handler: item/delete.delete
events:
- http:
path: item/{id}
method: delete
cors: true

Next Step: Defining our Functions

Before we start creating our function files, let me explain the common parts of each file so they don’t have be explained several times over.

Because each function (file) is deployed as its own lambda, each function needs to connect to the MongoDB database.

Remember how we set MONGO* environment variables? Here’s where we are going to use them. Note that outside of our function methods, we’ll write these lines of code:

# Fetch mongo env vars
usr = os.environ['MONGO_DB_USER']
pwd = os.environ['MONGO_DB_PASS']
mongo_db_name = os.environ['MONGO_DB_NAME']
mongo_collection_name = os.environ['MONGO_COLLECTION_NAME']
url = os.environ['MONGO_DB_URL']
# Connection String
client = pymongo.MongoClient("mongodb+srv://" + usr + ":" + pwd + "@" + url + "/test?retryWrites=true&w=majority")
db = client[mongo_db_name]
collection = db[mongo_collection_name]

Above, we connect to the Mongo Atlas database using PyMongo, and we build the connection string using the environment variables.

That allows us to reference the db and collection, which will later be used to query the database.

Create the functions

Create Function

Since we already explained the connection, the rest should be pretty simple if you have used Mongo before. We’re just attaching a uuid as an id attribute to whatever payload the user provides. Then the insert_one function inserts the item into the collection.

Append the following to item/create.py :

import json
import os
import uuid
import pymongo
# Fetch mongo env vars
usr = os.environ['MONGO_DB_USER']
pwd = os.environ['MONGO_DB_PASS']
mongo_db_name = os.environ['MONGO_DB_NAME']
mongo_collection_name = os.environ['MONGO_COLLECTION_NAME']
url = os.environ['MONGO_DB_URL']
# Connection String
client = pymongo.MongoClient("mongodb+srv://" + usr + ":" + pwd + "@" + url + "/test?retryWrites=true&w=majority")
db = client[mongo_db_name]
collection = db[mongo_collection_name]
def create(event, context):
# get request body
data = json.loads(event['body'])
# create item to insert
item = {
'_id': str(uuid.uuid1()),
'data': data,
}
# write item to database
collection.insert_one(item)
# create response
response = {
"statusCode": 200,
"body": json.dumps(item)
}
# return response
return response

Get Function

From the lambda event object, we fetch the pathParameters of id to determine the id of the object to retrieve from the database.

If you’re actually developing an API, you’ll want some error handling — like returning a 404 if an item is not returned. I actually did this for delete , which you’ll see shortly.

Append the following to item/get.py :

import json
import os
import pymongo
# Fetch mongo env vars
usr = os.environ['MONGO_DB_USER']
pwd = os.environ['MONGO_DB_PASS']
mongo_db_name = os.environ['MONGO_DB_NAME']
mongo_collection_name = os.environ['MONGO_COLLECTION_NAME']
url = os.environ['MONGO_DB_URL']
# Connection String
client = pymongo.MongoClient("mongodb+srv://" + usr + ":" + pwd + "@" + url + "/test?retryWrites=true&w=majority")
db = client[mongo_db_name]
collection = db[mongo_collection_name]
def get(event, context):
# get item_id to delete from path parameter
item_id = event['pathParameters']['id']
# delete item from the database
item = collection.find_one({"_id": item_id})
# create a response
response = {
"statusCode": 200,
"body": json.dumps(item)
}
# return response
return response

List Function

Below I show how to get the query string parameters from the lambda event object as well if you want to query. The rest should be straightforward, using the find method to return all items if no query string parameters are provided.

Append the following to item/list.py :

import json
import os
import pymongo
# Fetch mongo env vars
usr = os.environ['MONGO_DB_USER']
pwd = os.environ['MONGO_DB_PASS']
mongo_db_name = os.environ['MONGO_DB_NAME']
mongo_collection_name = os.environ['MONGO_COLLECTION_NAME']
url = os.environ['MONGO_DB_URL']
# Connection String
client = pymongo.MongoClient("mongodb+srv://" + usr + ":" + pwd + "@" + url + "/test?retryWrites=true&w=majority")
db = client[mongo_db_name]
collection = db[mongo_collection_name]
def list(event, context):
# create response body object
response_body = {}
# create array for reponse items
response_body['response_items'] = []
# return path parameters with filter key
response_body['filter'] = event['multiValueQueryStringParameters']
# build query with any path parameters
query = {}
if event['multiValueQueryStringParameters'] is not None:
for parameter in event['multiValueQueryStringParameters']:
query[parameter] = event['multiValueQueryStringParameters'][parameter][0]
# create list of items
cursor = collection.find(query)
for document in cursor:
response_body['response_items'].append(document)
# create response
response = {
"statusCode": 200,
"body": json.dumps(response_body)
}
# return response
return response

Delete Function

This function is very similar to the get function, but uses delete_one and provides support if an item has already been deleted or does not exist in the database by providing a 404 .

import os
import pymongo

# Fetch mongo env vars
usr = os.environ['MONGO_DB_USER']
pwd = os.environ['MONGO_DB_PASS']
mongo_db_name = os.environ['MONGO_DB_NAME']
mongo_collection_name = os.environ['MONGO_COLLECTION_NAME']
url = os.environ['MONGO_DB_URL']

# Connection String
client = pymongo.MongoClient("mongodb+srv://" + usr + ":" + pwd + "@" + url + "/test?retryWrites=true&w=majority")
db = client[mongo_db_name]
collection = db[mongo_collection_name]


def delete(event, context):
# get item_id to delete from path parameter
item_id = event['pathParameters']['id']

# delete item from the database
del_resp = collection.delete_one({"_id": item_id})

# if no item return 404
if del_resp.deleted_count == 0:

response = {
"statusCode": 404,
}

return response

# create a response
response = {
"statusCode": 204,
}

return response

Create the requirements

Earlier in the article, I said we’ll need a requirments.txt file when we added the plugin. Our dependencies are pymongo and dnspython .

Append the following to your requirements.txt file:

pymongo
dnspython

Create the Mongo Atlas Backend and Configure Environment Variables

If you haven’t created a Mongo Atlas DB before, you can find a step by step guide in Part 1: Cluster Creation of this article.

During this process, you will create:

  • A Mongo DB
  • A URL of the Mongo DB
  • A Mongo Collection
  • A Mongo Cluster Username
  • A Mongo Cluster Password

Before we can deploy the API, you need to set the environment variables on your machine.

Append the following to your ~/.bash_profile and source it.

export MONGO_DB_USER=    
export MONGO_DB_PASS=
export MONGO_DB_NAME=SampleDatabase
export MONGO_COLLECTION_NAME=SampleCollection
export MONGO_DB_URL=

If you followed the article, you will see the DB and DB Collection name created were SampleDatabase and SampleCollection respectively.

Deploy it!

  1. Install Serverless
npm install -g serverless

2. Install serverless-python-requirements locally

npm i — save serverless-python-requirements

3. Deploy the API

sls deploy

When the stack is finished, you should see something like this:

Serverless: Stack update finished...
Service Information
service: serverless-pymongo-item-api
stage: dev
region: us-east-1
stack: serverless-pymongo-item-api-dev
resources: 28
api keys:
None
endpoints:
POST - https://0xfyi15qci.execute-api.us-east-1.amazonaws.com/dev/item
GET - https://0xfyi15qci.execute-api.us-east-1.amazonaws.com/dev/item
GET - https://0xfyi15qci.execute-api.us-east-1.amazonaws.com/dev/item/{id}
DELETE - https://0xfyi15qci.execute-api.us-east-1.amazonaws.com/dev/item/{id}
functions:
create: serverless-pymongo-item-api-dev-create
list: serverless-pymongo-item-api-dev-list
get: serverless-pymongo-item-api-dev-get
delete: serverless-pymongo-item-api-dev-delete
layers:
None
Serverless: Removing old service artifacts from S3...
Serverless: Run the "serverless" command to setup monitoring, troubleshooting and testing.

Test the API

CREATE

curl --request POST \
--url https://0xfyi15qci.execute-api.us-east-1.amazonaws.com/dev/item \
--header 'content-type: application/json' \
--data '{
"attribute_1": "Pet",
"attribute_2": "Rock"
}'

Expected Response

204 status

{
"_id": "c6f03ca0-f792-11e9-9534-260a4b91bfe9",
"data": {
"attribute_1": "Pet",
"attribute_2": "Rock"
}
}

GET

curl --request GET \
--url https://0xfyi15qci.execute-api.us-east-1.amazonaws.com/dev/item/c6f03ca0-f792-11e9-9534-260a4b91bfe9 \
--header 'content-type: application/json'

Expected Response

200 status

{
"_id": "c6f03ca0-f792-11e9-9534-260a4b91bfe9",
"data": {
"attribute_1": "Pet",
"attribute_2": "Rock"
}
}

LIST

curl --request GET \
--url https://0xfyi15qci.execute-api.us-east-1.amazonaws.com/dev/item \
--header 'content-type: application/json'

Expected Response

200 status

{
"response_items": [
{
"_id": "c6f03ca0-f792-11e9-9534-260a4b91bfe9",
"data": {
"attribute_1": "Pet",
"attribute_2": "Rock"
}
},
{
"_id": "717c5f36-f799-11e9-a921-1e0e685be73c",
"data": {
"attribute_1": "Pete",
"attribute_2": "Rock"
}
}
],
"filter": null
}

Delete

curl --request DELETE \
--url https://0xfyi15qci.execute-api.us-east-1.amazonaws.com/dev/item/c6f03ca0-f792-11e9-9534-260a4b91bfe9 \
--header 'content-type: application/json'

Expected Response

204 status

The Startup

Medium's largest active publication, followed by +605K people. Follow to join our community.

Garrett Sweeney

Written by

The Startup

Medium's largest active publication, followed by +605K people. Follow to join our community.

More From Medium

More from The Startup

More from The Startup

More from The Startup

1.5K

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade