SFTT #2: Using Cognito in serverless integration testing | Lumigo
Welcome to the second edition of Serverless from the Trenches, our series of bite-sized blog posts aimed at developers and DevOps working in serverless. Each article will focus on a different technique or tool to solve a real-world problem and — hopefully — help make your work in serverless more productive.
This week we look at how to add Cognito to your integration tests flow, making for true black box testing.
Testing Cognito
We love integration tests here at Lumigo. We see them as a major part of our CI/CD process, and we believe that they play a pivotal role in serverless testing.
One of the test scenarios that we have here is creating an event in our system and then executing an API REST query to retrieve the event details. It’s pure black box testing. Our REST interface sits behind API-GW and is authenticated by Cognito.
If you’ve ever tried to create users in Cognito programmatically, you know that it’s hard. This is because setting the initial password is not enough, the developer needs to change it manually 😨, but manual and automation do not go together.
The following article will cover:
- Our testing scenario and how we incorporate Cognito into it
- Creating a user in Cognito via a python script, including their password 😃
- Actual testing code snippets
Testing scenario
So, before digging into the details, let’s first define our test scenario:
- We need to create a valid Cognito user. At Lumigo the developer uses the same Cognito user to run the integration test and to log into our development dashboard.
- We then embed the username and password created into the integration test configuration.
- In the test itself, we pull the configuration and authenticate it using amazon-cognito-identity-js
Creating a user
One of the hardest things about using Cognito is to create a user with a predefined password, without the need to change it after first login (FORCE_CHANGE_PASSWORD account status).
Luckily, there is a nice Python package called warrant which gives us the ability to play directly with Cognito in Python. Let’s use it to create our user and define its password.
import boto3
from warrant.aws_srp import AWSSRP
from warrant import dict_to_cognito
import click
import random
import string
def admin_create_user(
client,
user_pool_id: str,
username: str,
temporary_password="",
attr_map=None,
**kwargs,
):
"""
Create a user using admin super privileges.
:param username: User Pool username
:param temporary_password: The temporary password to give the user.
Leave blank to make Cognito generate a temporary password for the user.
:param attr_map: Attribute map to Cognito's attributes
:param kwargs: Additional User Pool attributes
:return response: Response from Cognito
"""
response = client.admin_create_user(
UserPoolId=user_pool_id,
Username=username,
UserAttributes=dict_to_cognito(kwargs, attr_map),
TemporaryPassword=temporary_password,
MessageAction="SUPPRESS",
)
response.pop("ResponseMetadata")
return response
@click.command()
@click.argument("cognito_pool_id")
@click.argument("cognito_client_id")
@click.argument("email")
@click.option("--profile", default="default", help="AWS credentials profile")
@click.option("--verbose", default=False, help="Verbose printing", is_flag=True)
def create_user(
cognito_pool_id: str,
cognito_client_id: str,
email: str,
profile: str,
verbose: bool,
):
click.echo("Creating a new user")
click.echo(f"Profile: {profile}")
click.echo(f"Pool ID: {cognito_pool_id}")
click.echo(f"Client ID: {cognito_client_id}")
click.echo(f"Email: {email}")
punctuation = "!#$%&()*+/<=>?@[]^{}~"
tmp_pass = "".join(
random.choices(string.digits + string.ascii_letters + punctuation, k=12)
)
password = f"A!1a{tmp_pass}"
click.echo(f"Password: {password}")
boto3.setup_default_session(profile_name=profile)
client = boto3.client("cognito-idp")
user_details = admin_create_user(
client,
cognito_pool_id,
email,
"Q!w2e3r4",
**{
"email_verified": "True",
"email": email,
},
)
if verbose:
click.echo(f"Creation response:{user_details}")
username = user_details["User"]["Username"]
click.echo("User created successfully")
aws = AWSSRP(
username=username,
password="Q!w2e3r4",
pool_id=cognito_pool_id,
client_id=cognito_client_id,
client=client,
)
token = aws.set_new_password_challenge(password)
if verbose:
click.echo(f"Tokens: {token}")
click.echo(f"Access Token: {token['AuthenticationResult']['AccessToken']}")
if __name__ == "__main__":
try:
create_user()
except Exception as err:
click.echo("Failed!", err=True)
click.echo(str(err), err=True)
exit(1)
Let’s quickly go over it
- We are using click for argument support
- As arguments, we need the user pool ID and client ID
- And you must supply the user’s email that will act as username
- The magic happens in two places:
- Line 66 — Create a Cognito user, this initial creation which leaves the user in FORCE_CHANGE_PASSWORD account status
- Line 88 — Which changes the user’s password to the one we actually want
Embedding the user’s password and username
After creating the user we need to embed it into a configuration file which our integration tests framework uses. For the integration tests we use NodeJS, therefore our best way to pass configuration is to use dotenv. We’ve created a simple prepare_env.sh script which prepares the .env files.
#!/usr/bin/env bash
set -eo pipefail
bold=$(tput bold 2>/dev/null || true)
normal=$(tput sgr0 2>/dev/null || true)
echo ".____ .__ .__ ";
echo "| | __ __ _____ |__| ____ ____ |__| ____ ";
echo "| | | | \/ \| |/ ___\ / _ \ | |/ _ \ ";
echo "| |___| | / Y Y \ / /_/ > <_> ) | ( <_> )";
echo "|_______ \____/|__|_| /__\___ / \____/ /\ |__|\____/ ";
echo " \/ \/ /_____/ \/ ";
echo
echo "Prepare your env file"
function usage() {
cat <<EOM
Usage:
If no parameters are used then local deployment is being chosen.
$(basename $0) [options]
[--env] - Optional. Environment to use. Default is USER environment.
[--region] - Optional. Deploy on this aws region. Default is us-west-2.
[--username] - Optional. User name to use for cognito registration. Default is test_email@nomail.com
EOM
exit 0
}
while [[ $# -gt 0 ]]
do
key="$1"
case $key in
--help)
usage
;;
--env)
opt_env="$2"
shift # past argument
shift # past value
;;
--region)
opt_region="$2"
shift # past argument
shift # past value
;;
--username)
opt_username="$2"
shift # past argument
shift # past value
;;
*)
echo "Unknown argument ${1}. Aborting."
exit 1
esac
done
env=${opt_env:-${USER}}
region=${opt_region:-us-west-2}
username=${opt_username:-test_email_$(date +%s|openssl md5 | tail -c 5)@nomail.com}
echo "Env: ${env}"
echo "Region: ${region}"
echo "Username: ${username}"
user_pool_id=$(aws cloudformation describe-stacks --region ${region} --stack-name ${env}-common-resources|jq -r '.Stacks[0]["Outputs"][]| select(.OutputKey | contains("CognitoUserPoolId"))|.OutputValue')
client_id=$(aws cloudformation describe-stacks --region ${region} --stack-name ${env}-common-resources|jq -r '.Stacks[0]["Outputs"][]| select(.OutputKey | contains("UserPoolClient"))|.OutputValue')
echo "User pool id: ${user_pool_id}"
echo "Client id: ${client_id}"
echo "${bold}Creating a new user${normal}"
# Do not fail if folder does not exist
rm -rf venv || true
virtualenv venv -p python3.7
source ./venv/bin/activate
pip install -r requirements.txt
user_password=$(python create_user.py ${user_pool_id} ${client_id} ${username} |grep Password|awk '{print $2}')
echo "User password: ${user_password}"
echo "${bold}Writing .env file${normal}"
cat > .env <<EOM
USER_NAME=${username}
PASSWORD=${user_password}
EOM
cat .env
echo "Done"
The trick in the script is to automate everything, i.e.
- Line 59 — Generate a random username.
- Line 65,66 — Pull the pool and client ID automatically through AWS CLI
- Line 73 — Creating virtual env and install requirements.txt
Authenticating with the user
After creating the user and embedding it in a .env file, it’s time to use it. Each test has the following structure:
- Authenticate user via Cognito and receive an authentication token
- Use the authentication token in the Authorization header
Let’s look at an example:
const CognitoIdentityServiceProvider = require("amazon-cognito-identity-js");
// Required in order to use the cognito js library to work.
global.fetch = require("node-fetch");
/**
* Authenticate a cognito user and return its authentication token. Use the auth token in the authorization header
* @param callback Callback function with error as first param and the actual user token in the second param.
*/
function authenticateUser(callback) {
console.info("Authenticating user");
const authenticationData = {
Username: userName,
Password: password
};
const authenticationDetails = new CognitoIdentityServiceProvider.AuthenticationDetails(
authenticationData
);
const poolData = {
UserPoolId: userPoolId, // Your user pool id here
ClientId: clientId // Your client id here
};
const userPool = new CognitoIdentityServiceProvider.CognitoUserPool(poolData);
const userData = {
Username: userName,
Pool: userPool
};
const cognitoUser = new CognitoIdentityServiceProvider.CognitoUser(userData);
cognitoUser.authenticateUser(authenticationDetails, {
onSuccess: function(result) {
const token = result.getIdToken().getJwtToken();
console.info(`User token: ${token}`);
callback(null, token);
},
onFailure: function(err) {
callback(err);
}
});
}
- We use amazon-cognito-identity-js
- Usually, amazon-cognito-identity-js is used in the client, therefore, you need to set global.fetch for it to work, see line 3
- Line 34 — at the end we are interested in the token itself which you can use in the Authorization header
To summarize the process:
- Each developer prepares their development environment once by running a script
- The script creates a Cognito user
- The Cognito user is used by the integration tests to receive an authentication token
- Which in turn is used in the authorization header
Tell us how you use Cognito in your integration flow! How do you solve the issues covered in this article? Agree or disagree with our approach? Share your thoughts on Twitter 🙂
Monitor & debug your serverless application effortlessly! Get alerted as soon as an issue occurs and instantly drill down to see a virtual stack trace & correlated logs.
Set up your free Lumigo account today & start fixing serverless issues in a fraction of the time! Find out more
Originally published at https://lumigo.io on May 9, 2019.