Building AI Enabled GraphQL Applications

How to use GraphQL to map to AWS Lambda functions that work with AI services.

For the final codebase, check out the GitHub repo here.
Follow me on Twitter to hear about future posts -> https://twitter.com/dabit3

In this post we’ll be building a React Native app that takes user input text, translates it into another language, synthesizes the language to actual speech and plays it back for us!

We’ll be using AWS AppSync as the GraphQL endpoint & AWS Amplify for the GraphQL client.


One of the most powerful things about AWS AppSync is the ability to use Lambda functions as first class endpoints to seamlessly integrate almost any service or database into your GraphQL API without much hassle.

We’ll see that in action in this post.

This app is built with four main parts:

  1. A client application (built with React Native)
  2. An Amazon S3 bucket that holds the translated MP3s
  3. A Lambda function that will handle the GraphQL query, do processing with Amazon Polly & Amazon Translate, place an MP3 into an S3 bucket, then return the id for the created MP3 so we can play it.
  4. The GraphQL API (built with AWS AppSync)

The application flow looks like this:

Enough talking, let’s get building!

React Native app

The React Native app is really made up of a single component. This component handles all of the functionality:

  1. Handling user text input and saving to the state
  2. Handling language input type and saving to the state
  3. Sending GraphQL queries to an API & handling response
  4. Playing audio based on GraphQL API response

The dependencies we’re using are:

To follow along, you can clone the repo:

git clone https://github.com/dabit3/appsync-lambda-ai.git
You can also view the App.js containing all of the functionality at this gist.

Once you’ve downloaded the project, install the dependencies:

yarn || npm install

Creating the S3 bucket

Because this app will be creating MP3s with the translations we will be using, we will need to have an S3 bucket to hold these MP3s.

To create the S3 bucket do the following:

  1. Visit https://s3.console.aws.amazon.com/ and click Create bucket
  2. Give the bucket a name and choose defaults for all other options
  3. Update line 46 in App.js to use the bucket name you just created (replace YOURBUCKETNAME):
const mp3Url = `https://s3.amazonaws.com/YOURBUCKETNAME/${sentence}

Building the Lambda function

Adding & configuring the Lambda function

Now that the React Native project is downloaded, we’ll create the Lambda function.

The Lambda function is where we see most of the action happen! The function takes the user’s sentence & preferred language, translates it, synthesizes it to real speech in the form of an MP3, stores the MP3 in and S3 bucket, then returns the key so we can identify the MP3 and play it back in the app!

  1. In the AWS dashboard, go to the AWS Lambda console
  2. Create a new function by clicking on Create Function
  3. Give the function a name, choose the runtime as Node.js 8.10, choose Create a custom role for the role, give the role a name of lambda_ai_role & click Allow.
  4. Click Create Function
  5. Next, we need to add permissions to the Lambda function in order to access other services (such as S3, Polly, & Translate). To do so, go to the IAM console, go to Roles, find & choose the lambda_ai_role, and add the following policies:
  • AmazonS3FullAccess
  • AmazonPollyFullAccess
  • TranslateReadOnly

Your lambda_ai_role summary should now look like this:

Your Lambda function designer in the dashboard should now look like this if the correct policies were added:

6. Now we need to update the Lambda function with the bucket name we’ll be using. In lambda/index.js on line 8, update the Bucket property with the S3 bucket name you created earlier.

7. Finally, we need to add the actual Lambda function we will be running! In the lambda folder of this project, there is a Lambda function (index.js) & a package.json file. From within this directory, install the dependencies and then zip the folder into a zip file using the following command:

yarn || npm install
zip -r ../translate.zip *

Upload this zip file to AWS Lambda as the function code and click Save.

A look at the Lambda function

Let’s take a look at and discuss the Lambda function we will be using:

  • The first thing we do is import the aws-sdk and create new instances of Translate, Polly, & S3.
  • Next, we create the handler function.
  • The first thing we do in the handler function is create a params object for Translate that will describe the Text & the Language Code we have coming in from the API.
  • Next, we call translate.translateText passing in this params object. This will return the translated text and we set the message variable to that text.
  • Now we will use the language code to get the proper voice to go with the language. Each language has a couple of voices that work properly with the language so we make sure to choose the right voice for the right language.
  • We then create a new params object for Polly containing a few things including the Text & VoiceID.
  • Now we call polly.synthesizeSpeech & pass in the params object we just created.
  • If polly.synthesizeSpeech is successful, we will get a data object returned with an AudioStream property.
  • We use the data.AudioStream along with a unique ID to store the Audio in an S3 bucket using S3.putObject. If the item is stored without error we get a success callback function.
  • If the S3.putObject method is successful, we call the callback of the Lambda function passing in the id of the S3 object (which we will use later to play back the MP3).

Building the GraphQL API

The last step is creating the GraphQL API.

  1. Go to https://console.aws.amazon.com/appsync/ and click on the orange Create API button.
  2. Give the API a name, choose Custom Schema & click create.
  3. Create a new AWS Lambda data source
  • Click Data Sources
  • Click NEW
  • Give the data source a name, choose AWS Lambda Function as the Data source type
  • For the region, choose the region where you created the AWS Lamba function
  • For the Function ARN, choose the Lambda function you just created
  • For the Role, choose New Role

4. Click on Schema and create the following schema:

type TranslatedSentence {
sentence: String!
}
type Query {
getTranslatedSentence(sentence: String!, code: String!): TranslatedSentence
}

5. Add a resolver to the getTranslatedSentence query

  • From within the Schema editor, on the Right (under Data Types), click on Attach next to the getTranslatedSentence field.
  • For the Data Source name, choose the new data source we just created
  • Click Save

6. Add AWS AppSync configuration to the project

Next, we need to edit the aws-exports.js file to specify our AWS AppSync configuration. In the root directory of the React Native project, update aws-exports.js with your AWS AppSync credentials:

const awsmobile = {
'aws_appsync_graphqlEndpoint': 'YOURENDPOINT',
'aws_appsync_region': 'YOURREGION',
'aws_appsync_authenticationType': 'API_KEY',
'aws_appsync_apiKey': 'YOURAPIKEY',
}

7. Run the React Native app

If building for iOS, run the following command from the terminal:

react-native run-ios

If building for Android, first open an Android emulator and then run the following command from the terminal:

react-native run-android

Conclusion

With AWS AppSync you can leverage a single GraphQL endpoint to work with multiple databases, data sources & AWS Lambda functions giving you the power to do almost anything you would like. In this post, we’ve scratched the surface by demonstrating how we can use a GraphQL query to work with an AWS Lambda function to interact with other AWS services such as Amazon Polly, Amazon Translate, & Amazon S3.

To learn more about AWS AppSync, check out this repo.

I encourage you to experiment more and reach out to me to show me cool stuff you are building with this stack!

My Name is Nader Dabit . I am a Developer Advocate at AWS Mobile working with projects like AWS AppSync and AWS Amplify, and the founder of React Native Training.
If you enjoyed this article, please clap n number of times and share it! Thanks for your time.