Intelligent Document Processing With AWS AI Services and MuleSoft RPA
Introduction
Insurance claims, invoices, various forms and applications — every organization deals with a wide range of unstructured documents. Although manual extraction of critical information is still common, many departments have automated these processes.
Amazon Textract, a machine learning (ML) based service, goes beyond simple optical character recognition (OCR) to identify and extract data from scanned files.
Today, I will demonstrate how to use MuleSoft RPA, the AWS Intelligent Document Processing (AWS IDP) component in MuleSoft RPA Builder, and Amazon Textract together. We will fully automate the process of reading an email, processing its attachment, and using its data to create a new record in a CRM system.
The Process
I started by creating a Process Automation (bot) to read emails and save the attached Invoice.pdf in a specific folder. My bot will extract data stored in the invoice, including the vendor’s name, contact information, invoice number, and purchased items (description, quantity, and price). To be precise, the bot will call the AWS IDP component in MuleSoft RPA Builder to do the job. Once the data is extracted, the bot will use the information to create and populate a new invoice record in Zoho CRM.
Note: For the sake of simplicity, I am not explaining how to create Zoho CRM and AWS accounts. You can use both accounts with a free trial period.
Bot tasks
I have added the following RPA Bot Tasks to the process automation:
- Read emails : read emails and save the attached PDFs in a specific directory (to Process)
- Log into Zoho CRM
- New invoice: navigate to the invoices page
- Extract PDF data using AWS: connect to AWS Service to extract data from each PDF file saved to the “To Process” directory
- Enter invoice details: create and enter the invoice details
- Move the file to the “Completed” folder: if there are more files to be read, go to step 3, “New invoice”
- Sign out of Zoho
Here is the format of the document I am using for testing, Invoice.pdf.
Extract Data From PDF Using the AWS IDP Component
ERP Docs With AWS
The ERP Docs With AWS component uses the Amazon Textract service to analyze invoices and receipts and extract relevant data in JSON format.
Note: You must have an AWS Account.
The configuration is simple and requires the AWS Client ID, Client Secret, Region Endpoint, and the file to analyze. The supported file types are PDF, JPEG, and PNG.
The AWS Service has three types of responses:
- Raw Text: the extracted raw text of the document
- Normalized JSON: the parsed result based on the normalized result and defined JSON path
- Raw JSON: the complete AWS response
JSON Query
The “JSON Query” component extracts data from JSON with JSONPath Expressions. I used this online tool to create and test the expressions.
By setting JSON Object as Normalized JSON, the AWS Service response will look like this:
[
{
"ExpenseIndex": 1,
"PageNumber": 1,
"SummaryFields": [
{
"Type": "VENDOR_NAME",
"Value": "RPA Mule Company",
"Label": "",
"Confidence": 99.923393249511719
},
{
"Type": "OTHER",
"Value": "Josephine Darakjy (Sample)",
"Label": "Payable to",
"Confidence": 92.0
},
{
"Type": "OTHER",
"Value": "Chanay (Sample)\n4B Blue Ridge Blvd\nBrighton, MI, 48116, United States",
"Label": "Invoice for",
"Confidence": 77.0
},
{
"Type": "OTHER",
"Value": "Project name 1",
"Label": "Project",
"Confidence": 64.0
},
{
"Type": "OTHER",
"Value": "29/09/2022",
"Label": "Submitted on",
"Confidence": 48.5
},
{
"Type": "TOTAL",
"Value": "£15,540.00",
"Label": "Total",
"Confidence": 99.946990966796875
},
{
"Type": "INVOICE_RECEIPT_ID",
"Value": "123456",
"Label": "Invoice #",
"Confidence": 99.971229553222656
},
{
"Type": "SUBTOTAL",
"Value": "£15,540.00",
"Label": "Subtotal",
"Confidence": 99.975227355957031
},
{
"Type": "DUE_DATE",
"Value": "14/10/2022",
"Label": "Due date",
"Confidence": 98.978599548339844
},
{
"Type": "OTHER",
"Value": "£0.00",
"Label": "Adjustments",
"Confidence": 77.353431701660156
}
],
"LineItemGroups": [
{
"ColumnName": "Description",
"Type": "ITEM",
"TypeConfidence": 70.0,
"CellContents": [
"Google Pixel 6a",
"OnePlus Nord CE 2 5G",
"OnePlus 10 Pro",
"Samsung Galaxy S22 Ultra"
]
},
{
"ColumnName": "Qty",
"Type": "QUANTITY",
"TypeConfidence": 70.0,
"CellContents": [
"5",
"2",
"3",
"7"
]
},
{
"ColumnName": "Unit price",
"Type": "OTHER",
"TypeConfidence": 70.0,
"CellContents": [
"£720.00",
"£320.00",
"£850.00",
"£1,250.00"
]
},
{
"ColumnName": "Total price",
"Type": "PRICE",
"TypeConfidence": 70.0,
"CellContents": [
"£3,600.00",
"£640.00",
"£2,550.00",
"£8,750.00"
]
},
{
"ColumnName": "",
"Type": "EXPENSE_ROW",
"TypeConfidence": 99.990859985351562,
"CellContents": [
"Google Pixel 6a 5 £720.00 £3,600.00",
"OnePlus Nord CE 2 5G 2 £320.00 £640.00",
"OnePlus 10 Pro 3 £850.00 £2,550.00",
"Samsung Galaxy S22 Ultra 7 £1,250.00 £8,750.00"
]
}
]
}
]
Below are some examples of evaluating and getting the values for Vendor Name, Project and Invoice Number.
- Vendor Name: $..SummaryFields[?(@.Type==’VENDOR_NAME’)].Value
- Project: $..SummaryFields[?(@.Label==’Project’)].Value
- Invoice Number: $..SummaryFields[?(@.Type==’INVOICE_RECEIPT_ID’)].Value
Once I evaluated the values, I saved them in their respective variables.
Next, the bot logs into Zoho CRM to create a new invoice using Click Web Element and Keystrokes components.
I have recorded every step of this exercise — executing the bot, sending the Invoice.pdf to AWS Service and, extracting the data from the returned JSON, and opening and creating a new Invoice in Zoho CRM.
What do you think? What was your experience using AWS components in MuleSoft RPA Builder? Let me know in the comments.
I hope you liked this post.
Originally published at https://blogs.infomentum.com.