Text and Image Generation in MuleSoft (The OpenAI Buzz)

OpenAI, an AI research and deployment company, has been in the horizon for a fairly long time now.

With its latest brainchild the ChatGPT, it has caught the world by storm — creating opportunities for making work seem more meaningful, while also injecting plausible concerns regarding the future of work, be it academia or white collar ones.

Context and Ideation

Taking a cue from the OpenAI API, that can be applied to virtually any task that involves understanding or generating natural language or code; today I will discuss the use of the brand new OpenAI Connector for Mule 4.x, that is currently hosted and maintained here, by Dejim Juang - dejimj@gmail.com

OpenAI connector for MuleSoft

Key Terms and Concepts

• Prompt — You input some text as a prompt, and the model will generate a text completion that attempts to match whatever context or pattern you gave it. For example, if you give the API the prompt, “Write a leave application”, it will return a completion leave application, with logical placeholders for you to add specifics. Designing your prompt is essentially how you “program” the model, usually by providing some instructions or a few examples.
• Tokens — The OpenAI models understand and process text by breaking it down into tokens. Tokens can be words or just chunks of characters. One limitation to keep in mind is that your text prompt and generated completion combined must be no more than the model’s maximum context length.
• Models — The available GPT-3 models to use are text-davinci-003, text-curie-001, text-babbage-001, text-ada-001. A limited codex beta model (descendants of the GPT-3 models, that can understand and generate code. Their training data contains both natural language and billions of lines of public code from GitHub) is also available called, code-davinci-002.
  - Read at length about all of them and many more here.

Using the OpenAI MuleSoft connector

• Clone the OpenAI connector repository
• Deploy the connector to your local Maven repo using mvn clean install. — If you are using Maven 3.8.1 or above, you will definitely run into issues with HTTP repositories being blocked, like the one below

To fix this error, you need to define an exception for a specific blocked repository (here, mulesoft-plugin-releases) in your <mirrors> section, in settings.xml, like below

Overriding the Mirror Exception

• This will successfully install the OpenAI connector and you can find it in your Users\{yourUsername}\.m2\repository\com\dejim\openai folder
• Finally add the connector dependency in your mule project’s pom.xml file for using it like any other MuleSoft connector :)

Adding the dependency in pom.xml

Setting up your OpenAI account

1. Login to your OpenAI account at https://platform.openai.com/login
2. On the top right click on your icon and select View API keys

API key Management

3. Click + Create new secret key

4. Once created, copy and store the key somewhere safe, as it will no longer be retrievable once generated and viewed.

Back to familiar territory

Only the OpenAI Config (Configuration) needs to be configured in your global elements, as show below. It needs only the API key from OpenAI that you generated in step 3 above.

Flows involved for achieving the integration

image-generation flow

Here, the Create Image File OpenAI connector operation generates an image file with the DALL-E model (a 12-billion parameter version of GPT-3 trained to generate images from text descriptions, using a dataset of text–image pairs) given a prompt.

The parameter response_format is set to b64_json for this operation and the output is a Base64 string.

Note- The other optional parameters like,

→ n (default 1) — number of images to generate

→ size (default 1024x1024) — dimensions of the generated images

and a few others, are set to their defaults by this connector. You can read more about them here.

The Transform Message component then transforms the Base64 string, obtained from the Create Image File connector, into a binary value, as shown below

which is then written using the File Write connector, to finally obtain the image.

I hit the request from POSTMAN for this image-generation flow, as below

and voila! (I obtained the below image at the File Write path “/Users/Saagnik/Desktop/image.png” using the prompt “laugh” - quite accurate, ain’t it!)

My obtained image (image.png) with the prompt “laugh”

Do share your “prompts” and their “response images”, in the comments!!

text-generation flow

Here, the Create Completion OpenAI connector operation will accept a user prompt and will return a generated text completion, that attempts to match whatever context or pattern (viz., prompt) you gave it. You can read more about prompt design here.

Let’s see the other parameters that this connector expects

• Max Tokens (default 16) — The maximum number of tokens to generate in the completion. Please note that the token count of your prompt plus max_tokens cannot exceed the model's context length.
• Temperature (default 1) — What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
• Model — ID of the model to use from among, text-davinci-003, text-curie-001, text-babbage-001, text-ada-001 or code-davinci-002.

Note- The other optional parameters like,

→ top_p — an alternative to sampling with temperature

→ n — how many completions to generate for each prompt

→ suffix — suffix that comes after a completion of inserted text

→ presence_penalty — number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics

and a few others, are set to their defaults by this connector. You can read more about them here.

I again hit the request from POSTMAN for this text-generation flow and obtain the text completion output on the right, as shown below

Prompt — “Write a letter for bank account closure”

The mule logs are also shown below,

Isn’t that excitingly accurate - how it also ends with precise amicable conclusion and a mention of the attachment details, too!

Do share your “prompts” and their “obtained completions”, in the comments!!

Note — Using OpenAI connector is not free (as expected!) and your account comes with a Free trial usage of $18.00 credits.

Please be mindful and judicious when you use this connector to avoid redundant usage.

Hope you found my article worth your time. Do share your own observations, by tinkering with the parameters and prompts of the OpenAI connector, in the comments. Cheers!

References :

1. OpenAI Connector for Mule 4.x, by Dejim Juang — https://github.com/djuang1/openai-connector
2. OpenAI Completions API Reference Guide — https://platform.openai.com/docs/api-reference/completions/create
3. OpenAI Create Image API Reference Guide — https://platform.openai.com/docs/api-reference/images/create
4. OpenAI Models Overview — https://platform.openai.com/docs/models/overview