Image Generator using DALL-E

Dall-E is a generative AI model that produces images. It uses deep learning methodologies to generate images based on natural language descriptions.

We can use Dall-E to generate images according to our needs. I have used these Dall-E generated images for my previous blogs too. Being a neural networks-based model, it tries to replicate how a person might think on a topic and gives out an image accordingly.

I have made a chatbot which allows the user to generate images using Dall-E. The user has to put in a description of the image for which the ChatGPT API makes a prompt. If this prompt is aligning to what the user needs in that image, the Dall-E wrapper is used to generate an image for this.

Check Out the Code for this Bot

GitHub - Sayalee-Damle/image-generator: Chatbot which generates image based on the prompt provided…

Pre-Requisites

We will be using Python 3.11 with LangChain and the ChatGPT API. For this we created an Anaconda environment using the command prompt. I have used ChatGPT API: gpt-3.5-turbo-16k-0613. We create a poetry shell for this project. To start the project, use the following commands.

pip install poetry
poetry init
poetry shell
poetry install
poetry build

A ‘pyproject.toml’ and a poetry.lock file will be created which will include all the installed dependencies.

Prompt for Image Generation

The first step towards generation is collecting the description of the user requirements. For image generation, Dall-E should understand the user requirements in depth. The user may not be very specific, so we generate a prompt in-depth. This prompt is generated using the description provided by the user.

The template used for this looks like this.

template_image_description = """prompt which desribes {image_desc} in 20 words"""

The prompt will be generated using this template PromptTemplate.

def prompt_factory() -> str:
    return PromptTemplate(
            input_variables=["image_desc"],
            
            template=t.template_image_description,
        )

This PromptTemplate is passed through the LLM chain to give out the prompt.

async def give_prompt(description: str)-> str:
    chain = await image_chain_factory()
    prompt_generated = await chain.arun(description)
    logger.info(prompt_generated)
    return prompt_generated

Image Generation

The user should check the generated prompt, if it is acceptable, it will be passed to the Dall-E Wrapper for image generation. The Dall-E wrapper can be installed using,

from langchain.utilities.dalle_image_generator import DallEAPIWrapper

The image will be generated using this function.

def generate_image(prompt: str) -> str:
    return DallEAPIWrapper().run(prompt)

ChatBot

This how the ChatBot for image generation looks like.

The bot tells us to describe the image that we wish to generate. I gave an example of ‘a field of sunflower’. The prompt generated is:

‘A vast field of sunflowers, their vibrant yellow petals stretching towards the sun, creating a breathtaking sea of beauty.’

It is a good prompt to generate the image, so I said yes. We get the URL for the image.

This URL is read and opened in the chatbot for the user to preview the image. The URL is also provided to the user if the user wants to download the image later.

For this chatbot I have used Tagging which is used to sense the sentiment of the message given by the user. So, if the user answers ‘yes’ to a question, the bot will sense a ‘positive’ response and take the further steps. Similarly, if it is ‘no’, a ‘negative’ response is logged. If it is neither of the above, it will exit.

A pydantic tagging chain is used in this project.This is the image generated by Dall-E.

If I am not satisfied with the image, a new image will be generated. Let’s try that out.

A new image is generated.

This is a better image which satisfies my requirements.

I have also taken in a scenario where the image generated can be downloaded from the chatbot itself. For this I have used the code given below.

def download_image_file(image_url, image_name):
    img_data = requests.get(image_url).content
    path = cfg.image_path /  f"{image_name}.jpg"
    with open(path, 'wb') as handler:
        handler.write(img_data)

I have given the user a provision to select the name of the image to be downloaded.

A new folder called ‘ImagesDall-E’ will be created which will contain the downloaded image.

Conclusion

Dall-E is a powerful tool used to generate images. You can use this bot just for checking out how Dall-E works and how amazing AI is!