DALL-E 3: Revolutionizing Image Creation and Editing
One of the most controversial aspects of Generative AI is AI-Generated Art. Questions regarding the training data for models like DALL-E 3 have garnered criticism from many in the artistic space. However, as AI tools increasingly become part of our daily lives, the ability to create and customize images with products like DALL-E 3 will become more ingrained. OpenAI has led the way in generative AI products, from their Playground and its APIs to incredible video creation through Sora. The DALL-E 3 model and its powerful features have continued this tradition of leading the way.
I will be covering the following topics in this article:
- What is DALL-E?
- How does DALL-E work?
- How to edit generated images in ChatGPT
- DALL-E 3 Applications
- The future of AI generated art
What is DALL-E?
DALL-E is a text-to-image model developed by OpenAI that allows users to prompt ChatGPT to create any graphic they need. Text-to-image, as the name suggests, means that input text will be used to direct the model about what to create. Since its initial release, DALL-E has seen a host of updates and improvements. The latest of which has brought editing images to the ChatGPT ecosystem. An important reminder is that both the DALL-E 2 and DALL-E 3 can be accessed through the Images API. While API isn’t the focus of this article, it’s important to note that all of these features can be accessed (and brought into existing products) by utilizing its endpoints.
“DALL·E 3 understands significantly more nuance and detail than our previous systems, allowing you to easily translate your ideas into exceptionally accurate images.” — OpenAI Blog
How does DALL-E work?
This answer could be an article of its own, yet it’s important to discuss because it may influence how you interact with the model. The following will be a brief high-level explanation. The DALL-E model was trained on a large database of images and caption pairs. Each caption describes the image it is paired with. This is meant to allow the model to “understand” relationships between images and their text description. More specifically, an image is broken down into a series of “patches”, these patches are similar to how tokens work with text generation. By “understanding” these patches, the model can create and edit images.
An important note on DALL-E 3’s safety policy. Obviously, a tool like DALL-E is powerful and could be used in unsafe ways. With this potential in mind, OpenAI has focused heavily on preventing the creation of violent, adult, and harmful content with their usage policies. Extensive internal testing has also been conducted by OpenAI’s Red Teamers to help stop the model from creating inappropriate content.
How to edit generated images in ChatGPT
While impressive on its own, DALL-E’s latest update might be its biggest yet. The ability to edit generated images allows users to become designers with the click of a button.
We’ll start by asking ChatGPT to create an image of a bear. Something like this will be created.
By clicking on the image, the following screen will appear.
Next, you can select the tool circled above to enter “Editing mode”. This will have you highlight (with a brush tool) the areas of the image that you wish to change. In this example, I wanted to add a small river so I selected the ground before entering my prompt on the right side. If you want to export the full prompt, you can do so by clicking the “Prompt” icon right of the “Edit” icon.
These actions will result in this new edited version of the image:
Application Use Cases
DALL-E is an incredible tool because of the diverse applications it has. In the past, individuals needed to search for (and sometimes purchase) specific images to use in their projects, but now they can generate their own images. This power, combined with the ability to edit these images quickly, means that a significant amount of people now have the power to create their own visual content.
A few specific use cases include:
- Cover Art: Many writers, myself included, use DALL-E or similar tools to have custom images for each article. This process saves significant time and allows people to focus on their main task.
- Education: Teachers and students can now have access to vast amounts of teaching aids when explaining a subject. Learning about Rome, ask ChatGPT to show you what Rome may have looked like at different periods of time.
- Marketing: Using the editing tool, users can create impactful visuals that help tell a story or convey a message. DALL-E can save time and help brainstorm ideas.
- For Fun: While writing this article, I was able to experiment with DALL-E and had a blast. It’s incredibly cool to experience the unique art and styles that can be created.
The future of AI generated art
AI generated art isn’t going anywhere. In fact, it will only become more prevalent in our daily lives as people and companies continue to incorporate this tool into other products. Imagine a homepage that is unique for each user, or a custom built book filled with your art. These types of products will capitalize on DALL-E’s success and ensure that this new art form sticks around. While not the focus of this article, OpenAI’s Image API can be connected to external APIs to create unique experiences. The GPT marketplace is also full of useful tools for creating art with DALL-E. For example, Keymate.AI’s GPT allows users to store prompts to their personal Keymate Memory. This allows users to store and manage all their prompts in a single location. The future of AI will be full of tools that will allow people to create and do things they couldn’t otherwise accomplish.
