Creating Personalized ChatGPT with Spring AI — Text to Image

Image Generation AI

In the first two installments of this series, we embarked on a journey into the realm of large language models (LLMs).

In part 1, we unpacked the complexities of these AI marvels trained on massive troves of text and code, and discovered how Spring AI acts as a bridge, allowing Java developers to seamlessly integrate LLMs into their applications. We then put this newfound knowledge into action, constructing a basic chatbot powered by Llama2. Part 2 saw us take things a step further, we customized this chatbot to provide insightful answers to user queries specifically related to a designated PDF document. Spring’s mighty vector store played a pivotal role in achieving this feat.

Beyond Text: Exploring Artistic Co-Creation with Your Chatbot

In this part, to delve deeper into the wonderful world of personalizing our chatbot. We will further enhance our chatbot and make it generate images from your prompts. There a lot of amazing Image generation AI but for this part we will be using Stable Diffusion 3. Stable Diffusion has a free 25 credit so you can follow along with having to worry about money.

🚀 Without Further Ado: Let’s Dive In!

Prerequisites

• Java 17 or new versions of Java
• Stable Diffusion API Key
• Basic knowledge of Spring
• Part 1 of this series
• Part 2 of this series

Let’s Code

1. Let’s add the below dependency in pom.xml to integrate Stability AI easily into our spring application.

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-stability-ai-spring-boot-starter</artifactId>
</dependency>

2. Create a new api key ( or use any existing one ) in your Stability AI account. Copy the key and add the it as a new property in your application.properties( or yml ).

API Key

spring.ai.stabilityai.api-key=YourAPIKey

3. Updates in OllamaAiService.java. Update the constructor to take StabilityAiImageClient and create a new method for generating image from a prompt.

private final ChatClient chatClient;
    private final VectorStore vectorStore;
    private final ImageClient aiImageClient;

    @Autowired
    public LlamaAiServiceImpl(ChatClient chatClient,
               VectorStore vectorStore, 
               ImageClient aiImageClient) {
        this.chatClient = chatClient;
        this.vectorStore = vectorStore;
        this.aiImageClient = aiImageClient;
    }

public InputStream getTextToImage(String prompt, String style) {
        ImageResponse response = aiImageClient.call(
                new ImagePrompt(prompt,
                        StabilityAiImageOptions.builder()
                                .withStylePreset(style)
                                .withN(4)
                                .withHeight(1024)
                                .withWidth(1024).build())

        );
        Image output = response.getResult().getOutput();
        InputStream stream = new ByteArrayInputStream(Base64.getDecoder()
                  .decode(output.getB64Json()));
        return stream;
    }

4. Create a new endpoint in OllamaRestController.java which invokes this new method and return the response.

@RestController
public class OllamaRestController {
   // For previous API endpoints, please check Part 1 and Part 2
   
   @ResponseBody
    @RequestMapping(value = "/generateImage", method = RequestMethod.GET, produces = MediaType.IMAGE_JPEG_VALUE)
    public ResponseEntity<InputStreamResource> generateImage(
    @RequestParam(value = "promptMessage",
     defaultValue = "Generate a random image") String prompt,
     @RequestParam(value = "style",
     defaultValue = "AnimeStyle") String prompt,
    ) {
        InputStream result = llamaAiService.generateImage(prompt, style);
        return ResponseEntity.ok()
                .contentType(MediaType.IMAGE_PNG)
                .body(new InputStreamResource(result));
    }
   
}

5. Let’s run Ollama and bring up our LLM model. If you have installed Ollama then you can run it by the below command

ollama run mistral // i.e. ollama run <model>

If you are running Ollama as docker image then you can use the below command

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 
--name ollama ollama/ollama

Once Ollama is running, test it out by running the below curl

curl --location '<http://localhost:11434/api/generate>' \\
--header 'Content-Type: application/json' \\
--data '{
  "model": "mistral",
  "prompt": "What is AI ?"
 }'

6. Now let’s bring up our Spring app and test the rest endpoints.

curl --location '<http://localhost:8080/generateImage?prompt=Superman%20and%20batman%20having%20a%20drink%20fight%20in%20AnimeStyle'--data> ''

Boom! You should have your own personal Picasso running in your system.

This series has taken us on a whirlwind exploration of building and personalizing chatbots with Spring AI. We started with the fundamentals, then honed our skills by crafting a document-specific information powerhouse. Now, we’ve unlocked the incredible potential of AI image generation, letting your chatbot paint pictures with words!

This is just the tip of the iceberg. As AI technology continues to evolve, the possibilities for chatbot development are truly boundless. Imagine chatbots that can not only converse and create art, but also act as intelligent assistants or even generate personalized learning experiences.

So, what are you waiting for? Dive into the exciting world of Spring AI and start building your own superpowered chatbot today! We can’t wait to see what creative marvels you bring to life.

Happy coding! And to stay updated on all things Spring AI and cutting-edge developments, be sure to follow us!

P.S : If you are curious about the result of the curl from Step 6.

It’s me and you guys after completing this whole series 🤤.