Creating Personalized ChatGPT with Spring AI — Part 1
Greetings, Java developers! The recent release of Spring AI 0.8.0 signifies a landmark moment — the democratization of AI development within the Java ecosystem. This series delves into this exciting initiative, empowering you to leverage AI’s capabilities in your Java projects.
Spring and the AI Revolution
Spring, the cornerstone framework for building robust Java applications, has embraced the transformative power of AI with the introduction of Spring AI. As highlighted in the official announcement on https://spring.io/blog, Spring AI aims to make AI development accessible to a wider range of developers, not just machine learning or AI specialists.
Series Focus:
This series caters to both novice and experienced Java developers. Whether you’re new to AI or seeking to enhance your existing skills, we’ll guide you through:
- Core AI Concepts Demystified: We’ll briefly touch upon different AI concepts used in building the chatbot, ensuring a clear understanding within the context of Java development.
- Spring AI in Action: We’ll showcase how to leverage Spring AI’s functionalities for tasks like creating intelligent chatbots using LLMs like Llama & Mistral. In the next part we will enhance our application to work on our own data set.
We will be using open source LLMs for our app so that everyone can follow along without worrying about api keys or any money.
So without further ado let’s get started.
What are LLMs
Unless you are living under rock you must have heard or read about LLMs. But what exactly are LLMs ?
Large language models (LLMs) represent a significant advancement in the field of natural language processing (NLP) within artificial intelligence (AI). These models are essentially complex algorithms trained on massive datasets of text and code, allowing them to process information and respond in a comprehensive and informative manner.
Key Functionalities:
- Text Generation: LLMs can generate different creative text formats, including poems, code, scripts, and even musical pieces.
- Machine Translation: They offer efficient language translation capabilities, facilitating communication across diverse linguistic barriers.
- Question Answering: LLMs function as powerful information retrieval systems, capable of answering your questions in an informative way by leveraging their vast knowledge base.
Here are some prominent examples of Large Language Models (LLMs) across different organizations:
Bard/Gemini from Google, Llama2 from Meta, GPT3/4 from Open AI and if you go to Hugging Face you’ll find that there are more than 590K more models.
This time seriously without further ado let’s get started.
Prerequisites
- Java 17 or new versions of Java
- Ollama — Ollama is a lightweight and extensible framework specifically designed to run large language models (LLMs) locally on your machine. It can be downloaded from it’s official site or docker lovers can run it as a docker image.
P.S : If you are using mac with M1/M2 chipset then it would be better if you install Ollama app rather than the docker image as you won’t be able to use your GPU cores if you run it as docker image. - Basic knowledge of Spring
Let’s Code
- Create a new spring application and add 3 dependencies, Web, Ollama ( from Spring AI ) and Lombok. You can use Spring initialzr or your own IDE to create the new project. If you want to save some time, I’ve created this sample project in Spring Initializr which you can use. Once created your pom.xml should look like this :
<?xml version="1.0" encoding="UTF-8"?>
...
<properties>
<java.version>17</java.version>
<spring-ai.version>0.8.1</spring-ai.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-ollama-spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
...
</project>2. Create a new java class for storing the app configurations.
@Configuration
public class AIConfiguration {
@Bean
@Primary
OllamaApi ollamaApi() {
return new OllamaApi("http://localhost:11434/");
}
@Bean
@Primary
ChatClient chatClient(OllamaApi ollamaApi) {
return new OllamaChatClient(ollamaApi).withModel("mistral")
.withDefaultOptions(OllamaOptions.create()
.withModel(OllamaOptions.DEFAULT_MODEL)
.withTemperature(0.4f));
}
}Let’s briefly understand what these 2 Beans are
OllamaApi — It acts as a client-side interface for interacting with a remote Ollama server. Also it facilitates sending requests to the Ollama server and receiving responses related to LLM execution.
ChatClient — It acts as a bridge between your Java code and various Large Language Models (LLMs) — the AI engines that power these conversational interactions.
We can also use Spring’s magic to create these two beans by adding the following properties to your application.properties. But I like to have better control our my beans that’s why I’ve created these explicitly.
spring.ai.ollama.base-url=http://localhost:11434/
spring.ai.ollama.chat.options.model=llama23. Create a new service which will use the chatClient and interact with the model.
@Service
public class OllamaAiService{
private final ChatClient chatClient;
@Autowired
public OllamaAiService(ChatClient chatClient) {
this.chatClient = chatClient;
}
// Just for current article we are returning string, in
// the next article we will deal with complex types
public String generateMessage(String promptMessage) {
String chatResponse = chatClient.call(promptMessage);
return chatResponse;
}
public String generateJoke(String topic) {
String chatResponse = chatClient.call(String
.format("Tell me a joke about %s", topic));
return chatResponse;
}
}4. Create a controller which will invoke these service methods and returns the response.
@RestController
public class OllamaRestController {
private final OllamaAiService llamaAiService;
@Autowired
public OllamaRestController(OllamaAiService llamaAiService) {
this.llamaAiService = llamaAiService;
}
@GetMapping("chat")
public ResponseEntity<String> generate(
@RequestParam(value = "prompt",
defaultValue = "Why don't you follow us?")
String prompt) {
String aiResponse = llamaAiService.generateMessage(prompt);
return ResponseEntity.status(HttpStatus.OK).body(aiResponse);
}
@GetMapping("chat/joke/{topic}")
public ResponseEntity<String> generateJoke(@RequestParam(value = "topic",
defaultValue = "Tech")
String topic) {
String aiResponse = llamaAiService.generateJoke(topic);
return ResponseEntity.status(HttpStatus.OK).body(aiResponse);
}
}5. Let’s run Ollama and bring up our LLM model.
If you have installed Ollama then you can run it by the below command
//ollama run <model>
ollama run llama2If you are running Ollama as docker image then you can use the below command
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434
--name ollama ollama/ollamaOnce Ollama is running, test it out by running the below curl
curl --location 'http://localhost:11434/api/generate' \
--header 'Content-Type: application/json' \
--data '{
"model": "llama2",
"prompt": "What is AI ?"
}'6. Now let’s bring up our spring app and test the rest endpoints.
curl --location 'http://localhost:8080/chat?prompt=Will%20AI%20replace%20us%20%3F' \
--data ''
curl --location 'http://localhost:8080/chat/joke?prompt=AI' \
--data ''If everything’s is working as expected ( which never happens on the first try ) then by this time you should be on floor after reading a hilarious joke by your own app.
This is a simple application which can be easily extended by applying some prompt engineering magic. But this can be a homework for you 😛.
Also if you want to use Open AI or any other model please do checkout official page of Spring AI.
Phew!! That concludes the first segment of our series. In this initial part, we carefully guided you through the fundamental steps necessary for setting up a chatbot, using the powerful combination of Spring AI and Language Learning Models (LLMs).
Our primary objective was to provide you with a comprehensive understanding of the basic structure and functionality of such chatbots. We sincerely hope that you found this tutorial not only informative but also easy to follow and implement, thereby making the process of building a chatbot straightforward and enjoyable.
Looking ahead, the upcoming part of this series promises to delve deeper into the intricate aspects of chatbot creation, specifically focusing on how to customize your chatbot to suit your specific needs and preferences. Furthermore, we will also cover the training of your chatbot using our proprietary datasets, a crucial step in ensuring that your chatbot can effectively understand and respond to user inputs.
Stay tuned for Part 2, where we dive into the exciting world of personalizing your chatGPT!
