Spring AI Tutorial (with Examples)

Large Language Models (LLMs) like GPT-4, Claude, Gemini, and Mistral that power Generative AI applications like ChatGPT have evolved and increased in number, and have accelerated a new class of intelligent applications and making existing applications more intelligent. People who make decisions for businesses are trying hard to understand how to use these intelligent programs in their products and services. Spring AI helps Java developers make these intelligent programs by providing well-known autoconfiguration and client-API-based programming approaches.

Spring AI supports almost all major model providers such as OpenAI, Microsoft, Amazon, Google, and Huggingface. It supports plain chat-based interactions as well as conversion from Text to Image, etc., in synchronous and asynchronous (stream API) methods.

1. Prerequisite: Setup the OpenAI Project API Key

The Project API key (previously User API key) is a unique encoded string that identifies and authenticates a user or application. You can obtain the key for your account at page: platform.openai.com/api-keys.

Next, we need to configure this key into environment variables to refer to it in our project, without exposing it to source code. It enhances the security of the application.

export OPENAI_API_KEY=[api_key_copied_from_openai_site]

On a Windows machine, we can add the environment variable in System properties:

Later, we can refer to this API key in the project’s properties file:


2. Project Dependencies

In a Spring boot application, we can enable support for Spring AI using the dependency: spring-ai-openai-spring-boot-starter.




As of today, Spring AI is still in development mode so we can pull it from the milestone repository. You can check the official GitHub page for the latest version.

	  <name>Spring Milestones</name>
	  <name>Spring Snapshots</name>

Additionally, we will need to include spring-web, spring-webflux, httpclient dependencies for creating the web components and handling HTTP connections to the backend LLMs.


	<!-- other dependnecies -->

3. What does Spring AI Module provide?

Spring AI is a new addition to the Spring Framework ecosystem for developing generative AI-focused applications that focus on generating new content in response to input prompts. In the backend, it makes an API call to OpenAI and presents the response back.

At its heart, Spring AI offers a text-based generative AI system. It means that users input the text, and the application responds with relevant output in different formats (String, Map, List, XML, JSON, Image, Video etc.). For response generation, Spring AI provides integration with most of the prominent generative AI models, including OpenAI, Azure Open AI, Bedrock (Amazon), Ollama, and Vertex AI (Google).

3.1. Client API

The following is a list of important classes/interfaces used in interacting with OpenAI LLMs:

ChatClientThe core interface for text-based interactions. This is used for simple requests, prompt-based requests, and requests requiring a response in a specific format (such as JSON, XML etc).
ImageClientIt provides a client for calling the OpenAI image generation API. When we send some text as a request using this client, we get the URL of the generated image in the response.
SpeechClientIt provides a client for calling the OpenAI text-to-speech generation API (TTS-1). When we send some text as a request using this client, we get the speech file path in the response.
import org.springframework.ai.chat.ChatClient;
import org.springframework.ai.image.ImageClient;
import org.springframework.ai.openai.OpenAiChatClient;
import org.springframework.ai.openai.OpenAiAudioSpeechClient;
import org.springframework.ai.openai.OpenAiImageClient;
import org.springframework.ai.openai.api.OpenAiApi;
import org.springframework.ai.openai.api.OpenAiAudioApi;
import org.springframework.ai.openai.api.OpenAiImageApi;
import org.springframework.ai.openai.audio.speech.SpeechClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

public class AppConfiguration {

  SpeechClient speechClient(@Value("${spring.ai.openai.api-key}") String apiKey) {
    return new OpenAiAudioSpeechClient(new OpenAiAudioApi(apiKey));

  ImageClient imageClient(@Value("${spring.ai.openai.api-key}") String apiKey) {
    return new OpenAiImageClient(new OpenAiImageApi(apiKey));

  ChatClient chatClient(@Value("${spring.ai.openai.api-key}") String apiKey) {
    return new OpenAiChatClient(new OpenAiApi(apiKey));

3.2. Output Converter

We will not be interacting with LLMs in only plain text format. Sometimes, our applications will need the outputs in a very specific structure (such as fixed JSON schema). In such cases, we can use the specific converters based on the usecase.

The converters perform two main tasks:

  • It instructs the prompt for the desired response format and structure. This ensures that the response is generated in a rigorous format.
  • Upon receiving the response, it parses the response in the required format such as a Java bean, a list, or a map of values
BeanOutputConverterIt provides the JSON format (derived from the fields of a Java bean) in which the LLM output must be requested, and transforms the LLM output to a specific Java bean instance using JSON schema.
ListOutputConverterIt instructs the prompt to request LLM output in JSON format in a list of comma-separated values, and later transform the response into a java.util.List instance.
MapOutputConverterIt instructs the prompt to request LLM output in JSON format and the structure should be of type java.util.HashMap, and later transforms the response into a java.util.Map instance.

3.3. Retrieval Augmented Generation (RAG)

In cases where we want the LLM to respond based on information available in a specified document, Spring AI provides the document readers. This is especially useful in Retrieval Augmented Generation (RAG).

Spring AI’s document reader API supports various formats, including simple text, JSON, Apache Tika, and PDFs. These are the main classes/interfaces part of Spring AI’s document processing capability:

  • Document: text-based representation of the data and metadata.
  • DocumentReader: is responsible for loading a List from a data source such as PDF, Markdown, and JSON.
  • DocumentWriter: persist the documents into a vector database.
  • DocumentTransformer: is responsible for processing the data in various ways (splitting, concatenation etc).
  • Embedding: a representation of the data as a List that is used by the vector database to compute the ‘similarity’ of a user’s query to relevant documents.
  • ContentFormatter: converts the Document text and metadata into an AI-prompt-friendly text representation.

3.4. Vector Store Integration

A vector store (or vector database) performs ‘similarity searches’, just like exact matches in traditional relational databases. A vector store enables us to organize the document data into smaller chunks. We store our documents in a vector store in RAG-related usecases.

When given a vector as a query, a vector database returns vectors that are “similar” to the query vector. This returned vector is a sub-document that we can send to LLM instead of sending the whole document which is a costly operation.

Spring AI allows integration with several leading vector stores, including Chroma, Pinecone, Redis, Weavite, Milvus, Azure, and PostgreSQL (PG Vector).

4. Spring AI Chat Completion Example

Let us begin with the simplest example. We ask the LLM (OpenAI GPT) to tell us a Joke. The prompt is a simple string, and we do not specify anything for the response format.

The tellSimpleJoke() method utilizes the ChatClient API (discussed in section 3.1.) which sends a simple prompt “Tell me a joke” to LLM. Whatever we get in the response, we simply put it in a Map and send it as a response to the API consumer.

public class OpenAiChatController {

  private final ChatClient chatClient;

  public OpenAiChatController(ChatClient chatClient) {
    this.chatClient = chatClient;

  public Map<String, String> tellSimpleJoke() {

    return Map.of("generation", chatClient.call("Tell me a joke"));

Let’s test this API:

For streaming the response from Model, we can call the chatClient.stream() model if the client implementation class implements the StreamingChatClient interface. For example, OpenAiChatClient class implements both interfaces i.e. ChatClient and StreamingChatClient.

5. Spring AI PromptTemplate Example

A PromptTemplate is quite similar to template strings in Java 21. In a prompt template, we have replaceable tokens in the string, which are replaced in runtime based on the parameters. Consider a similar prompt template as defined in application.properties. This ‘subject’ and ‘language’ are template parameters that will be provided in runtime.

app.joke.simple.promptTemplate=Tell me a joke about {subject} in {language}.

Spring AI’s PromptTemplate class helps in resolving such prompts and passing it to the ChatClient.call() method.

public class OpenAiChatController {

	private final ChatClient chatClient;
	private String promptTemplate;

	public OpenAiChatController(ChatClient chatClient,
	          @Value("${app.joke.simple.promptTemplate}") String promptTemplate) {

		this.chatClient = chatClient;
		this.promptTemplate = promptTemplate;

	public String tellSimpleJokeWithPrompt(@RequestParam("subject") String subject,
	                @RequestParam("language") String language) {

	  PromptTemplate pt = new PromptTemplate(promptTemplate);
	  Prompt renderedPrompt = pt.create(Map.of("subject", subject, "language", language));

	  ChatResponse response = chatClient.call(renderedPrompt);
	  return response.getResult().getOutput().getContent();

In the above code, we have asked for the joke’s subject and language from the API user as request parameters. Let’s test it out:

6. Spring AI Structured Output Example

Simple text-based outputs are sufficient for Chatbot-type applications. But in data-intensive intelligent applications, we must request and respond in complex formats (XML, JSON etc), just like we do in typical REST APIs using Java Beans.

In the following example, we have modified the previous API to request and respond in the format in JSON with fields in the JokeResponse record type:

public record JokeResponse(String subject, String language, String joke) {

The API utilizes the BeanOutputConverter class that derives the JSON format from the specified Java bean. Later it uses its parse() method to convert the JSON response into JokeResponse object.

public class OpenAiChatController {

	private final ChatClient chatClient;
	private String jsonPromptTemplate;

	public OpenAiChatController(ChatClient chatClient,
	          @Value("${app.joke.formatted.promptTemplate}") String promptTemplate) {

		this.chatClient = chatClient;
		this.jsonPromptTemplate = jsonPromptTemplate;

  public JokeResponse tellSpecificJokeInJsonFormat(@RequestParam("subject") String subject,
                 @RequestParam("language") String language) {

    BeanOutputConverter<JokeResponse> parser = new BeanOutputConverter<>(JokeResponse.class);
     * Your response should be in JSON format.
     * Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
     * Do not include markdown code blocks in your response.
     * Remove the ```json markdown from the output.
     * Here is the JSON Schema instance your output must adhere to:
     * ```{
     *   "$schema" : "https://json-schema.org/draft/2020-12/schema",
     *   "type" : "object",
     *   "properties" : {
     *     "joke" : {
     *       "type" : "string"
     *     },
     *     "language" : {
     *       "type" : "string"
     *     },
     *     "subject" : {
     *       "type" : "string"
     *     }
     *   }
     * }```
     * */
    String format = parser.getFormat();

    PromptTemplate pt = new PromptTemplate(jsonPromptTemplate);
    Prompt renderedPrompt = pt.create(Map.of("subject", subject, "language", language, "format", format));

    ChatResponse response = chatClient.call(renderedPrompt);
    return parser.parse(response.getResult().getOutput().getContent());

The request format is appended at the end of prompt to the LLM.

app.joke.formatted.promptTemplate=Tell me a joke about {subject} in {language}. {format}

Let’s test this API:

7. Spring AI Image Generation Example

The image generation with OpenAI uses the DALL-E models. We provide the message to ImageClient and it responds with the URL of the generated image.

public class OpenAiImageController {

  private final ImageClient imageClient;

  public OpenAiImageController(ImageClient imageClient) {
    this.imageClient = imageClient;

  public String imageGen(@RequestParam String message) {
    ImageOptions options = ImageOptionsBuilder.builder()

    ImagePrompt imagePrompt = new ImagePrompt(message, options);
    ImageResponse response = imageClient.call(imagePrompt);
    String imageUrl = response.getResult().getOutput().getUrl();

    return "redirect:" + imageUrl;

Let us test the API:

We can download the image from the URL in our application. For testing, copy the URL in a browser which should display the generated image:

We should take care when setting the parameters in ImageOptions as invalid parameters will raise a runtime error. For example, if we request an image of size 900×600 then it will throw NonTransientAiException:

    "timestamp": "2024-05-09T09:42:37.369+00:00",
    "status": 500,
    "error": "Internal Server Error",
    "trace": "org.springframework.ai.retry.NonTransientAiException: 400 - {\n  \"error\": {\n    \"code\": null,\n    \"message\": \"'900x600' is not one of ['256x256', '512x512', '1024x1024', '1024x1792', '1792x1024'] - 'size'\",\n    \"param\": null,\n    \"type\": \"invalid_request_error\"\n  }
    "message": "400 - {\n  \"error\": {\n    \"code\": null,\n    \"message\": \"'900x600' is not one of ['256x256', '512x512', '1024x1024', '1024x1792', '1792x1024'] - 'size'\",\n    \"param\": null,\n    \"type\": \"invalid_request_error\"\n  }\n}\n",
    "path": "/image-gen"

8. Conclusion

In this Spring AI tutorial, we learned the basics of the Spring AI module and its core terminology. We learned to interact with OpenAI’s chat and image generation services using simple text prompts as well as strictly formatted JSON prompts. Practice with these APIs to understand and take benefit of them in your next intelligent AI-enabled application.

Happy Learning !!

Source Code on Github


Notify of
Inline Feedbacks
View all comments

About Us

HowToDoInJava provides tutorials and how-to guides on Java and related technologies.

It also shares the best practices, algorithms & solutions and frequently asked interview questions.