Spring AI Interview Questions and Answers

Preparing for a Spring AI interview means understanding how a Java application uses an AI model, supplies the information it needs, and checks the results. You should be able to explain why a chatbot needs memory, how a model requests a tool, and what to check when a RAG application gives the wrong answer.

This guide covers 30 Spring AI interview questions, from ChatClient and prompt templates to structured output, advisors, MCP, RAG, and testing. The examples follow OrderDesk, an online store’s support assistant that answers policy questions and looks up orders. Version-specific answers refer to Spring AI 2.0.

1. Spring AI Basics

1.1. What is Spring AI?

Spring AI is a framework for adding AI features to Spring applications. It provides Java APIs to send prompts to AI models, work with their responses, and connect them to application data and services.

For example, an online store can use Spring AI to build a support assistant that answers questions from its return policy and calls an order service to check delivery status. Spring AI provides the integrations for model access, document retrieval, chat memory, and tool calling needed to build that flow.

Its common interfaces let us use supported providers, such as OpenAI and Ollama, through familiar Spring patterns, including dependency injection, starters, and external configuration.

1.2. Does Spring AI let us switch model providers without changing code?

For features supported by both providers, we can often keep the application code and change the provider starter and configuration. This works because our code uses Spring AI interfaces, such as ChatModel, rather than calling a provider’s HTTP API directly.

For example, a service that sends a text prompt through ChatClient may work with either OpenAI or Ollama. However, the replacement model must support the features the service uses. Image input, tool calling, structured output, and provider-specific settings can require code changes.

We also need to test the responses after switching. Two models can accept the same request but differ in how well they follow instructions, extract values, or choose a tool.

1.3. What is the difference between ChatModel and ChatClient?

ChatModel is the interface for sending a request to a chat model. Its synchronous call() method accepts a Prompt and returns a ChatResponse, which contains the generated answer and response metadata.

ChatClient is built on top of ChatModel. It provides a fluent API for setting messages, applying advisors, and reading the result as text or a Java object. The following examples show both APIs returning the answer text:

ChatResponse response = chatModel.call(
    new Prompt("How can I track my order?"));

String answer = response.getResult().getOutput().getText();

String answer = chatClient.prompt()
    .system("You are OrderDesk, an online store support assistant.")
    .user("How can I track my order?")
    .call()
    .content();

Use ChatClient for application features that combine prompts with memory, retrieval, or tools. Use ChatModel when you want to work directly with prompts and model responses. See the ChatClient reference.

1.4. What changed about tool execution in Spring AI 2.0?

Spring AI 2.0 moved automatic tool execution from individual ChatModel implementations into ChatClient‘s advisor chain. The auto-configured client registers a ToolCallingAdvisor to manage the tool-calling loop, unless that feature is disabled.

For example, when a customer asks for an order’s status, the model can request the order lookup tool. The advisor coordinates tool execution, sends the result back to the model, and continues until the model returns an answer without further tool requests.

If we call ChatModel directly, a tool request is returned to our code without being executed automatically. We must handle that request ourselves or use ChatClient. This is an important migration check for applications that previously relied on Spring AI 1.x running tools inside the model implementation. See the upgrade notes.

1.5. What is the difference between call() and stream()?

Use call() when the application needs the complete response before continuing. In this example, content() sends the request and waits for the answer as a String:

String answer = chatClient.prompt()
    .user(question)
    .call()
    .content();

Use stream() when the application should receive the response in pieces as the model generates it. Here, content() returns a Flux<String>, a Reactor type that can emit multiple values over time:

Flux<String> answer = chatClient.prompt()
    .user(question)
    .stream()
    .content();

The streaming request starts when the returned Flux is subscribed to. A web endpoint can pass those pieces to the browser, allowing it to display the answer progressively.

Calling call() alone does not contact the model; a result method such as content() triggers execution. Streaming improves the time until the user sees the first part of an answer, but does not necessarily shorten total generation time. See the ChatClient response APIs.

2. Prompts and Structured Output

2.1. What is the difference between a system message and a user message?

A system message tells the model how the application expects it to behave. A user message contains the question or task it should respond to. The request can also include assistant messages from earlier exchanges.

For example, OrderDesk can use a system message that says, “Answer store-policy questions using the supplied policy text.” The customer then asks, “Can I return an opened product?” in a user message.

Keep application instructions separate from customer input. This makes prompts easier to maintain, but it does not guarantee that a model will follow every instruction. Checks such as whether a customer owns an order must run in Java code. See the Prompts reference.

2.2. What is a prompt template?

A prompt template is reusable text with placeholders for values supplied at runtime. Spring AI fills those placeholders before sending the request to the model.

String answer = chatClient.prompt()
    .user(u -> u
        .text("Write a short update for {customer} about order {orderId}.")
        .param("customer", "Priya")
        .param("orderId", "ORD-1042"))
    .call()
    .content();

Here, the same instruction can be used for different customers and orders. We can change the wording in one place instead of rebuilding it through string concatenation in every service method.

The template renderer interprets the placeholders. If a template also contains literal braces, such as a JSON example, configure its delimiters or escaping appropriately. Parameter substitution is not input validation or protection against prompt injection. See the prompt-template documentation.

2.3. How does Spring AI map a response to a Java object?

The entity() method asks for a structured response and converts it into the Java type we provide. This is useful when application code needs fields it can validate or store, rather than a paragraph of text.

For example, OrderDesk can extract the details of a support request into a record:

public record TicketTriage(
    String orderId,
    String category,
    String summary
) {}

TicketTriage ticket = chatClient.prompt()
    .system("""
        Classify the support request as PAYMENT, DELIVERY, RETURN, or OTHER.
        Extract the order ID and write a one-sentence summary.
        Use NONE when the request has no order ID.
        """)
    .user("I was charged twice for order ORD-1042.")
    .call()
    .entity(TicketTriage.class);

The default approach uses output-format instructions and a converter. It can fail if the model returns an incompatible response. Also, a successfully created record may still contain incorrect values, so validate the result before using it. See the output converters reference.

2.4. How is native structured output different?

With native structured output, Spring AI sends the output schema through a provider feature designed for structured responses. The provider and selected model must support that feature.

In Spring AI 2.0, we can request it when converting the response:

TicketTriage ticket = chatClient.prompt()
    .system("""
        Classify the request as PAYMENT, DELIVERY, RETURN, or OTHER.
        Copy the order ID exactly, or use NONE if it is missing.
        Write a one-sentence summary.
        """)
    .user(message)
    .call()
    .entity(TicketTriage.class,
        spec -> spec.useProviderStructuredOutput());

The schema describes the response structure; the prompt still needs to explain the task. For example, a schema can require an orderId string without establishing which order the customer meant.

Use native output to improve format reliability, then apply the same validation and authorization checks as for other application input. See the Structured Output reference.

2.5. What should happen when structured output is invalid?

First identify whether the failure concerns the response format or its contents.

If the model returns malformed JSON or a value that cannot be converted into the target type, handle the conversion failure. Schema validation can catch structural problems, and a bounded retry can give the model a chance to correct its response.

If conversion succeeds but the order does not exist, retrying JSON generation will not solve the problem. Check the order in the application and ask the customer to clarify it when appropriate.

Set a retry limit and a clear failure response. The customer should receive a useful next step rather than an internal parsing error. See Spring AI’s output-conversion guidance.

3. Advisors and Chat Memory

3.1. What is an advisor in Spring AI?

An advisor is a component in the ChatClient processing chain. It can examine or change a request before passing it to the next component, and process the response returned by that component.

For example, a retrieval advisor can find store-policy documents and add them to the request. A memory advisor can add earlier conversation messages. A custom advisor can apply shared request checks without repeating them in each service method.

Implement CallAdvisor for synchronous processing and StreamAdvisor for streaming processing. An advisor can also stop the chain and return a response itself, so an advised request does not always reach the model. See the Advisors API reference.

3.2. In what order do advisors run?

Spring AI sorts advisors by their order values. The advisor with the lower value handles the request first and, when it delegates to the rest of the chain, processes the returning response last.

Request:  Advisor A -> Advisor B -> Model
Response: Advisor A <- Advisor B <- Model

The order in which advisors are written in a configuration list does not establish their execution order. Advisors with equal order values have no guaranteed relative order.

Ordering matters when advisors interact. For example, a request guard should reject an unacceptable request before a retrieval advisor performs an unnecessary search. Around a tool-calling loop, advisor placement also determines whether it handles the overall interaction or individual model calls. See the advisor-order documentation.

3.3. What is the difference between ChatMemory, ChatMemoryRepository, and a memory advisor?

ChatMemory manages the messages available as conversation context. ChatMemoryRepository provides their storage. A memory advisor connects that memory to ChatClient requests.

Component	Example responsibility
ChatMemory	Keep a window of recent messages
ChatMemoryRepository	Read and write messages in memory or a database
MessageChatMemoryAdvisor	Add retained messages to a request and record the exchange

For OrderDesk, a database repository can preserve messages across restarts, while a message-window implementation limits the conversation context. We still need to configure the memory advisor on the client; storing messages alone does not add them to a prompt. See the Chat Memory reference.

3.4. Does a persistent repository give the model unlimited memory?

No. A repository can preserve messages, but a model request can contain only a limited amount of context. The memory strategy determines which retained messages are included.

For example, MessageWindowChatMemory maintains a bounded window. Older messages are evicted as the conversation grows, so information mentioned early in a long chat may no longer be available to the model.

A message count is also different from a token limit. Ten short messages and ten pasted documents can have very different sizes.

If the application needs a complete conversation record for support or auditing, store that history separately from the context window used for generation. See the memory and history distinction.

3.5. Why is the conversation ID important?

The conversation ID tells the memory advisor which conversation to load and update. Every memory-enabled request must identify the appropriate conversation.

String answer = chatClient.prompt()
    .user(message)
    .advisors(a -> a.param(
        ChatMemory.CONVERSATION_ID, conversationId))
    .call()
    .content();

This example assumes the client already has a memory advisor configured.

Use a different ID for each conversation, and check that it belongs to the authenticated user before using it. One user may have several chats, so using the user ID alone would combine those chats into one context.

A shared or unchecked conversation ID can mix customers’ messages and expose private information. See Spring AI’s conversation-ID guidance.

4. Tool Calling and MCP

4.1. How does tool calling work?

Tool calling lets a model request an operation implemented by the application. The request contains the tool name and arguments; application code performs the operation.

For example, OrderDesk receives, “Where is order ORD-1042?” The interaction can proceed as follows:

1. The application sends the question and available tool definitions to the model.
2. The model requests the order-status tool with ORD-1042 as its argument.
3. The application validates the argument, checks access, and calls the order service.
4. The tool result is sent back to the model so it can explain the status.

In Spring AI 2.0, ToolCallingAdvisor coordinates this loop and ToolCallingManager handles tool execution. The model chooses an operation from the tools it is given; our application controls what those tools can do. See the Tool Calling reference.

4.2. How do we define and register a tool?

A common approach is to annotate a Java method with @Tool and pass an object containing that method to ChatClient.

The following excerpt assumes an injected OrderService. Its lookup method validates access using the authenticated customer’s identity:

@Tool(description = "Find the delivery status of an order belonging to the current customer")
public String getOrderStatus(String orderId) {
    return orderService.findStatusForCurrentCustomer(orderId);
}

Register the object for a request:

String answer = chatClient.prompt()
    .user("Where is order ORD-1042?")
    .tools(orderTools)
    .call()
    .content();

The description helps the model decide when to request the tool. Exposing an order ID as an argument does not authorize access: the service must enforce that check. Register only tools appropriate for the task. See the tool-definition documentation.

4.3. How should we handle tools that change data?

Treat a tool that submits a refund or cancels an order as an application operation with real consequences. It needs authorization, validated arguments, and any confirmation required by the business process.

For OrderDesk, the assistant can first prepare a refund proposal. The application shows the amount and order to the customer, records their confirmation, and only then permits execution. The model’s claim that the customer agreed is not sufficient evidence of confirmation.

Make repeated requests safe as well. An idempotency key, for example, can prevent a retried refund submission from creating a second refund. Use timeouts and audit records so failures can be investigated without blindly repeating the action.

4.4. What is MCP, and how is it different from tool calling?

Model Context Protocol (MCP) standardizes communication between an AI application and servers that expose capabilities such as tools, resources, and prompts.

Tool calling and MCP solve different parts of the interaction. Tool calling allows a model to request an operation. MCP allows an application to discover and invoke capabilities exposed by another process or service.

For example, an MCP server could expose OrderDesk’s order lookup. A Java assistant and an assistant written in another language could then use the same server. Spring AI can adapt discovered MCP tools into callbacks available to ChatClient.

An MCP connection does not grant unrestricted access. The server must authenticate callers and enforce access rules for each operation. See the Spring AI MCP overview.

5. Embeddings, Vector Stores, and RAG

5.1. What is an embedding?

An embedding represents an input as a vector of numbers. An application can compare these vectors to find inputs that are similar according to the embedding model.

For example, a customer might ask, “When can I send this jacket back?” A policy document might call the same topic the “return window.” Embeddings can help retrieve that document even though the question does not use its exact wording.

float[] vector = embeddingModel.embed(
    "Most items can be returned within 10 days of delivery.");

Document and query vectors must belong to compatible embedding spaces. Matching vector dimensions alone is not enough. When changing the embedding model, plan to re-embed documents unless compatibility is explicitly established. See the Embeddings Model API.

5.2. What does a VectorStore do?

A VectorStore stores documents with their embeddings and retrieves documents similar to a query. Spring AI provides a common API for these operations across supported stores.

List<Document> matches = vectorStore.similaritySearch(
    SearchRequest.builder()
        .query("How long do I have to return a jacket?")
        .topK(3)
        .similarityThreshold(0.5)
        .build());

Here, topK(3) requests at most three matches. The threshold excludes results below the configured similarity level, so fewer than three documents may be returned.

The value 0.5 is an example, not a recommended setting for every application. A similarity score measures a relationship between vectors; it is not the probability that a generated answer is correct. Evaluate retrieval using representative questions before choosing a threshold. See the Vector Databases reference.

5.3. What is RAG?

Retrieval-augmented generation (RAG) adds a retrieval step before answer generation. The application finds information relevant to a question and includes it in the model’s request.

For example, when a customer asks about returning a jacket, OrderDesk retrieves the applicable return policy. The model then receives both the question and the policy text, allowing it to answer using information supplied by the store.

RAG does not retrain the model. Updating the indexed documents can make new information available to later requests. However, retrieval may select an outdated or irrelevant passage, and the model can still misread the evidence. RAG needs retrieval tests and answer evaluation. See the RAG reference.

5.4. How do QuestionAnswerAdvisor and RetrievalAugmentationAdvisor differ?

QuestionAnswerAdvisor provides a straightforward way to search a vector store and add the matching documents to a prompt. It is useful when that retrieval flow meets the application’s needs.

RetrievalAugmentationAdvisor allows more of the retrieval flow to be assembled from separate components. These can transform the query, retrieve documents, and construct the context sent to the model.

For example, a follow-up question such as “Does that apply to sale items?” may need the earlier conversation to become a useful search query. A modular flow can include a query transformation for that purpose.

Choose the advisor based on the steps you need to customize. A more elaborate flow adds work and potentially extra model calls, so evaluate whether it improves retrieval. See the Spring AI RAG documentation.

5.5. Why do we split documents into chunks?

Chunking divides a long document into smaller pieces that can be retrieved independently. It helps the application send relevant passages to the model without including the entire source document.

For example, a store handbook may cover delivery, payments, and returns. A question about returns should retrieve that section rather than consume the context window with the whole handbook.

Chunk size is a tradeoff. Very small chunks may separate a rule from its exceptions. Large chunks may include unrelated sections and weaken retrieval precision. Preserve headings and useful metadata so each chunk remains understandable.

In Spring AI’s ETL flow, readers extract documents, transformers prepare or split them, and writers load them into a destination such as a vector store. See the ETL Pipeline reference.

5.6. How would you investigate an incorrect RAG answer?

Trace the question through ingestion, retrieval, prompt construction, and generation. This identifies the failed stage before you change a model or rewrite the prompt.

For OrderDesk, I would check:

1. Source: Is the correct policy available and current?
2. Index: Was it successfully read, split, embedded, and stored?
3. Retrieval: Did the search return the relevant passage? Check the query, filters, threshold, and result limit.
4. Prompt: Did the retrieved passage reach the model without being dropped or truncated?
5. Answer: Does the response accurately describe the supplied policy?

If the relevant passage never reaches the model, changing the generation prompt alone will not fix retrieval. If the evidence is missing or ambiguous, provide a fallback such as clarification or support escalation.

5.7. How do you prevent RAG from exposing another customer’s documents?

Restrict retrieval to documents the authenticated user is allowed to access. Apply this restriction before private content is included in the model request.

For example, OrderDesk can add a server-controlled tenant filter to each vector search. If access rules are more detailed than a tenant boundary, retrieval must enforce those rules too, using the store’s filtering capabilities or an authorized retrieval layer.

Do not let the model or browser replace mandatory access filters. A prompt saying “ignore other customers’ documents” is insufficient once those documents have already been retrieved and sent to the model.

Test this with users from different tenants and with deliberately altered document and conversation IDs. See the vector-store filtering documentation.

6. Testing and Production

6.1. How do you test a Spring AI application?

Test the application logic separately from the model’s behavior. A fake model gives predictable responses for unit tests, while integration and evaluation tests check the parts that depend on real providers or generated answers.

Test	Example check
Unit test	A missing order ID triggers clarification instead of a lookup
Provider integration test	The configured model accepts the required structured-output request
Retrieval test	A returns question retrieves the current return policy
Tool test	A customer cannot retrieve another customer’s order
Answer evaluation	The answer states the return window supported by the retrieved policy

Use exact assertions for deterministic application behavior. For live-model responses, check facts, structure, and task outcomes rather than one exact sentence. Keep a repeatable set of questions to detect regressions when prompts, models, or retrieval settings change.

6.2. What is an LLM evaluator?

An LLM evaluator asks a model to assess a generated response against defined criteria. It can help evaluate many responses that would otherwise need manual review.

Spring AI provides evaluators such as RelevancyEvaluator and FactCheckingEvaluator. Relevancy concerns whether the response addresses the request in context; fact checking concerns whether supplied evidence supports its claims.

For example, give the judge a policy stating a ten-day return window. Test both a correct ten-day answer and an incorrect thirty-day answer. This helps reveal a judge that accepts everything.

The judge is another model and can be wrong. Compare its decisions with human-reviewed examples, including difficult cases, before relying on its scores. See the Evaluation Testing reference.

6.3. What should we monitor in production?

Monitor the complete AI interaction: response time, failures, token usage, retrieval operations, and tool calls. A slow answer may come from document search or a downstream service rather than the model itself.

Spring AI integrates with Micrometer observations for components including ChatClient, advisors, models, and vector stores. With the appropriate metrics and tracing setup, these observations help identify where time is spent and which operations fail.

Track answer quality separately as well. A request that returns HTTP 200 can still give an incorrect answer.

Avoid collecting full prompts and responses by default merely to investigate latency. They may contain private messages and documents. Decide what needs recording, who can access it, and how long it is retained. See the Observability reference.

6.4. How would you reduce an AI application’s cost and response time?

Measure the request before optimizing it. Compare model time, retrieval time, tool time, input tokens, and output tokens to find the largest contributor.

For OrderDesk, possible changes include:

• Send the relevant conversation context instead of every previous message.
• Retrieve fewer irrelevant passages and remove repeated prompt text.
• Use a smaller model for tasks it performs reliably, such as a tested classification task.
• Bound retries and tool iterations so one request cannot continue indefinitely.
• Set a suitable response-length budget for the task.

Check the same evaluation questions after each change. Shortening the context can remove a policy exception, and a cheaper model can reduce extraction accuracy. Optimize for the cost and time of a correct completed task, including any retries.

For implementation examples covering these topics, follow the Spring AI Tutorial and the OrderDesk source project.

Happy Learning !!