Building AI Agents with Spring AI: 5 Workflow Patterns

A workflow pattern is a fixed way to combine several model calls in Java code, so that each call does one small job. In this example, we build the five common patterns in OrderDesk, the support assistant of a demo online store. We’ll use Spring Boot 4.1, Spring AI 2.0, and Java 21, with a local model in Ollama.

1. How Workflow Patterns Work

One model call with a long prompt often does too much at once, and a small model loses track. A workflow splits the job: our code calls the model several times, and each call gets a short, focused prompt. The code, not the model, decides the order of the calls.

These five patterns cover most cases:

1. Chain: each call’s output is the next call’s input.
2. Routing: one call classifies the input, and the code sends it to the right handler.
3. Parallelization: independent calls on the same input run at the same time.
4. Orchestrator and workers: one call splits the task into subtasks, and one call handles each subtask.
5. Evaluator and optimizer: one call writes, another call judges, and the writer improves its answer until the judge passes it.

An autonomous agent is different: the model itself decides which step comes next. A workflow is easier to test and debug because the steps are visible in the code. Anthropic’s Building effective agents describes the same five patterns.

In Spring AI, every pattern is plain Java around ChatClient, so there’s no workflow API to learn. Two ChatClient features do most of the work. The entity() method turns an answer into a Java record that our code can branch on. The advisors() method adds RAG to a single call, and tools() adds tools to a single call.

2. The Example App

This article adds five endpoints to OrderDesk, the app of this series. The code is in the com.howtodoinjava.orderdesk.agents package. Every new class goes in a sub-package of com.howtodoinjava.orderdesk, so that Spring Boot finds it. The full app is on GitHub in the orderdesk folder.

The basic setup (Ollama with qwen2.5:3b, the Spring Boot project, the Spring AI BOM and the Ollama starter) is explained in Getting Started with Spring AI.

2.1. What the Patterns Reuse

The patterns call three beans from earlier articles. If you only followed Getting Started, take the files listed at the end of this section from the GitHub folder.

The first bean is supportClient, the ChatClient that answers customers. Its system prompt holds the store’s rules. It comes from Getting Started, in ChatClientConfig:

    static final String SUPPORT_RULES = """
        You are OrderDesk, the support assistant of an online store.
        Answer in at most three sentences.
        Delivery: Pune, Mumbai and Delhi in 2 days; every other city in India in 5 days.
        Delivery is free above INR 499; below that it costs INR 49.
        We do not deliver outside India.
        """;

    @Bean
    ChatClient supportClient(ChatClient.Builder builder) {
        return builder
            .defaultSystem(SUPPORT_RULES)
            .build();
    }

The second bean is a QuestionAnswerAdvisor, which adds RAG to a call. Retrieval-augmented generation (RAG) means that the advisor searches the store’s policy sections in a vector store and adds the best matches to the prompt. The bean is in RagConfig:

    @Bean
    QuestionAnswerAdvisor questionAnswerAdvisor(VectorStore vectorStore) {
        return QuestionAnswerAdvisor.builder(vectorStore)
            .searchRequest(SearchRequest.builder()
                .topK(3)
                .similarityThreshold(0.5)
                .build())
            .build();
    }

1. topK(3) adds at most three policy sections.
2. similarityThreshold(0.5) adds a section only if its similarity score with the search text is at least 0.5.
3. The search text is the text of the user message. Section 3.4 shows why that matters.

QuestionAnswerAdvisor is in this dependency:

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-vector-store-advisor</artifactId>
</dependency>

The vector store needs an embedding model (a model that turns text into an array of numbers for the search). OrderDesk uses nomic-embed-text in Ollama:

ollama pull nomic-embed-text

spring.ai.ollama.embedding.options.model=nomic-embed-text

The third bean is OrderTools, the order lookup tools. Its getOrderStatus() method reads one order’s status and city from the orders table.

Files from earlier articles:

1. RagConfig.java: the questionAnswerAdvisor bean shown above, from Spring AI RAG Advisors.
2. SimpleVectorStoreConfig.java: the in-memory vector store, saved to data/vector-store.json, from Spring AI VectorStore Example.
3. PolicyIngestion.java and IngestController.java: POST /ingest loads the four policy files (returns, shipping, warranty, and payments) into the vector store, from Spring AI ETL Pipeline Example.
4. OrderTools.java, schema.sql, and data.sql: the order tools and the orders table with its sample rows, from Spring AI Tool Calling.

2.2. A Second ChatClient and the Controller

The internal steps, such as classifying or judging, shouldn’t follow the store’s customer rules, so we add a second, plain client:

package com.howtodoinjava.orderdesk.agents;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class WorkflowConfig {

    // A plain client without OrderDesk's support rules, for internal steps
    // such as classifying, checking and judging
    @Bean
    ChatClient workerClient(ChatClient.Builder builder) {
        return builder
            .defaultSystem("You are a precise assistant inside a customer support system.")
            .build();
    }
}

Spring AI auto-configures ChatClient.Builder and gives each injection point a new builder, so defaultSystem() here doesn’t change supportClient.

All five endpoints live in one controller, WorkflowController. Here are its fields and constructor:

package com.howtodoinjava.orderdesk.agents;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import com.howtodoinjava.orderdesk.tools.OrderTools;

@RestController
public class WorkflowController {

    private static final Logger log = LoggerFactory.getLogger(WorkflowController.class);

    private final ChatClient supportClient;
    private final ChatClient workerClient;
    private final QuestionAnswerAdvisor policyAdvisor;
    private final OrderTools orderTools;

    WorkflowController(@Qualifier("supportClient") ChatClient supportClient,
                       @Qualifier("workerClient") ChatClient workerClient,
                       QuestionAnswerAdvisor policyAdvisor, OrderTools orderTools) {
        this.supportClient = supportClient;
        this.workerClient = workerClient;
        this.policyAdvisor = policyAdvisor;
        this.orderTools = orderTools;
    }

OrderDesk now has more than one ChatClient bean, so Spring can’t pick one by type. @Qualifier(“workerClient”) names the bean we want, and a bean’s name is its @Bean method name. If a class from Getting Started injects a ChatClient without @Qualifier, add @Qualifier(“supportClient”) to it. The policyAdvisor and orderTools fields need no qualifier because there’s one bean of each type.

The methods in the next sections go into this class. Start Ollama and the app as shown in Getting Started, and restart the app after adding the code. If the vector store is empty, load the policies once:

curl -X POST localhost:8080/ingest

3. The Five Patterns in OrderDesk

Each pattern below has its code, one real run, and what went wrong in that run. All runs use qwen2.5:3b on a CPU.

3.1. Chain

A chain fits a task with fixed steps. OrderDesk turns an angry complaint into a reply in three steps:

    // 1. Chain: each step's output is the next step's input
    @GetMapping("/agents/chain")
    List<String> chain(@RequestParam String complaint) {
        String summary = workerClient.prompt()
            .user("Summarize this customer complaint in one sentence: " + complaint)
            .call().content();
        String draft = supportClient.prompt()
            .user("Write a reply to this complaint: " + summary)
            .advisors(policyAdvisor)
            .call().content();
        String polite = workerClient.prompt()
            .user("Rewrite this reply so it is polite and at most three sentences. "
                + "Keep every fact. Reply: " + draft)
            .call().content();
        return List.of("SUMMARY: " + summary, "DRAFT: " + draft, "FINAL: " + polite);
    }

1. The summary keeps the facts and drops the angry words. The draft’s RAG search uses the text we send, so it searches with those facts.
2. The draft uses supportClient with policyAdvisor, so it quotes the store’s policy sections.
3. The rewrite shortens the draft for the customer.

curl -G localhost:8080/agents/chain --data-urlencode "complaint=I ordered a jacket four days ago and it arrived today with a broken zip. This is the second time! I want my money back."

["SUMMARY: The customer ordered a jacket four days ago and received a defective product with a broken zip, requesting a refund due to previous issues.","DRAFT: I'm sorry to hear about the issue with your jacket. Please upload a photo of the damaged item and the parcel on the order page within 48 hours of delivery. We will then proceed to either replace the item or issue a full refund, and you will not be responsible for the return pickup. Please note that refunds typically go back to the original payment method within 5 to 7 working days after we receive the item. If you choose store credit, it will be added to your OrderDesk wallet at once.","FINAL: I apologize about the jacket issue. Please upload photos of the damaged item and the parcel on the order page within 48 hours. We will then process either a replacement or refund, and you won't need to return the item. Refunds usually go back to your original payment within 5-7 working days, and store credit is added immediately to your OrderDesk wallet."]

The draft is right: the customer isn’t responsible for the return pickup. But the rewrite turned that into “you won’t need to return the item”, which is wrong, and it kept four sentences instead of three. Each step can change a fact, even with “Keep every fact” in the prompt. So check the last step’s output against the step that holds the facts, or skip a rewrite step that only changes the tone.

3.2. Routing

Routing fits input that needs different handlers. OrderDesk’s questions are about policies, about one order, or about something else:

    // 2. Routing: classify first, then send the question to the right handler
    enum Route { POLICY, ORDER, OTHER }

    record RouteDecision(Route route, String reason) {
    }

    @GetMapping("/agents/route")
    List<String> route(@RequestParam String question) {
        RouteDecision decision = workerClient.prompt()
            .user("""
                Classify the customer question.
                POLICY: about returns, delivery, warranty or payment rules.
                ORDER: about one specific order or a customer's orders.
                OTHER: anything else.
                Question: """ + question)
            .call()
            .entity(RouteDecision.class);
        log.info("Route: {}", decision);

        String answer = switch (decision.route()) {
            case POLICY -> supportClient.prompt().user(question)
                .advisors(policyAdvisor).call().content();
            case ORDER -> supportClient.prompt().user(question)
                .tools(orderTools).call().content();
            case OTHER -> "OrderDesk can help with orders and store policies only.";
        };
        return List.of("ROUTE: " + decision.route() + " (" + decision.reason() + ")",
            "ANSWER: " + answer);
    }

The entity(RouteDecision.class) call makes the model answer in JSON that matches the record, and Spring AI converts it. The enum limits the route to three values, so the switch covers every case. Each handler gets only what it needs: RAG for policies, tools for orders, and no model call at all for the rest.

If the model’s JSON has no valid route, entity() throws an exception or route() is null, and the request fails. So production code should check the decision before the switch.

curl -G localhost:8080/agents/route --data-urlencode "question=Can I pay cash when the parcel arrives?"
curl -G localhost:8080/agents/route --data-urlencode "question=Has order ORD-1040 been delivered?"
curl -G localhost:8080/agents/route --data-urlencode "question=Can you recommend a good movie?"

["ROUTE: POLICY (payment rules)","ANSWER: Cash on delivery is available for orders up to INR 5,000, and it costs INR 30 extra per order. For orders above INR 499, delivery is free. If you have a damaged or wrong item, please report it within 48 hours of delivery."]
["ROUTE: ORDER (Has order ORD-1040 been delivered?)","ANSWER: The order ORD-1040 has been shipped from Ahmedabad, but the delivery status is not yet updated. Please check back later or contact customer support for more information."]
["ROUTE: OTHER (Can you recommend a good movie?)","ANSWER: OrderDesk can help with orders and store policies only."]

All three routes were right. The movie question cost only the classify call. The order answer has one slip: Ahmedabad is the order’s delivery city in the orders table, not where the order was shipped from.

3.3. Parallelization

Parallelization fits independent calls on the same input. OrderDesk checks a draft reply against three policies at once:

    // 3. Parallelization: independent checks run at the same time
    record PolicyCheck(String policy, boolean ok, String issue) {
    }

    @GetMapping("/agents/parallel")
    List<PolicyCheck> parallel(@RequestParam String reply) throws Exception {
        List<String> rules = List.of(
            "returns: items can be returned within 10 days of delivery",
            "shipping: delivery is free above INR 499, otherwise INR 49",
            "payments: cash on delivery is allowed up to INR 5,000");
        try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
            List<CompletableFuture<PolicyCheck>> checks = rules.stream()
                .map(rule -> CompletableFuture.supplyAsync(() -> workerClient.prompt()
                    .user("""
                        Policy: %s
                        Support reply: %s
                        Does the reply contradict this policy? Set ok to false only if it
                        states something different from the policy.""".formatted(rule, reply))
                    .call()
                    .entity(PolicyCheck.class), executor))
                .toList();
            return checks.stream().map(CompletableFuture::join).toList();
        }
    }

Each check runs on its own virtual thread (a cheap Java 21 thread for blocking work), and join() waits for all three. The total time is close to the slowest check, not the sum of three.

But Ollama answers several requests at the same time only when its OLLAMA_NUM_PARALLEL environment variable allows it. With a value of 1, the three checks still wait for each other on a local model. A hosted model answers them together. Our run below took about 71 seconds on a CPU.

We sent a reply with two mistakes (30 days, and cash for any amount) and one correct fact (free delivery above INR 499):

curl -G localhost:8080/agents/parallel --data-urlencode "reply=You can return any item within 30 days. Delivery is free above INR 499, and you can pay cash on delivery for any order amount."

[{"policy":"items can be returned within 10 days of delivery","ok":false,"issue":"The reply states that items can be returned within 30 days, which contradicts the policy that items can be returned within 10 days of delivery."},{"policy":"Delivery is free above INR 499, and you can pay cash on delivery for any order amount.","ok":false,"issue":"The support reply states that delivery is free above INR 499, which contradicts the stated policy that delivery is free above INR 499. Additionally, the reply mentions that you can pay cash on delivery for any order amount, which is not part of the stated policy."},{"policy":"Policy: payments: cash on delivery is allowed up to INR 5,000","ok":false,"issue":"The reply states that you can pay cash on delivery for any order amount, which contradicts the policy that allows cash on delivery up to INR 5,000."}]

The returns and payments checks found the two real mistakes. The shipping check failed a fact that matches the rule (“free above INR 499 contradicts free above INR 499”), and it copied the reply into the policy field. A small model used as a checker gives false alarms. Our code already knows which rule it sent to each check, so use that rule as the label, not the policy field from the model.

3.4. Orchestrator and Workers

This pattern fits a task whose parts are known only after reading it. One call plans the subtasks, then one worker call answers each:

    // 4. Orchestrator and workers: one call plans the subtasks, workers do them
    record Plan(List<String> subtasks) {
    }

    @GetMapping("/agents/orchestrate")
    List<String> orchestrate(@RequestParam String request) {
        Plan plan = workerClient.prompt()
            .user("""
                Split this request into at most four separate questions that a support
                agent can answer one by one from the store policies. Request: """ + request)
            .call()
            .entity(Plan.class);
        log.info("Plan: {}", plan.subtasks());

        List<String> results = new ArrayList<>();
        for (String subtask : plan.subtasks()) {
            // each worker gets the whole request too, so details such as the city survive
            String answer = supportClient.prompt()
                .user("Customer request: " + request + "\nAnswer only this part: " + subtask)
                .advisors(policyAdvisor).call().content();
            results.add(subtask + " -> " + answer);
        }
        return results;
    }

Each worker also gets the whole request, because a planner can shorten a subtask and drop details. In our first version, workers got only the subtask. The plan shortened one to “Tell me delivery charge”, and the workers lost the city and the price.

curl -G localhost:8080/agents/orchestrate --data-urlencode "request=I live in Nagpur and want to buy a phone for INR 6,000. How long will delivery take, what will it cost, what warranty do I get, and can I pay cash?"

["How long will delivery take for a phone priced at INR 6,000 from Nagpur? -> Delivery to Nagpur will take 5 days for a phone priced at INR 6,000.","What will be the cost of delivery for a phone priced at INR 6,000 from Nagpur? -> The cost of delivery for a phone priced at INR 6,000 from Nagpur will be free since your order qualifies for free delivery as it is above INR 499.","What warranty do I get for a phone priced at INR 6,000 from Nagpur? -> For a phone priced at INR 6,000, the warranty information is not provided in the given context. I cannot answer what warranty you get for the phone.","Can I pay cash for a phone priced at INR 6,000 from Nagpur? -> I'm sorry, but I can't answer your question about paying cash for a phone priced at INR 6,000 from Nagpur. The context provided does not include information about cash on delivery options for orders above INR 5,000."]

In this run, the planner kept the city and the price in every subtask, so this output doesn’t show the effect of passing the whole request; the first run did. The delivery time (5 days) and the free delivery are right.

But two workers said they couldn’t answer. The policyAdvisor bean searches with the text of the user message, and here that text is the whole long request plus the subtask. The search returns at most three sections with a score of at least 0.5.

The warranty worker says that none of its sections held the warranty. The cash worker names the INR 5,000 limit but doesn’t apply it: the right answer is “No, because INR 6,000 is above the limit”. So the workers are only as good as what each one retrieves and reads. A shorter search text per worker (only the subtask) or a lower threshold would help here.

3.5. Evaluator and Optimizer

This pattern fits output that a clear rule can judge. One call drafts and another judges. If the judge fails the draft, a third call improves it with the judge’s feedback. The code judges at most three drafts:

    // 5. Evaluator and optimizer: draft, judge, improve, at most three rounds
    record Evaluation(boolean pass, String feedback) {
    }

    @GetMapping("/agents/refine")
    List<String> refine(@RequestParam String complaint) {
        List<String> rounds = new ArrayList<>();
        String reply = supportClient.prompt()
            .user("Write a reply to this customer: " + complaint)
            .advisors(policyAdvisor)
            .call().content();
        for (int round = 1; round <= 3; round++) {
            Evaluation evaluation = workerClient.prompt()
                .user("""
                    Judge this support reply. It passes only if it is at most three sentences,
                    is polite, and gives the customer one clear next step.
                    Customer: %s
                    Reply: %s""".formatted(complaint, reply))
                .call()
                .entity(Evaluation.class);
            rounds.add("ROUND " + round + ": " + reply + " | PASS=" + evaluation.pass()
                + " | " + evaluation.feedback());
            if (evaluation.pass() || round == 3) {
                break;   // passed, or no rounds left to judge an improved reply
            }
            reply = supportClient.prompt()
                .user("""
                    Improve this reply using the feedback. Keep every fact.
                    Reply: %s
                    Feedback: %s""".formatted(reply, evaluation.feedback()))
                .call().content();
        }
        return rounds;
    }
}

The limit stops a judge that never passes from looping forever. In round 3, the loop ends right after the judge, because no round is left to judge another draft. The improve step has no policyAdvisor, so its facts come only from the draft it improves.

curl -G localhost:8080/agents/refine --data-urlencode "complaint=My headphones stopped working after two weeks. I opened the seal of course. I want a refund."

["ROUND 1: I'm sorry, but you cannot return headphones once the seal is opened. According to the policy provided, we do not accept returns for earphones and headphones after the seal is broken. However, if the headphones are damaged or not as described, you can report this within 48 hours of delivery by uploading a photo of the item and the parcel on the order page. We will then send a replacement or provide a full refund. Please let me know if you need further assistance. | PASS=true | The reply is at most three sentences, polite, and provides a clear next step for the customer to upload a photo and report the issue for a replacement or refund."]

The judge passed a reply of five sentences and said it was “at most three sentences”. It also didn’t notice that the 48-hour rule can’t help a customer two weeks after delivery. A judge is a model call too, and a small model judges loosely. Rules that code can check, such as a sentence count, should be checked in code, and the model should judge only what code can’t.

4. Which Pattern to Use

This decision diagram picks a pattern for a task:

The patterns also combine. For example, the chain could start with the router. Similarly, the parallel checks could check each worker’s answer.

When the steps can’t be written down in advance, a workflow isn’t enough. Then an agent framework lets the model or a planner choose the next step. The next article shows that with Embabel.

5. Common Traps

5.1. Every Call Adds Time

Each pattern multiplies the model calls. On our CPU, the chain took about 100 seconds for three calls, and the orchestrator about 75 seconds for five. Use the fewest calls that do the job, and run independent calls in parallel.

5.2. A Model Judge Isn’t a Test

Check in code what code can check, such as a sentence count. Treat a small model’s verdict as a hint.

6. FAQs

6.1. Is a workflow an AI agent?

Not in the strict sense, because our code fixes the steps. Many production “agents” are workflows, because they are easier to test.

6.2. Should every step use the same model?

No. A classify or judge step often works with a small, fast model, while the step that writes the customer’s reply may need a bigger one. A separate ChatClient bean per step, like workerClient, can set its own model with defaultOptions() on its builder.

6.3. How do I test a workflow?

Test each step on its own with fixed inputs. Test the routing and the parsing of the records with a fake ChatModel: a lambda that returns fixed text, passed to ChatClient.create(). It needs no real model. The article Testing Spring AI Apps: Mocks, Testcontainers and Evaluators shows it.

7. Conclusion

A workflow pattern splits one big prompt into several small model calls, and Java code decides their order. In OrderDesk, routing worked every time. The other patterns ran, but each run made a mistake that a person would catch.

The one trap to remember is that every step is a model call that can be wrong. Keep the number of steps small, pass the facts along, and check in code what code can check.

All articles in order: Spring AI Tutorial

8. References

• Spring AI Reference: Building Effective Agents
• Anthropic: Building Effective Agents
• Spring AI Reference: ChatClient

Drop me your questions related to Spring AI agents in the comments.

Happy Learning !!

Source Code on Github