Building AI Agents with Spring AI: 5 Workflow Patterns

A customer-support request can involve several jobs: identify the topic, look up an order, find the relevant policy, and write a clear reply. Asking a model to do all of this in one prompt can make the result harder to control.

A workflow separates these jobs into smaller model calls and connects them with Java code. In this article, we implement five workflow patterns in OrderDesk using Spring AI: chaining, routing, parallelization, orchestrator and workers, and evaluator and optimizer. The examples use Spring Boot 4.1, Spring AI 2.0, Java 21, and a local Ollama model.

1. What Is an AI Workflow?

An AI workflow defines how an application moves from an input to a result. Each model call has a focused responsibility, while application code controls branching, tool access, repetition, and stopping conditions. Some decisions, such as choosing a route or proposing subtasks, can still come from the model.

The five patterns below organize model calls in different ways:

1. Chain: each call’s output is the next call’s input.
2. Routing: one call classifies the input, and the code sends it to the right handler.
3. Parallelization: independent calls on the same input run at the same time.
4. Orchestrator and workers: one call splits the task into subtasks, and one call handles each subtask.
5. Evaluator and optimizer: a writer produces an answer, a reviewer evaluates it, and the writer revises it until it passes or reaches a configured limit.

In an autonomous agent, the model has more freedom to select its next action as the task develops. Workflows place more of that control in application code, which makes the execution path easier to inspect and test. Anthropic’s Building effective agents describes these five patterns and the distinction between workflows and agents.

We implement these patterns with ordinary Java around ChatClient. The entity() method maps a model response to a Java type that the application can use for decisions. The advisors() method attaches an advisor, such as the RAG advisor used here, and tools() makes tools available to a particular call. Structured conversion still needs validation; it does not guarantee a correct model decision.

2. Overview of the Example Application

This article adds five endpoints to OrderDesk, the app of this series. The code is in the com.howtodoinjava.orderdesk.agents package. Every new class goes in a sub-package of com.howtodoinjava.orderdesk, so that Spring Boot finds it. The full app is on GitHub in the orderdesk folder.

The basic setup (Ollama with qwen2.5:3b, the Spring Boot project, the Spring AI BOM and the Ollama starter) is explained in Getting Started with Spring AI.

2.1. Shared Clients, Policies, and Order Tools

The examples reuse three components: a customer-facing ChatClient, a policy-retrieval advisor, and order lookup tools. If your project contains only the Getting Started example, add the source files linked at the end of this section before using the workflow endpoints.

The first bean is supportClient, the ChatClient that answers customers. Its system prompt holds the store’s rules. It comes from Getting Started, in ChatClientConfig:

    static final String SUPPORT_RULES = """
        You are OrderDesk, the support assistant of an online store.
        Answer in at most three sentences.
        Delivery: Pune, Mumbai and Delhi in 2 days; every other city in India in 5 days.
        Delivery is free above INR 499; below that it costs INR 49.
        We do not deliver outside India.
        """;

    @Bean
    ChatClient supportClient(ChatClient.Builder builder) {
        return builder
            .defaultSystem(SUPPORT_RULES)
            .build();
    }

The second bean is a QuestionAnswerAdvisor, which adds RAG to a call. Retrieval-augmented generation (RAG) means that the advisor searches the store’s policy sections in a vector store and adds the best matches to the prompt. The bean is in RagConfig:

    @Bean
    QuestionAnswerAdvisor questionAnswerAdvisor(VectorStore vectorStore) {
        return QuestionAnswerAdvisor.builder(vectorStore)
            .searchRequest(SearchRequest.builder()
                .topK(3)
                .similarityThreshold(0.5)
                .build())
            .build();
    }

1. topK(3) adds at most three policy sections.
2. similarityThreshold(0.5) adds a section only if its similarity score with the search text is at least 0.5.
3. The search text is the text of the user message. Section 3.4 shows why that matters.

QuestionAnswerAdvisor is in this dependency:

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-vector-store-advisor</artifactId>
</dependency>

The vector store needs an embedding model (a model that turns text into an array of numbers for the search). OrderDesk uses nomic-embed-text in Ollama:

ollama pull nomic-embed-text

spring.ai.ollama.embedding.options.model=nomic-embed-text

The third bean is OrderTools, the order lookup tools. Its getOrderStatus() method reads one order’s status and city from the orders table.

Files from earlier articles:

1. RagConfig.java: the questionAnswerAdvisor bean shown above, from Spring AI RAG Advisors.
2. SimpleVectorStoreConfig.java: the in-memory vector store, saved to data/vector-store.json, from Spring AI VectorStore Example.
3. PolicyIngestion.java and IngestController.java: POST /ingest loads the four policy files (returns, shipping, warranty, and payments) into the vector store, from Spring AI ETL Pipeline Example.
4. OrderTools.java, schema.sql, and data.sql: the order tools and the orders table with its sample rows, from Spring AI Tool Calling.

2.2. Configure the Worker Client and Controller

Internal tasks need different instructions from customer replies. A classifier should return a route, and a reviewer should return an evaluation; neither should format its output as a customer-support message. We therefore create a separate workerClient without the support client’s default system prompt:

package com.howtodoinjava.orderdesk.agents;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class WorkflowConfig {

    // A plain client without OrderDesk's support rules, for internal steps
    // such as classifying, checking and judging
    @Bean
    ChatClient workerClient(ChatClient.Builder builder) {
        return builder
            .defaultSystem("You are a precise assistant inside a customer support system.")
            .build();
    }
}

Spring AI auto-configures ChatClient.Builder and gives each injection point a new builder, so defaultSystem() here doesn’t change supportClient.

All five endpoints live in one controller, WorkflowController. Here are its fields and constructor:

package com.howtodoinjava.orderdesk.agents;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import com.howtodoinjava.orderdesk.tools.OrderTools;

@RestController
public class WorkflowController {

    private static final Logger log = LoggerFactory.getLogger(WorkflowController.class);

    private final ChatClient supportClient;
    private final ChatClient workerClient;
    private final QuestionAnswerAdvisor policyAdvisor;
    private final OrderTools orderTools;

    WorkflowController(@Qualifier("supportClient") ChatClient supportClient,
                       @Qualifier("workerClient") ChatClient workerClient,
                       QuestionAnswerAdvisor policyAdvisor, OrderTools orderTools) {
        this.supportClient = supportClient;
        this.workerClient = workerClient;
        this.policyAdvisor = policyAdvisor;
        this.orderTools = orderTools;
    }

Use @Qualifier(“workerClient”) and @Qualifier(“supportClient”) to select the intended ChatClient explicitly. By default, each bean’s name comes from its @Bean method name. Review other ChatClient injection points when adding the second bean so their choice remains unambiguous. The advisor and order tools each have a single bean in this example.

The methods in the next sections go into this class. Start Ollama and the app as shown in Getting Started, and restart the app after adding the code. If the vector store is empty, load the policies once:

curl -X POST localhost:8080/ingest

3. The Five Patterns in OrderDesk

Each section explains when to use the pattern, shows the implementation, and examines a recorded example response. The samples use qwen2.5:3b on a CPU. They illustrate both the control flow and the kinds of mistakes a workflow must handle; they are not a guarantee of repeatable model output.

3.1. Chaining: Process a Request in Stages

Use a chain when a task has a known sequence of dependent steps. Each step consumes the previous result. Here, OrderDesk summarizes a complaint, drafts a response using policy retrieval, and rewrites the draft for the customer:

    // 1. Chain: each step's output is the next step's input
    @GetMapping("/agents/chain")
    List<String> chain(@RequestParam String complaint) {
        String summary = workerClient.prompt()
            .user("Summarize this customer complaint in one sentence: " + complaint)
            .call().content();
        String draft = supportClient.prompt()
            .user("Write a reply to this complaint: " + summary)
            .advisors(policyAdvisor)
            .call().content();
        String polite = workerClient.prompt()
            .user("Rewrite this reply so it is polite and at most three sentences. "
                + "Keep every fact. Reply: " + draft)
            .call().content();
        return List.of("SUMMARY: " + summary, "DRAFT: " + draft, "FINAL: " + polite);
    }

1. The summary is intended to remove emotional wording while retaining the facts. Its output becomes the input to the policy search.
2. The drafting call uses supportClient with policyAdvisor to retrieve relevant policy sections and prepare a response.
3. The final call asks for a shorter, more polite version of that draft.

curl -G localhost:8080/agents/chain --data-urlencode "complaint=I ordered a jacket four days ago and it arrived today with a broken zip. This is the second time! I want my money back."

["SUMMARY: The customer ordered a jacket four days ago and received a defective product with a broken zip, requesting a refund due to previous issues.","DRAFT: I'm sorry to hear about the issue with your jacket. Please upload a photo of the damaged item and the parcel on the order page within 48 hours of delivery. We will then proceed to either replace the item or issue a full refund, and you will not be responsible for the return pickup. Please note that refunds typically go back to the original payment method within 5 to 7 working days after we receive the item. If you choose store credit, it will be added to your OrderDesk wallet at once.","FINAL: I apologize about the jacket issue. Please upload photos of the damaged item and the parcel on the order page within 48 hours. We will then process either a replacement or refund, and you won't need to return the item. Refunds usually go back to your original payment within 5-7 working days, and store credit is added immediately to your OrderDesk wallet."]

The draft says the customer is not responsible for arranging the return pickup. The final rewrite changes that to “you won’t need to return the item,” which means something different. It also contains four sentences despite the three-sentence instruction.

This is the main risk of chaining: each additional transformation can alter an earlier fact. Compare the final answer with the evidence used to draft it. If a separate rewrite adds little value, remove that step and ask the drafting call for the required tone and length.

3.2. Routing: Choose the Right Handler

Use routing when different requests need different capabilities. A policy question needs retrieval, an order-status question needs a database tool, and an unrelated question can receive a fixed response. The classifier chooses among POLICY, ORDER, and OTHER:

    // 2. Routing: classify first, then send the question to the right handler
    enum Route { POLICY, ORDER, OTHER }

    record RouteDecision(Route route, String reason) {
    }

    @GetMapping("/agents/route")
    List<String> route(@RequestParam String question) {
        RouteDecision decision = workerClient.prompt()
            .user("""
                Classify the customer question.
                POLICY: about returns, delivery, warranty or payment rules.
                ORDER: about one specific order or a customer's orders.
                OTHER: anything else.
                Question: """ + question)
            .call()
            .entity(RouteDecision.class);
        log.info("Route: {}", decision);

        String answer = switch (decision.route()) {
            case POLICY -> supportClient.prompt().user(question)
                .advisors(policyAdvisor).call().content();
            case ORDER -> supportClient.prompt().user(question)
                .tools(orderTools).call().content();
            case OTHER -> "OrderDesk can help with orders and store policies only.";
        };
        return List.of("ROUTE: " + decision.route() + " (" + decision.reason() + ")",
            "ANSWER: " + answer);
    }

The entity(RouteDecision.class) call requests a structured response and maps it to the record. The enum represents the three supported routes, and the switch selects the handler. Each handler receives only the capabilities it needs: policy retrieval, order tools, or a fixed response without another model call.

Validate the mapped decision before entering the switch. Invalid output can cause conversion to fail or leave the route missing. Decide how the application should handle these cases, such as a controlled retry or a fallback response.

curl -G localhost:8080/agents/route --data-urlencode "question=Can I pay cash when the parcel arrives?"
curl -G localhost:8080/agents/route --data-urlencode "question=Has order ORD-1040 been delivered?"
curl -G localhost:8080/agents/route --data-urlencode "question=Can you recommend a good movie?"

["ROUTE: POLICY (payment rules)","ANSWER: Cash on delivery is available for orders up to INR 5,000, and it costs INR 30 extra per order. For orders above INR 499, delivery is free. If you have a damaged or wrong item, please report it within 48 hours of delivery."]
["ROUTE: ORDER (Has order ORD-1040 been delivered?)","ANSWER: The order ORD-1040 has been shipped from Ahmedabad, but the delivery status is not yet updated. Please check back later or contact customer support for more information."]
["ROUTE: OTHER (Can you recommend a good movie?)","ANSWER: OrderDesk can help with orders and store policies only."]

The classifier selected the expected route in all three sample requests. The unrelated movie question required only the classification call. However, the order response describes Ahmedabad as the shipping origin, although the database field represents the delivery city. Correct routing selects the right handler; it does not verify the handler’s final answer.

3.3. Parallelization: Run Independent Checks Together

Use parallelization when tasks do not depend on one another’s results. Here, three reviewers independently check the same draft against returns, shipping, and payment rules:

    // 3. Parallelization: independent checks run at the same time
    record PolicyCheck(String policy, boolean ok, String issue) {
    }

    @GetMapping("/agents/parallel")
    List<PolicyCheck> parallel(@RequestParam String reply) throws Exception {
        List<String> rules = List.of(
            "returns: items can be returned within 10 days of delivery",
            "shipping: delivery is free above INR 499, otherwise INR 49",
            "payments: cash on delivery is allowed up to INR 5,000");
        try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
            List<CompletableFuture<PolicyCheck>> checks = rules.stream()
                .map(rule -> CompletableFuture.supplyAsync(() -> workerClient.prompt()
                    .user("""
                        Policy: %s
                        Support reply: %s
                        Does the reply contradict this policy? Set ok to false only if it
                        states something different from the policy.""".formatted(rule, reply))
                    .call()
                    .entity(PolicyCheck.class), executor))
                .toList();
            return checks.stream().map(CompletableFuture::join).toList();
        }
    }

Each check runs on a separate virtual thread, and join() waits for its result. Java 21 virtual threads suit this blocking-call structure. When the model service can process the calls concurrently, the waiting time can approach that of the slowest check rather than the sum of all three.

Submitting concurrent requests does not guarantee concurrent inference. Ollama’s OLLAMA_NUM_PARALLEL setting, model size, and available hardware affect whether requests run together or wait in a queue. Hosted services also have concurrency and rate limits. The recorded CPU example below took about 71 seconds.

We sent a reply with two mistakes (30 days, and cash for any amount) and one correct fact (free delivery above INR 499):

curl -G localhost:8080/agents/parallel --data-urlencode "reply=You can return any item within 30 days. Delivery is free above INR 499, and you can pay cash on delivery for any order amount."

[{"policy":"items can be returned within 10 days of delivery","ok":false,"issue":"The reply states that items can be returned within 30 days, which contradicts the policy that items can be returned within 10 days of delivery."},{"policy":"Delivery is free above INR 499, and you can pay cash on delivery for any order amount.","ok":false,"issue":"The support reply states that delivery is free above INR 499, which contradicts the stated policy that delivery is free above INR 499. Additionally, the reply mentions that you can pay cash on delivery for any order amount, which is not part of the stated policy."},{"policy":"Policy: payments: cash on delivery is allowed up to INR 5,000","ok":false,"issue":"The reply states that you can pay cash on delivery for any order amount, which contradicts the policy that allows cash on delivery up to INR 5,000."}]

The returns and payments reviewers detect the two intended mistakes. The shipping reviewer incorrectly rejects the valid free-delivery rule and copies the reply into the policy field.

Treat reviewer output as another model response that can be wrong. The application already knows which policy each check evaluates, so it can use that known value as the label. A production implementation also needs timeouts, concurrency limits, and a defined response when one check fails to complete.

3.4. Orchestrator and Workers: Break Down a Request

Use this pattern when the required subtasks depend on the incoming request. The orchestrator proposes a plan, and workers answer its subtasks. Unlike a chain, the subtask list is generated at runtime. In this implementation, the worker calls run sequentially and the endpoint returns their answers as a list; it does not make a final synthesis call:

    // 4. Orchestrator and workers: one call plans the subtasks, workers do them
    record Plan(List<String> subtasks) {
    }

    @GetMapping("/agents/orchestrate")
    List<String> orchestrate(@RequestParam String request) {
        Plan plan = workerClient.prompt()
            .user("""
                Split this request into at most four separate questions that a support
                agent can answer one by one from the store policies. Request: """ + request)
            .call()
            .entity(Plan.class);
        log.info("Plan: {}", plan.subtasks());

        List<String> results = new ArrayList<>();
        for (String subtask : plan.subtasks()) {
            // each worker gets the whole request too, so details such as the city survive
            String answer = supportClient.prompt()
                .user("Customer request: " + request + "\nAnswer only this part: " + subtask)
                .advisors(policyAdvisor).call().content();
            results.add(subtask + " -> " + answer);
        }
        return results;
    }

Each worker receives both its assigned subtask and the original request. This preserves details such as city and order value if the planner leaves them out of a shortened subtask. There is a tradeoff: the longer combined message also becomes the policy-retrieval query and may make it less focused.

curl -G localhost:8080/agents/orchestrate --data-urlencode "request=I live in Nagpur and want to buy a phone for INR 6,000. How long will delivery take, what will it cost, what warranty do I get, and can I pay cash?"

["How long will delivery take for a phone priced at INR 6,000 from Nagpur? -> Delivery to Nagpur will take 5 days for a phone priced at INR 6,000.","What will be the cost of delivery for a phone priced at INR 6,000 from Nagpur? -> The cost of delivery for a phone priced at INR 6,000 from Nagpur will be free since your order qualifies for free delivery as it is above INR 499.","What warranty do I get for a phone priced at INR 6,000 from Nagpur? -> For a phone priced at INR 6,000, the warranty information is not provided in the given context. I cannot answer what warranty you get for the phone.","Can I pay cash for a phone priced at INR 6,000 from Nagpur? -> I'm sorry, but I can't answer your question about paying cash for a phone priced at INR 6,000 from Nagpur. The context provided does not include information about cash on delivery options for orders above INR 5,000."]

In this sample, the planner retains the city and order value in each subtask. The delivery estimate of five days and free delivery match the example’s policy. Keeping the original request available protects against other plans that omit those details.

Two workers still fail to give a useful answer. Their policy retrieval uses the combined user message and subtask, with at most three sections passing the configured similarity threshold of 0.5. A worker can miss the evidence it needs or misunderstand evidence that was retrieved.

The warranty worker reports insufficient information. The cash-on-delivery worker mentions the INR 5,000 limit but fails to apply it to the INR 6,000 order. The expected conclusion is that this order exceeds the limit.

Inspect the retrieved sections before changing the prompt or search settings. A focused retrieval query may improve relevance; changing the threshold requires checking the results. Also validate the generated plan before executing it: asking for at most four subtasks in a prompt does not enforce that limit in Java.

3.5. Evaluator and Optimizer: Review and Revise an Answer

Use this pattern when an answer can be reviewed against clear criteria and improved with feedback. A writer produces the first draft, an evaluator returns a verdict and feedback, and the writer revises a failed draft. This implementation evaluates at most three drafts:

    // 5. Evaluator and optimizer: draft, judge, improve, at most three rounds
    record Evaluation(boolean pass, String feedback) {
    }

    @GetMapping("/agents/refine")
    List<String> refine(@RequestParam String complaint) {
        List<String> rounds = new ArrayList<>();
        String reply = supportClient.prompt()
            .user("Write a reply to this customer: " + complaint)
            .advisors(policyAdvisor)
            .call().content();
        for (int round = 1; round <= 3; round++) {
            Evaluation evaluation = workerClient.prompt()
                .user("""
                    Judge this support reply. It passes only if it is at most three sentences,
                    is polite, and gives the customer one clear next step.
                    Customer: %s
                    Reply: %s""".formatted(complaint, reply))
                .call()
                .entity(Evaluation.class);
            rounds.add("ROUND " + round + ": " + reply + " | PASS=" + evaluation.pass()
                + " | " + evaluation.feedback());
            if (evaluation.pass() || round == 3) {
                break;   // passed, or no rounds left to judge an improved reply
            }
            reply = supportClient.prompt()
                .user("""
                    Improve this reply using the feedback. Keep every fact.
                    Reply: %s
                    Feedback: %s""".formatted(reply, evaluation.feedback()))
                .call().content();
        }
        return rounds;
    }
}

The loop stops when a draft passes or the third evaluation finishes. That means at most two revisions, with no extra revision after the final evaluation. The limit prevents indefinite retries.

The revision call does not attach policyAdvisor, so it performs no fresh policy retrieval. It still uses supportClient and its default system instructions, along with the previous draft and feedback. Important supporting evidence should remain available throughout the revision process.

curl -G localhost:8080/agents/refine --data-urlencode "complaint=My headphones stopped working after two weeks. I opened the seal of course. I want a refund."

["ROUND 1: I'm sorry, but you cannot return headphones once the seal is opened. According to the policy provided, we do not accept returns for earphones and headphones after the seal is broken. However, if the headphones are damaged or not as described, you can report this within 48 hours of delivery by uploading a photo of the item and the parcel on the order page. We will then send a replacement or provide a full refund. Please let me know if you need further assistance. | PASS=true | The reply is at most three sentences, polite, and provides a clear next step for the customer to upload a photo and report the issue for a replacement or refund."]

The evaluator approves a five-sentence reply while describing it as “at most three sentences.” It also overlooks that a 48-hour reporting rule does not apply two weeks after delivery.

A model verdict is not proof that the answer meets the requirements. Enforce deterministic constraints in application code where practical, and assess factual claims against the relevant policies or order data. Use model review for criteria that need judgment, while retaining a clear stopping condition and a fallback for drafts that never pass.

4. Choosing a Workflow Pattern

Start with the structure of the task. The following diagram summarizes which pattern to consider:

Patterns can be combined when each addition has a clear purpose. A router might select a policy-answering chain, or independent reviewers might check a worker’s answer. Keep the overall execution path easy to inspect, and avoid adding model calls without a measurable benefit.

If the application cannot define suitable steps or branches in advance, a more dynamic agent may be appropriate. That introduces additional decisions about tool access, execution limits, and recovery. The next article explores this approach with Embabel.

5. Practical Limits to Plan For

5.1. Control Latency and the Number of Calls

Every added model call contributes latency and consumes tokens. In the recorded CPU examples, the three-call chain took about 100 seconds and the five-call orchestrator took about 75 seconds. These are observations from that setup, rather than comparative benchmarks. Use only the stages the task needs, bound the number of subtasks and revisions, and run independent work concurrently when the model service supports it.

5.2. Validate Model Decisions and Final Answers

Validate routes, required fields, plan size, and loop limits in application code. Keep the source evidence available when checking factual answers. A reviewer model can provide useful feedback, but its approval should not replace checks that the application can perform directly.

6. FAQs

6.1. Is a workflow an AI agent?

Terminology varies. The examples here are workflows because Java code defines their execution structure and stopping conditions. A model may classify a request or generate a plan within that structure. An autonomous agent gives the model broader control over which action happens next.

6.2. Should every step use the same model?

No. Choose a model for each task based on measured accuracy, latency, and cost. A small model may handle a narrow classification task well, while policy reasoning or evaluation may require a more capable model. The sample reviewer mistakes show why a smaller judge should not be assumed sufficient. Separate ChatClient beans can use different model options.

6.3. How do I test a workflow?

Test the Java control flow with fixed model responses: expected routes, missing fields, invalid structured output, failed workers, and the retry limit. Then evaluate real model outputs on representative customer requests and known policy answers. The article Testing Spring AI Apps: Mocks, Testcontainers and Evaluators covers the available approaches.

7. Conclusion

Spring AI workflows combine focused model calls with explicit Java control flow. Chains handle dependent stages, routing selects a handler, parallelization handles independent work, orchestration generates subtasks, and evaluation adds a bounded revision loop.

The sample responses show that a working execution path can still produce an incorrect answer. Start with the smallest workflow that serves the request, preserve the evidence between steps, validate model decisions, and check the final result against the application’s requirements.

All articles in order: Spring AI Tutorial

8. References

• Spring AI Reference: Building Effective Agents
• Anthropic: Building Effective Agents
• Spring AI Reference: ChatClient

Drop me your questions related to Spring AI agents in the comments.

Happy Learning !!

Source Code on Github