Spring AI Custom CallAdvisor & StreamAdvisor Example

Spring AI 1.0.0 introduced advisor interfaces—specifically CallAdvisor and StreamAdvisor to help developers intercept and enrich model interactions without altering core business logic. These advisors act similarly to Spring AOP (Aspect-Oriented Programming) patterns in Spring Framework.

Internally, CallAdvisor and StreamAdvisor provide hooks to wrap around LLM invocations, both for synchronous and streaming scenarios. This article provides a complete working example demonstrating how to implement these interfaces and how to use the overridden methods adviseCall() and adviseStream().

1. CallAdvisor and StreamAdvisor API

The CallAdvisor allows us to intercept blocking/synchronous model calls to an LLM before and after execution. Inside the advisecall() method, we can modify input, log data, handle errors, and manipulate output.

public interface CallAdvisor extends Advisor {

  ChatClientResponse adviseCall(ChatClientRequest chatClientRequest, CallAdvisorChain callAdvisorChain);
}

Similar modifications can be done when streaming outputs from an LLM model using StreamAdvisor interface.

public interface StreamAdvisor extends Advisor {

  Flux<ChatClientResponse> adviseStream(ChatClientRequest chatClientRequest, StreamAdvisorChain streamAdvisorChain);
}

2. Advisor Lifecycle & Composition

Spring AI supports composable chains of advisors. This means you can register multiple advisors and they will be invoked in order. It follows a decorator pattern, where each advisor wraps the next.

The chain looks like:

Client -> Advisor1 -> Advisor2 -> … -> ModelCall

Spring AI API provides several built-in advisors, as well as we can create custom advisors using CallAdvisor and StreamAdvisor. And then we can use these advisors as follows:

String responseContent = chatClient.prompt()
      .user("... message...")
      .advisors(advisor1)
      .advisors(advisor2)
      .advisors(customAdvisor)
      .advisors(advisor3)
      .call()
      .content();

3. Creating a Custom CallAdvisor and StreamAdvisor

Create a class and implement the CallAdvisor and StreamAdvisor interfaces. Then, implement the methods similar to Spring AOP style.

In the following example, we are modifying the prompt and instructing the LLM to respond as if it is talking to a 16-year-old child.

import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.ai.chat.client.ChatClientRequest;
import org.springframework.ai.chat.client.ChatClientResponse;
import org.springframework.ai.chat.client.advisor.api.CallAdvisor;
import org.springframework.ai.chat.client.advisor.api.CallAdvisorChain;
import org.springframework.ai.chat.client.advisor.api.StreamAdvisor;
import org.springframework.ai.chat.client.advisor.api.StreamAdvisorChain;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.util.Assert;
import reactor.core.publisher.Flux;

public class CustomAdvisor implements CallAdvisor, StreamAdvisor {

  private final static Logger logger = LoggerFactory.getLogger(CustomAdvisor.class);

  @Override
  public ChatClientResponse adviseCall(ChatClientRequest chatClientRequest, CallAdvisorChain callAdvisorChain) {
    Assert.notNull(chatClientRequest, "the chatClientRequest cannot be null");

    ChatClientRequest formattedChatClientRequest = augmentWithCustomInstructions(chatClientRequest);
    return callAdvisorChain.nextCall(chatClientRequest);
  }

  @Override
  public Flux<ChatClientResponse> adviseStream(ChatClientRequest chatClientRequest, StreamAdvisorChain streamAdvisorChain) {
    Assert.notNull(chatClientRequest, "the chatClientRequest cannot be null");

    ChatClientRequest formattedChatClientRequest = augmentWithCustomInstructions(chatClientRequest);
    return streamAdvisorChain.nextStream(chatClientRequest);
  }


  private static ChatClientRequest augmentWithCustomInstructions(ChatClientRequest chatClientRequest) {

    String customInstructions = "Please respond as you are explaining it to a 16-year-old child. " +
      "Use simple words and short sentences. " +
      "If you don't know the answer, say 'I don't know'.";

    Prompt augmentedPrompt = chatClientRequest.prompt()
      .augmentUserMessage(userMessage -> userMessage.mutate()
        .text(userMessage.getText() + System.lineSeparator() + customInstructions)
        .build());

    return ChatClientRequest.builder()
      .prompt(augmentedPrompt)
      .context(Map.copyOf(chatClientRequest.context()))
      .build();
  }

  @Override
  public String getName() {
    return "CustomAdvisor";
  }

  @Override
  public int getOrder() {
    return Integer.MAX_VALUE;
  }
}

Avoid any kind of blocking calls in StreamAdvisor.

To invoke the advisor, we must pass the advisor reference to the ChatClient instance.

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.openai.OpenAiChatModel;
import org.springframework.ai.openai.api.OpenAiApi;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class CustomAdvisorExample implements CommandLineRunner {

  @Value("${spring.ai.openai.api-key}")
  String apiKey;

  public static void main(String[] args) {
    SpringApplication.run(CustomAdvisorExample.class, args);
  }

  @Override
  public void run(String... args) {

    OpenAiApi openAiApi = OpenAiApi.builder().apiKey(apiKey).build();
    OpenAiChatModel chatModel = OpenAiChatModel.builder().openAiApi(openAiApi).build();

    ChatClient chatClient = ChatClient
      .builder(chatModel)
      .build();

    String responseContent = chatClient.prompt()
      .user("My name is: Lokesh. Say my name in french and explain its meaning.")
      .advisors(new CustomAdvisor())
      .call()
      .content();

    System.out.printf("response: %s\n", responseContent);
  }
}

4. Conclusion

Spring AI’s CallAdvisor and StreamAdvisor are very handy tools that make it easy to extend your AI pipeline. Whether you’re building a chatbot, knowledge assistant, or a prompt orchestration service, advisors let you hook into the model lifecycle in clean, reusable ways.

Happy Learning !!

Source Code Download