A chat response can take several seconds to finish. Streaming lets the application show the answer as it arrives instead of keeping the user waiting for the complete response.
In this article, we expose Spring AI output as Server-Sent Events (SSE), read it with curl and a browser, and support longer questions through POST. We also explain why plain text chunks can lose leading spaces and how JSON events preserve them.
1. Complete Responses and Streaming Responses
The synchronous call().content() chain returns the completed answer as a String. The streaming stream().content() chain returns a Flux<String>, whose values arrive as the response is generated. A chunk may be a word, part of a word, whitespace, or a larger fragment; the client must not assume token or word boundaries.
The recorded example sends the same question through both endpoint styles: “Explain your return policy for a damaged parcel in four sentences.” The timings illustrate the difference between receiving the first content and completing the full answer:
/ask: whole answer after 19.5 s, 528 bytes
/stream-json: first piece after 0.20 s, 117 pieces, last after 22.8 sThe /ask endpoint uses call(). The /stream-json endpoint uses stream(), and we build it in Section 4.
The streaming run completes later in this example, but shows its first content much earlier. Streaming improves perceived responsiveness by delivering partial output; it does not guarantee a shorter generation time. These timings are specific to the recorded local setup.
2. How Server-Sent Events Carry the Answer
SSE sends a sequence of text events over one HTTP response with content type text/event-stream. The connection remains open while events are produced, so the browser can display new content without requesting each fragment separately.
In the response body, each event is a line that starts with data:, followed by an empty line:
data:Our
data: delivery
data: chargesThe browser has a built-in JavaScript class for this format, called EventSource. It opens the connection, splits the body into events, and calls our JavaScript function once per event. On the server side, Spring MVC writes this format for us when an endpoint returns a Flux and says it produces text/event-stream.
SSE only goes one way, from the server to the browser. That’s all a chat answer needs, because the browser sends the question once, at the start, in the request.
3. The Example App
This article is part of the OrderDesk series. OrderDesk is the support assistant of a demo online store. The code for this article lives in the com.howtodoinjava.orderdesk.streaming package and the src/main/resources/static/chat.html page. The full app is on GitHub in the orderdesk folder.
The example uses Spring Boot 4.1.1, Spring AI 2.0.1, and Java 21, with the qwen2.5:3b model running in Ollama on our own computer, so we don’t need an API key. For the basic setup (Ollama, the model, the Spring Boot project, the Spring AI BOM, and the Ollama starter), see Getting Started with Spring AI.
Streaming adds nothing to that setup. Reactor comes with the Spring AI starter, and the spring-boot-starter-web dependency from Getting Started knows how to send a Flux as SSE. So we don’t add the WebFlux starter or any other dependency.
The project from Getting Started also has the supportClient bean, a ChatClient defined in the ChatClientConfig class, and the streaming endpoints use this bean. Its system prompt (the fixed instructions sent to the model with every question) holds the store’s delivery rules. These rules explain the answers we see later:
package com.howtodoinjava.orderdesk.chat;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class ChatClientConfig {
static final String SUPPORT_RULES = """
You are OrderDesk, the support assistant of an online store.
Answer in at most three sentences.
Delivery: Pune, Mumbai and Delhi in 2 days; every other city in India in 5 days.
Delivery is free above INR 499; below that it costs INR 49.
We do not deliver outside India.
""";
@Bean
ChatClient supportClient(ChatClient.Builder builder) {
return builder
.defaultSystem(SUPPORT_RULES)
.build();
}
}The system prompt has no return policy. Keep that in mind for Step 4.
The /ask endpoint from Section 1 also comes from Getting Started. It’s in ChatController and uses ChatClient’s call():
@GetMapping("/ask")
String ask(@RequestParam String question) {
return supportClient.prompt()
.user(question)
.call()
.content();
}Put new classes under the package of the main class, OrderDeskApplication, so that Spring Boot finds them. In our project, that package is com.howtodoinjava.orderdesk. If your Getting Started project uses another base package, use yours.
4. Build the Endpoint and Browser Client
Now we build the streaming endpoints and read them in three ways: first with curl, then with EventSource in the browser, and finally with fetch() for long questions.
| Step | Command | What you have afterwards |
|---|---|---|
| 1 | none (code only) | A GET /stream endpoint that sends SSE |
| 2 | curl -N ... | The raw events, as the server sends them |
| 3 | open http://localhost:8080/chat.html | The answer in the browser: first without spaces, then with the JSON fix |
| 4 | click “Ask (POST)” in chat.html | A fetch() reader for questions too long for a URL |
Steps 2 and 3 each need only Step 1. Step 4 also uses the page from Step 3.
Step 1: Create a Streaming Endpoint
Let’s start with the controller. Create StreamController in the com.howtodoinjava.orderdesk.streaming package. It gets the supportClient bean in its constructor and has three endpoints. In this step, we look at the first one, /stream, and Steps 3 and 4 use the other two.
package com.howtodoinjava.orderdesk.streaming;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;
@RestController
public class StreamController {
private static final Logger log = LoggerFactory.getLogger(StreamController.class);
private final ChatClient supportClient;
StreamController(@Qualifier("supportClient") ChatClient supportClient) {
this.supportClient = supportClient;
}
// For EventSource in the browser, which can only send GET requests
@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
Flux<String> stream(@RequestParam String question) {
return supportClient.prompt()
.user(question)
.stream()
.content();
}
// Each piece wrapped in JSON, so the browser keeps its leading space
@GetMapping(value = "/stream-json", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
Flux<Map<String, String>> streamJson(@RequestParam String question) {
return supportClient.prompt()
.user(question)
.stream()
.content()
.map(piece -> Map.of("text", piece))
.doOnCancel(() -> log.info("Client disconnected, model call cancelled"));
}
// For fetch() in the browser, when the question is too long for a URL
@PostMapping(value = "/stream-json", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
Flux<Map<String, String>> streamJsonPost(@RequestBody String question) {
return streamJson(question);
}
}The @Qualifier(“supportClient”) annotation picks the supportClient bean by name. With only the code from Getting Started, the app has one ChatClient bean, so the qualifier is optional. The finished OrderDesk app has several ChatClient beans, and then the name matters.
The /stream endpoint differs from the /ask endpoint in two places only:
- It calls .stream().content() instead of .call().content(), so it returns a Flux<String> right away, before the model has written anything.
- The produces attribute is MediaType.TEXT_EVENT_STREAM_VALUE, which is the string “text/event-stream”. It tells Spring MVC to write each String from the Flux as one SSE event.
Spring MVC subscribes to the returned Flux and writes emitted values into the HTTP response as SSE events. The response stays open until completion, failure, cancellation, or timeout. The streaming pipeline starts when it is subscribed to, rather than merely when the Flux is constructed.
Start Ollama and the app as shown in Getting Started with Spring AI. If the app is already running, restart the app after adding the code.
Step 2: Test the Stream with curl
Next, we look at the raw events with curl. The -N flag makes curl print each event as soon as it arrives:
curl -N "localhost:8080/stream?question=Explain%20your%20delivery%20charges%20in%20two%20sentences."These are the first events of the answer, exactly as the server sent them:
data:Our
data: delivery
data: charges
data: are
data: IN
data:R
data:
data:4
data:9
data: forEach piece from the model became one event. The space after data: belongs to the piece, because the model sends ” delivery” with a space at the start. Also, the pieces aren’t always whole words: “INR 49″ arrived as ” IN”, “R”, ” “, “4”, and “9”.
The browser in the next step reads these spaces differently.
Step 3: Read the Stream in the Browser
Now let’s show the answer on a web page. Spring Boot serves every file in src/main/resources/static as it is. Create the static folder if it doesn’t exist yet. The chat.html page is then available at http://localhost:8080/chat.html. It has a text box, three buttons, and one <p> element for the answer. The third button, “Ask (POST)”, calls askLong(), which we look at in Step 4:
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>OrderDesk chat</title>
<style>
body { font-family: sans-serif; max-width: 640px; margin: 40px auto; }
#answer { white-space: pre-wrap; border: 1px solid #ccc; padding: 12px; min-height: 80px; }
</style>
</head>
<body>
<h1>OrderDesk</h1>
<input id="question" size="60" value="Explain your delivery charges in two sentences.">
<button onclick="askPlain()">Ask (plain text)</button>
<button onclick="askJson()">Ask (JSON)</button>
<button onclick="askLong(document.getElementById('question').value)">Ask (POST)</button>
<p id="answer"></p>
<script>
const answer = document.getElementById("answer");
const question = () => encodeURIComponent(document.getElementById("question").value);
// Plain text events: the browser drops the space after "data:"
function askPlain() {
answer.textContent = "";
const source = new EventSource("/stream?question=" + question());
source.onmessage = (event) => answer.textContent += event.data;
source.onerror = () => source.close(); // the server closed the stream
}
// JSON events: the space stays inside the "text" field
function askJson() {
answer.textContent = "";
const source = new EventSource("/stream-json?question=" + question());
source.onmessage = (event) => answer.textContent += JSON.parse(event.data).text;
source.onerror = () => source.close();
}
// POST with fetch(), for questions too long for a URL
async function askLong(text) {
answer.textContent = "";
const response = await fetch("/stream-json", {
method: "POST",
headers: { "Content-Type": "text/plain" },
body: text
});
const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
let buffer = "";
while (true) {
const { value, done } = await reader.read();
if (done) break;
buffer += value;
const events = buffer.split("\n\n");
buffer = events.pop(); // an event cut in half waits for the next read
for (const event of events) {
if (event.startsWith("data:")) {
answer.textContent += JSON.parse(event.slice(5)).text;
}
}
}
}
</script>
</body>
</html>The askPlain() function opens an EventSource on /stream. For each event, the browser calls onmessage, and event.data holds the text after data:. We append that text to the answer.
The onerror handler matters too. When the answer is complete, the server closes the response. EventSource treats a closed connection as an error and, by default, reconnects after a few seconds. For a chat answer, a reconnect would send the same question again. So we close the EventSource ourselves when that happens.
Open the page and click “Ask (plain text)”. The page shows:
OurdeliverychargesarestructuredsuchthattheyarefreeforordersaboveINR499,andforordersbeloworincludingINR499,thechargeisINR49.
In this sample, leading spaces disappear because SSE parsing removes one optional space immediately after the data: delimiter. The emitted fragment’s first space occupies that position, so appending parsed events joins words together. The raw curl output makes this easy to miss.
Wrap each fragment in a JSON object such as {“text”:” delivery”}. The whitespace is then inside a JSON string and survives SSE parsing. The /stream-json endpoint maps each fragment to a Map containing text; its cancellation hook is discussed in Section 5.
Spring MVC converts each Map to JSON with Jackson, the JSON library that comes with spring-boot-starter-web, before it writes the event. Let’s look at the raw events with curl:
curl -N "localhost:8080/stream-json?question=Explain%20your%20return%20policy%20for%20a%20damaged%20parcel%20in%20four%20sentences."These are the first two events:
data:{"text":"Our"}
data:{"text":" return"}In the page, the one line of JavaScript that changes is in askJson(). It reads /stream-json and parses each event with JSON.parse(event.data).text. Click “Ask (JSON)”, and the answer keeps its spaces:
Our delivery charges are INR 49 for orders below INR 499 and are free for orders above INR 499.
The wording differs from the plain-text answer because each click gets a new answer.
Step 4: Send a Long Question with fetch()
Use POST when the question belongs in a request body, for example a long message or pasted order history. Native EventSource uses GET and cannot set an arbitrary body, so this example reads the streaming POST response with fetch().
StreamController already has the endpoint for this. The POST /stream-json endpoint reads the question from the request body and returns the same JSON events as GET /stream-json. The @RequestBody String parameter reads the raw text of the body as one String.
The browser side uses fetch() instead of EventSource. The fetch() function is the browser’s JavaScript function for sending any HTTP request. It gives us the response body as a ReadableStream, a browser object that hands over the body in chunks as they arrive, and we split those chunks into events ourselves. This is the askLong() function in chat.html from Step 3.
TextDecoderStream converts incoming bytes to text. Network chunks do not align with SSE events: a read can contain several events or only part of one. The example retains incomplete text in a buffer and processes complete event frames. Its parser targets the simple data-only format emitted by these endpoints.
Unlike EventSource, this code doesn’t reconnect when the stream ends. The loop stops when done is true.
We can also test the POST endpoint with curl. The -X POST flag sets the method, and –data sends the question as the body:
curl -N -X POST localhost:8080/stream-json -H "Content-Type: text/plain" --data "I bought a kettle on 2 October, order ORD-1042. It arrived with a cracked lid and the box was wet. I want to return it. Explain your return policy for a damaged parcel in four sentences."The events start like this:
data:{"text":"Our"}
data:{"text":" return"}
data:{"text":" policy"}To try it in the browser, paste the same question into the text box and click “Ask (POST)”. The answer streams in with its spaces:
Our return policy allows for the return of damaged or defective items within 14 days of delivery. For your kettle, if the lid is cracked and the box was wet, you should initiate a return. Please include all original packaging and any damaged parts with your return. We will process your refund or exchange as per your preference.
The transport works, but the sample answer invents a fourteen-day return window. The system prompt contains delivery rules, not a return policy. Streaming affects delivery of the answer rather than factual accuracy. Supply the actual policy through instructions, retrieval, or tools before using the response as support guidance.
5. When the Client Disconnects
Sometimes the user closes the tab, or clicks “Ask” again, before the answer is complete. The browser then closes the connection. When Spring MVC tries to write the next piece, it finds the connection closed and cancels the Flux. Cancelling the Flux stops Spring AI from reading more of the model’s answer.
The /stream-json endpoint has a doOnCancel() call that logs this moment. The doOnCancel() method runs the given code when the subscriber cancels the Flux. In this case, the subscriber is Spring MVC. To test it, we let curl give up after 2 seconds:
curl -N --max-time 2 "localhost:8080/stream-json?question=Explain%20your%20return%20policy%20for%20a%20damaged%20parcel%20in%20four%20sentences."The app logged this line:
INFO ... c.h.o.streaming.StreamController : Client disconnected, model call cancelledUse doOnCancel() when you need to clean up, for example to stop a timer or to record that the answer was not finished. The POST endpoint calls streamJson(), so it has the same doOnCancel(). The /stream endpoint has no doOnCancel(), so it logs nothing, but its Flux is cancelled in the same way.
6. Proxies That Buffer the Stream (Not Tested Here)
In production, there is often a reverse proxy between the browser and the app, such as nginx. A reverse proxy is a server that receives the browser’s request and forwards it to our app.
The problem with a proxy is that it may collect the whole response in a buffer before it sends it on. In that case, the user waits 20 seconds again, even though our app streams.
Compression can do the same. To compress well, gzip waits for more text before it sends anything.
These are the usual settings to check:
- In nginx, turn buffering off for the streaming path with proxy_buffering off;.
- Turn gzip off for the streaming path in the proxy.
- Spring Boot doesn’t compress text/event-stream by default. Keep it that way: don’t add it to server.compression.mime-types.
An nginx location for the JSON stream could look like this:
location /stream-json {
proxy_pass http://localhost:8080;
proxy_buffering off;
gzip off;
}Other proxies, load balancers, and CDNs have their own buffering settings. If the stream works with curl against the app but not through the proxy, look at the proxy first.
7. Streaming Structured Output
Sometimes, we want the answer as a Java object, for example a record with a delivery quote. With call(), the entity() method converts the answer for us. Spring AI 2.0 has no streaming version of entity() yet. A half-written JSON object can’t be converted, so you would have to collect the whole answer first, and then streaming gives no benefit.
Use ChatClient’s stream() for text that a person reads, and call().entity() for data that your code reads. Spring AI Structured Output Converters shows the converters.
8. FAQs
8.1. Do I need Spring WebFlux to stream from Spring AI?
No. A Spring MVC controller can return a Flux with produces = “text/event-stream”, and Spring MVC sends it as SSE. Reactor comes with Spring AI, so the spring-boot-starter-web dependency is enough.
8.2. Why does my streamed answer have no spaces in the browser?
The browser removes one space after data: in each event, and many pieces start with a space. The curl output still shows the spaces, which makes the bug confusing. Wrap each piece in JSON, as the /stream-json endpoint does, and read it with JSON.parse(event.data).
8.3. Why does EventSource send the same question again after the answer?
When the server closes the stream, EventSource reconnects by default. Close it in the onerror handler, as chat.html does.
8.4. Can I use advisors with ChatClient’s stream()?
Yes. Advisors that implement StreamAdvisor run on streaming calls. Spring AI Custom CallAdvisor & StreamAdvisor Example shows how to write one.
8.5. What happens if an answer takes longer than the request timeout?
Spring MVC runs a streaming response as an async request, and an async request has a timeout. If spring.mvc.async.request-timeout is not set, the server’s own default applies. For Tomcat, the server that comes with spring-boot-starter-web, that default is 30 seconds. If your answers take longer, set spring.mvc.async.request-timeout in application.properties to a value above your longest answer. We did not test this setting; our answers finished in under 23 seconds.
9. Conclusion
Use stream().content() to obtain incremental text, return the Flux from the MVC endpoint, and declare text/event-stream. The recorded example demonstrates an earlier first fragment, rather than a guaranteed performance improvement for every request.
Preserve chunk contents exactly. JSON event payloads protect leading spaces, and browser code should append parsed text without trimming it.
Handle cancellation and close browser connections deliberately. Before deployment, check request timeouts and proxy buffering so the stream reaches the user incrementally rather than arriving as one delayed response.
All articles in order: Spring AI Tutorial
10. References
Drop me your questions related to Spring AI streaming in the comments.
Happy Learning !!
