Spring AI with Google Gemini: Setup and Thinking Budget

Spring AI lets a Spring Boot application use Google Gemini and a local Ollama model through the same ChatClient API. We create one client for each backend and let the caller select it with a request parameter.

This tutorial covers the Google GenAI starter, API-key configuration, response metadata, and a Gemini thinking-budget option. The recorded answers use Ollama. The Gemini setup and authentication errors illustrate the hosted integration without claiming a successful response using a valid account key.

1. How Spring AI Connects to Gemini and Ollama

OrderDesk is the demo store’s support assistant used throughout this series. This separate example retains its Ollama backend and adds a Gemini client, making both available through one endpoint.

Both backends accept messages and return an answer. Their deployment, pricing, options, and performance differ. Choose a model by evaluating your actual requests rather than assuming every hosted model will outperform every local one.

• Ollama runs inference locally. These calls avoid a hosted inference charge, while using the machine’s compute and memory.
• Gemini runs on Google’s service. Prompts leave the local application, and usage is governed by the chosen model, account tier, quotas, and pricing.

The application exposes both choices for comparison. It does not route questions automatically or fall back to Ollama after a Gemini error.

If each provider had its own Java API, you’d write the same code twice for two providers. Spring AI avoids this by defining a common interface called ChatModel. Each provider module implements this interface with its own class: GoogleGenAiChatModel for Gemini and OllamaChatModel for Ollama. The Spring AI Chat Model API page describes this interface.

The controller uses ChatClient, which wraps one ChatModel. Calls such as prompt(), user(…), and call() build and send a request without making the controller construct Google-specific HTTP messages:

In our app, the caller chooses the model with model=hosted or model=local. There is no automatic routing between the two.

The recorded answer later in the article comes from Ollama. Gemini error output shows a rejected request rather than a model-generated answer. No successful Gemini response or provider usage comparison is claimed.

2. Configure the Spring Boot Application

This example is a separate module in the OrderDesk series called orderdesk-gemini. It isn’t part of the main OrderDesk app, so you can either create a new Spring Boot project or clone the module from the spring-ai-examples repository.

All classes belong to the com.howtodoinjava.orderdesk.gemini package. The module uses Spring Boot 4.1.1, Spring AI 2.0.1, and Java 21.

The local model needs Ollama running with the qwen2.5:3b model. If you need help installing both, follow the Getting Started with Spring AI tutorial.

2.1. Dependencies

Here is the full pom.xml. The Spring AI BOM (bill of materials) sets the version of every Spring AI module, so the starters need no version:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>4.1.1</version>
    <relativePath/>
  </parent>

  <groupId>com.howtodoinjava</groupId>
  <artifactId>orderdesk-gemini</artifactId>
  <version>1.0.0</version>

  <properties>
    <java.version>21</java.version>
  </properties>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-bom</artifactId>
        <version>2.0.1</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-webmvc</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-starter-model-ollama</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-starter-model-google-genai</artifactId>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
      </plugin>
    </plugins>
  </build>
</project>

The spring-boot-starter-webmvc dependency is the Spring Boot 4 starter for REST controllers. The spring-ai-starter-model-ollama starter adds the OllamaChatModel bean, and spring-ai-starter-model-google-genai adds the GoogleGenAiChatModel bean.

2.2. Choose Gemini Developer API or Vertex AI

You can use Gemini in two ways, and the Spring AI starter supports both:

1. Gemini Developer API: you create an API key and send it with each request. This is the quick way to start, and we use it here.
2. Vertex AI: Gemini runs in your Google Cloud project. Instead of a key, you set a project ID and a location, and the app logs in with Google Cloud credentials.

The basic ChatClient call can remain the same while authentication and service settings change. The Spring AI Google GenAI reference lists the Vertex AI properties and authentication options.

2.3. Configure an API Key

Create a key in Google AI Studio. Before you use it, check the free limits and prices on Google’s Gemini API pricing page. The key is a secret, so keep it out of your code and your Git repository. Put it in an environment variable, in the same terminal where you start the app:

export SPRING_AI_GOOGLE_GENAI_API_KEY=your-gemini-key

On Windows PowerShell, use $env:SPRING_AI_GOOGLE_GENAI_API_KEY=”your-gemini-key”. Spring Boot binds this environment variable to the property spring.ai.google.genai.api-key.

The app needs this key even if you only want to try the local model. Without a key (and without a Vertex AI project and location), the app doesn’t start:

Caused by: java.lang.IllegalStateException: Incomplete Google GenAI configuration: Provide 'api-key' for Gemini API or 'project-id' and 'location' for Vertex AI.

Both model beans are configured during startup. Choosing Ollama for an individual request does not disable Gemini configuration.

For the local-only demonstration, a nonempty placeholder such as dummy-key passes this sample’s startup check. It does not provide Gemini access: Google rejects it when the hosted client sends a request.

com.google.genai.errors.ClientException: 400 . API key not valid. Please pass a valid API key.

The requests with model=local still work because they never reach Google. We recorded the local answer in Section 4 this way.

2.4. Select Models and Set Response Limits

Here is the full application.properties:

spring.application.name=orderdesk-gemini
server.port=8084

# Two chat models in one app, so we build the ChatClients ourselves
spring.ai.chat.client.enabled=false

# The local model, as in the other OrderDesk articles
spring.ai.ollama.base-url=http://localhost:11434
spring.ai.ollama.chat.model=qwen2.5:3b
spring.ai.ollama.chat.temperature=0.2

# The key comes from the environment variable SPRING_AI_GOOGLE_GENAI_API_KEY
spring.ai.google.genai.chat.model=gemini-2.5-flash
spring.ai.google.genai.chat.max-output-tokens=500
spring.ai.google.genai.chat.temperature=0.2

The Ollama settings select the local server and qwen2.5:3b. The Gemini settings select a hosted model and a 500-token output limit. Temperature affects sampling where supported; a low value does not ensure factual accuracy.

The sample explicitly selects gemini-2.5-flash. Check Google’s model documentation for availability, lifecycle, and supported options before changing the ID. A model change may also require different thinking settings.

We explain the line spring.ai.chat.client.enabled=false in the next section.

3. Create Separate Gemini and Ollama Clients

With two model beans present, this application builds its clients explicitly. Setting spring.ai.chat.client.enabled=false disables the auto-configured builder, and ChatConfig binds each client to its intended model. The ChatClient reference covers other multiple-model configurations.

ChatConfig creates one client per model. Both clients get the same system prompt (the instructions the model gets before each question):

package com.howtodoinjava.orderdesk.gemini;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.ollama.OllamaChatModel;
import org.springframework.ai.google.genai.GoogleGenAiChatModel;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ChatConfig {

    static final String SUPPORT = "You are OrderDesk, the support assistant of an online store. "
        + "Answer in at most three sentences.";

    // The hosted model: for questions that need a stronger model
    @Bean
    ChatClient hostedClient(GoogleGenAiChatModel chatModel) {
        return ChatClient.builder(chatModel).defaultSystem(SUPPORT).build();
    }

    // The local model in Ollama: free, for simple questions
    @Bean
    ChatClient localClient(OllamaChatModel chatModel) {
        return ChatClient.builder(chatModel).defaultSystem(SUPPORT).build();
    }
}

Injecting GoogleGenAiChatModel and OllamaChatModel by concrete type identifies each backend. Each is passed to its own ChatClient.builder(chatModel) call, and both clients receive the same support instructions.

Next, the Reply record holds what we return: the answer, the model name, and the token counts. These are the values that differ between providers:

package com.howtodoinjava.orderdesk.gemini;

// The answer plus the facts that differ between models
public record Reply(String model, String answer, Integer inputTokens, Integer outputTokens) {
}

The controller picks a client from the model request parameter. The default is hosted:

package com.howtodoinjava.orderdesk.gemini;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.metadata.ChatResponseMetadata;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class SupportController {

    private final ChatClient hostedClient;
    private final ChatClient localClient;

    SupportController(@Qualifier("hostedClient") ChatClient hostedClient,
                      @Qualifier("localClient") ChatClient localClient) {
        this.hostedClient = hostedClient;
        this.localClient = localClient;
    }

    // model=hosted (default) or model=local
    @GetMapping("/ask")
    Reply ask(@RequestParam String question,
              @RequestParam(defaultValue = "hosted") String model) {
        ChatClient client = "local".equals(model) ? localClient : hostedClient;
        ChatResponse response = client.prompt().user(question).call().chatResponse();
        ChatResponseMetadata metadata = response.getMetadata();
        return new Reply(metadata.getModel(), response.getResult().getOutput().getText(),
            metadata.getUsage().getPromptTokens(), metadata.getUsage().getCompletionTokens());
    }
}

@Qualifier tells Spring which bean to inject when multiple beans match the required type. In this example, it identifies the bean by the name of the @Bean method.

The chatResponse() method returns the full response rather than just the text. Its getMetadata() method gives us the model name and a Usage object with the token counts.

The main class is a plain Spring Boot application:

package com.howtodoinjava.orderdesk.gemini;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class GeminiApplication {

    public static void main(String[] args) {
        SpringApplication.run(GeminiApplication.class, args);
    }
}

4. Call the Endpoint and Understand the Response

Start Ollama. Then start the app with mvn spring-boot:run in the terminal where you set the key. Let’s ask the local model first:

curl -G localhost:8084/ask --data-urlencode "question=Can I return a kettle after 5 days?" \
  --data-urlencode model=local

{"model":"qwen2.5:3b","answer":"Yes, you can return a kettle within 5 days if it meets the store's return policy, which typically allows for quick exchanges or refunds for items received within a short timeframe. Please check the specific return policy on the product page or contact customer service for detailed instructions.","inputTokens":43,"outputTokens":55}

The recorded Ollama response took 9.9 seconds on the original laptop CPU. It is not a Gemini timing.

The answer asks the customer to check the return policy because that policy is missing from the prompt. A different provider would still need the store’s rules. Other OrderDesk examples use retrieval-augmented generation (RAG) to find relevant policy text and add it to the model’s context.

Omitting model=local selects Gemini. The controller is designed to return the same JSON fields for either backend, but no successful Gemini response is included because no valid key was available.

5. Control Thinking for a Gemini Request

Some Gemini models support a reasoning phase before producing their final answer. The controls depend on the model family. They can help with tasks that need several checks or calculations, but do not supply missing business facts.

Additional reasoning can consume tokens and increase latency. Evaluate whether it improves the result enough to justify that overhead. A thinking budget is a tuning option, rather than a guarantee of response time or quality.

For compatible Gemini 2.5 models, thinkingBudget controls the reasoning-token budget. The Spring AI Google GenAI reference describes these settings:

1. A positive number sets the maximum number of thinking tokens.
2. 0 requests that thinking be disabled, where supported by the selected model.
3. If you don’t set it, the model decides how much to think.

The next endpoint requests a zero thinking budget for one call using GoogleGenAiChatOptions. It illustrates the option for the model configured in this example:

package com.howtodoinjava.orderdesk.gemini;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.google.genai.GoogleGenAiChatOptions;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class ThinkingController {

    private final ChatClient hostedClient;

    ThinkingController(@Qualifier("hostedClient") ChatClient hostedClient) {
        this.hostedClient = hostedClient;
    }

    // Gemini 2.5 Flash thinks by default; a budget of 0 turns thinking off for this call
    @GetMapping("/ask/fast")
    String fast(@RequestParam String question) {
        return hostedClient.prompt()
            .user(question)
            .options(GoogleGenAiChatOptions.builder().thinkingBudget(0))
            .call()
            .content();
    }
}

The .options(…) call overrides this request’s thinking setting. Other configured defaults continue to apply. The ordinary /ask endpoint makes no override, so it retains the selected model’s configured or default behavior.

In Spring AI 2.0, options(…) takes the options builder, so we don’t call build() (see the Spring AI upgrade notes).

Two more points from the same Spring AI page:

• The Gemini 3 models use a different option, thinkingLevel, instead of thinkingBudget. You can’t set both in one request. If you switch to a Gemini 3 model, change the option, too.
• gemini-2.5-flash-lite doesn’t think by default. With that model, set a positive budget if you want thinking.

No successful Gemini request was recorded for /ask/fast, so this example does not establish a latency or cost saving from the setting.

6. Deployment Considerations

A few things change when real customers use the hosted model:

1. Monitor reported usage and request volume. Set an output limit and evaluate thinking settings against quality, latency, and model pricing.
2. Handle authentication errors, quotas, timeouts, and outages. An Ollama fallback needs explicit implementation; the request parameter provides no automatic recovery.
3. Customer messages leave your computer. Check that this is allowed for your data, and don’t send what you don’t need.
4. Model names change over time. Keep the model in application.properties, so switching to a newer Gemini model is a configuration change, not a code change.

The common ChatClient API lets the application reuse its general approach to memory, advisors, and retrieval. Verify provider and model support before relying on features such as tools or structured responses.

All articles in order: Spring AI Tutorial

Drop me your questions related to Spring AI with Google Gemini in the comments.

Happy Learning !!

Source Code on Github