Spring AI Example: Generate Image from Text

LLM models such as Dall-E, Stable Diffusion (used by Stability), Midjourney, Imagen (by Google), GauGAN (by Nvidia), Pixray, etc. are capable of generating images from the supplied input text or prompt. Spring AI module has builtin support for text-to-image generation using the following providers:

  • DALL-E (by OpenAI)
  • Stable Diffusion (by Stability)

This tutorial discusses the basics of Spring AI API for image generation and demonstrates their usage with simple-to-follow examples.

1. Overview of Spring AI Image Generation API

In Spring AI, the following is the list of all the classes and interfaces used in text-to-image generation:

Core Interface/ClassDescription
ImageClientIt is a functional interface with a single method ‘call(prompt)’ and returns the ImageResponse.
ImageMessageIt encapsulates the text to use and the weight that the text should have in influencing the generated image.

ImagePrompt
It encapsulates a list of ImageMessage objects and optional request options.
ImageOptionsIt encapsulates the optional request options to be passed to the Image generation model
ImageResponseIt holds the AI Model’s output containing one of potentially multiple ImageGeneration output objects resulting from a single prompt.
ImageGenerationIt represents the output response and related metadata about a single result.
ImageGenerationMetadataIt represents the metadata associated with a single ImageResponse.

The general syntax to use an ImageClient for generating images for the provided text/prompt is:

ImageResponse response = imageClient.call(
  new ImagePrompt("A cat chasing a mouse",
	  ImageOptions.builder()
	    .withQuality("hd")
	    .withN(1)
	    .withHeight(1024)
	    .withWidth(1024)
	    .build())
);

2. Maven

We start with adding the required dependencies in the project. To add the support of OpenAI APIs, we add the spring-ai-openai-spring-boot-starter dependency. When we add this dependency, OpenAiImageClient is automatically configured as imageClient bean. To disable the autoconfiguration, set the property ‘spring.ai.openai.image.enabled‘ to ‘false‘.

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
</dependency>

To add the support of Stability APIs, we add the spring-ai-stability-ai-spring-boot-starter dependency. It autoconfigures the imageClient bean with an instance of StabilityAiImageClient. To disable the autoconfiguration, set the property ‘spring.ai.stabilityai.image.enabled‘ to ‘false‘.

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-stability-ai-spring-boot-starter</artifactId>
</dependency>

3. Creating ImageClient

Spring AI provides two classes implementing the ImageClient interface.

We can create ImageClient bean in the application’s configuration file. We can create an instance of OpenAiImageClient or StabilityAiImageClient, based on the project’s needs.

@Configuration
public class AppConfiguration {

	//Define any one bean as per requirements

	// For OpenAI
	@Bean
	ImageClient imageClient(@Value("${spring.ai.openai.api-key}") String apiKey) {
	  return new OpenAiImageClient(new OpenAiImageApi(apiKey));
	}

	//For Stability
	/*@Bean
	ImageClient imageClient(@Value("${spring.ai.stability.api-key}") String apiKey) {
	  return new StabilityAiImageClient(new StabilityAiApi(apiKey));
	}*/
}

The API key is read from the application.properties file which in turn the value from environment variables. This helps in keeping the key out of the source code, thus improving the application’s security.

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.stability.api-key=${STABILITY_API_KEY}

4. Image Generation Controller

Next, we write a REST controller that accepts the user inputs into the application and responds with the URL of the generated image. The API consumer can then use the image URL to download the image in his application or browser.

In the following example, we are generating an image using OpenAI’s image generation API. The OpenAI generates a single image for an input text in model DALL-E 3.

import org.springframework.ai.image.*;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class OpenAiImageController {

  private final ImageClient imageClient;

  public OpenAiImageController(ImageClient imageClient) {
    this.imageClient = imageClient;
  }

  @GetMapping("/image-gen")
  public String imageGen(@RequestParam String message) {

    ImageOptions options = ImageOptionsBuilder.builder()
        .withModel("dall-e-3")
        .withHeight(1024)
        .withWidth(1024)
        .build();

    ImagePrompt imagePrompt = new ImagePrompt(message, options);
    ImageResponse response = imageClient.call(imagePrompt);
    String imageUrl = response.getResult().getOutput().getUrl();

    return "redirect:" + imageUrl;
  }
}

The same code can be used for Stability API as well. We only need to configure the StabilityAiImageClient in place of OpenAiImageClient as discussed in the previous section.

5. Base URL Property

By default, the URL for image generation is selected with the default value of properties: spring.ai.openai.image.base-url or spring.ai.stabilityai.base-url. If we wish to change the endpoint URL, we can override these properties:

spring.ai.openai.image.base-url=api.openai.com

//OR

spring.ai.stabilityai.image.base-url=api.stability.ai/v1

There are several other image generation properties to govern the interaction between our application, and OpenAI APi / Stability API. You can read more about them in the official docs.

6. Demo

Let us test the image generation API by sending an input prompt and verifying the generated image’s content. The prompt we are sending is: A cat chasing a mouse.

As expected, we get the URL of the generated image. Copy the URL in a browser window for verification:

Great, we have successfully generated an image from the input prompt using Spring AI’s OpenAI image generation API.

7. Conclusion

This short Spring AI tutorial discussed the basics of image generation API related to OpenAI and Stability AI’s APIs. We also demonstrated the uses of API with an example. In the demo, we created an image from the input prompt and verified the generated image as well.

Happy Learning !!

Source Code on Github

Comments

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments

About Us

HowToDoInJava provides tutorials and how-to guides on Java and related technologies.

It also shares the best practices, algorithms & solutions and frequently asked interview questions.