DynamoDB with Spring Cloud AWS DynamoDbTemplate

This article will teach us to integrate with DynamoDB using the Spring-cloud-aws module. We will also look at different Spring-provided APIs that help us interact and perform various CRUD operations in DynamoDB.

1. Intro to AWS DynamoDB

DynamoDB is a fully managed, key-value NoSQL database by Amazon, designed to run high-performance applications at any scale. DynamoDB offers built-in security, continuous backups, automated multi-Region replication, in-memory caching, and data import and export tools. To learn more about DynamoDB, we can check the official documentation.

To avoid creating an AWS account and incurring any cost of running a live instance, let’s install and deploy DynamoDB locally. Check out the instructions on how to set up DynamoDB locally. 

2. Connecting with DynamoDB

2.1. Maven

To configure and connect with DynamoDB, we must provide the latest version of spring-cloud-aws dependencies.

To simplify the management of dependency versions for a given release of Spring Cloud AWS, we will use the Spring Cloud AWS Bill of Materials (BOM). By utilizing the BOM from our application’s build script, there is no need for us to specify and maintain dependency versions manually. The BOM version we use determines the versions of dependencies utilized, ensuring we are using tested and supported versions by default unless we choose to override them.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-aws-dependencies</artifactId>
            <version>3.0.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Since we will be integrating DynamoDB into our application, so we will provide a starter dependency of spring-cloud-aws-starter-dynamodb.

<dependency>
    <groupId>io.awspring.cloud</groupId>
    <artifactId>spring-cloud-aws-starter-dynamodb</artifactId>
</dependency>

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <exclusions>
        <exclusion>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Because of classloader related issue in AWS SDK DynamoDB Enhanced client, to use Spring Cloud AWS DynamoDB module together with Spring Boot DevTools we need to explicitly exclude the spring-boot-devtools dependency.

2.2. DynamoDB Client Configuration

Next, let’s add the following connection properties to the application.properties file. Note that the access and secret keys can have any random value for our local setup. When accessing a local instance of DynamoDB, these values are not used for authentication.

For connecting to a remotely installed instance, we can provide the actual keys and endpoint that we get after creating a database instance and a database user in AWS.

aws.dynamodb.accessKey=accessKey
aws.dynamodb.secretKey=secretKey
aws.dynamodb.endpoint=http://localhost:8000

Next, create beans for DynamoDbClient and DynamoDbEnhancedClient using it to connect with DynamoDB.

@Configuration
public class DynamoDBConfiguration {

  @Value("${aws.dynamodb.accessKey}")
  private String accessKey;

  @Value("${aws.dynamodb.secretKey}")
  private String secretKey;

  @Value("${aws.dynamodb.endpoint}")
  private String endpoint;

  @Bean
  public DynamoDbClient getDynamoDbClient() {
    return DynamoDbClient.builder()
        .endpointOverride(URI.create(endpoint))
        .region(Region.AP_SOUTH_1)
        .credentialsProvider(StaticCredentialsProvider.create(
            AwsBasicCredentials.create(accessKey, secretKey)))
        .build();
  }

  @Bean
  public DynamoDbEnhancedClient getDynamoDbEnhancedClient() {
    return DynamoDbEnhancedClient.builder()
        .dynamoDbClient(getDynamoDbClient())
        .build();
  }
}

The DynamoDbEnhancedClient provides a programming model that closely resembles the Java Persistence API (JPA), whereby a class can be transformed into an entity by applying specific annotations.

3. Creating a Table in DynamoDB

3.1 Model/Entity

In order to mark the MovieDetails class as a DynamoDB bean, we are utilizing the @DynamoDbBean annotation. Additionally, we are using the @DynamoDbPartitionKey annotation for declaring the variable as a partition key on the getter method and @DynamoDbAttribute maps the attribute to the specific column.

@NoArgsConstructor
@AllArgsConstructor
@Setter
@DynamoDbBean
public class MovieDetails {

  private String id;
  private String title;
  private LocalDate year;
  private String genre;
  private String country;
  private Integer duration;
  private String language;

  @DynamoDbPartitionKey
  public String getId() {
    return id;
  }

  @DynamoDbAttribute("title")
  public String getTitle() {
    return title;
  }
  
  @DynamoDbAttribute("release_year")
  public LocalDate getYear() {
    return year;
  }

  @DynamoDbAttribute("genre")
  public String getGenre() {
    return genre;
  }

  @DynamoDbAttribute("country")
  public String getCountry() {
    return country;
  }

  @DynamoDbAttribute("duration")
  public Integer getDuration() {
    return duration;
  }

  @DynamoDbAttribute("language")
  public String getLanguage() {
    return language;
  }
}

3.2. Creating a Table

The Spring AWS DynamoDB starter does not automatically create a table from bean like traditional JPA. So we have 2 options to create a table in DynamoDB:

• Using aws command
• Using DynamoDbClient createTable() API

Using AWS Command

The below command creates a table “movie_details” with column “id” as its hash key (primary / partition key).

aws dynamodb create-table \
    --table-name movie_details \
    --attribute-definitions \
        AttributeName=id,AttributeType=S \
    --key-schema \
        AttributeName=id,KeyType=HASH \
    --provisioned-throughput \
        ReadCapacityUnits=1,WriteCapacityUnits=1 \
    --region ap-south-1 \
    --output json
    --endpoint-url http://localhost:8080

Using DynamoDbClient.createTable()

In order for DynamoDbEnhancedClient to determine the appropriate table name for a given entity, it relies on a bean of type DynamoDbTableNameResolver. By default, this resolver will convert the entity’s class name into snake case format.

dynamoDbTable = dynamoDbEnhancedClient.table("movie_details", TableSchema.fromBean(MovieDetails.class));

dynamoDbTable.createTable();

The below code creates the same table, but more declaratively.

ArrayList<AttributeDefinition> attributeDefinitions = new ArrayList<>();
attributeDefinitions.add(AttributeDefinition.builder().attributeName("id").attributeType("S").build());

ArrayList<KeySchemaElement> tableKeySchema = new ArrayList<>();
tableKeySchema.add(KeySchemaElement.builder().attributeName("id").keyType(KeyType.HASH).build());

String tableName = "movie_details";
CreateTableRequest createTableRequest = CreateTableRequest.builder().tableName(tableName)
	.provisionedThroughput(ProvisionedThroughput.builder().readCapacityUnits((long) 1)
		.writeCapacityUnits((long) 1).build())
	.attributeDefinitions(attributeDefinitions).keySchema(tableKeySchema)
	.build();

try {
	dynamoDbClient.createTable(createTableRequest);
}
catch (ResourceInUseException e) {
	// table already exists, do nothing
}

4. Using DynamoDbTemplate

The DynamoDbTemplate is a high-level API to use DynamoDB built on top of DynamoDB Enhanced Client. It offers a comprehensive set of methods for carrying out standard CRUD operations on entities, as well as additional convenience methods for querying data from DynamoDB.

4.1. API

We can use the conventional save(), update(), load() or delete() methods for performing the database operations.

save(Object object) ;

update(Object  object);

load(Key key, Class<T> class);

delete(Key key, Class<T> class);
delete(Object object);

• The save() saves the specified object to the table.
• The update() updates the attributes corresponding to the mapped class properties.
• The load() method retrieves an item from a table in DynamoDB. To retrieve the desired item, the primary key (partition key) of that item must be provided.
• The delete() method allows the deletion of an item from a table. To perform this operation, an object instance of the mapped class or partition key of the object must be passed in.

4.2. Demo

Let us see how we can use the DynamoDbTemplate for various operations.

//save
MovieDetails movieDetails = new MovieDetails("MOV001", "Avengers", null, "Action", "US", 175, "English");
MovieDetails savedMovie =  dynamoDbTemplate.save(movieDetails);

Assertions.assertEquals(movieDetails.getId(), savedMovie.getId());

//update
savedMovie.setTitle("Avengers Endgame");
MovieDetails updatedMovie = dynamoDbTemplate.update(movieDetails);

Assertions.assertEquals(savedMovie.getTitle(), updatedMovie.getTitle());

//load
MovieDetails fetchedDetails = dynamoDbTemplate.load(
                Key.builder().partitionValue(updatedMovie.getId()).build(), MovieDetails.class);
Assertions.assertNotNull(fetchedDetails);

//delete
dynamoDbTemplate.delete(updatedMovie);

5. Scan and Query

We have two more ways to fetch data from DynamoDB:

• query
• scan

5.1. Query

A query operation searches only the primary key attribute values and supports a subset of comparison operators on key attribute values to refine the search process. Query operations only support an equal operator evaluation of the Primary Key, but conditional (=, <, <=, >, >=, Between, Begin) on the Sort Key.

query(QueryEnhancedRequest queryEnhancedRequest, Class<T> class);

Let us see an example:

public PageIterable<MovieDetails> queryData(String partitionKey,  String genre) {

    Map<String, AttributeValue> expressionValues = new HashMap<>();
    expressionValues.put(":value", AttributeValue.fromS(genre));

    Expression filterExpression = Expression.builder()
                .expression("genre = :val1")
                .expressionValues(expressionValues)
                .build();

    QueryConditional queryConditional = QueryConditional
                .keyEqualTo(
                        Key.builder()
                                .partitionValue(partitionKey)
                        .build());

    QueryEnhancedRequest queryRequest = QueryEnhancedRequest.builder()
                .queryConditional(queryConditional)
                .filterExpression(filterExpression)
                .build();


    PageIterable<MovieDetails> movieList = dynamoDbTemplate.query(queryRequest,MovieDetails.class);
    return  movieList;
}

5.2. Scan

A scan operation scans the entire table. We can specify filters to apply to the results to refine the values returned to us, after the complete scan. Scan operations are generally slower and more expensive as the operation has to iterate through each item in our table to get the items we are requesting.

scan(ScanEnhancedRequest scanEnhancedRequest, Class<T> class);

Let us see an example:

public PageIterable<MovieDetails> scanDataByGenre(String genre) {

    Map<String, AttributeValue> expressionValues = new HashMap<>();
    expressionValues.put(":val1", AttributeValue.fromS(genre));

    Expression filterExpression = Expression.builder()
                .expression("genre = :val1")
                .expressionValues(expressionValues)
                .build();

    ScanEnhancedRequest scanEnhancedRequest = ScanEnhancedRequest.builder()
                .filterExpression(filterExpression).build();

    PageIterable<MovieDetails> movieList = dynamoDbTemplate.scan(scanEnhancedRequest, MovieDetails.class);
    return  movieList;
}

5. Conclusion

In this article, we learned how to configure and connect with DynamoDB using the Spring Cloud AWS module. We also looked at different operations supported in DynamoDB and how to use them using DynamoDbTemplate.

Happy Learning !!

Sourcecode on Github