Read a File from AWS S3 using S3Client

Learn to read a text file stored in AWS S3 bucket. We will learn to read a public file, as well as, non-public file using the access/secret keys.

1. Setup

For demo purposes, we have stored a text file ‘text.txt‘ in AWS S3 bucket ‘howtodoinjava-s3-bucket‘. We have made the file public so we can access the file directly through the URL “https://howtodoinjava-s3-bucket.s3.amazonaws.com/test.txt”.

The file content is a simple string to keep things simple.

Hello World!

2. Read a File using S3Client from AWS SDK

2.1. Maven

Before starting to code, add the AWS SDK dependencies in pom.xml. The software.amazon.awssdk:s3 provides the required classes to interact with S3 bucket and its objects. So include its latest version.

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>software.amazon.awssdk</groupId>
      <artifactId>bom</artifactId>
      <version>2.17.261</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
	<dependency>
		<groupId>software.amazon.awssdk</groupId>
		<artifactId>s3</artifactId>
	</dependency>
</dependencies>

2.2. Providing Credentials

If we are reading the file using AWS SDK, we must provide the IM credentials (access key id and secret access key). We can provide credentials in different ways.

For example, we can configure the profile-specific credentials in credentials file in the <USER_DIR>/.aws location.

[default]
aws_access_key_id = YOUR-KEY-ID
aws_secret_access_key = YOUR-KEY-VALUE

To access these credentials, use ProfileCredentialsProvider with S3ClientBuilder.

S3Client s3 = S3Client.builder()
    .credentialsProvider(ProfileCredentialsProvider.create())
    .region(region)
    .build();

Another option is to read the credentials from System properties and create an instance of AwsBasicCredentials ourselves.

var awsBasicCredentials = AwsBasicCredentials.create("YOUR-KEY-ID", "YOUR-KEY-VALUE");

S3Client s3 = S3Client.builder()
    .credentialsProvider(new AwsCredentialsProvider() {
      @Override
      public AwsCredentials resolveCredentials() {
        return awsBasicCredentials;
      }
    })
    .region(region)
    .build();

2.3. Reading the File

To read the file, create the request with GetObjectRequest and invoke the request with S3Client.getObject(). It returns an instance of ResponseInputStream that can be used to read the file content.

@Test
void testReadFileUsingS3Client() throws IOException {

  String bucketName = "howtodoinjava-s3-bucket";
  String key = "test.txt";
  Region region = Region.US_EAST_1;

  S3Client s3 = S3Client.builder()
      .credentialsProvider(ProfileCredentialsProvider.create())
      .region(region)
      .build();

  GetObjectRequest getObjectRequest = GetObjectRequest.builder()
      .bucket(bucketName)
      .key(key)
      .build();

  ResponseInputStream response = s3.getObject(getObjectRequest);

  Assertions.assertEquals("Hello World!", new String(response.readAllBytes()));
}

3. Read a Public File using URL

If the file is public, we can read this file similar to any other file available over the network. The following example reads the file using URL.

@Test
void testReadFileFromUrl() throws IOException {

  String FILE_URL = "https://howtodoinjava-s3-bucket.s3.amazonaws.com/test.txt";

  BufferedReader bufferedReader = new BufferedReader(
      new InputStreamReader(new URL(FILE_URL).openConnection().getInputStream()));

  String line;
  StringBuilder content = new StringBuilder();
  while ((line = bufferedReader.readLine()) != null)
  {
    content.append(line);
  }
  Assertions.assertEquals("Hello World!", content.toString());
}

4. Conclusion

This short tutorial taught us to read a file from AWS S3 bucket using the S3Client. We also learned to read the file over the network if the file is publicly available over the network.

Happy Learning !!

Sourcecode on Github