Reading a File with Channels and Buffers

Learn to read small and large files from the filesystem using the Java NIO APIs Path, FileChannel, ByteBuffer and MappedByteBuffer.

  • We are using the RandomAccessFile instance that behaves like a large array of bytes stored in the file system. It uses file pointers that act as a cursor to maintain the current read location in the file.
  • A ByteBuffer represents the buffered bytes in the memory during the read/write operations.
  • A MappedByteBuffer is a direct byte buffer whose content is a memory-mapped region of a file.

1. Reading Small Files with ByteBuffer and FileChannel

Use this technique to read a small file. The idea is to create a ByteBuffer large enough where all the file content fits into the buffer, and the file can be read in a single read() operation.

try(RandomAccessFile aFile = new RandomAccessFile("test.txt", "r");
  FileChannel inChannel = aFile.getChannel();) {
  
  long fileSize = inChannel.size();

  //Create buffer of the file size
  ByteBuffer buffer = ByteBuffer.allocate((int) fileSize);
  inChannel.read(buffer);
  buffer.flip();

  // Verify the file content
  for (int i = 0; i < fileSize; i++) {
    System.out.print((char) buffer.get());
  }
} catch (IOException e) {
  e.printStackTrace();
}

2. Reading Large Files with ByteBuffer and FileChannel

Use this technique to read a large file where all the file content will not fit into the buffer at a time. To avoid OutOfMemory issues, we can read the file in chunks with a fixed size small buffer.

try (RandomAccessFile aFile = new RandomAccessFile("test.txt", "r");
  FileChannel inChannel = aFile.getChannel();) {

  //Buffer size is 1024
  ByteBuffer buffer = ByteBuffer.allocate(1024);

  while (inChannel.read(buffer) > 0) {
    buffer.flip();
    for (int i = 0; i < buffer.limit(); i++) {
      System.out.print((char) buffer.get());
    }
    buffer.clear(); // do something with the data and clear/compact it.
  }
} catch (IOException e) {
  e.printStackTrace();
}

3. Reading a File using MappedByteBuffer

MappedByteBuffer extends the ByteBuffer class with operations that are specific to memory-mapped file regions.

try (RandomAccessFile aFile = new RandomAccessFile("test.txt", "r");
  FileChannel inChannel = aFile.getChannel();) {

  MappedByteBuffer buffer = inChannel
    .map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size());

  buffer.load();
  for (int i = 0; i < buffer.limit(); i++) {
    System.out.print((char) buffer.get());
  }
  buffer.clear(); // do something with the data and clear/compact it.

} catch (IOException e) {
  e.printStackTrace();
}

All the above techniques will read the content of the file and print it to the console.

Happy Learning !!

Source Code on Github

Was this post helpful?

Join 7000+ Awesome Developers

Get the latest updates from industry, awesome resources, blog updates and much more.

* We do not spam !!

15 thoughts on “Reading a File with Channels and Buffers”

  1. MappedByteBuffer buffer = inChannel.map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size());

    —-> inChannel.size()
    it looks like size of the file. What will happen if file is having 8-10 GB. Will mappedbytebuffer is suggestible and how internally it will work without out of memory

    Reply
  2. Nice examples. I just wonder that on your first example “1) Read a small file in buffer of file size” why did you not use finally{} block for aFile resource?

    Reply
    • When you close the reader it will close the file for you, theoretically you would miss the case when the reader throws an exception in the constructor after the file has opened, but I believe the only exceptions that could occur at that time would be things like out of memory and similar which would forcibly close down your JVM anyways.

      Reply
  3. Hi Lokesh,

    I want to divide an xml file according to size using NIO. We can read char or byte only from buffer, so how can i check whether the xml file is proper or not.

    Animesh

    Reply
    • Brilliant question. And that means I do not know the answer 🙂 I will try to find a solution for this problem when time permits. I you get any solution, then please update me as well. Thanks !!

      Reply
  4. Hi,
    Could you please explain why in the example “1) Read a small file in buffer of file size”, you write:
    20 buffer.rewind();
    21 buffer.flip();

    Wouldn’t be enough to use buffer.flip(), like in the example “2) Read a large file in chunks with fixed size buffer”?

    Reply
    • You are right. I must have forget to comment that line. Actually both methods are almost similar. In fact, you can use rewind( ) to go back and reread the data in a buffer that has already been flipped.

      Reply
  5. Hi Lokesh, thanks for the informative write-up. The key difference is that in NIO you are reading a buffer not a stream. And NIO is by design non-blocking. So the thread is not blocked.

    Reply
  6. Just a question. I have to process huge text files (billions of records, each) line by line. Should I read them with IO, NIO, or NIO2? Which class?

    Reply
  7. Sorry but the old famous way do not the same – here is reading line by line – and with a Reader.
    The three other sources reading byte by byte – with a stream.

    No good!

    Reply

Leave a Comment

HowToDoInJava

A blog about Java and related technologies, the best practices, algorithms, and interview questions.