Java NIO Vectored IO

Java NIO Channels provide an important new capability known as scatter/gather (referred to in some circles as vectored I/O). Scatter/gather is a simple yet powerful concept.

Scatter/gather is a technique through which bytes can be read from a stream into a set of buffers (vectors) with a single read() invocation and bytes can be written from a set of buffers to a stream with a single write() invocation.

Most modern operating systems support native vectored I/O. When you request a scatter/gather operation on a channel, the request will be translated into appropriate native calls to fill or drain the buffers directly. This is a big win because buffer copies and system calls are reduced or eliminated.

Until recently Java did not have the capability of performing vectored I/O operation. So we are used to reading bytes directly into a single byte array or do multiple reads to fetch the data.

1. Scatter/Gather APIs

A scattering read from a channel is a read operation that reads data into more than one buffer. Thus, the channel “scatters” the data from the channel into multiple buffers.

A gathering write to a channel is a write operation that writes data from more than one buffer into a single channel. Thus, the channel “gathers” the data from multiple buffers into one channel.

Scatter and gather can be really useful in situations where you need to work with various parts of the transmitted data separately.

public interface ScatteringByteChannel extends ReadableByteChannel
{
	public long read (ByteBuffer [] dsts) throws IOException;
	public long read (ByteBuffer [] dsts, int offset, int length) throws IOException;
}

public interface GatheringByteChannel extends WritableByteChannel
{
	public long write(ByteBuffer[] srcs) throws IOException;
	public long write(ByteBuffer[] srcs, int offset, int length) throws IOException;
}

You can see that each interface adds two new methods that take an array of buffers as arguments.

Now let’s write a quick example to understand how to use this feature.

2. Java Scatter/Gather IO Example

In this example, I have created two buffers. One buffer will store a random number and another will store a random string. I will use GatheringByteChannel to read-write the data stored in both buffers in a file channel.

Then I will read the data back from the file into two separate buffers using ScatteringByteChannel and print the content in the console to verify that data stored and retrieved are matched.

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.GatheringByteChannel;
import java.nio.channels.ScatteringByteChannel;

public class ScatteringAndGatheringIOExample 
{
	public static void main(String params[]) 
	{
		String data = "Scattering and Gathering example shown in howtodoinjava.com";
		
		gatherBytes(data);
		scatterBytes();
	}

	/*
	 * gatherBytes() reads bytes from different buffers and writes to file
	 * channel. Note that it uses a single write for both the buffers.
	 */
	public static void gatherBytes(String data) 
	{
		//First Buffer holds a random number
		ByteBuffer bufferOne = ByteBuffer.allocate(4);
		
		//Second Buffer holds data we want to write
		ByteBuffer buffer2 = ByteBuffer.allocate(200);

		//Writing Data sets to Buffer
		bufferOne.asIntBuffer().put(13);
		buffer2.asCharBuffer().put(data);
		
		//Calls FileOutputStream(file).getChannel()
		GatheringByteChannel gatherer = createChannelInstance("test.txt", true);

		//Write data to file
		try 
		{
			gatherer.write(new ByteBuffer[] { bufferOne, buffer2 });
		} 
		catch (Exception e) 
		{
			e.printStackTrace();
		}
	}

	/*
	 * scatterBytes() read bytes from a file channel into a set of buffers. Note that
	 * it uses a single read for both the buffers.
	 */
	public static void scatterBytes() 
	{
		//First Buffer holds a random number
		ByteBuffer bufferOne = ByteBuffer.allocate(4);
		
		//Second Buffer holds data we want to write
		ByteBuffer bufferTwo = ByteBuffer.allocate(200);

		//Calls FileInputStream(file).getChannel()
		ScatteringByteChannel scatterer = createChannelInstance("test.txt", false);
		
		try 
		{
			//Reading from the channel
			scatterer.read(new ByteBuffer[] { bufferOne, bufferTwo });
		} 
		catch (Exception e) 
		{
			e.printStackTrace();
		}

		
		//Read the buffers seperately
		bufferOne.rewind();
		bufferTwo.rewind();

		int bufferOneContent = bufferOne.asIntBuffer().get();
		String bufferTwoContent = bufferTwo.asCharBuffer().toString();
		
		//Verify the content
		System.out.println(bufferOneContent);
		System.out.println(bufferTwoContent);
	}
	
	
	public static FileChannel createChannelInstance(String file, boolean isOutput) 
	{
		FileChannel fc = null;
		try 
		{
			if (isOutput) {
				fc = new FileOutputStream(file).getChannel();
			} else {
				fc = new FileInputStream(file).getChannel();
			}
		} 
		catch (Exception e) {
			e.printStackTrace();
		}
		return fc;
	}
}

3. Conclusion

Scatter/gather can be an extraordinarily powerful tool when used properly. It allows you to delegate to the operating system the grunt work of separating out the data you read into multiple buckets, or assembling disparate chunks of data into a whole. This can be a huge win because the operating system is highly optimized for this sort of thing.

It saves you the work of moving things around, thereby avoiding buffer copies, and reduces the amount of code you need to write and debug. Since you are basically assembling data by providing references to data containers, the various chunks can be assembled in different ways by building multiple arrays of buffer references in different combinations.

Happy Learning !!

Leave a Reply

0 Comments
Inline Feedbacks
View all comments

About Us

HowToDoInJava provides tutorials and how-to guides on Java and related technologies.

It also shares the best practices, algorithms & solutions, and frequently asked interview questions.