Java Virtual Threads: Project Loom

In Java, virtual threads (JEP-425) are JVM-managed lightweight threads that help in writing high-throughput concurrent applications (throughput means how many units of information a system can process in a given amount of time).

In contrast to platform threads, the virtual threads are not wrappers of OS threads. They are lightweight Java entities (with their own stack memory with a small footprint – only a few hundred bytes) that are cheap to create, block, and destroy. We can create many of them at the same time (millions) so they sustain a massive throughput.

Virtual threads are stored in the JVM heap (and they take advantage of Garbage Collector) instead of the OS stack. Moreover, virtual threads are scheduled by the JVM via a work-stealing ForkJoinPool scheduler. Practically, JVM schedules and orchestrates virtual threads to run on platform threads in such a way that a platform thread executes only one virtual thread at a time.

virtual-threads-history-new

1. Traditional Thread Model and Its Problems

Before digging into virtual threads, let us first understand how the threads work in traditional threads in Java.

1.1. Classic / Platform Threads

In Java, a classic thread is an instance of java.lang.Thread class. Moving forward, we will call them platform threads, as well.

Traditionally, Java has treated the platform threads as thin wrappers around operating system (OS) threads. Creating such platform threads has always been costly (due to a large stack and other resources that are maintained by the operating system), so Java has been using the thread pools to avoid the overhead in thread creation.

The number of platform threads also has to be limited because these resource-hungry threads can affect the performance of the whole machine. This is mainly because platform threads are mapped 1:1 to OS threads.

1.2. Scalability Issues with Platform Threads

Platform threads have always been easy to model, program and debug because they use the platform’s unit of concurrency to represent the application’s unit of concurrency. It is called thread-per-request pattern.

However, this pattern limits the throughput of the server because the number of concurrent requests (that server can handle) becomes directly proportional to the server’s hardware performance. So, the number of available threads has to be limited even in multi-core processors.

Apart from the number of threads, latency is also a big concern. If you watch closely, in today’s world of microservices, a request is served by fetching/updating data on multiple systems and servers. While the application waits for the information from other servers, the current platform thread remains in an idle state. This is a waste of computing resources and a major hurdle in achieving a high throughput application.

1.3. Issues with Reactive Programming

Reactive style programming solved the problem of platform threads waiting for responses from other systems. The asynchronous APIs do not wait for the response, rather they work through the callbacks. Whenever a thread invokes an async API, the platform thread is returned to the pool until the response comes back from the remote system or database. Later, when the response arrives, the JVM will allocate another thread from the pool that will handle the response and so on. This way, multiple threads are involved in handling a single async request.

In async programming, the latency is removed but the number of platform threads are still limited due to hardware limitations, so we have a limit on scalability. Another big issue is that such async programs are executed in different threads so it is very hard to debug or profile them.

Also, we have to adopt a new programming style away from typical loops and conditional statements. The new lambda-style syntax makes it hard to understand the existing code and write programs because we must now break our program into multiple smaller units that can be run independently and asynchronously.

So we can say that virtual threads also improve the code quality by adapting the traditional syntax while having the benefits of reactive programming.

2. Virtual Threads Look Promising

Similar to traditional threads, a virtual thread is also an instance of java.lang.Thread that runs its code on an underlying OS thread, but it does not block the OS thread for the code’s entire lifetime. Keeping the OS threads free means that many virtual threads can run their Java code on the same OS thread, effectively sharing it.

It is worth mentioning that we can create a very high number of virtual threads (millions) in an application without depending on the number of platform threads. These virtual threads are managed by JVM, so they do not add extra context-switching overhead as well because they are stored in RAM as normal Java objects.

Similar to traditional threads, the application’s code runs in a virtual thread for the entire duration of a request (in thread-per-request style) but the virtual thread consumes an OS thread only when it performs the calculations on the CPU. They do not block the OS thread while they are waiting or sleeping.

Virtual threads help in achieving the same high scalability and throughput as the asynchronous APIs with the same hardware configuration, without adding the syntax complexity.

Virtual threads are best suited to executing code that spends most of its time blocked, waiting for data to arrive on a network socket or waiting for an element in queue for example.

3. Difference between Platform Threads and Virtual Threads

  • Virtual threads are always daemon threads. The Thread.setDaemon(false) method cannot change a virtual thread to be a non-daemon thread. Note that JVM terminates when all started non-daemon threads have terminated. This means JVM will not wait for virtual threads to complete before exiting.
Thread virtualThread = ...; //Create virtual thread

//virtualThread.setDaemon(true);  //It has no effect
  • Virtual threads always have the normal priority and the priority cannot be changed, even with setPriority(n) method. Calling this method on a virtual thread has no effect.
Thread virtualThread = ...; //Create virtual thread

//virtualThread.setPriority(Thread.MAX_PRIORITY);  //It has no effect
  • Virtual threads are not active members of thread groups. When invoked on a virtual thread, Thread.getThreadGroup() returns a placeholder thread group with the name “VirtualThreads“.
  • Virtual threads do not support the stop(), suspend(), or resume() methods. These methods throw an UnsupportedOperationException when invoked on a virtual thread.

4. Comparing the Performance of Platform Threads and Virtual Threads

Let us understand the difference between both kinds of threads when they are submitted with the same executable code.

To demo it, we have a very simple task that waits for 1 second before printing a message in the console. We are creating this task to keep the example simple so we can focus on the concept.

final AtomicInteger atomicInteger = new AtomicInteger();

Runnable runnable = () -> {
  try {
    Thread.sleep(Duration.ofSeconds(1));
  } catch(Exception e) {
      System.out.println(e);
  }
  System.out.println("Work Done - " + atomicInteger.incrementAndGet());
};

Now we will create 10,000 threads from this Runnable and execute them with virtual threads and platform threads to compare the performance of both. We will use the Duration.between() api to measure the elapsed time in executing all the tasks.

First, we are using a pool of 100 platform threads. In this way, Executor will be able to run 100 tasks at a time and other tasks will need to wait. As we have 10,000 tasks so the total time to finish the execution will be approximately 100 seconds.

Instant start = Instant.now();

try (var executor = Executors.newFixedThreadPool(100)) {
  for(int i = 0; i < 10_000; i++) {
    executor.submit(runnable);
  }
}

Instant finish = Instant.now();
long timeElapsed = Duration.between(start, finish).toMillis();  
System.out.println("Total elapsed time : " + timeElapsed);	
Total elapsed time : 101152 //Approx 101 seconds

As of today, virtual threads are a preview API and disabled by default. Use $ java --source 19 --enable-preview Main.java to run the code.

Next, we will replace the Executors.newFixedThreadPool(100) with Executors.newVirtualThreadPerTaskExecutor(). This will execute all the tasks in virtual threads instead of platform threads.

Instant start = Instant.now();

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
  for(int i = 0; i < 10_000; i++) {
    executor.submit(runnable);
  }
}

Instant finish = Instant.now();
long timeElapsed = Duration.between(start, finish).toMillis();  
System.out.println("Total elapsed time : " + timeElapsed);	
Total elapsed time : 1589 //Approx 1.5 seconds

Notice the blazing fast performance of virtual threads that brought down the execution time from 100 seconds to 1.5 seconds with no change in the Runnable code.

5. How to Create Virtual Threads

From the API perspective, a virtual thread is another flavor of java.lang.Thread. If we dig a little bit via getClass(), we see that a virtual thread class is java.lang.VirtualThread which is a final non-public class that extends the BaseVirtualThread class which is a sealed abstract class that extends java.lang.Thread.

final class VirtualThread extends BaseVirtualThread {…}

sealed abstract class BaseVirtualThread extends Thread
  permits VirtualThread, ThreadBuilders.BoundVirtualThread {…}

5.1. Using Thread.startVirtualThread()

We can create and start a virtual thread for our task via the startVirtualThread(Runnable task). This method creates a new virtual thread to execute a given Runnable task and schedules it to execute.

Runnable runnable = () -> System.out.println("Inside Runnable");   // Task to run

Thread vThread = Thread.startVirtualThread(runnable);

//or

Thread vThread = Thread.startVirtualThread(() -> {
	//Code to execute in virtual thread
	System.out.println("Inside Runnable");
});

5.2. Using Thread.Builder

If we want to explicitly start the thread after creating it, we can use Thread.ofVirtual() that returns a VirtualThreadBuilder instance. Its start() method starts a virtual thread.

It is worth noting that Thread.ofVirtual().start(runnable) is equivalent to Thread.startVirtualThread(runnable).

Runnable runnable = () -> System.out.println("Inside Runnable");
Thread virtualThread = Thread.ofVirtual().start(runnable);

We can use the Thread.Builder reference to create and start multiple threads.

Runnable runnable = () -> System.out.println("Inside Runnable");

Thread.Builder builder = Thread.ofVirtual().name("JVM-Thread");

Thread t1 = builder.start(runnable); 
Thread t2 = builder.start(runnable);

A similar API Thread.ofPlatform() exists for creating platform threads as well.

Thread.Builder builder = Thread.ofPlatform().name("Platform-Thread");

Thread t1 = builder.start(() -> {...}); 
Thread t2 = builder.start(() -> {...});

5.3. Using Executors.newVirtualThreadPerTaskExecutor()

This method creates one new virtual thread per task. The number of threads created by the Executor is unbounded.

In the following example, we are submitting 10,000 tasks and waiting for all of them to complete. The code will create 10,000 virtual threads to complete these 10,000 tasks.

Note that the following syntax is part of structured concurrency, another new feature proposed in Project Loom. We will discuss it in a separate post.

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    IntStream.range(0, 10_000).forEach(i -> {
        executor.submit(() -> {
            Thread.sleep(Duration.ofSeconds(1));
            return i;
        });
    });
}

6. Creating an Unstarted Virtual Thread

Creating an unstarted virtual thread can be done via unstarted(Runnable task) as follows:

Thread vThread = Thread.ofVirtual().unstarted(task);

Or, via Thread.Builder as follows:

Thread.Builder builder = Thread.ofVirtual();
Thread vThread = builder.unstarted(task);

This time, the thread is not scheduled for execution. It will be scheduled for execution only after we explicitly call the start() method:

vThread.start();

Note that the unstarted() method is available for platform threads as well via Thread.ofPlatform().unstarted(task) method.

7. Waiting for a Virtual Thread to Terminate

A virtual thread runs always as a daemon thread. When a given task is executed by a virtual thread, the main thread is not blocked. In order to wait for the virtual thread to terminate we have to call one of the join() methods. 

When we use join() without arguments then it waits indefinitely. Use join(Duration duration) or join(long millis) to wait in a time-bound manner. These methods throw an InterruptedException so you have to catch it and handle it or just throw it.

vThread.join();

Now, because of join(), the main thread cannot terminate before the virtual thread. It has to wait until the virtual thread completes.

8. Best Practices to Follow

Before learning best practices, make it very clear that Virtual threads ARE NOT faster than platform threads and they don’t boost in-memory computational capabilities. However, virtual threads can be launched much faster than platform threads.

Another point to remember is that Virtual threads DO NOT release a task. A virtual thread takes a task and should return a result or gets interrupted.

8.1. DO NOT Pool the Virtual Threads

Java thread pool was designed to avoid the overhead of creating new OS threads because creating them was a costly operation. But creating virtual threads is not expensive, so, there is never a need to pool them. It is advised to create a new virtual thread everytime we need one.

Note that after using the virtual threads, our application may be able to handle millions of threads, but other systems or platforms handle only a few requests at a time. For example, we can have only a few database connections or network connections to other servers.

In these cases also, do not use the thread pool. Instead, use semaphores to make sure only a specified number of threads are accessing that resource.

private static final Semaphore SEMAPHORE = new Semaphore(50);

SEMAPHORE.acquire();

try {
  // semaphore limits to 50 concurrent access requests
  //Access the database or resource
} finally {
  SEMAPHORE.release();
}

8.2. Avoid using Thread-local Variables

Virtual threads support thread-local behavior the same way as platform threads, but because the virtual threads can be created in millions, thread-local variables should be used only after careful consideration.

For example, if we scale a million virtual threads in the application, there will be a million ThreadLocal instances along with the data they refer to. Such a large number of instances can put enough burden on the physical memory and it should be avoided.

Extent-Local variables [JEP-429] are a better alternative. Note that in Java 21 [JEP-444], virtual threads now support thread-local variables all of the time. It is no longer possible, as it was in the preview releases, to create virtual threads that cannot have thread-local variables.

8.3. Use ReentrantLock instead of Synchronized Blocks

There are two specific scenarios in which a virtual thread can block the platform thread (called pinning of OS threads).

  • When it executes code inside a synchronized block or method, or
  • When it executes a native method or a foreign function.

Such synchronized block does not make the application incorrect, but it limits the scalability of the application similar to platform threads.

As a best practice, if a method is used very frequently and it uses a synchronized block then consider replacing it with the ReentrantLock mechanism.

So instead of using synchronized block like this:

public synchronized void m() {
	try {
	 	// ... access resource
	} finally {
	 	//
	}
}

use ReentrantLock like this:

private final ReentrantLock lock = new ReentrantLock();

public void m() {
	lock.lock();  // block until condition holds
	try {
	 	// ... access resource
	} finally {
	 	lock.unlock();
	}
}

It is suggested that there is no need to replace synchronized blocks and methods that are used infrequently (e.g., only performed at startup) or that guard in-memory operations.

9. Conclusion

Traditional Java threads have served very well for a long time. With the growing demand of scalability and high throughput in the world of microservices, virtual threads will prove a milestone feature in Java history.

With virtual thread, a program can handle millions of threads with a small amount of physical memory and computing resources, otherwise not possible with traditional platform threads. It will also lead to better-written programs when combined with structured concurrency.

Happy Learning !!

Source Code on Github

Comments

Subscribe
Notify of
guest
2 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments

About Us

HowToDoInJava provides tutorials and how-to guides on Java and related technologies.

It also shares the best practices, algorithms & solutions and frequently asked interview questions.

Our Blogs

REST API Tutorial

Dark Mode

Dark Mode