Project Looms changes the existing Thread implementation from the mapping of an OS thread, to an abstraction that can either represent such a thread or a virtual thread. In itself, that is an interesting move on a platform that historically put a lot more value on backward-compatibility in comparison to innovation. Compared to other recent Java versions, this feature is a real game-changer. Developers in general should start getting familiar with it as soon as possible.

Loom will decrease any resource consumption that native/kernel thread limited. Which is pretty much every HTTP service. It’s going to be huge and everything will be using it. From “big data” frameworks like Spark, REST, databases like Cassandra. It’s hard to think of a common use case for Java that won’t benefit from virtual threads.

This article walks you through a experiment that uses a Jiny BIO application with Virtual Threads (similar as Golang’s Goroutine). Having access to early access builds is the perfect opportunity to take a look what it takes to use virtual threads as worker threads, to see how memory consumption and the number of throughput it could serve over time.

Prepare stuff

Project Loom is in its early stages which don’t allow for exact benchmarking. Instead, the state of the project should be considered to change over time. From the project page:

Early-access (EA) functionality might never make it into a general-availability (GA) release.

EA functionality might be changed or removed at any time.

Involved components:

• Jiny Core 0.3.3

To run the experiment, you need to use a Loom EAP build (Java 16).

How the application was customized

• Jiny BIO mode (which is a traditional blocking IO mode)

public class App {
    public static void main(String[] args) {
        var app = HttpServer.port(1234);
        app.get("/", ctx -> of("Hello World"));
        app.start();
    }
}

• Jiny NIO mode (which is a non-blocking IO mode)

public class App {
    public static void main(String[] args) {
        var app = NIOHttpServer.port(1234);
        app.get("/", ctx -> ofAsync("Hello World"));
        app.start();
    }
}

• Jiny Fiber

It’s easy to integrate a BIO Jiny application with Fiber virtual thread, Jiny has a method to customize the worker executor:

public class App {
    public static void main(String[] args) {
        var app = HttpServer.port(1234)
                      // Use Fiber thread
                      .setExecutor(Executors.newVirtualThreadExecutor());
        app.get("/", ctx -> of("Hello World"));
        app.start();
    }
}

Observations

The measurement uses wrk (yes, there’s the coordinated omission problem, but for this case it’s something we can live with) for warmup and measurement. There are quite significant differences between using Virtual and Kernel threads.

• Jiny BIO with 382MB memory consumed and can serve 56705 req/sec

➜ wrk -t8 -c5000 -d30s http://test-wrk.tuhuynh.com
Running 30s test @ http://test-wrk.tuhuynh.com
  8 threads and 5000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.89ms    4.82ms 460.52ms   90.45%
    Req/Sec     7.20k     4.57k   28.15k    77.83%
  1706741 requests in 30.10s, 263.68MB read
  Socket errors: connect 4757, read 826, write 11, timeout 0
Requests/sec:  56705.78
Transfer/sec:      8.76MB

• Jiny NIO with 249MB memory consumed and can serve 111842 req/sec

➜ wrk -t8 -c5000 -d30s http://test-wrk.tuhuynh.com
Running 30s test @ http://test-wrk.tuhuynh.com
  8 threads and 5000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.15ms    2.48ms 398.85ms   98.82%
    Req/Sec    14.06k     9.76k   41.96k    64.67%
  3357931 requests in 30.02s, 518.78MB read
  Socket errors: connect 4757, read 644, write 40, timeout 0
Requests/sec: 111842.56
Transfer/sec:     17.28MB

• Jiny BIO (uses Fiber virtual thread) with 248M memory consumed and can serve 106584 req/sec

➜ wrk -t8 -c5000 -d30s http://test-wrk.tuhuynh.com
Running 30s test @ http://test-wrk.tuhuynh.com
  8 threads and 5000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.25ms    1.06ms  77.13ms   95.49%
    Req/Sec    13.40k     6.79k   26.58k    44.29%
  3199300 requests in 30.02s, 494.28MB read
  Socket errors: connect 4757, read 344, write 0, timeout 0
Requests/sec: 106584.14
Transfer/sec:     16.47MB

My machine’s specs:

Macbook Pro 16-inch 2019 (macOS Big Sur)
Intel i7-9750H (12) @ 2.60GHz, 16GB memory

Conclusion

Virtual threads require less memory than kernel one (249 MB RSS vs 382 MB RSS), and in Blocking IO mode with Fiber the Jiny application can perform similar as Non-Blocking IO Jiny application.

Reference: “Project Loom: Modern Scalable Concurrency for the Java” - Ron Pressler

With BIO:

With NIO:

With old BIO API & Fiber thread:

How about Reactive/Coroutine?

The main question is, now that the JVM API offers an abstraction over OS threads, what will become of other abstractions such as Reactive and coroutines?

Developers who are about to learn about Reactive and coroutines should probably take a step back, and evaluate whether they should instead learn the new Thread API - or not.