.png)
4 months ago
7
Virtual Threads, one of the most talked about features in Java 21. They are way more lightweight than traditional platform threads and let you handle high concurrency with less resource usage. At Cashfree Payments, we recently migrated a few of our microservices to Java 21 and began integrating Virtual Threads for I/O-heavy operations like bank API calls. We quickly learned that using Virtual Threads the wrong way can silently erode performance, or worse, introduce instability. This post is a reflection of that journey.
These are seven lessons, including mistakes, anti-patterns, and edge cases, we ran into (or deliberately avoided) while working with Virtual Threads. Whether you’re just getting started or already knee-deep in migration, these seven lessons can save you from expensive surprises.
Let’s dive in.
Problem: Virtual Threads are optimised for I/O-bound tasks but not recommended for CPU-intensive tasks. Unlike Platform Threads, they do not have OS-level thread scheduling leverage, which might give you suboptimal CPU utilisation. Moreover, in CPU-intensive workloads, Virtual Threads can face starvation due to the unavailability of underneath carrier threads.
Example of incorrect usage:
Evaluation: First things first, before applying Virtual Threads, check the nature of the workload. If it is an I/O bound task (tasks spend most of their time waiting), then only prefer Virtual Threads. For our use case, we mostly have bank-related network calls with high traffic and moderate latency (i.e. 2–5s, which is very high in CPU clock). That means our workload is I/O-Intensive with high concurrency, suitable for Virtual Thread.
Smarter approach: For CPU-heavy tasks, use a fixed-size thread pool instead. The recommended pool size should be equal to the number of available cores.
Problem: Certain operations can pin a Virtual Thread to a carrier thread (Carrier threads are known as those Platform Threads that execute virtual threads), due to which other Virtual Threads can be blocked from executing, even in I/O-Intensive Workloads. This loses the benefits of Virtual Threads.
Common blocking operations that cause pinning:
- synchronised blocks
- Object.wait()
- Native method calls
Example of incorrect usage:
Evaluation: Before switching to Virtual Threads, check if there are any blocking operations highlighted above in your codebase. If you find any, either try to eliminate them or use non-blocking alternatives. In our case, we didn’t have any obvious blocking calls. But even then, we were not 100% sure that the whole code was free of them since some third-party libraries might still be using blocking operations behind the scenes. We took the best approach here to try and test. We haven’t found anything suspicious related to performance degradation. Still, if you notice some unnecessary timeouts or the number of live Platform Threads is going up significantly, then there is a probability that some carrier threads are getting stuck. To track this down, you can use the JVM flag -Djdk.tracePinnedThreads=full, which helps you pinpoint the problem.
Smarter approach: Either avoid synchronised in Virtual Threads or use alternative concurrency mechanisms like ReentrantLock. Java 24 has improvements to reduce pinning in synchronised blocks.
Problem: Platform threads store their stacks in native memory (refers to overall system memory), whereas Virtual Threads store their stacks in heap memory (refers to a specific region within the Java Virtual Machine (JVM), dedicated to storing Java objects and managed by JVM Garbage Collector). If the heap size is not properly configured, Virtual Threads may lead to lower concurrency than Platform Threads. In some cases, even a small number of Virtual Threads can cause an OutOfMemoryError, depending upon the workload. Resulting in reduced concurrency instead of improving it.
Evaluation:
The graph above shows the production heap usage of one of our Cashfree microservices that runs on Spring Boot + Java 21. The graph is split into two parts by the black vertical line. On the left, you can see heap memory usage before we started using Virtual Threads in place of Tomcat Threads, and on the right, after the switch. As you can observe, there’s a significant increase in heap memory usage after moving to Virtual Threads, which was very obvious. This ends up impacting the heap availability for actual traffic. An increased heap does not mean that overall memory consumption increases in the case of Virtual Threads. Instead, it means that the stack which used to take space in native memory is now shifted to the heap.
Smarter approach: Adjust JVM options to allocate more heap memory to achieve high concurrency. We did the same for our microservice.
This sets the max heap limit to 35% of available RAM.
Problem: You might create unnecessary scheduling overhead if Virtual Threads are used with traditional thread pool executors (like FixedThreadPool). Submitting Virtual Threads to such a thread pool double-schedules them:
- First, the Platform Thread pool schedules the task.
- Then, inside the task, a Virtual Thread is scheduled again by the JVM.
Example of incorrect usage:
Evaluation: If you want to convert your thread pool to use Virtual Thread, it doesn’t mean submitting Virtual Threads to the pool. This will create unnecessary overhead, as mentioned earlier. Instead, you must take a different approach: don’t pool them. The nature of a Virtual Thread is to be created, processed, and destroyed quickly. The idea is to use a Virtual Thread per task and then let it destroy once finished. A better approach is already mentioned below. But before switching to a new executor, keep in mind that you have now unlocked a ton of concurrency, which might become an issue for any resources your tasks depend on. We’ll dive into that in point #6 later.
Smarter approach: Use a special executor built for Virtual Thread with the same analogy that is discussed above:
Problem: Using ThreadLocal for caching in a virtual thread context can backfire. Unlike traditional platform threads, virtual threads are short-lived and not reused. This actually means every time you spawn a new virtual thread, it gets its own fresh ThreadLocal state. So, even if you’ve cached something expensive, the next thread won’t see it as it starts from scratch.
Here’s an example that highlights the issue:
Example of incorrect usage:
Evaluation: Check for Thread Local used in the code. You might find two types of usage of Thread Local:
Context-specific information: This is when you store things like a user object in ThreadLocal to avoid passing it around in every method. In such cases, using Thread Local with Virtual Thread, you must clean up the ThreadLocal using remove() in a ‘finally’ block, or you’ll risk a memory leak.
Caching: While some developers use ThreadLocal to cache expensive objects, this approach falls flat with Virtual Threads. Unlike Platform Threads, Virtual Threads aren’t reused. The former are short-lived and lightweight. As a result, caching via ThreadLocal provides no real benefit.
Smarter approach: There is not one general alternative to offer, but in the above case, a better direction would be to use ExecutorService executor = Executors.newFixedThreadPool(4); so that expensive objects can be reused across the threads. Using this approach, you only need to compute once per pool thread, cached.
You should also consider exploring Scoped Values (introduced in Java 21) instead of Thread Local for a temporary state (reference).
Problem: Even though the JVM can handle millions of virtual threads, throwing that kind of concurrency at your system without guardrails is asking for trouble. Here’s where things can go south:
- Memory Pressure: Sure, each virtual thread has a tiny stack, around 1–2 KB, but that adds up fast when you scale into the millions. Suddenly, your heap can start sweating.
- CPU Starvation: Running too many tasks at the same time can overload the CPU and cause a lot of context switching.
- Unbounded I/O Load: When tasks talk to databases or call other services, too many virtual threads can easily overload them
Example of incorrect usage:
Evaluation: Whenever you introduce virtual threads into your application, always remember: you’re essentially handing out an unlimited pass to spawn unlimited concurrent tasks. That sounds powerful, but without control, it can overwhelm both your internal resources (like memory or CPU) and external systems (like databases or APIs). So if you want a limit on concurrency, then you shouldn’t depend on executors anymore. Instead, take control of concurrency yourself. Java also gives you dedicated built-in options for this.
Smarter approach: If you’re spawning virtual threads in an unbounded loop, either throttle them or use backpressure mechanism to only allow as much concurrency as your system can handle gracefully. Below I have used Semaphore of the same.
Problem: Unlike platform threads, however, virtual threads won’t appear cleanly in traditional monitoring tools like jstack, top, or basic thread dumps. This lack of visibility makes debugging and performance tuning challenging. One common misconception teams make regarding virtual threads is assuming they’d give out a performance boost, like magic, without measuring real-world workloads. However, issues like excessive heap usage or thread starvation can go unnoticed without using profiling tools.
Smarter approach:
- Use Java Flight Recorder (JFR) or Async Profiler to track Virtual Threads.
- Enable detailed thread tracking with JVM flags:
-Djdk.tracePinnedThreads=full
Virtual threads are powerful, but just like any other tool, they need to be used with caution. They do well in I/O-bound use cases, but they’re not a silver bullet. If you jump in without having an understanding of how they behave, especially under stress, you might end up with slower performance, memory issues, or tricky debugging problems.
At Cashfree Payments, we’ve seen real impact using them for high-latency tasks like bank API calls. But we’ve also learned that Virtual Threads need a different mindset. You can’t just replace old threads and expect magic. You need to know where they fit, where they don’t, and how to keep things in check. Bottom line: Virtual Threads are powerful but only when used thoughtfully.
Watch out for this space for more engineering insights!
