Core Java

Virtual Threads One Year in Production: What Teams Got Wrong and What Actually Improved

Project Loom shipped in Java 21 in September 2023. Two-plus years of real production data — including Netflix’s deadlock post-mortem, thread-local memory explosions, and framework readiness surprises — is now enough to write honestly about it.

1. The Original Promise — and the Production Gap

When Project Loom finally shipped as a stable feature in JEP 444 with Java 21, the excitement was immediate. The pitch was compelling: write plain, sequential, blocking code — the same style most Java engineers already know — and get reactive-style scalability for free. No reactive streams, no callback chains, no mental model shift. Just threads. Millions of them.

And yet, something interesting happened in the months that followed. Teams that adopted virtual threads in production discovered that the reality was more nuanced than the launch blog posts suggested. Not because virtual threads are bad — they are genuinely good — but because the migration surface was larger than most teams anticipated. Thread pinning hit harder than the documentation implied. Thread-local variables caused memory problems that nobody had warned about. A few teams hit deadlocks severe enough that their engineers published detailed post-mortems.

Two years on, enough of that production data has accumulated to write an honest retrospective. Consequently, this article is structured around what actually went wrong first, then what actually improved, and finally what the Java 24 and 25 fixes mean for teams still sitting on the fence.

Who this is for?

This article is primarily for engineering teams on Java 21 or later who are evaluating virtual threads for a production service, or who have already adopted them and want to understand the landscape better. It assumes familiarity with basic Java concurrency concepts — threads, executors, and synchronized.

2. Mistake 1: Underestimating the Pinning Problem

By far the most widely discussed issue with virtual threads in Java 21 was pinning. The concept is straightforward: a virtual thread is normally lightweight because it can unmount from its carrier (OS) thread whenever it blocks, freeing that carrier to serve another virtual thread. Pinning is what happens when that unmounting cannot occur — the virtual thread gets stuck on the carrier thread for the duration of the blocking operation. The end result is that your lightweight threads start behaving exactly like the platform threads you were trying to replace.

In Java 21, pinning happened in two main scenarios: when a virtual thread performed a blocking operation inside a synchronized block, and when it called certain blocking native methods. The first scenario was the one that burned most teams, because synchronized is everywhere — not just in their own code, but deep inside third-party libraries and the JDK itself.

Post-mortem · Netflix Engineering · July 2024

How pinning caused a complete application deadlock at Netflix

Netflix published a detailed post-mortem describing exactly this scenario. Their application had five virtual threads and one platform thread all contending for the same lock. Four of the virtual threads had become pinned to OS carrier threads inside a synchronized block while waiting for a ReentrantLock. Those four pinned threads occupied all available carrier threads in the fork-join pool. A fifth virtual thread was then signaled to unpark — but it could not proceed because there were no free carrier threads left to schedule it onto. The application froze. What made diagnosis particularly difficult was that Java 21’s thread dumps did not include locking or parking information for virtual threads — a gap that forced the Netflix team to resort to heap dump inspection with Eclipse MAT to identify the lock owner.

What teams got wrong initially was assuming that because frameworks like Spring Boot 3.2 had added virtual thread support, the pinning problem was handled. In reality, most popular libraries fixed their explicit synchronized usage quickly — but internal JDK classes, caching libraries like Caffeine (via ConcurrentHashMap), and third-party JDBC drivers still contained hidden pinning scenarios. The good news is that the fix eventually arrived.

Detect pinning events with JFR (Java 21–23)

java -Djdk.tracePinnedThreads=full \
     -XX:StartFlightRecording=filename=app.jfr,settings=default \
     -jar your-app.jar

jfr print --events jdk.VirtualThreadPinned app.jfr

Fixed in Java 24

JEP 491 — Synchronize Virtual Threads without Pinning — landed in Java 24 and resolved nearly all synchronized pinning scenarios, including the ConcurrentHashMap.computeIfAbsent case that caught many teams. On Java 25, which includes this fix, the jdk.VirtualThreadPinned JFR event simply stops firing for synchronized blocks. The JEP authors explicitly state that teams no longer need to replace synchronized with ReentrantLock as a workaround. The old advice was right for Java 21 — it is now obsolete.

3. Mistake 2: Thread-Local Caching That Silently Exploded

This was the subtler mistake — and in some ways the more dangerous one, precisely because it did not produce an immediate error. It just slowly made things worse.

The pattern is common in Java codebases: an expensive object — a SimpleDateFormat, a database connection wrapper, a JSON serializer — gets cached in a ThreadLocal to avoid creating it repeatedly. With a platform thread pool of 200 threads, that cache holds at most 200 instances. This is entirely intentional and efficient: the pool is small, the objects are reused, and memory stays manageable.

Virtual threads break this assumption completely. Because virtual threads are never pooled and never reused by unrelated tasks, every new task that calls code relying on a ThreadLocal cache creates a fresh instance of that expensive object. At low concurrency, nobody notices. At 50,000 concurrent virtual threads — a number that is entirely plausible and even advertised as a selling point — you now have 50,000 instances of an object that was designed to have 200. Memory spikes. GC pressure rises. The heap fills up with objects that were supposed to be shared.

The real danger here is that the problematic ThreadLocal often lives inside a library you do not control. The official Oracle documentation calls out asynchronous frameworks specifically: they sometimes cache expensive objects in thread locals under the assumption that they are used by a small pool. Mixing virtual threads with such frameworks therefore creates invisible memory pressure that only shows up at scale.

How to spot thread-local bloat?

Run your service under realistic load with -XX:NativeMemoryTracking=summary and monitor heap growth over time. If heap growth is non-linear as concurrent request count increases, thread-local caching is a likely culprit. Profile with JFR to identify which object types are accumulating unexpectedly: jfr print --events jdk.OldObjectSample app.jfr

The correct fix depends on context. For objects that genuinely need to be per-task rather than per-thread, the answer is often simply to instantiate them locally within the task — virtual thread creation is so cheap that the cost of object creation is now the bottleneck, not thread creation. For objects that need sharing across related tasks, Scoped Values (finalized in Java 25) offer a safer, immutable alternative to ThreadLocal that works correctly with virtual threads because values are scoped to a structured task lifetime rather than a thread’s lifetime.

4. Mistake 3: Using Virtual Threads for CPU-Bound Work

Perhaps the most fundamental misunderstanding about virtual threads is treating them as a general-purpose performance upgrade. They are not — they are a specific solution to a specific problem: the cost of blocking I/O on OS threads. The moment your virtual thread is doing actual computation rather than waiting, it cannot unmount from its carrier thread anyway. It sits on a carrier the entire time it runs, which means the scalability benefit disappears entirely.

Worse, because the default virtual thread scheduler uses a fork-join pool sized to the number of available CPU cores, saturating it with CPU-bound virtual threads actually starves other virtual threads that are waiting to resume after I/O. The result can be worse throughput than a plain thread pool tuned for the workload. As SpringJavaLab’s Java 25 benchmarks note, virtual threads do not make slow systems fast — they make I/O-blocking-constrained systems more scalable. There is a significant difference.

Do not use virtual threads for…

CPU-intensive computation (image processing, cryptography, compression, parsing), parallel stream operations, and any workload where threads are rarely or never blocked on I/O. For these, use a bounded ExecutorService with platform threads sized to your core count instead.

Virtual thread benefit by workload type

Throughput improvement over equivalent platform thread pool — real benchmark data ranges. Sources: Bell SW Spring Boot benchmark · Shinde inter-service call study · Java Code Geeks memory study

5. Mistake 4: Unbounded Concurrency Without Guardrails

There is a seductive logic to virtual threads: if they are cheap, why limit them? Spin up as many as you need. In theory, the JVM can handle millions. In practice, this thinking regularly causes production incidents.

The problem is that while virtual threads themselves are cheap — around 1–2 KB of heap per thread at rest — the downstream systems they talk to are not. A database has a fixed connection pool. A downstream REST API has a rate limit or its own thread-per-connection model. A message broker has throughput caps. When 100,000 virtual threads simultaneously try to acquire a database connection from a pool of 20, you do not get magical database scalability — you get 99,980 threads queued, contending, and consuming heap while they wait. You have effectively moved the bottleneck from thread creation to resource contention, and you have made it harder to reason about.

Rate-limit virtual thread concurrency with a semaphore

Semaphore dbSemaphore = new Semaphore(20); // match your DB pool size

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (var request : incomingRequests) {
        executor.submit(() -> {
            dbSemaphore.acquire();
            try {
                return database.fetch(request);
            } finally {
                dbSemaphore.release();
            }
        });
    }
}

Structured concurrency helps here

Java 25 includes a refined version of StructuredTaskScope that makes it easier to manage the lifetime of groups of virtual threads as a unit — including bounding their concurrency. While still in preview, it is the long-term direction for well-governed virtual thread usage. The key principle: always think about the downstream systems, not just the threads themselves.

6. What Actually Improved — the Real Numbers

After all of the caveats above, it is important to be clear: for the right workload, virtual threads delivered real and substantial gains. The improvements were not marketing numbers. They showed up in production metrics.

The clearest wins came in I/O-bound services that were previously constrained by a platform thread pool limit. The pattern is consistent across multiple published benchmarks: a Spring Boot service configured with spring.threads.virtual.enabled=true — a single line change — handling HTTP requests that each perform one or more downstream blocking I/O calls showed throughput improvements ranging from 2x to 3x compared to the default 200-thread Tomcat pool under high concurrency. One independent study measured a 43% reduction in memory usage and a 4x improvement in tail latency on the same hardware.

Requests per second: platform threads vs virtual threads (I/O-bound Spring Boot service)

Measured at increasing concurrent user counts — 200-thread platform pool vs virtual thread executor. Based on published benchmarks from sohamkamani.com and Bell SW, normalized to relative throughput for comparability

The gains are real, but they are conditional. Specifically, they require the application to have a meaningful I/O wait ratio — blocking that gives the carrier thread time to work on other things. Services that were already reactive, or that were CPU-bound, saw minimal improvement. Meanwhile, services that had been limping along with max-threads=200 and a queue that regularly hit its ceiling saw transformative results.

7. The Evolution: Java 23 → 24 → 25

One important context that is easy to miss in retrospectives like this is that Java 21 was not the end of the virtual thread story — it was the beginning. The subsequent releases addressed specific pain points with notable speed.

Sept 2023

Java 21 — Virtual threads finalized (JEP 444)

Stable API, carrier thread model established. Pinning on synchronized is a known limitation. Thread dumps lack lock ownership info for virtual threads. Scoped Values in preview.

Sept 2024

Java 23 — Thread dump improvements

Virtual thread dumps now include lock ownership and parking information — directly addressing one of Netflix’s post-mortem findings. Diagnosing deadlocks becomes tractable without heap dump surgery.

Mar 2025

Java 24 — Pinning fundamentally fixed (JEP 491)

synchronized blocks no longer pin virtual threads. ConcurrentHashMap.computeIfAbsent pinning resolved. The JEP authors explicitly say to stop replacing synchronized with ReentrantLock as a workaround.

Sept 2025

Java 25 — Scoped Values finalized (JEP 487)

The recommended alternative to ThreadLocal for virtual thread-friendly context propagation becomes a stable, production-ready API. Structured concurrency continues in preview with a refined scope API.

The practical implication of this timeline is significant: teams running Java 21 have been living with both the pinning problem and the thread dump opacity problem simultaneously, and many of them have not upgraded yet. Moving to Java 25 means you get JEP 491’s pinning fix, improved observability, and finalized Scoped Values — all at once. The remaining virtual thread rough edges have been progressively sanded down.

8. Framework Readiness and Migration Checklist

One of the most common early sources of confusion was whether your framework actually supported virtual threads at the level you expected. “Support” turned out to mean different things to different frameworks.

Framework / LibraryVirtual thread supportMinimum versionWhat to configure
Spring Boot (Tomcat)Full — one property3.2 + Java 21spring.threads.virtual.enabled=true
Spring Boot (WebFlux)Not needed / not recommendedAlready non-blocking; virtual threads add no benefit and can mix badly
QuarkusFull via @RunOnVirtualThread3.0 + Java 21Annotate REST endpoints with @RunOnVirtualThread
MicronautFull4.2 + Java 21Configure executor in application.yml
HelidonFull (Helidon SE natively VT)4.0 + Java 21Default in Helidon SE 4.0+
JDBC (generic)Partial — driver-dependentVariesVerify your driver does not pin on socket I/O; use connection pool semaphore
HikariCPCompatible5.1.0+Size pool to downstream DB capacity; not to virtual thread count
R2DBC (reactive JDBC)Not applicableAlready non-blocking; do not mix with virtual thread executors
Apache Kafka clientsCompatible3.6+Verify consumer thread model; internal locking patterns resolved by JEP 491 on Java 24+
gRPC (Java)Needs explicit executor config1.58+Pass virtual thread executor to ServerBuilder.executor()

When you are ready to migrate, the practical checklist is straightforward. Start by enabling JFR virtual thread pinning recording on a staging environment under realistic load — this surfaces any remaining pinning before it reaches production. Then enable virtual threads via the framework property, apply a semaphore or bounded scope to any resource pools your threads access, and audit ThreadLocal usage for any caching patterns that assume thread reuse. Finally, monitor heap growth under load in staging and compare it against your platform thread baseline.

Enable virtual threads in Spring Boot (application.properties)

spring.threads.virtual.enabled=true

Verify virtual threads are active at runtime

Thread.ofVirtual().start(() ->
    System.out.println("Is virtual: " + Thread.currentThread().isVirtual())
).join();

Use Scoped Values instead of ThreadLocal for context propagation (Java 25)

static final ScopedValue REQUEST_ID = ScopedValue.newInstance();

ScopedValue.where(REQUEST_ID, "abc-123").run(() -> {
    processRequest(); // REQUEST_ID.get() returns "abc-123" anywhere in the call stack
});

9. What We Have Learned

Virtual threads delivered on the core promise: I/O-bound services running on Java 21 with a single configuration change saw 2–3x throughput improvements in real production conditions. That is not a benchmark artifact — it is a genuine win for a specific, common class of server application. The simplicity win is real too: teams that had been maintaining reactive codebases for scalability now have a credible alternative that does not require a different concurrency mental model.

At the same time, the first year exposed four recurring mistakes. Pinning caused real deadlocks — including at Netflix — because synchronized code inside third-party libraries was invisible until something went wrong. Thread-local caching silently bloated memory at scale because the assumption that threads are reused, which is fundamental to that pattern, is false for virtual threads. CPU-bound workloads saw no benefit and occasionally saw regressions. And unbounded concurrency moved the bottleneck from thread creation to resource contention without eliminating it.

Java 24 and 25 addressed the most severe of these directly. JEP 491 fixed synchronized pinning, Java 23 improved thread dump observability, and Java 25’s finalized Scoped Values give teams the right tool for context propagation. Teams who adopted virtual threads in Java 21 and hit these rough edges can now revisit those assumptions. The platform has caught up to the promise significantly faster than most expected.

The honest summary for 2026: if you have an I/O-bound service on Java 25 and you have not enabled virtual threads yet, you are leaving throughput and simplicity on the table for no good reason. If you have a CPU-bound service or a reactive stack, virtual threads are still not your answer — and that is perfectly fine.

Eleftheria Drosopoulou

Eleftheria is an Experienced Business Analyst with a robust background in the computer software industry. Proficient in Computer Software Training, Digital Marketing, HTML Scripting, and Microsoft Office, they bring a wealth of technical skills to the table. Additionally, she has a love for writing articles on various tech subjects, showcasing a talent for translating complex concepts into accessible content.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Back to top button