Home/Learn/Operating Systems/Thread Pools

Thread Pools

Intermediate

Processes & Threads

A thread pool maintains a set of pre-created worker threads that pick up tasks from a queue, avoiding the overhead of creating and destroying threads for every task.

Overview

Creating a Java platform thread is expensive — the JVM must allocate a stack (default 512 KB to 1 MB), the OS must create a kernel thread data structure, and the scheduler must be informed. For a web server handling thousands of requests per second, creating a new thread per request would be catastrophic. Thread pools solve this by creating N worker threads once, maintaining a work queue, and reusing threads across many tasks. Java's Executor framework (java.util.concurrent) provides ready-made pools: FixedThreadPool, CachedThreadPool, ScheduledThreadPool, SingleThreadExecutor, and ForkJoinPool. The underlying ThreadPoolExecutor exposes fine-grained control over core pool size, max pool size, keep-alive time, work queue type, and rejection policy.

ThreadPoolExecutor: Full Control

ThreadPoolExecutor is the engine behind all Executors factory methods. Understanding its parameters is critical: corePoolSize (threads always alive), maximumPoolSize (peak threads), keepAliveTime (how long excess threads survive idle), workQueue (where tasks wait), and RejectedExecutionHandler (what happens when queue is full and max threads reached).

Java — ThreadPoolExecutor with all 7 parameters

// ThreadPoolExecutor with all parameters explicitly configured
ThreadPoolExecutor executor = new ThreadPoolExecutor(
    4,                                   // corePoolSize: always-on threads
    16,                                  // maximumPoolSize: peak threads under load
    60L, TimeUnit.SECONDS,               // keepAliveTime: idle extra threads live 60s
    new ArrayBlockingQueue<>(1000),      // bounded work queue (prevents memory blow-up)
    new ThreadFactory() {
        private final AtomicInteger count = new AtomicInteger(1);
        public Thread newThread(Runnable r) {
            Thread t = new Thread(r, "worker-" + count.getAndIncrement());
            t.setDaemon(false);
            return t;
        }
    },
    new ThreadPoolExecutor.CallerRunsPolicy() // rejection: caller thread runs the task
);

// Submit a task and get a Future
Future<String> future = executor.submit(() -> {
    Thread.sleep(100);
    return "result-" + Thread.currentThread().getName();
});
System.out.println(future.get(5, TimeUnit.SECONDS));

// Graceful shutdown
executor.shutdown();                             // stop accepting new tasks
executor.awaitTermination(30, TimeUnit.SECONDS); // wait for in-flight tasks

Executors Factory Methods

The Executors utility class provides common configurations. FixedThreadPool: N threads, unbounded LinkedBlockingQueue — good for CPU-bound tasks. CachedThreadPool: 0 core threads, unlimited max, SynchronousQueue — good for many short I/O tasks. ScheduledThreadPool: for recurring/delayed tasks. ForkJoinPool: work-stealing pool for recursive divide-and-conquer (used by parallel streams and CompletableFuture).

Java — Executors factory methods and use cases

// 1. FixedThreadPool — CPU-bound, predictable resource usage
ExecutorService fixed = Executors.newFixedThreadPool(
    Runtime.getRuntime().availableProcessors()
);

// 2. CachedThreadPool — many short-lived I/O tasks
// WARNING: can create thousands of threads under load — use with caution
ExecutorService cached = Executors.newCachedThreadPool();

// 3. ScheduledThreadPool — periodic tasks
ScheduledExecutorService scheduled = Executors.newScheduledThreadPool(2);
scheduled.scheduleAtFixedRate(
    () -> System.out.println("Heartbeat: " + Instant.now()),
    0, 5, TimeUnit.SECONDS   // initial delay=0, period=5s
);

// 4. ForkJoinPool — parallel streams, CompletableFuture default pool
ForkJoinPool fjp = ForkJoinPool.commonPool();
System.out.println("Parallelism: " + fjp.getParallelism());

// CompletableFuture uses ForkJoinPool.commonPool() by default
CompletableFuture<String> cf = CompletableFuture.supplyAsync(() -> "async result");
System.out.println(cf.get());

// ALWAYS shutdown executors in production
fixed.shutdown();
cached.shutdown();
scheduled.shutdown();

Rejection Policies and Shutdown

When the work queue is full and the pool is at maximum capacity, the RejectedExecutionHandler decides what to do. Java provides four: AbortPolicy (throw RejectedExecutionException — default), CallerRunsPolicy (caller thread runs the task, provides back-pressure), DiscardPolicy (silently drop), DiscardOldestPolicy (drop oldest queued task). For production, CallerRunsPolicy is often best because it slows the producer.

Java — rejection policies, monitoring, and shutdown strategies

// Rejection policy comparison
// AbortPolicy (default): throws RejectedExecutionException
ThreadPoolExecutor abort = new ThreadPoolExecutor(
    1, 1, 0L, TimeUnit.MS, new SynchronousQueue<>(),
    new ThreadPoolExecutor.AbortPolicy()
);

// CallerRunsPolicy: back-pressure — producer runs rejected task itself
ThreadPoolExecutor backPressure = new ThreadPoolExecutor(
    2, 4, 60L, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(10),
    new ThreadPoolExecutor.CallerRunsPolicy()   // slows producer naturally
);

// Monitoring pool health
System.out.println("Pool size:    " + backPressure.getPoolSize());
System.out.println("Active tasks: " + backPressure.getActiveCount());
System.out.println("Queue size:   " + backPressure.getQueue().size());
System.out.println("Completed:    " + backPressure.getCompletedTaskCount());

// Shutdown vs shutdownNow
backPressure.shutdown();       // graceful: no new tasks, wait for queued tasks
// backPressure.shutdownNow(); // forceful: interrupts running threads, returns queued tasks

Key Points to Remember

1Thread creation is expensive (~512KB–1MB stack + kernel thread overhead) — pools reuse threads.
2ThreadPoolExecutor parameters: corePoolSize, maxPoolSize, keepAlive, queue, threadFactory, rejectionPolicy.
3FixedThreadPool uses an unbounded queue — can OOM if tasks accumulate; prefer bounded queues in production.
4CachedThreadPool can create unlimited threads — dangerous under sustained high load.
5CallerRunsPolicy provides natural back-pressure: the producer thread runs rejected tasks, slowing submission.
6Always call shutdown() or shutdownNow() to release thread resources; use awaitTermination() for graceful drain.

Interview Questions

What happens when all threads are busy and the queue is full in a ThreadPoolExecutor?

MediumAmazon

Why is newCachedThreadPool dangerous for long-running or CPU-bound tasks?

MediumGoogle

What is the difference between shutdown() and shutdownNow()?

EasyAtlassian

Design a thread pool configuration for a web server handling 10,000 concurrent I/O-bound requests.

HardNetflix

Ask Aria about Thread Pools

Your personal AI tutor — ask anything about this concept

Revision Status

Personal Notes

Discussion

Loading discussion…

Daemon Threads

Concurrency vs Parallelism