Thread pool: Reusing threads

In short: A fixed group of prepared threads that work through tasks from a queue instead of starting a new thread for every task.

In more detail: Creating a thread costs time and memory. Whoever starts a new thread for every small task wastes resources and can overload the system with many simultaneous tasks. A thread pool keeps a limited number of workers ready and puts incoming tasks into a queue.

In Depth

How it works

1. At startup N worker threads are created.
2. New tasks go into a task queue.
3. A free worker takes the next task, executes it and then fetches the next one. If all workers are busy, the task waits in the queue. This keeps the number of simultaneous threads limited.

Example in Java

ExecutorService pool = Executors.newFixedThreadPool(4);
Future<Integer> result = pool.submit(() -> compute());
System.out.println(result.get());
pool.shutdown();

Python offers concurrent.futures.ThreadPoolExecutor, Go relies on lightweight goroutines, and Node.js internally uses a pool for file access.

The right size

For compute-intensive tasks, roughly the number of CPU cores fits. For tasks that often wait for network or disk, the pool can be much larger because the threads mostly sleep. A rule of thumb: threads ≈ cores × (1 + wait time / compute time). Measurements beat any formula.

Pitfalls

An unbounded queue can fill memory under overload; a bounded queue with a rule for what happens on overflow (reject, run in the caller) is better. Tasks that wait for each other can jam a pool (deadlock) if all workers are blocked. Errors in tasks must be handled, otherwise they are lost unnoticed. Shut pools down cleanly on exit.

See also: Thread, Concurrency, Deadlock, Queue