Thread Pool

I mentioned in my last post that I would cover some interesting topics that all relate to thread. Now here is the second one, on thread pool.

Thread pool is an efficient way to achieve concurrency since it saves the cost of creating new threads all the time by reusing threads in a pool. To get a good understanding of how to leverage ThreadPool class in the .NET framework, you can start by reading Jeffery Richter's good article on MSDN. However that was .NET 1.1. In .NET 2.0, the ThreadPool class supports a total of 4 types of uses as opposed to 3, which can be seen clearly from the enumeration in SSCLI2:

enum ThreadpoolThreadType

{

    WorkerThread,

    CompletionPortThread,

    WaitThread,

    TimerMgrThread

};

With the new ThreadPool class, we can

• 1) call a method on a worker thread by QueueUserWorkItem


• 2) call a method on an IO Completion port thread by UnsafeQueueNativeOverlapped


• 3) call a method on an IO Completion port thread when a kernel object is signaled by RegisterWaitForSingleObject


• 4) call a method on a worker thread by using System.Threading.Timer - note this is the API seemingly irrelevant to ThreadPool

While IO Completion Port (IOCP) is not new to native Windows, the inclusion of its API in thread pool is the latest feature added to .NET 2.0. IOCP is a mechanism on Windows to allow asynchronous IO. The IOCP threads will be woken up and start code execution if IO operation is complete. The Win32 API for this is GetQueuedCompletionStatus, which essentially doesn't return (thread back to the pool) until IO operation is done. And more interestingly, IOCP isn't just limited to being used for IO - ThreadPool.UnsafeQueueNativeOverlapped can be used just like ThreadPool.QueueUserWorkItem. Here is a code snippet for queueing a delegate to the IOCP threads via Overlapped data structure:

unsafe

{

    Overlapped overlapped = new Overlapped(0, 0, IntPtr.Zero, null);

    NativeOverlapped* pOverlapped = overlapped.Pack(IocpThreadProc, null);

    ThreadPool.UnsafeQueueNativeOverlapped(pOverlapped);

}

Note that unsafe keyword is used as native pointer is created. And here is the delegate:

unsafe static void IocpThreadProc(uint x, uint y, NativeOverlapped* p)

{  

    try

    {

        Sort();

    }

    finally

    {

        Overlapped.Free(p);

    }

}

Note that Free must be called in the finally block to ensure that memory is not leaked.

The philosophy of IOCP is that the optimal number of concurrent running threads should be equal to the number of processors, because anything more would introduce the overhead of context switches between different threads. Peeking into the win32threadpool.cpp file of SSCLI2, one can find that while both the number of worker threads and the number of IOCP threads can grow under the ceiling of upper limits set by ThreadPool.SetMaxThreads, IOCP has constraints from CPU utilization also. In other words, even if you set the max of IOCP threads to a number much higher than the number of processros, there won't be more concurrent threads than the number of processors if there is very high CPU utilization. On the other hand, the number of worker threads, under the situation of high CPU utilitzation, will likely grow under the ceiling with requests queued up. This blog has shown empirical results proving this. The example is a bit of extreme in that a large number of highly computation-intensive requests are being queued to the thread pool. However it shouldn't be viewed that IOCP is always the better one to use. Remember now you can still call ThreadPool.SetMaxThreads to throttle the number of concurrent worker threads to achieve good results too. Measuring and tuning would be required in real world situations. While it's great that we have an additional option to be able to queue any requests to IOCP through UnsafeQueueNativeOverlapped, keep in mind that IOCP was designed to be a great mechanism to handle IO asynchronously. Examples, such as System.IO.FileStream, System.Net.Sockets.Socket, System.ServiceModel.Channels.MsmqQueue, etc. can be found in the FCL. So I'd conjecture that while using a custom IO device, if asynchronous IO is desired, the same pattern can be followed by passing the handle of the IO object to ThreadPool.BindHandle to associate the IO device to the IOCP of the ThreadPool.

As to using System.Threading.Timer, this great MSDN Article discusses it pretty well in comparison with System.Timers.Timer and System.Windows.Forms.Timer. It's important to remember that System.Timers.Timer is a wrapper of System.Threading.Timer. System.Windows.Forms.Timer uses a completely different mechanism, which is SetTimer Win32 API, posting WM_TIMER message to the application queue. This means that your delegate added to the Tick event will be running on the same UI thread that created the main application windows forms. As the UI thread processes a lot of messages, including mouse and keyboard messages, this mechanism will surely not get you a lot of mileage for concurrency if that's what you're looking for.

As server apps like ASP.NET, WCF already have concurrency built in, ThreadPool should not be overly used in there. ThreadPool can be ideal in UI applications, Windows service, and batch applications, etc, if they need a lot of concurrency.