Limiting the thread pool worker thread size

  • Thread starter Thread starter PGP
  • Start date Start date
P

PGP

Hello,

I am trying to use the threadpool for database worker threads and since i
run into database connection limits, i decided to limit the thread pool
threads to a small number. I started out with 5 so i have couple of threads
for book keeping and the remaining (3) threads for database work. I do queue
more than 3 work items though. What i am seeing in this restricted setup is
that once the 3 database threads are engaged, the threading deadlocks
somehow.

Now i look back to threadpool's documentation and find that the threadpool
is also used by timers on wait operations. I know there is minimal details
here, but before i go into details does this setup raise some warning flags
in your minds?


TIA
Priyesh
 
I am trying to use the threadpool for database worker threads and since i
run into database connection limits, i decided to limit the thread pool
threads to a small number. I started out with 5 so i have couple of threads
for book keeping and the remaining (3) threads for database work. I do queue
more than 3 work items though. What i am seeing in this restricted setup is
that once the 3 database threads are engaged, the threading deadlocks
somehow.
Click to expand...
Now i look back to threadpool's documentation and find that the threadpool
is also used by timers on wait operations. I know there is minimal details
here, but before i go into details does this setup raise some warning flags
in your minds?
Click to expand...

You can limit number of ThreadPool theads SetMaxThreads but be aware
that your threads are not only ones in there. .NET uses ThreadPool for
other things also.

It is hard to give you any good answer without knowning what the
problem really is but here are few options that I can see:

1) Go to producer/consumer pattern. Have only one producer (Database
thread) feed data to other processing threads. This shouldn't slow
down your processing by much (if anything) since you made SQL server's
connections compete for resources (especially disk to retrieve data)
anyhow.

2) Think about using parallel extensions library. Good multithreading
is really hard to do and you have most of problems already solved
there (e.g. having four threads competing for two cores). It is in CTP
state, but personaly, I would dare to put it in production.

3) Think about your problem again. Maybe there is another solution
looming somewhere.
 
Thanks guys. This is exactly what i was looking to hear. My colleague
commented this was the only time he saw that adding threads helped out of a
deadlock and it's an interesting observation. I have taken the limit off now
and am looking at alternate methods to limit the queue like Pete and Josip
suggests.
 
Back
Top