Socket.BeginSend and EndSend causes thread count to runaway

  • Thread starter Thread starter Ravi Ambros Wallau
  • Start date Start date
R

Ravi Ambros Wallau

Hi:
My company has developed an application that, in a very basic
explanation, send files throught a socket to a known server, using a
protocol that is implemented in a full assynchronous manner.
Saying full assynchronous means that file I/O is assyncronous (using the
method BeginRead, from the class System.IO.Stream), as well as network I/O
(using the methods exposed in the class System.Net.Sockets.Socket, that are
BeginRead, EndRead, BeginReceive, and EndReceive).
In a normal behavior, there are no more than 30 threads owned by this
application. I can have 3 threads ready, 2 or 3 threads running, and 20~23
waiting (will be expurged if not used again in a minute, a .Net defined
interval, as it seems to me).
But, sometimes, the number of threads increases up to 1000, or 1500
threads in our production servers. By monitoring the number of available
threads in the "normal" behavior, I can note that the number of free workes
threads never is lower than ~47, and the number of completion port threads
is never lower than ~997 threads. This is the normal behavior, and I don't
have the production data. But, as I understand, the total number of threads
in the application can only increase to 1000, or 1500, if many threads are
in use by the application, but are not worker threads (allocated to the
completion port).
I'll try to collect more that about this situation in the production
environment, but it's a very strange behavior. Is there some bug that makes
some I/O operation to be blocked for a long time? Is there some problem with
the environment being used (Windows 2000 Server, Windows XP)? If the link to
the remote server is slow, will I/O operations take long to complete?
Show me the light, I'm a little bit lost.

Thanks,
Ravi Wallau.
 
During those periods when you have 1000 or 1500 threads engaged how many
connections are you actually supporting, that is, how many nondormant
clients connected to your server actively accessing data?

Randy Neall
 
The problem is in the client, not in the server (yeah, I know).
There are no more than two or three connections made at the same time.
The interesting thing is that, in somecases, the number of threads waiting
in the system increases in the order of two per second.
I'll take another look in my source. I don't know if the problem is related
to a connection that can't be made (a SocketException), or something related
to the FileSystemWatcher class.
Tks.
 
Suppose that I have a FileSystemWatcher, monitoring only creation file
events, and an antivirus is running... Could the antivirus causes a lot of
events? And, if so, these events can raise the number of threads to this
high number I have?
 
We, too, have two-way asynchronous socket-based communication over the
internet where we control both ends of the discussion with .Net c# programs.
At any time you have only 25 threads per processor to burn. We had a bug
resolved today where we hit that limit and our application froze. How you
are able to report thousands of threads is a complete mystery to me. What
are you looking at that convinces you that you have 1500 threads going at
once? We discovered our problem in the debugger by opening the threads
window.

Randy Neall
 
Hi:
We're trying to discover the problem. There are three ways that we know:
1. Using the Windows Task Manager;
2. Using the Performance Monitor;
3. Using the namespace System.Diagnostics, and looking at Process class.

We suspect that the problem is related to some thread that is "locked",
but we don't know yet what causes this. I don't suspect anymore that
assyncrhonous socket IO is causing this, we are looking at FileSystemWatcher
class, but we don't know yet the real causes.
 
Back
Top