GC and active sockets

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

I'm using .net 2.0 beta2 and noticing some strange behaviour around socket
garbage collection.

Problem:
If a socket is a part of a class which has been instantiated, socket
connected, and class de-referenced Socket will stay connected forever.

Environment:
Sockets are doing async read/write

Question:
How do i get it solved :) also - is putting something on IOCP thread will
cause garbage collection to fail because of internal blocking that is going
on. How can i clean up connected socket which is a member of a class which
suddenly lost refernece. Under normal conditions I do clean up, however,
there are scenarios where abnormal termination of a thread can cause socket's
parent class loose reference.
 
Bleh... I think I found the answer as to why, no solution as of yet though....

When Socket.BeginReceive is called, buffer memory appears to be pinned,
therefore it will never unpin it until some data shows and the call
continues, the problem is that Socket.BeginReceive will call itself in the
end of its call hence new buffer will get pinned, so unless GC tries to
collect it at the moment when the call is completed but new one has not been
initiated yet .... we have a hanging socket

Anyone has any idea if there is a way to get reference count to an instance
of a class from .NET?
 
Couldn't you just make sure that you included a distructor ~ClassName() into
your object that closes the socket on it's destruction?
 
The destructor is there; however, since in .NET destructor = finalizer, the
object has to be considered for finalization or finalizer wont run.

It seems as if GC is not considering object for finalization since there is
an IOCP wait operation in progress (beginread/beginreceive). I have tried
numerous ways and

The only way to interrupt the wait is to send the data to the socket as
there is no native way to shutdown beginXXX once it has started.
 
Arthur M. said:
The destructor is there; however, since in .NET destructor = finalizer,
the
object has to be considered for finalization or finalizer wont run.

It seems as if GC is not considering object for finalization since there
is
an IOCP wait operation in progress (beginread/beginreceive). I have tried
numerous ways and

The only way to interrupt the wait is to send the data to the socket as
there is no native way to shutdown beginXXX once it has started.
Click to expand...

Could you please post a small but complete sample that illusrates the issue?

Willy.
 
In my specific case there is a socket server listenting on port 1111, you can
use anything that accepts a connection.

Note that function openConnections() never really closes a tcp client, but
instead lets them go out of scope.

Granted in true production environment this should never happen, however,
garbage collection should prevent exactly that - memory/resource leak.

My original code involves working directly with sockets, to ensure it is not
my mistake, I have reproduced it with .NETs internal TCPClient hence no
try/catch no watches for shutdowns and buffers that are thrown away

Garbage collection timer is there to ensure garbage collection is forced to
run.

Code Below:


static void openConnections()
{
for ( int op_counter = 0; op_counter < 30; op_counter++ )
{
System.Net.Sockets.TcpClient pt = new
System.Net.Sockets.TcpClient();
Singularity.Communication.Client(System.Net.IPAddress.Any, 0,
System.Net.IPAddress.Loopback, 1111);
pt.Connect(new
System.Net.IPEndPoint(System.Net.IPAddress.Loopback, 1111));
System.Net.Sockets.NetworkStream ns = pt.GetStream();
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

}
}

static void rcvd(IAsyncResult ar)
{
((System.Net.Sockets.NetworkStream) (ar.AsyncState)).EndRead(ar);
( (System.Net.Sockets.NetworkStream) ( ar.AsyncState )
).BeginRead(new byte[100], 1, 0, rcvd, ( (System.Net.Sockets.NetworkStream) (
ar.AsyncState ) ));
}

static Timer tmr = new Timer();

static void Main()
{
tmr.Interval = 1000;
tmr.Enabled = true;
tmr.Start();
tmr.Tick += new EventHandler(tmr_Tick);
//f.Show();
//f.FormClosed += new FormClosedEventHandler(f_FormClosed);
//f = null;
openConnections();
Application.Run();
}

static void tmr_Tick(object sender, EventArgs e)
{
Console.WriteLine("Collecting");
tmr.Stop();
GC.Collect();
GC.WaitForPendingFinalizers();
tmr.Start();
}
 
In my specific case there is a socket server listenting on port 1111, you can
use anything that accepts a connection.

Note that function openConnections() never really closes a tcp client, but
instead lets them go out of scope.

Granted in true production environment this should never happen, however,
garbage collection should prevent exactly that - memory/resource leak.

My original code involves working directly with sockets, to ensure it is not
my mistake, I have reproduced it with .NETs internal TCPClient hence no
try/catch no watches for shutdowns and buffers that are thrown away

Garbage collection timer is there to ensure garbage collection is forced to
run.

Code Below:


static void openConnections()
{
for ( int op_counter = 0; op_counter < 30; op_counter++ )
{
System.Net.Sockets.TcpClient pt = new
System.Net.Sockets.TcpClient();
Singularity.Communication.Client(System.Net.IPAddress.Any, 0,
System.Net.IPAddress.Loopback, 1111);
pt.Connect(new
System.Net.IPEndPoint(System.Net.IPAddress.Loopback, 1111));
System.Net.Sockets.NetworkStream ns = pt.GetStream();
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

}
}

static void rcvd(IAsyncResult ar)
{
((System.Net.Sockets.NetworkStream) (ar.AsyncState)).EndRead(ar);
( (System.Net.Sockets.NetworkStream) ( ar.AsyncState )
).BeginRead(new byte[100], 1, 0, rcvd, ( (System.Net.Sockets.NetworkStream) (
ar.AsyncState ) ));
}

static Timer tmr = new Timer();

static void Main()
{
tmr.Interval = 1000;
tmr.Enabled = true;
tmr.Start();
tmr.Tick += new EventHandler(tmr_Tick);
//f.Show();
//f.FormClosed += new FormClosedEventHandler(f_FormClosed);
//f = null;
openConnections();
Application.Run();
}

static void tmr_Tick(object sender, EventArgs e)
{
Console.WriteLine("Collecting");
tmr.Stop();
GC.Collect();
GC.WaitForPendingFinalizers();
tmr.Start();
}
 
Arthur M. said:
In my specific case there is a socket server listenting on port 1111, you
can
use anything that accepts a connection.

Note that function openConnections() never really closes a tcp client, but
instead lets them go out of scope.

Granted in true production environment this should never happen, however,
garbage collection should prevent exactly that - memory/resource leak.

My original code involves working directly with sockets, to ensure it is
not
my mistake, I have reproduced it with .NETs internal TCPClient hence no
try/catch no watches for shutdowns and buffers that are thrown away

Garbage collection timer is there to ensure garbage collection is forced
to
run.

Code Below:


static void openConnections()
{
for ( int op_counter = 0; op_counter < 30; op_counter++ )
{
System.Net.Sockets.TcpClient pt = new
System.Net.Sockets.TcpClient();
Singularity.Communication.Client(System.Net.IPAddress.Any, 0,
System.Net.IPAddress.Loopback, 1111);
pt.Connect(new
System.Net.IPEndPoint(System.Net.IPAddress.Loopback, 1111));
System.Net.Sockets.NetworkStream ns = pt.GetStream();
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

}
}

static void rcvd(IAsyncResult ar)
{
((System.Net.Sockets.NetworkStream)
(ar.AsyncState)).EndRead(ar);
( (System.Net.Sockets.NetworkStream) ( ar.AsyncState )
).BeginRead(new byte[100], 1, 0, rcvd, (
(System.Net.Sockets.NetworkStream) (
ar.AsyncState ) ));
}

static Timer tmr = new Timer();

static void Main()
{
tmr.Interval = 1000;
tmr.Enabled = true;
tmr.Start();
tmr.Tick += new EventHandler(tmr_Tick);
//f.Show();
//f.FormClosed += new FormClosedEventHandler(f_FormClosed);
//f = null;
openConnections();
Application.Run();
}

static void tmr_Tick(object sender, EventArgs e)
{
Console.WriteLine("Collecting");
tmr.Stop();
GC.Collect();
GC.WaitForPendingFinalizers();
tmr.Start();
}
Click to expand...


While ns goes out of scope in openConnections it is passed to BeginRead :
....
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

as argument and used in rcvd, so you keep a reference to your NetworkStream
(ns) the wrapped TcpClient (tp) and his underlying socket.
What would you expect from the GC to free here?

Willy.
 
That was exactly my point; once an object goes out of scope and effectively
is unreachable through any means available to developer, the only reference
point to that object is a blocking IOCP thread.

I agree that from pure system's point of view everything works properly,
reference A goes away but reference B is still there. From development stand
point of view, however, once i place something into IOCP pool there is no way
for me to signal interrupt therefore, there is a logical assumption that if i
remove all references to the object and the only one remains is IOCP
reference, it should get garbage collected. Granted, leaving objects out of
scope without clean up is bad programming habbit as is, but isn't garbage
collection supposed to protect against those type of mistakes?

I think:
( no refernce = garbage) equally (no reference = there is no way for a
developer to access an object) and therefore (no way for a developer to
access an object = garbage)

Whether it is an adjustment to GC or adjustment to IOCP implementation (give
me an ability to get BeginXXX operation out of wait state) is question
someone in Microsoft (in my opinion) needs to ask and answer.




Willy Denoyette said:
Arthur M. said:
In my specific case there is a socket server listenting on port 1111, you
can
use anything that accepts a connection.

Note that function openConnections() never really closes a tcp client, but
instead lets them go out of scope.

Granted in true production environment this should never happen, however,
garbage collection should prevent exactly that - memory/resource leak.

My original code involves working directly with sockets, to ensure it is
not
my mistake, I have reproduced it with .NETs internal TCPClient hence no
try/catch no watches for shutdowns and buffers that are thrown away

Garbage collection timer is there to ensure garbage collection is forced
to
run.

Code Below:


static void openConnections()
{
for ( int op_counter = 0; op_counter < 30; op_counter++ )
{
System.Net.Sockets.TcpClient pt = new
System.Net.Sockets.TcpClient();
Singularity.Communication.Client(System.Net.IPAddress.Any, 0,
System.Net.IPAddress.Loopback, 1111);
pt.Connect(new
System.Net.IPEndPoint(System.Net.IPAddress.Loopback, 1111));
System.Net.Sockets.NetworkStream ns = pt.GetStream();
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

}
}

static void rcvd(IAsyncResult ar)
{
((System.Net.Sockets.NetworkStream)
(ar.AsyncState)).EndRead(ar);
( (System.Net.Sockets.NetworkStream) ( ar.AsyncState )
).BeginRead(new byte[100], 1, 0, rcvd, (
(System.Net.Sockets.NetworkStream) (
ar.AsyncState ) ));
}

static Timer tmr = new Timer();

static void Main()
{
tmr.Interval = 1000;
tmr.Enabled = true;
tmr.Start();
tmr.Tick += new EventHandler(tmr_Tick);
//f.Show();
//f.FormClosed += new FormClosedEventHandler(f_FormClosed);
//f = null;
openConnections();
Application.Run();
}

static void tmr_Tick(object sender, EventArgs e)
{
Console.WriteLine("Collecting");
tmr.Stop();
GC.Collect();
GC.WaitForPendingFinalizers();
tmr.Start();
}
Click to expand...


While ns goes out of scope in openConnections it is passed to BeginRead :
....
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

as argument and used in rcvd, so you keep a reference to your NetworkStream
(ns) the wrapped TcpClient (tp) and his underlying socket.
What would you expect from the GC to free here?

Willy.
Click to expand...
 
Objects don't go out of scope, references do go out of scope.
Your local references go out of scope, but as YOU passed a NetworkStream
reference to the BeginRead delegate which posts a request to the completion
thread pool which stores the NetworkStream reference for later use, you have
yet another reference to the NetworkStream Object ( and as such to the
TcpClient and the Socket instances), so the GC can't collect. At this point
YOU say he! this is a "bad programming habit" I would like the GC to
consider the NS object as garbage, I say "it's a bug in your code" and the
GC can't and isn't supposed to protect you from bugs in your code.

When you're done with the TcpClient, you simply have to close it by calling
TcpClient.Close(), this will effectively remove the request from the pool,
close the socket (Dispose) the NetworkStream (Dispose) and the TcpClient
objects.

Willy.



Arthur M. said:
That was exactly my point; once an object goes out of scope and
effectively
is unreachable through any means available to developer, the only
reference
point to that object is a blocking IOCP thread.

I agree that from pure system's point of view everything works properly,
reference A goes away but reference B is still there. From development
stand
point of view, however, once i place something into IOCP pool there is no
way
for me to signal interrupt therefore, there is a logical assumption that
if i
remove all references to the object and the only one remains is IOCP
reference, it should get garbage collected. Granted, leaving objects out
of
scope without clean up is bad programming habbit as is, but isn't garbage
collection supposed to protect against those type of mistakes?

I think:
( no refernce = garbage) equally (no reference = there is no way for a
developer to access an object) and therefore (no way for a developer to
access an object = garbage)

Whether it is an adjustment to GC or adjustment to IOCP implementation
(give
me an ability to get BeginXXX operation out of wait state) is question
someone in Microsoft (in my opinion) needs to ask and answer.




Willy Denoyette said:
Arthur M. said:
In my specific case there is a socket server listenting on port 1111,
you
can
use anything that accepts a connection.

Note that function openConnections() never really closes a tcp client,
but
instead lets them go out of scope.

Granted in true production environment this should never happen,
however,
garbage collection should prevent exactly that - memory/resource leak.

My original code involves working directly with sockets, to ensure it
is
not
my mistake, I have reproduced it with .NETs internal TCPClient hence no
try/catch no watches for shutdowns and buffers that are thrown away

Garbage collection timer is there to ensure garbage collection is
forced
to
run.

Code Below:


static void openConnections()
{
for ( int op_counter = 0; op_counter < 30; op_counter++ )
{
System.Net.Sockets.TcpClient pt = new
System.Net.Sockets.TcpClient();
Singularity.Communication.Client(System.Net.IPAddress.Any, 0,
System.Net.IPAddress.Loopback, 1111);
pt.Connect(new
System.Net.IPEndPoint(System.Net.IPAddress.Loopback, 1111));
System.Net.Sockets.NetworkStream ns = pt.GetStream();
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

}
}

static void rcvd(IAsyncResult ar)
{
((System.Net.Sockets.NetworkStream)
(ar.AsyncState)).EndRead(ar);
( (System.Net.Sockets.NetworkStream) ( ar.AsyncState )
).BeginRead(new byte[100], 1, 0, rcvd, (
(System.Net.Sockets.NetworkStream) (
ar.AsyncState ) ));
}

static Timer tmr = new Timer();

static void Main()
{
tmr.Interval = 1000;
tmr.Enabled = true;
tmr.Start();
tmr.Tick += new EventHandler(tmr_Tick);
//f.Show();
//f.FormClosed += new FormClosedEventHandler(f_FormClosed);
//f = null;
openConnections();
Application.Run();
}

static void tmr_Tick(object sender, EventArgs e)
{
Console.WriteLine("Collecting");
tmr.Stop();
GC.Collect();
GC.WaitForPendingFinalizers();
tmr.Start();
}
Click to expand...


While ns goes out of scope in openConnections it is passed to BeginRead :
....
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

as argument and used in rcvd, so you keep a reference to your
NetworkStream
(ns) the wrapped TcpClient (tp) and his underlying socket.
What would you expect from the GC to free here?

Willy.
Click to expand...
Click to expand...
 
I think you need to re-read the thread! there is no way for ME to interrupt
a function placed in IO completion pool therefore, there is no way for me to
remove the reference once external conditions require me to, therefore I HAVE
TO wait for IOCP thread to drop the reference before i can clean up and avoid
memory leaks.

There is a logic flaw here, I cant terminate an IOCP thread until external
conditions set it off and it completes. If situation is such that conditions
will never arise ... memory leak. This is a BUG and if you dont consider it
to be a bug with GC, it is a bug with overal automatic resource management.

With socket, I do have an ability to call close and thereby release the
handle, but with external resources, it is not always possible.




Willy Denoyette said:
Objects don't go out of scope, references do go out of scope.
Your local references go out of scope, but as YOU passed a NetworkStream
reference to the BeginRead delegate which posts a request to the completion
thread pool which stores the NetworkStream reference for later use, you have
yet another reference to the NetworkStream Object ( and as such to the
TcpClient and the Socket instances), so the GC can't collect. At this point
YOU say he! this is a "bad programming habit" I would like the GC to
consider the NS object as garbage, I say "it's a bug in your code" and the
GC can't and isn't supposed to protect you from bugs in your code.

When you're done with the TcpClient, you simply have to close it by calling
TcpClient.Close(), this will effectively remove the request from the pool,
close the socket (Dispose) the NetworkStream (Dispose) and the TcpClient
objects.

Willy.



Arthur M. said:
That was exactly my point; once an object goes out of scope and
effectively
is unreachable through any means available to developer, the only
reference
point to that object is a blocking IOCP thread.

I agree that from pure system's point of view everything works properly,
reference A goes away but reference B is still there. From development
stand
point of view, however, once i place something into IOCP pool there is no
way
for me to signal interrupt therefore, there is a logical assumption that
if i
remove all references to the object and the only one remains is IOCP
reference, it should get garbage collected. Granted, leaving objects out
of
scope without clean up is bad programming habbit as is, but isn't garbage
collection supposed to protect against those type of mistakes?

I think:
( no refernce = garbage) equally (no reference = there is no way for a
developer to access an object) and therefore (no way for a developer to
access an object = garbage)

Whether it is an adjustment to GC or adjustment to IOCP implementation
(give
me an ability to get BeginXXX operation out of wait state) is question
someone in Microsoft (in my opinion) needs to ask and answer.




Willy Denoyette said:
In my specific case there is a socket server listenting on port 1111,
you
can
use anything that accepts a connection.

Note that function openConnections() never really closes a tcp client,
but
instead lets them go out of scope.

Granted in true production environment this should never happen,
however,
garbage collection should prevent exactly that - memory/resource leak.

My original code involves working directly with sockets, to ensure it
is
not
my mistake, I have reproduced it with .NETs internal TCPClient hence no
try/catch no watches for shutdowns and buffers that are thrown away

Garbage collection timer is there to ensure garbage collection is
forced
to
run.

Code Below:


static void openConnections()
{
for ( int op_counter = 0; op_counter < 30; op_counter++ )
{
System.Net.Sockets.TcpClient pt = new
System.Net.Sockets.TcpClient();
Singularity.Communication.Client(System.Net.IPAddress.Any, 0,
System.Net.IPAddress.Loopback, 1111);
pt.Connect(new
System.Net.IPEndPoint(System.Net.IPAddress.Loopback, 1111));
System.Net.Sockets.NetworkStream ns = pt.GetStream();
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

}
}

static void rcvd(IAsyncResult ar)
{
((System.Net.Sockets.NetworkStream)
(ar.AsyncState)).EndRead(ar);
( (System.Net.Sockets.NetworkStream) ( ar.AsyncState )
).BeginRead(new byte[100], 1, 0, rcvd, (
(System.Net.Sockets.NetworkStream) (
ar.AsyncState ) ));
}

static Timer tmr = new Timer();

static void Main()
{
tmr.Interval = 1000;
tmr.Enabled = true;
tmr.Start();
tmr.Tick += new EventHandler(tmr_Tick);
//f.Show();
//f.FormClosed += new FormClosedEventHandler(f_FormClosed);
//f = null;
openConnections();
Application.Run();
}

static void tmr_Tick(object sender, EventArgs e)
{
Console.WriteLine("Collecting");
tmr.Stop();
GC.Collect();
GC.WaitForPendingFinalizers();
tmr.Start();
}




While ns goes out of scope in openConnections it is passed to BeginRead :
....
ns.BeginRead(new byte[100], 1, 0, rcvd, ns);

as argument and used in rcvd, so you keep a reference to your
NetworkStream
(ns) the wrapped TcpClient (tp) and his underlying socket.
What would you expect from the GC to free here?

Willy.
Click to expand...
Click to expand...
Click to expand...
 
Arthur M. said:
I think you need to re-read the thread! there is no way for ME to
interrupt
a function placed in IO completion pool therefore, there is no way for me
to
remove the reference once external conditions require me to, therefore I
HAVE
TO wait for IOCP thread to drop the reference before i can clean up and
avoid
memory leaks.

There is a logic flaw here, I cant terminate an IOCP thread until external
conditions set it off and it completes. If situation is such that
conditions
will never arise ... memory leak. This is a BUG and if you dont consider
it
to be a bug with GC, it is a bug with overal automatic resource
management.

With socket, I do have an ability to call close and thereby release the
handle, but with external resources, it is not always possible.
Click to expand...

The GC has nothing to do with resource clean-up, he's only there to manage
the objects stored on the managed heap, not the external resources like
handles , connection, files , unmanaged memory ....
Resource management is done by applying the Dispose pattern.

Please post a complete sample that illustrates the issue.

Willy.
BTW functions aren't placed in IOCP pools, the callback will be executed on
the IOCP thread when an event is signalled on the socket. Such event can be
a close or data put on the stream by the TCP stack.
 
Back
Top