Connection Pooling Maxed Out

[msnew.microsoft.com]

Hi

We migrated our web server from a single processor, 1 GB RAM to a
dual-processor blade server with 2 GB RAM and we hit this problem. We did
not make any changes to the pool size i.e. using the default settings.
We had encountered this problem occasionally (1-2 times a month) much
earlier in the previous server and we tidied up our application to
explicitly close the connections and the problem went away.

When we moved to the new servers (incl the database and the application
servers), this problem occurs everyday from the 4th day onwards (since the
move) with increasing frequency. Each time we did the IISReset. At the
SQLServer, we noticed about 40 connections at the time of the problem.

As a temporary measure, we increased the max pool size to 400 and we
discovered that the no of connections could hit the 400 mark. What we could
not understand is why this did not happen in the old servers if there was
still a problem with our application. We are using DOTNET Framework 1.0
SP2.

As this is not a public domain application, the usage pattern should be
fairly consistent, especially in the span of 1-2 weeks.

-Tang

 

[Miha Markic [MVP C#]]

Hi,

For the warmup - are you sure you are closing the connections?

 

[Tang Thim Sing]

Hi Miha

There are still places where we haven't done this completely but what's
puzzling is why this didn't happen on the previously on the old servers.
Our application is rather huge and we did one round of cleanup (closing the
connections) for the more commonly-used pages when we first realised this
problem i.e. when this was happening 1-2 times a month. After that round of
cleanup, the problem went away.

Now, it is hitting us several times a day, even during off-peak periods. If
those places where we haven't closed the connections are the cause, we
should have been hit earlier on.

-Tang

Hi,

For the warmup - are you sure you are closing the connections?

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

Hi

We migrated our web server from a single processor, 1 GB RAM to a
dual-processor blade server with 2 GB RAM and we hit this problem. We did
not make any changes to the pool size i.e. using the default settings.
We had encountered this problem occasionally (1-2 times a month) much
earlier in the previous server and we tidied up our application to
explicitly close the connections and the problem went away.

When we moved to the new servers (incl the database and the application
servers), this problem occurs everyday from the 4th day onwards (since the
move) with increasing frequency. Each time we did the IISReset. At the
SQLServer, we noticed about 40 connections at the time of the problem.

As a temporary measure, we increased the max pool size to 400 and we
discovered that the no of connections could hit the 400 mark. What we could
not understand is why this did not happen in the old servers if there was
still a problem with our application. We are using DOTNET Framework 1.0
SP2.

As this is not a public domain application, the usage pattern should be
fairly consistent, especially in the span of 1-2 weeks.

-Tang

 

[Miha Markic [MVP C#]]

Hi Tang,

Not necessarily.
The connection is also closed when the instance is garbage collected.
I guess that garbage collector works different of the new machine since it
is a multiproc and has more ram.
The only good way is to make sure you explicitly close all connections.

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

Hi Miha

There are still places where we haven't done this completely but what's
puzzling is why this didn't happen on the previously on the old servers.
Our application is rather huge and we did one round of cleanup (closing the
connections) for the more commonly-used pages when we first realised this
problem i.e. when this was happening 1-2 times a month. After that round of
cleanup, the problem went away.

Now, it is hitting us several times a day, even during off-peak periods. If
those places where we haven't closed the connections are the cause, we
should have been hit earlier on.

-Tang

Hi,

For the warmup - are you sure you are closing the connections?

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

Hi

We migrated our web server from a single processor, 1 GB RAM to a
dual-processor blade server with 2 GB RAM and we hit this problem. We did
not make any changes to the pool size i.e. using the default settings.
We had encountered this problem occasionally (1-2 times a month) much
earlier in the previous server and we tidied up our application to
explicitly close the connections and the problem went away.

When we moved to the new servers (incl the database and the application
servers), this problem occurs everyday from the 4th day onwards (since the
move) with increasing frequency. Each time we did the IISReset. At the
SQLServer, we noticed about 40 connections at the time of the problem.

As a temporary measure, we increased the max pool size to 400 and we
discovered that the no of connections could hit the 400 mark. What we could
not understand is why this did not happen in the old servers if there was
still a problem with our application. We are using DOTNET Framework 1.0
SP2.

As this is not a public domain application, the usage pattern should be
fairly consistent, especially in the span of 1-2 weeks.

-Tang

 

[Rajesh Patel]

is it good to call Garbage collector explicitly in application. This may
solve the problem!


Regards,

Rajesh Patel

Hi Tang,

Not necessarily.
The connection is also closed when the instance is garbage collected.
I guess that garbage collector works different of the new machine since it
is a multiproc and has more ram.
The only good way is to make sure you explicitly close all connections.

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

Hi Miha

There are still places where we haven't done this completely but what's
puzzling is why this didn't happen on the previously on the old servers.
Our application is rather huge and we did one round of cleanup (closing the
connections) for the more commonly-used pages when we first realised this
problem i.e. when this was happening 1-2 times a month. After that

round

of
cleanup, the problem went away.

Now, it is hitting us several times a day, even during off-peak periods. If
those places where we haven't closed the connections are the cause, we
should have been hit earlier on.

-Tang

Hi,

For the warmup - are you sure you are closing the connections?

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

Hi

We migrated our web server from a single processor, 1 GB RAM to a
dual-processor blade server with 2 GB RAM and we hit this problem.

We
did

not make any changes to the pool size i.e. using the default settings.
We had encountered this problem occasionally (1-2 times a month) much
earlier in the previous server and we tidied up our application to
explicitly close the connections and the problem went away.

When we moved to the new servers (incl the database and the application
servers), this problem occurs everyday from the 4th day onwards

(since
the

move) with increasing frequency. Each time we did the IISReset. At the
SQLServer, we noticed about 40 connections at the time of the problem.

As a temporary measure, we increased the max pool size to 400 and we
discovered that the no of connections could hit the 400 mark. What we
could
not understand is why this did not happen in the old servers if

there
was

still a problem with our application. We are using DOTNET Framework 1.0
SP2.

As this is not a public domain application, the usage pattern should be
fairly consistent, especially in the span of 1-2 weeks.

-Tang

 

[Miha Markic [MVP C#]]

Hi Rajesh,

is it good to call Garbage collector explicitly in application. This may
solve the problem!

No, it isn't a very good idea to force garbage collection.

 

[Tang Thim Sing]

Hi Miha

That's what we are doing right now, but it's scary to know that there's very
little room for missing out a few connections here and there. We were
hoping there was some place that we can tweak (at the OS or IIS).

Thanks

Hi Tang,

Not necessarily.
The connection is also closed when the instance is garbage collected.
I guess that garbage collector works different of the new machine since it
is a multiproc and has more ram.
The only good way is to make sure you explicitly close all connections.

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

Hi Miha

There are still places where we haven't done this completely but what's
puzzling is why this didn't happen on the previously on the old servers.
Our application is rather huge and we did one round of cleanup (closing the
connections) for the more commonly-used pages when we first realised this
problem i.e. when this was happening 1-2 times a month. After that

round

of
cleanup, the problem went away.

Now, it is hitting us several times a day, even during off-peak periods. If
those places where we haven't closed the connections are the cause, we
should have been hit earlier on.

-Tang

Hi,

For the warmup - are you sure you are closing the connections?

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

Hi

We migrated our web server from a single processor, 1 GB RAM to a
dual-processor blade server with 2 GB RAM and we hit this problem.

We
did

not make any changes to the pool size i.e. using the default settings.
We had encountered this problem occasionally (1-2 times a month) much
earlier in the previous server and we tidied up our application to
explicitly close the connections and the problem went away.

When we moved to the new servers (incl the database and the application
servers), this problem occurs everyday from the 4th day onwards

(since
the

move) with increasing frequency. Each time we did the IISReset. At the
SQLServer, we noticed about 40 connections at the time of the problem.

As a temporary measure, we increased the max pool size to 400 and we
discovered that the no of connections could hit the 400 mark. What we
could
not understand is why this did not happen in the old servers if

there
was

still a problem with our application. We are using DOTNET Framework 1.0
SP2.

As this is not a public domain application, the usage pattern should be
fairly consistent, especially in the span of 1-2 weeks.

-Tang

 

[Miha Markic [MVP C#]]

Hi Tang,

Hi Miha

That's what we are doing right now, but it's scary to know that there's very
little room for missing out a few connections here and there. We were
hoping there was some place that we can tweak (at the OS or IIS).

That's how it works .
You might check your progam with a profiler - it might show you the leaked
connections and their origin in code.

 

[Tang Thim Sing]

Hi Miha

We have gone through our entire application and made sure we closed and also
disposed the connections.

We are monitoring the counters for the .NET CLR data for # of pooled
connections and we are still seeing connections when all our web pages have
been closed off. Also at the SQL Server, we are monitoring the connections
via SP_WHO and we see the same number of connections under SLEEPING status.

We noticed that after a while the number of these connections dropped,
presumably being reclaimed by the system. At the Profiler we saw the action
'AUDIT Logout' corresponding to the dropping of these sleeping connections.
However, there are still a number of connections remaining.

-Tang

 

[William \(Bill\) Vaughn]

Warning: the SqlClient performance counters are a bit buggy and can falsely
report active connections when there are none. We've seen that the pool is
correctly cleared when the processes that create them end. Keep in mind that
VS.NET also opens and holds connections for its own use. You might be seeing
those.

Another factor that's often overlooked is when you pass a DataReader to
another scope and don't set the CommandBehavior.CloseConnection AND OR the
receiving scope does not close the DataReader.
Of course, if the application is not "done" with the connection, it can't be
closed. That is, if you execute a query that has not completed, you can't
close the connection--unless you cancel the query and abandon the
resultsets. If the query is complex, takes a long time to execute, returns
too many rows, gets blocked or locked out, or is delayed for any reason the
system can't release the connection. When another application comes in that
needs a free connection, if all of the connections are busy, the only
alternative is to open another connection. Eventually, you run out of
connections as the user load increases.


hth
--
____________________________________
William (Bill) Vaughn
Author, Mentor, Consultant
Microsoft MVP
www.betav.com
Please reply only to the newsgroup so that others can benefit.
This posting is provided "AS IS" with no warranties, and confers no rights.
__________________________________