Context Switches, Web Gardens and InProc Session State

  • Thread starter Thread starter Brendan Kay
  • Start date Start date
B

Brendan Kay

Hi there,

We have an ASP.Net application that is generating huge numbers of Context
Switches when a certain number of active users are present. We tried changing
the configuration of the Application Pool to use a Web Garden and the number
of Context Switches decreased by more than 95% (that's the good news).

Our problem is that we have something in our session state that cannot be
serialized and therefore we cannot use StateServer for the Session mode.

We have been trying to find a way to get all requests associated with a
specific session to be dispatched to the one worker process. The Microsoft KB
article at http://support.microsoft.com/default.aspx/kb/822171 says:

"If an application keeps a connection alive and uses the same source port,
the requests are routed back to the same worker process."

We cannot seem to get this to occur for us. Does this apply to http requests
or is it only referring to ftp and other protocols served by IIS?

We have found plenty of places that state categorically that InProc session
state cannot be used with Web Gardens but we are hoping that the KB article
above knows something that they don't. Can anyone help us to tie particular
sessions to particular worker processes?

Thanks,

Brendan
 
Brendan Kay said:
Hi there,

We have an ASP.Net application that is generating huge numbers of Context
Switches when a certain number of active users are present. We tried changing
the configuration of the Application Pool to use a Web Garden and the number
of Context Switches decreased by more than 95% (that's the good news).
Click to expand...

Define 'Huge'. Are you sure that it was a problem? How many CPU cores do
you have and can you add more? I'm not sure how using a Web Garden actually
resolves the problem. What is the CPU usage rate like?

Our problem is that we have something in our session state that cannot be
serialized and therefore we cannot use StateServer for the Session mode.

We have been trying to find a way to get all requests associated with a
specific session to be dispatched to the one worker process. The Microsoft KB
article at http://support.microsoft.com/default.aspx/kb/822171 says:

"If an application keeps a connection alive and uses the same source port,
the requests are routed back to the same worker process."

We cannot seem to get this to occur for us. Does this apply to http requests
or is it only referring to ftp and other protocols served by IIS?
Click to expand...

Are your clients connecting via a proxy or firewall, (even if they are
internal clients I've often seen configurations which unintentionaly route
HTTP through a firewall/proxy to an internal server)?

That said its not possible to guarantee a fixed connection. Additionally
browser will use more than one connection, the likely hood is then that the
browser will have one connection on one process and another to the other
process.

We have found plenty of places that state categorically that InProc session
state cannot be used with Web Gardens but we are hoping that the KB article
above knows something that they don't. Can anyone help us to tie particular
sessions to particular worker processes?
Click to expand...

Web Gardens are not compatible with inproc session state. There is no good
mechanism in place that can affiliate a session with a process.


Load balancing and a Web Farm is more intelligent its able to affiliate a
client with a server.


Other options would be:-

Divide the site into separate applications what each run in their own pool

Allow commonly request chunks of dynamically generated HTML that don't often
differ from one request to the next to be cached.
 
Anthony Jones said:
Define 'Huge'. Are you sure that it was a problem? How many CPU cores do
you have and can you add more? I'm not sure how using a Web Garden actually
resolves the problem. What is the CPU usage rate like?
Click to expand...

Hi Anthony, thanks for your reply.

You are quite right that I should have provided more details - sorry about
the vagueness of the first post.

When running on a single worker process the Context Switches per second gets
up to 300K+. This is on a dual quad core machine so it a lot of context
switches per core.

When the application pool is configured to use 8 worker processes (one per
core) the context switches fall to less than 10K.

In both cases the CPU utilization stays around 30-50%. It seems to be pretty
clear that the context switches are the problem for us (unless I am missing
something else ;-).

I am not really sure why there would be such a big difference in context
switches because of the number of worker process (for exactly the same load)
but it is dramatic. The average response time per page goes from about 1.3s
to 12s.
Are your clients connecting via a proxy or firewall, (even if they are
internal clients I've often seen configurations which unintentionaly route
HTTP through a firewall/proxy to an internal server)?
Click to expand...

No, the clients are connecting directly (we have done tests that go through
a firewall and the results are the same).
That said its not possible to guarantee a fixed connection. Additionally
browser will use more than one connection, the likely hood is then that the
browser will have one connection on one process and another to the other
process.

Web Gardens are not compatible with inproc session state. There is no good
mechanism in place that can affiliate a session with a process.
Click to expand...

That is a shame (but not a huge surprise). We were hoping the knowledge base
article 822171 might be referring to some technique for keeping the
connection alive and having all the requests for a particular session routed
to the same worker process.
Load balancing and a Web Farm is more intelligent its able to affiliate a
client with a server.
Click to expand...

We are looking into that now.
Other options would be:-

Divide the site into separate applications what each run in their own pool
Click to expand...

This is another option we are looking at. We are seeing whether the
application can be run as several IIS applications and we may put our own
dispatcher in front of it.
Allow commonly request chunks of dynamically generated HTML that don't often
differ from one request to the next to be cached.
Click to expand...

Unfortunately all our pages are generated dynamically so caching is not an
option but we do believe we will find an alternate solution.

Thanks again for your thoughts.
 
Hi Mark,

Thanks for the reply. We will look into the web farm option as well as
options for inserting our own dispatcher.

Brendan
 
Brendan Kay said:
Hi Anthony, thanks for your reply.

You are quite right that I should have provided more details - sorry about
the vagueness of the first post.

When running on a single worker process the Context Switches per second gets
up to 300K+. This is on a dual quad core machine so it a lot of context
switches per core.

When the application pool is configured to use 8 worker processes (one per
core) the context switches fall to less than 10K.

In both cases the CPU utilization stays around 30-50%. It seems to be pretty
clear that the context switches are the problem for us (unless I am missing
something else ;-).

I am not really sure why there would be such a big difference in context
switches because of the number of worker process (for exactly the same load)
but it is dramatic. The average response time per page goes from about 1.3s
to 12s.
Click to expand...

Did you mean from 12s prior to using a web garden to 1.3s using a web
garden?
Do the worker processes run in 64bit?
How much memory do you have on the box?
No, the clients are connecting directly (we have done tests that go through
a firewall and the results are the same).


That is a shame (but not a huge surprise). We were hoping the knowledge base
article 822171 might be referring to some technique for keeping the
connection alive and having all the requests for a particular session routed
to the same worker process.


We are looking into that now.
pool

This is another option we are looking at. We are seeing whether the
application can be run as several IIS applications and we may put our own
dispatcher in front of it.
Click to expand...

I'm not sure what you mean by dispatcher or why you would need one?

The problem would be if there is some state in one sub application or
session thereof, that is needed in another sub application. Typically this
would be user authentication data. You would need to handle this yourself
and would probably need to form an umbrella session cookie that has a
path="/" (the sub application session cookies will be rooted in their
directories). It can get a bit hairy, and ultimately more costly that
installing a Web Farm. OTH, if most access is anonymous and you have
significant independant parts to your site then its fairly straightforward.
Unfortunately all our pages are generated dynamically so caching is not an
option but we do believe we will find an alternate solution.
Click to expand...

They may be generated dynamically but is the underlying data so volatile
that the same URL is unlikely to generate the same markup even when hit
several times a second. Just because something is generated dynamically
does not mean it can't be cached. Even for data that some would consider
quite volatile a 2 minute cache period for a high frequency URL could have a
significant impact.
 
Hi Anthony,

Thanks again for your continuing help.
Did you mean from 12s prior to using a web garden to 1.3s using a web
garden?
Do the worker processes run in 64bit?
How much memory do you have on the box?
Click to expand...

Yep, avarage page response time is about 1.3s when configured as a web
garden and about 12s when configured as a single worker thread.

The server has 4Gb of memory and the actual memory used by each worker
process is only about 100Mb. The server is running 32bit (and we can't change
that right now).
I'm not sure what you mean by dispatcher or why you would need one?

The problem would be if there is some state in one sub application or
session thereof, that is needed in another sub application. Typically this
would be user authentication data. You would need to handle this yourself
and would probably need to form an umbrella session cookie that has a
path="/" (the sub application session cookies will be rooted in their
directories). It can get a bit hairy, and ultimately more costly that
installing a Web Farm. OTH, if most access is anonymous and you have
significant independant parts to your site then its fairly straightforward.
Click to expand...

If we do go with the dispatcher it will be a front end application that
receives the request and creates an outer session for it. This session would
simply remember which URL it is passed on to and store the appropriate
session id/cookie to pass on. For any requests that come in from the same
outer session it would pass them on to the inner session with the appropriate
cookie (or URL).

In theory we believe it will work and we think the performance impact will
be much less than the context switches we are currently getting but we have
not actually coded it yet so there may be something that stops it being used.
They may be generated dynamically but is the underlying data so volatile
that the same URL is unlikely to generate the same markup even when hit
several times a second. Just because something is generated dynamically
does not mean it can't be cached. Even for data that some would consider
quite volatile a 2 minute cache period for a high frequency URL could have a
significant impact.
Click to expand...

In our case the dynamically generated pages use data from another source
that cannot be cached and must always display the current version (this is
ultimately the data and state that actually stops us from serializing the
session state). We can't see a way to use caching but we hope the web farm or
dispatcher will give us what we need.

Thanks again for your help.

Brendan
 
Back
Top