Load Testing Errors

  • Thread starter Thread starter Shabam
  • Start date Start date
S

Shabam

A web application of mine developed using C# + MS SQL runs fine normally.
However when I stress test it with a load testing software (using about 60
simultaneous users) some instances start erroring out. I see two different
errors. One is a "Object reference not set to an instance of an object."
error, which appears to always contain the same information, and the other
is a "There is no row at position X.", where X is a number.

Is this an indication of bad coding or is this just a normal consequence of
overloading a web application? How can the above two errors happen when the
server is being overloaded when normally the application works fine?


ERROR #1:

Server Error in '/' Application.
----------------------------------------------------------------------------
----

Object reference not set to an instance of an object.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.NullReferenceException: Object reference not set
to an instance of an object.

Source Error:

An unhandled exception was generated during the execution of the current web
request. Information regarding the origin and location of the exception can
be identified using the exception stack trace below.

Stack Trace:


[NullReferenceException: Object reference not set to an instance of an
object.]
System.Data.DataView.GetRecord(Int32 recordIndex) +22
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750


ERROR #2:

Server Error in '/' Application.
----------------------------------------------------------------------------
----

There is no row at position 5.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.IndexOutOfRangeException: There is no row at
position 5.

Source Error:

An unhandled exception was generated during the execution of the current web
request. Information regarding the origin and location of the exception can
be identified using the exception stack trace below.

Stack Trace:


[IndexOutOfRangeException: There is no row at position 5.]
System.Data.DataView.GetRecord(Int32 recordIndex) +60
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750
 
Try to determine at what line of code the error occurs.

Are you using exception handling? Sounds like you are not..

Jeff

Shabam said:
A web application of mine developed using C# + MS SQL runs fine normally.
However when I stress test it with a load testing software (using about 60
simultaneous users) some instances start erroring out. I see two different
errors. One is a "Object reference not set to an instance of an object."
error, which appears to always contain the same information, and the other
is a "There is no row at position X.", where X is a number.

Is this an indication of bad coding or is this just a normal consequence of
overloading a web application? How can the above two errors happen when the
server is being overloaded when normally the application works fine?


ERROR #1:

Server Error in '/' Application.
-------------------------------------------------------------------------- --
----

Object reference not set to an instance of an object.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.NullReferenceException: Object reference not set
to an instance of an object.

Source Error:

An unhandled exception was generated during the execution of the current web
request. Information regarding the origin and location of the exception can
be identified using the exception stack trace below.

Stack Trace:


[NullReferenceException: Object reference not set to an instance of an
object.]
System.Data.DataView.GetRecord(Int32 recordIndex) +22
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750


ERROR #2:

Server Error in '/' Application.
-------------------------------------------------------------------------- --
----

There is no row at position 5.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.IndexOutOfRangeException: There is no row at
position 5.

Source Error:

An unhandled exception was generated during the execution of the current web
request. Information regarding the origin and location of the exception can
be identified using the exception stack trace below.

Stack Trace:


[IndexOutOfRangeException: There is no row at position 5.]
System.Data.DataView.GetRecord(Int32 recordIndex) +60
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750
Click to expand...
 
This is not normal for a heavily loaded application - ASP.NET can quite
happily put up with continuous heavy loads, as can SQL Server. These
problems are indicative of programming errors.

There are many reasons an application might fail under load when it normally
works, but the two main ones you run into most often are:

* multithreading problems
* resource starvation issues

I notice your exception is occurring inside a DataView. So this makes me
wonder if perhaps it is the first one - are you sharing a single DataSet
instance in your application?

I've seen problems when using a DataSet to cache information at the
application level. You can do this, but you need to make sure that only one
thread at a time uses the DataSet in question. (This is rather tricky if
you're binding to the DataSet, as you appear to be doing in this case - you
would need to enforce sequential access to the DataSet around your call to
DataBind.)

An easy mistake to make with the DataSet is to think "I'm only reading data
from this DataSet, so I don't need multi-threading protection."
Unfortunately, anything that uses views onto the DataSet (e.g. data binding)
does in fact modify the internal index cache of the DataTables in the
DataSet, even if you're only reading data. I've seen code fail with
NullReferenceException errors for exactlyt this reason when reading from a
DataSet on multiple threads in the past, which is why I mention this.

So if you are sharing data in static (or Shared) fields or are using the
Application state, then it could well be a multithreading thing.


Alternatively, there may be some resource that you're running out of when
the system is under load, and you're not detecting this condition correctly,
or are not taking the correct steps to avoid the problem. However, I'm not
quite sure what could be happening that that would result in the errors you
have posted, so I can't offer a useful hypothesis.


Another possibility is that there's something about the way you're using the
database that means you're just getting unexpected results when lots of
stuff is happening concurrently. The errors suggest that you're expecting
to see data but that it's missing. Perhaps you're getting null values back
from your request when you were expecting non-null values. It is
conceivable that you might see such a problem if you failed to use a
transaction when one was necessary - a database request that works fine when
the system is not under load can return inconsistent or unexpected results
under heavy load if it doesn't ensure that its work is isolated through
transactions.

(In other words, just because your request happens to work on an idle system
doesn't mean that the request is bug-free.)


But whatever the problem is, this is not 'normal' - .NET web servers don't
simply start throwing random exceptions under load. ASP.NET is a lot more
robust than that!


--
Ian Griffiths - http://www.interact-sw.co.uk/iangblog/
DevelopMentor - http://www.develop.com/

Shabam said:
A web application of mine developed using C# + MS SQL runs fine normally.
However when I stress test it with a load testing software (using about 60
simultaneous users) some instances start erroring out. I see two
different
errors. One is a "Object reference not set to an instance of an object."
error, which appears to always contain the same information, and the other
is a "There is no row at position X.", where X is a number.

Is this an indication of bad coding or is this just a normal consequence
of
overloading a web application? How can the above two errors happen when
the
server is being overloaded when normally the application works fine?


ERROR #1:

Server Error in '/' Application.
----------------------------------------------------------------------------
----

Object reference not set to an instance of an object.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.NullReferenceException: Object reference not set
to an instance of an object.

Source Error:

An unhandled exception was generated during the execution of the current
web
request. Information regarding the origin and location of the exception
can
be identified using the exception stack trace below.

Stack Trace:


[NullReferenceException: Object reference not set to an instance of an
object.]
System.Data.DataView.GetRecord(Int32 recordIndex) +22
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750


ERROR #2:

Server Error in '/' Application.
----------------------------------------------------------------------------
----

There is no row at position 5.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.IndexOutOfRangeException: There is no row at
position 5.

Source Error:

An unhandled exception was generated during the execution of the current
web
request. Information regarding the origin and location of the exception
can
be identified using the exception stack trace below.

Stack Trace:


[IndexOutOfRangeException: There is no row at position 5.]
System.Data.DataView.GetRecord(Int32 recordIndex) +60
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750
Click to expand...
 
Hi...

Which tool you are using? Also make sure that you have required data in DB
while doing stress test etc,,,

Let me know if you have any specific questions

With Best REgards
Naveen K S

Shabam said:
A web application of mine developed using C# + MS SQL runs fine normally.
However when I stress test it with a load testing software (using about 60
simultaneous users) some instances start erroring out. I see two different
errors. One is a "Object reference not set to an instance of an object."
error, which appears to always contain the same information, and the other
is a "There is no row at position X.", where X is a number.

Is this an indication of bad coding or is this just a normal consequence of
overloading a web application? How can the above two errors happen when the
server is being overloaded when normally the application works fine?


ERROR #1:

Server Error in '/' Application.
----------------------------------------------------------------------------
----

Object reference not set to an instance of an object.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.NullReferenceException: Object reference not set
to an instance of an object.

Source Error:

An unhandled exception was generated during the execution of the current web
request. Information regarding the origin and location of the exception can
be identified using the exception stack trace below.

Stack Trace:


[NullReferenceException: Object reference not set to an instance of an
object.]
System.Data.DataView.GetRecord(Int32 recordIndex) +22
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750


ERROR #2:

Server Error in '/' Application.
----------------------------------------------------------------------------
----

There is no row at position 5.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.IndexOutOfRangeException: There is no row at
position 5.

Source Error:

An unhandled exception was generated during the execution of the current web
request. Information regarding the origin and location of the exception can
be identified using the exception stack trace below.

Stack Trace:


[IndexOutOfRangeException: There is no row at position 5.]
System.Data.DataView.GetRecord(Int32 recordIndex) +60
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750
Click to expand...
 
That's exactly what I thought when I read his question. If you are caching a
DataSet/DataTable, and modifying it in your code, that will affect other
threads using that same cached object.

One tip: if you need a DataView from a cached DataTable/DataSet, don't use
DefaultView -- instead always create a new dataview with new
DataView(dtMyCachedDataTable).

Or, maybe he's caching something & not repopulating it when it expires?

--
Ben Strackany
www.developmentnow.com



Ian Griffiths said:
This is not normal for a heavily loaded application - ASP.NET can quite
happily put up with continuous heavy loads, as can SQL Server. These
problems are indicative of programming errors.

There are many reasons an application might fail under load when it normally
works, but the two main ones you run into most often are:

* multithreading problems
* resource starvation issues

I notice your exception is occurring inside a DataView. So this makes me
wonder if perhaps it is the first one - are you sharing a single DataSet
instance in your application?

I've seen problems when using a DataSet to cache information at the
application level. You can do this, but you need to make sure that only one
thread at a time uses the DataSet in question. (This is rather tricky if
you're binding to the DataSet, as you appear to be doing in this case - you
would need to enforce sequential access to the DataSet around your call to
DataBind.)

An easy mistake to make with the DataSet is to think "I'm only reading data
from this DataSet, so I don't need multi-threading protection."
Unfortunately, anything that uses views onto the DataSet (e.g. data binding)
does in fact modify the internal index cache of the DataTables in the
DataSet, even if you're only reading data. I've seen code fail with
NullReferenceException errors for exactlyt this reason when reading from a
DataSet on multiple threads in the past, which is why I mention this.

So if you are sharing data in static (or Shared) fields or are using the
Application state, then it could well be a multithreading thing.


Alternatively, there may be some resource that you're running out of when
the system is under load, and you're not detecting this condition correctly,
or are not taking the correct steps to avoid the problem. However, I'm not
quite sure what could be happening that that would result in the errors you
have posted, so I can't offer a useful hypothesis.


Another possibility is that there's something about the way you're using the
database that means you're just getting unexpected results when lots of
stuff is happening concurrently. The errors suggest that you're expecting
to see data but that it's missing. Perhaps you're getting null values back
from your request when you were expecting non-null values. It is
conceivable that you might see such a problem if you failed to use a
transaction when one was necessary - a database request that works fine when
the system is not under load can return inconsistent or unexpected results
under heavy load if it doesn't ensure that its work is isolated through
transactions.

(In other words, just because your request happens to work on an idle system
doesn't mean that the request is bug-free.)


But whatever the problem is, this is not 'normal' - .NET web servers don't
simply start throwing random exceptions under load. ASP.NET is a lot more
robust than that!


--
Ian Griffiths - http://www.interact-sw.co.uk/iangblog/
DevelopMentor - http://www.develop.com/

Shabam said:
A web application of mine developed using C# + MS SQL runs fine normally.
However when I stress test it with a load testing software (using about 60
simultaneous users) some instances start erroring out. I see two
different
errors. One is a "Object reference not set to an instance of an object."
error, which appears to always contain the same information, and the other
is a "There is no row at position X.", where X is a number.

Is this an indication of bad coding or is this just a normal consequence
of
overloading a web application? How can the above two errors happen when
the
server is being overloaded when normally the application works fine?


ERROR #1:

Server Error in '/' Application.
--------------------------------------------------------------------------
Click to expand...
--
----

Object reference not set to an instance of an object.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.NullReferenceException: Object reference not set
to an instance of an object.

Source Error:

An unhandled exception was generated during the execution of the current
web
request. Information regarding the origin and location of the exception
can
be identified using the exception stack trace below.

Stack Trace:


[NullReferenceException: Object reference not set to an instance of an
object.]
System.Data.DataView.GetRecord(Int32 recordIndex) +22
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750


ERROR #2:

Server Error in '/' Application.
--------------------------------------------------------------------------
Click to expand...
--
----

There is no row at position 5.
Description: An unhandled exception occurred during the execution of the
current web request. Please review the stack trace for more information
about the error and where it originated in the code.

Exception Details: System.IndexOutOfRangeException: There is no row at
position 5.

Source Error:

An unhandled exception was generated during the execution of the current
web
request. Information regarding the origin and location of the exception
can
be identified using the exception stack trace below.

Stack Trace:


[IndexOutOfRangeException: There is no row at position 5.]
System.Data.DataView.GetRecord(Int32 recordIndex) +60
System.Data.DataView.IsOriginalVersion(Int32 index) +9
System.Data.DataRowView.GetColumnValue(DataColumn column) +23
System.Data.DataColumnPropertyDescriptor.GetValue(Object component) +25
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName) +72
System.Web.UI.DataBinder.GetPropertyValue(Object container, String
propName, String format) +11
System.Web.UI.WebControls.ListControl.OnDataBinding(EventArgs e) +403
System.Web.UI.Control.DataBind() +26
FN.advancedsearch.populateListcontrols()
FN.advancedsearch.Page_Load(Object sender, EventArgs e)
System.Web.UI.Control.OnLoad(EventArgs e) +67
System.Web.UI.Control.LoadRecursive() +35
System.Web.UI.Page.ProcessRequestMain() +750
Click to expand...
Click to expand...
 
Ben Strackany said:
That's exactly what I thought when I read his question. If you are caching a
DataSet/DataTable, and modifying it in your code, that will affect other
threads using that same cached object.

One tip: if you need a DataView from a cached DataTable/DataSet, don't use
DefaultView -- instead always create a new dataview with new
DataView(dtMyCachedDataTable).

Or, maybe he's caching something & not repopulating it when it expires?
Click to expand...

Thanks for the input. The application is not developed by me, but by some
programmers for hire. Currently I don't yet have access to the source code,
and even if I did I wouldn't be the person to be auditting it.

The issues you guys have brought up, are they pretty much the ones I should
be asking the programmers to look into? Are there any other things they
should look for? Thanks!
 
Another thing to check when you get null reference errors in an ASP.NET
application, but ONLY under load, is whether the objects on which you're
getting the null reference exceptions are in any way retreived from the
cache.

The following seemingly innocuous pseudo code can work fine until the site
is under stress:

if (someObjectKey is in the cache) {
// Following line can give null reference exception under load
Get the Object by key and do something with it
} else {
Create new object and put it in the cache
}

The reason is that there is an outside chance that between the if test and
the retreival, the object will expire in the cache. And the busier the site
is, the more often it will happen.

The following code avoids this issue:

myType myReference = theCache(keyValue);

if (myReference == null) {
Create new object and put it into the cache
} else {
use the object via myReference
}

If the object isn't in the cache you'll get a null reference and can check
for it; if it IS in the cache you'll get the desired object. If the object
expires in the cache before you get to use it, there's no problem because
you got your reference before that happened.

Generally the failure rate for the above problem code is very low, just a
tiny fraction of a percent of the time -- < 10 times out of 250K page views
would be typical. If the failure rate is higher than that, especially at
lower volumes, then either your cache expires very frequently, or there is
some other issue to look for.

I mention this only because it is a hard bug to reproduce and even harder to
figure out, yet it's easy to prevent.

--Bob
 
Back
Top