System.StackOverflowException on Deployment

[Guest]

Hey-

I have a pretty resource intensive application that peforms huge
queries(600,000+ records, only 2 varchar fields are returned though) and
then inserts the records into a red black tree. Some records are related
though, and the related records are not inserted as new nodes but are
instead inserted into to a linked list in the corresponding node. Originally
I was using a typical binary search tree, but need to make my application
able to handle much more incoming data so I converted to a RB tree. The
depth of the RB tree never exceeds 30, which is quite reasonable in terms of
the number of recursive calls. I have used the editbin tool to increase the
stack size to 10MB. This application is running on machines that are
dedicated solely to it. On my development box it runs perfect, however when
I deploy it I run into stack overflow exceptions when I run large queries on
the same db that my dev box hits. The OS, DB, RAM size, etc.. are the same
on all boxes. I did not recompile after running the editbin tool, I merely
copied the necesasry files to the other box and ran the app there. Are there
any glaring issues in my code below, or areas where I can make it more
robust? I am at a loss trying to figure this out. Are there any free tools
that I can use to monitor stack info on boxes w/o VS installed?

Thanks!
-Stumped and frustrated by stack overflows


data retrieval chunk:
SqlCommand command = new SqlCommand( login, connection );

command.CommandTimeout = 6000000;

command.ExecuteNonQuery();

command = new SqlCommand( query, connection );

SqlDataAdapter adapt = new SqlDataAdapter();

try

{

adapt.SelectCommand = command;

adapt.Fill( data, tableName );

}

catch( Exception yy )

{

new ErrorLog( yy.ToString() );

return false;

}



RB Tree Node insertion/update chunk:

foreach( DataRow row in data.Tables[ tableName ].Rows )

Tree.Add( row[ "id" ].ToString().Trim(), new CustomData( row[
"id" ].ToString().Trim(), row[ "Reference" ].ToString().Trim() ) );

 

[Nicholas Paldino [.NET/C# MVP]]

I don't think that you should be using the editbin tool to modify the
assembly after it is compiled. Rather, instead of using a DataSet (which
means that you have to hold all the data in memory and then cycle through
it), why not get a DataReader and then cycle through the records returned,
adding the values to the tree as you cycle through the reader? It should
eliminate the need for you to expand the stack size, and also increase the
performance of your app.

Hope this helps.

 

[Guest]

Thanks for the suggestion Nicholas, I will convert my code to using a
DataReader now and post my results

Thanks!

I don't think that you should be using the editbin tool to modify the
assembly after it is compiled. Rather, instead of using a DataSet (which
means that you have to hold all the data in memory and then cycle through
it), why not get a DataReader and then cycle through the records returned,
adding the values to the tree as you cycle through the reader? It should
eliminate the need for you to expand the stack size, and also increase the
performance of your app.

Hope this helps.


--
- Nicholas Paldino [.NET/C# MVP]
- (e-mail address removed)

Hey-

I have a pretty resource intensive application that peforms huge
queries(600,000+ records, only 2 varchar fields are returned though) and
then inserts the records into a red black tree. Some records are related
though, and the related records are not inserted as new nodes but are
instead inserted into to a linked list in the corresponding node. Originally
I was using a typical binary search tree, but need to make my application
able to handle much more incoming data so I converted to a RB tree. The
depth of the RB tree never exceeds 30, which is quite reasonable in

terms

of
the number of recursive calls. I have used the editbin tool to increase the
stack size to 10MB. This application is running on machines that are
dedicated solely to it. On my development box it runs perfect, however when
I deploy it I run into stack overflow exceptions when I run large

queries

on
the same db that my dev box hits. The OS, DB, RAM size, etc.. are the same
on all boxes. I did not recompile after running the editbin tool, I merely
copied the necesasry files to the other box and ran the app there. Are there
any glaring issues in my code below, or areas where I can make it more
robust? I am at a loss trying to figure this out. Are there any free tools
that I can use to monitor stack info on boxes w/o VS installed?

Thanks!
-Stumped and frustrated by stack overflows


data retrieval chunk:
SqlCommand command = new SqlCommand( login, connection );

command.CommandTimeout = 6000000;

command.ExecuteNonQuery();

command = new SqlCommand( query, connection );

SqlDataAdapter adapt = new SqlDataAdapter();

try

{

adapt.SelectCommand = command;

adapt.Fill( data, tableName );

}

catch( Exception yy )

{

new ErrorLog( yy.ToString() );

return false;

}



RB Tree Node insertion/update chunk:

foreach( DataRow row in data.Tables[ tableName ].Rows )

Tree.Add( row[ "id" ].ToString().Trim(), new CustomData( row[
"id" ].ToString().Trim(), row[ "Reference" ].ToString().Trim() ) );

 

[Guest]

Nicholas,

I apologize for wasting your time I'm kickin myself in the head over
here, it turns out that I wasn't transferring the latest and greatest
executable over to the live box. I was using PCAnywhere's autotransfer
feature because I have components scattered in different folders due to .net
remoting and multiple related apps and I somehow forgot to add an exe. So
all my issues were related to testing an outdated version... Nonetheless, I
converted to using a DataReader instead of the DataSet and no longer have to
increase the stack size and am able to monitor incoming data progress so I
thank you for that!

Thanks again

Thanks for the suggestion Nicholas, I will convert my code to using a
DataReader now and post my results

Thanks!

message news:%[email protected]...

I don't think that you should be using the editbin tool to modify the
assembly after it is compiled. Rather, instead of using a DataSet (which
means that you have to hold all the data in memory and then cycle through
it), why not get a DataReader and then cycle through the records returned,
adding the values to the tree as you cycle through the reader? It should
eliminate the need for you to expand the stack size, and also increase the
performance of your app.

Hope this helps.


--
- Nicholas Paldino [.NET/C# MVP]
- (e-mail address removed)

Hey-

I have a pretty resource intensive application that peforms huge
queries(600,000+ records, only 2 varchar fields are returned though) and
then inserts the records into a red black tree. Some records are related
though, and the related records are not inserted as new nodes but are
instead inserted into to a linked list in the corresponding node. Originally
I was using a typical binary search tree, but need to make my application
able to handle much more incoming data so I converted to a RB tree. The
depth of the RB tree never exceeds 30, which is quite reasonable in

terms

of
the number of recursive calls. I have used the editbin tool to

increase
the

stack size to 10MB. This application is running on machines that are
dedicated solely to it. On my development box it runs perfect, however when
I deploy it I run into stack overflow exceptions when I run large

queries

on
the same db that my dev box hits. The OS, DB, RAM size, etc.. are the same
on all boxes. I did not recompile after running the editbin tool, I merely
copied the necesasry files to the other box and ran the app there. Are there
any glaring issues in my code below, or areas where I can make it more
robust? I am at a loss trying to figure this out. Are there any free tools
that I can use to monitor stack info on boxes w/o VS installed?

Thanks!
-Stumped and frustrated by stack overflows


data retrieval chunk:
SqlCommand command = new SqlCommand( login, connection );

command.CommandTimeout = 6000000;

command.ExecuteNonQuery();

command = new SqlCommand( query, connection );

SqlDataAdapter adapt = new SqlDataAdapter();

try

{

adapt.SelectCommand = command;

adapt.Fill( data, tableName );

}

catch( Exception yy )

{

new ErrorLog( yy.ToString() );

return false;

}



RB Tree Node insertion/update chunk:

foreach( DataRow row in data.Tables[ tableName ].Rows )

Tree.Add( row[ "id" ].ToString().Trim(), new CustomData( row[
"id" ].ToString().Trim(), row[ "Reference" ].ToString().Trim() ) );