DataTable.Select(filter) Parameter too long?

  • Thread starter Thread starter Filipe Correia
  • Start date Start date
F

Filipe Correia

Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe
 
I'm not seeing any recursion issues based on the length of elements in the
filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);
 
The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe


Mark said:
I'm not seeing any recursion issues based on the length of elements in the
filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:

Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe
Click to expand...
Click to expand...
 
You really have a 64K parameter??
Click to expand...
yep :)
It's a generated filter string. I have an array of rows from dataset "A"
and I need to select the same rows (the ones with the same primary key)
from dataset "B". So, I generate the filter based on one set of rows and
use it to select() the same rows from the other dataset, which results,
in some extreme cases, in a 64k parameter (or in a 54k one now, please
check my other post).

Filipe
 
You should do all this on the server. You should never pass a parameter like
that...
 
Jeff said:
You should do all this on the server. You should never pass a parameter like
that...
Click to expand...
In this particular situation I'm doing this in memory because I'm
comparing some data on the server with some data on a local dataset. To
do that I' pulling the data I want from the server to a second dataset
and comparing both datasets in memory.

I'm also using long filters in other situations, but that's a long story...

Anyhow, I can't find anything on the docs about the select method having
a limit, so I'm not yet sure if this is a bug or not.


Filipe
 
Below there's some code that reproduces the StackOverflowException I'm
getting. Any thoughts on what the reason might be?

If you use 500 as the value for maxLimit the exception just goes away :S

thanks in advance,
Filipe

----------------------------------------------------------------
DataSet ds = new DataSet("ds");
DataTable dt = ds.Tables.Add("table");
dt.Columns.Add( "ID", typeof(int));
dt.Columns.Add( "IDUpper", typeof(int));

object[] b = new object[2];
string filter = "";
int maxLimit = 1000;

for (int i = 0; i <= maxLimit; i++){
b[0] = i;
b[1] = maxLimit - i;
dt.Rows.Add(b);
if (filter.Length > 0){
filter = filter += " OR ";
}
filter = filter += string.Format("ID='{0}' AND IDUpper='{1}'", i,
maxLimit - i);
}

DataRow[] rows = dt.Select(filter);
----------------------------------------------------------------




Filipe said:
The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe


Mark said:
I'm not seeing any recursion issues based on the length of elements in
the filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:

Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe
Click to expand...
Click to expand...
Click to expand...
 
Yep, shouldn't do that. Not a bug, you're doing it wrong. Let the engine do
what it's designed to do
 
That would be a limitation in the DataSet when evaluating the expression
because each binary operation calling the next on the stack.

Filipe Correia said:
Below there's some code that reproduces the StackOverflowException I'm
getting. Any thoughts on what the reason might be?

If you use 500 as the value for maxLimit the exception just goes away :S

thanks in advance,
Filipe

----------------------------------------------------------------
DataSet ds = new DataSet("ds");
DataTable dt = ds.Tables.Add("table");
dt.Columns.Add( "ID", typeof(int));
dt.Columns.Add( "IDUpper", typeof(int));

object[] b = new object[2];
string filter = "";
int maxLimit = 1000;

for (int i = 0; i <= maxLimit; i++){
b[0] = i;
b[1] = maxLimit - i;
dt.Rows.Add(b);
if (filter.Length > 0){
filter = filter += " OR ";
}
filter = filter += string.Format("ID='{0}' AND IDUpper='{1}'", i,
maxLimit - i);
}

DataRow[] rows = dt.Select(filter);
----------------------------------------------------------------




Filipe said:
The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe


Mark said:
I'm not seeing any recursion issues based on the length of elements in
the filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:


Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe
Click to expand...
Click to expand...
Click to expand...
 
I tried enclosing parts of the filter expression in parentisis in a way
it won't afect it's semantics, and it worked. :)
Not the best of solutions, but it's an efective workaround non the less.

Cheers,
Filipe

Mark said:
That would be a limitation in the DataSet when evaluating the expression
because each binary operation calling the next on the stack.

:

Below there's some code that reproduces the StackOverflowException I'm
getting. Any thoughts on what the reason might be?

If you use 500 as the value for maxLimit the exception just goes away :S

thanks in advance,
Filipe

----------------------------------------------------------------
DataSet ds = new DataSet("ds");
DataTable dt = ds.Tables.Add("table");
dt.Columns.Add( "ID", typeof(int));
dt.Columns.Add( "IDUpper", typeof(int));

object[] b = new object[2];
string filter = "";
int maxLimit = 1000;

for (int i = 0; i <= maxLimit; i++){
b[0] = i;
b[1] = maxLimit - i;
dt.Rows.Add(b);
if (filter.Length > 0){
filter = filter += " OR ";
}
filter = filter += string.Format("ID='{0}' AND IDUpper='{1}'", i,
maxLimit - i);
}

DataRow[] rows = dt.Select(filter);
----------------------------------------------------------------




Filipe said:
The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe


Mark Ashton wrote:


I'm not seeing any recursion issues based on the length of elements in
the filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:



Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe
Click to expand...
Click to expand...
Click to expand...
 
Hack?

Filipe Correia said:
I tried enclosing parts of the filter expression in parentisis in a way it
won't afect it's semantics, and it worked. :)
Not the best of solutions, but it's an efective workaround non the less.

Cheers,
Filipe

Mark said:
That would be a limitation in the DataSet when evaluating the expression
because each binary operation calling the next on the stack.

:

Below there's some code that reproduces the StackOverflowException I'm
getting. Any thoughts on what the reason might be?

If you use 500 as the value for maxLimit the exception just goes away :S

thanks in advance,
Filipe

----------------------------------------------------------------
DataSet ds = new DataSet("ds");
DataTable dt = ds.Tables.Add("table");
dt.Columns.Add( "ID", typeof(int));
dt.Columns.Add( "IDUpper", typeof(int));

object[] b = new object[2];
string filter = "";
int maxLimit = 1000;

for (int i = 0; i <= maxLimit; i++){
b[0] = i;
b[1] = maxLimit - i;
dt.Rows.Add(b);
if (filter.Length > 0){
filter = filter += " OR ";
}
filter = filter += string.Format("ID='{0}' AND IDUpper='{1}'", i,
maxLimit - i);
}

DataRow[] rows = dt.Select(filter);
----------------------------------------------------------------




Filipe Correia wrote:

The filter expression I'm using is something like:

"Field1 = '1' AND Field2 = '5' OR Field1 = '2' AND Field2 = '6' OR
Field1 = '3' AND Field2 = '7' ..."

Haven't tried with a filter expression like the one you used but I've
removed all the spaces on both sides of the equal signs in my filter,
having reduced it to 57286 chars long. Still no luck though.

I'm begining to think it might have something to do with having too many
ORs and ANDs in the expression, rather than having to do with the
expression's length.
I'm going to try testing with less conditions in the expression and see
how it goes.

thanks,
Filipe


Mark Ashton wrote:


I'm not seeing any recursion issues based on the length of elements in
the filterExpression. Here is what I tried.

DataTable table = new DataTable();
table.Columns.Add("c1", typeof(string));
table.Rows.Add(new object[] { "hello world" });

DataRow[] rows = table.Select(string.Format("c1='{0}'", new
string('a', ((int)UInt16.MaxValue)+20)));
Console.WriteLine("{0}", rows.Length);


:



Hi all,

I've found something that I think might be a bug in the DataTable
implementation.

I'm getting a StackOverflowException when executing the following
method, using a string parameter 64254 characters long:

DataTable.Select(string filterExpression)

Does anyone confirm this is a bug in DataTable.Select()?
Are there any known workarounds?

thanks in advance,
Filipe
Click to expand...
Click to expand...
Click to expand...
 
Back
Top