DataTable.Select method bug in .Net Framework 1.1 SP1

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

Since Microsoft removed the original thread from this newsgroup... here it is
again.

Just to add some comments: we are MSDN Universal Subscribers and didn't get
support from Microsoft. We are being ignored for almost 5 months. We have
tested Microsoft .Net Framework 2.0 Beta 2 and the problem is still there.

This is worst than a virus and Microsoft does nothing.

By the way: did you know that Windows 2003 SP1 installs .Net Framework SP1?
You can't find this information in any of Microsoft's documents about the SP1
of Windows 2003 but it does...

Has anyone from Microsoft the decency to answer this post and to solve this
problem?

Here is one of the original posts (25/02/2005):

There is a bug in .Net Framework 1.1 SP1 with the DataTable.Select method.

Microsoft's description of the problem may be found here

http://support.microsoft.com/default.aspx?scid=kb;en-us;891742

and you can ask for a fix to that issue. The problem is that the fix does
not work properly!

Some problems were solved with this fix but some persist. And the problem
relates to the overload of DataTable.Select with a filter and a sort
expression: in that case the result of the method returns rows that do not
meet the filter conditions.

We have a detailed description of this problem at :

http://pwp.netcabo.pt/0434926301/

and have included a new test harness (source code included) to test the
problems we are experiencing.

As we can't tell for sure what gets solved and what doesn't with this fix we
decided to continue limiting the support for our applications for clients not
running the SP1 of the .Net Framework 1.1.

The purpose of this thread is to notify microsoft for the problem so that an
effective fix is released and to alert the developer community for this very
harmful bug.

The first thread I know of reporting this problem dates from 9/9/2004 and
Microsoft still hasn't fixed the problem...
 
Lynch,

Please, don't give the impression if the thread is deleted because of the
subject

http://groups-beta.google.com/group...da136edf5dcea?q=lynch&rnum=1#859da136edf5dcea

I see more times in this thread that David Sceppa has tried to help you. I
can assure you that he is from Microsoft.

Luckily enough have these newsgroups using newsreaders again a retention
time from 2 months. If it was longer (as it has been the last months) than
some newsgroups including probably this one are impossible to reach for dial
up users with newsreaders.

If you want archieve information than use Google (or probably the Webversion
what I don't know).

Cor
 
lynch,

You might be interested in the assembly I've been working on at
http://www.queryadataset.com. It lets you perform complex SQL SELECT
statements including UNION, JOINS, GROUP BY, HAVING, ORDER BY, sub-queries,
etc against the tables in a dataset.

The web-site allows you to upload your own XML data fragment, DataSet or
resultset and issue queries using the QueryADataSet assembly. You can
select the XML document included in your test harness (User Uploaded -
ApplicationData.XML) and try the following query:

select * from [ENT_H_EMPLOYEE] where EntityId = 1 AND Deleted = 0 AND
FieldId = 70 AND HistoryStartDate >= '02/16/2005' order by entityid

The result, as you expect, should be 0 rows and 1 row if the date condition
is removed. While the downloadable demo does not generate the correct
results to this query, due to a bug, the web-site clearly demonstrates this
query working. If you are interested, I can provide an updated .NET 1.1
version of the assembly for you to download and try.

Hope this helps
Adrian Moore
http://www.queryadataset.com
 
Thank you Adrian.

I would be very much interested in the updated .Net 1.1 version of the
assembly you've been working on. It would be a great help.

Since there are no answers from Microsoft concerning to this issue, we have
been considering writing our own filtering classes. If you could provide a
working solution for this problem we most certainly would consider using it.

PS - if you want to contact me directly please do at:

(e-mail address removed)

Thank you very much.

Adrian Moore said:
lynch,

You might be interested in the assembly I've been working on at
http://www.queryadataset.com. It lets you perform complex SQL SELECT
statements including UNION, JOINS, GROUP BY, HAVING, ORDER BY, sub-queries,
etc against the tables in a dataset.

The web-site allows you to upload your own XML data fragment, DataSet or
resultset and issue queries using the QueryADataSet assembly. You can
select the XML document included in your test harness (User Uploaded -
ApplicationData.XML) and try the following query:

select * from [ENT_H_EMPLOYEE] where EntityId = 1 AND Deleted = 0 AND
FieldId = 70 AND HistoryStartDate >= '02/16/2005' order by entityid

The result, as you expect, should be 0 rows and 1 row if the date condition
is removed. While the downloadable demo does not generate the correct
results to this query, due to a bug, the web-site clearly demonstrates this
query working. If you are interested, I can provide an updated .NET 1.1
version of the assembly for you to download and try.

Hope this helps
Adrian Moore
http://www.queryadataset.com


lynch said:
Since Microsoft removed the original thread from this newsgroup... here it
is
again.

Just to add some comments: we are MSDN Universal Subscribers and didn't
get
support from Microsoft. We are being ignored for almost 5 months. We have
tested Microsoft .Net Framework 2.0 Beta 2 and the problem is still there.

This is worst than a virus and Microsoft does nothing.

By the way: did you know that Windows 2003 SP1 installs .Net Framework
SP1?
You can't find this information in any of Microsoft's documents about the
SP1
of Windows 2003 but it does...

Has anyone from Microsoft the decency to answer this post and to solve
this
problem?

Here is one of the original posts (25/02/2005):

There is a bug in .Net Framework 1.1 SP1 with the DataTable.Select method.

Microsoft's description of the problem may be found here

http://support.microsoft.com/default.aspx?scid=kb;en-us;891742

and you can ask for a fix to that issue. The problem is that the fix does
not work properly!

Some problems were solved with this fix but some persist. And the problem
relates to the overload of DataTable.Select with a filter and a sort
expression: in that case the result of the method returns rows that do not
meet the filter conditions.

We have a detailed description of this problem at :

http://pwp.netcabo.pt/0434926301/

and have included a new test harness (source code included) to test the
problems we are experiencing.

As we can't tell for sure what gets solved and what doesn't with this fix
we
decided to continue limiting the support for our applications for clients
not
running the SP1 of the .Net Framework 1.1.

The purpose of this thread is to notify microsoft for the problem so that
an
effective fix is released and to alert the developer community for this
very
harmful bug.

The first thread I know of reporting this problem dates from 9/9/2004 and
Microsoft still hasn't fixed the problem...
Click to expand...
Click to expand...
 
Hi.

David Sceppa did try to help us solve the problem.

And Microsoft, as I stated, released an Hotfix that should correct this
problem.
But it doesn't work!

I would like to remind you that this is not Open Source.
All companies that develop for Microsoft .Net Framework are helping
Microsoft sell it's products and are investing a great deal of money on that
development. It is only fair if not mandatory for Microsoft to fix the
problems it creates.

This has become a problem since .Net Framework 1.1 SP1 and now for .Net
Framework 2.0 Beta 2.

This issue has been around since September 2004!!! It is time it get's
fixed. Don't you think so?

Thanks

Lynch
 
Lynch, I'd love to help you out, as I am sure others will too. Unfortunately
I donot have enough hardware to support this issue. :-( . I.e. I donot have
a machine with .NET 1.1 and another with .NET 1.1 and SP1.

BTW, before you point out, Microsoft does not pay me zilch for anything,
neither they control what I say.

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync
 
Sahil,

can't you use Microsoft Virtual PC for testing?
I'm also being affected by this issue. SP1 introduces this bug in the Select
method and I can't imagine myself writing a new Select method from
scratch(!). Any and all help in solving this matter for good would be
tremendously appreciated!

André
 
Hi Sahil:

Thanks for your interest.

It's always the same: those that want to help never get anything in return. ;)

Still if any of you could lend me a hand, I would appreciate it.

Thanks

PS - Still hoping Microsoft solves this one.
 
Lynch

I don't like the impression you gave. I don't like it when somebody is
pointing his finger while there is so much help given.

I tried to do it with this code in which is your first message. And I saw
nothing wrong. Maybe can you try it as well and than also simulate until it
goes wrong?

Or maybe I oversee something than I will be glad to hear what?

\\\\Needs only a datagrid and a form

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles MyBase.Load
Dim dt As New DataTable
dt.Columns.Add("EntityID", GetType(System.Int32))
dt.Columns.Add("Deleted", GetType(System.Int32))
dt.Columns.Add("FieldId", GetType(System.Int32))
dt.Columns.Add("HistoryStartDate", GetType(System.DateTime))
dt.LoadDataRow(New Object() {7016, 0, 70, New DateTime(2004, 7, 1)},
True)
dt.LoadDataRow(New Object() {7016, 1, 70, New DateTime(2004, 7, 1)},
True)
dt.LoadDataRow(New Object() {7016, 1, 71, New DateTime(2004, 7, 1)},
True)
dt.LoadDataRow(New Object() {7015, 0, 70, New DateTime(2004, 7, 1)},
True)
dt.LoadDataRow(New Object() {7016, 0, 70, New DateTime(2003, 7, 1)},
True)
dt.LoadDataRow(New Object() {7016, 0, 70, New DateTime(2005, 7, 1)},
True)
Dim dr() As DataRow = _
dt.Select("EntityId = 7016 AND Deleted = 0 AND FieldId = 70 AND
HistoryStartDate >= #07/01/2004 00:00:00#")
Dim dt2 As New DataTable
dt2 = dt.Clone
For Each dr1 As DataRow In dr
dt2.ImportRow(dr1)
Next
DataGrid1.DataSource = dt2
End Sub
///

I hope this helps,

Cor
 
Hi Cor.

Thanks for your help.

About the bug:

The test you are running works well. As I have said before, the hotfix from
Microsoft solved that problem. But some problems remain as I state in
http://pwp.netcabo.pt/0434926301/. For the purpose os reproducing that
problem we made available a test harness with source code. Just read the text
try the TH with and without .Net Framework 1.1 SP1 and you will reproduce the
problem.

All the help from the comunity members is very much welcome and very much
appreciated. As you just did and as I often do. I thank you for the time you
are taking to look at this problem, helping me and the entire .net comunity.

If you have more ideas they're very welcome.

About Microsoft:

But Microsoft MUST solve the problem. It is their obligation to solve the
problem or just stop supporting that feature.

The last post from Microsoft concerning this issue is from Amir Massourian
stating:
"Thanks Lynch for reporting, we will investigate this issue" (26/02/2005).
As good as it might be to have someone aknowledge the problem do you think
this is a realy helpfull post since it has already been 4 months since that
reply? What is Microsoft doing about that? Plain ignoring? Shouldn't the
developers comunity ask Microsoft to solve the problems instead of trying
again and again to find a workaround for Microsoft's bugs?

Thanks
 
My Home PC is weird, it doesn't run Virtual PC. I know I need to buy a new
computer, but it takes a lot of research to find something I will be married
with for the next 3-4 years, and right now in the finishing touches of my
book, I just don't have the time.

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync
 
Lynch,

What we do get back in return is good Karma.

Anyway, solving this issue programmatically shouldn't be very hard, and it's
not like workarounds donot exist.

--

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync
 
Hi Sahil.

Yes there is a workaround! And do you know something: my company payed me
money to find the workaroud you are talking about! And Microsoft still didn't
solve the problem...
If we all thought the way you do some time from now there will be no
workaround for the framework. It would all just be a huge bug!
I want Microsoft to solve this one. They made us loose money so thats the
least they can do, for now any way...

PS - By the way, did you ever trie to feed your daughter on karma?????

If you want me to I will submit the workaround(its realy simple, by the way).

Lynch
 
Lynch,

Karma does go a long way. That is a long tangential discussion which is hard
to explain, but easy to believe in.
Yes I think you should post a solution if you know one.

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync
 
Fantastic .. Thank you !!!
:)


--

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync
----------------------------------------------------------------------------

lynch said:
Hi Sahil,

We are using the DataView methods instead of the DataTable.Select.
We made a method that returns the same result as the DataTable.Select so
that we have less impact on the rest of the code:

public static DataRow[] Select(DataTable table, string filter, string
sort)
{
if(table == null)
return null;

DataView filteredData = new DataView(table);

try
{
if(sort != null && sort != String.Empty)
filteredData.Sort = sort;

if(filter != null && filter != String.Empty)
filteredData.RowFilter = filter;

DataRow[] resultSet = new DataRow[filteredData.Count];

int i=0;

foreach(DataRowView rowView in filteredData)
{
resultSet = rowView.Row;
i++;
}

return resultSet;
}
finally
{
filteredData.Dispose();
}
}





Sahil Malik said:
Lynch,

Karma does go a long way. That is a long tangential discussion which is
hard
to explain, but easy to believe in.
Yes I think you should post a solution if you know one.

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync
Click to expand...
Click to expand...
 
I must say, although it works, the performance of this workaround is
excruciatingly bad.
In some tests I did, the time for executing a Select via a DataView on a
DataTable obtained through a database query (table with indexes, etc.) was up
15 times the time for the original DataTable.Select method.
In my particular case, this is not an option. I can't see anyone willing to
wait 3 minutes for a particular query (with some processing, correct).
When can we expect a *working* fix for all the problems related to the
DataTable.Select method?

André

Sahil Malik said:
Fantastic .. Thank you !!!
:)


--

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync
----------------------------------------------------------------------------

lynch said:
Hi Sahil,

We are using the DataView methods instead of the DataTable.Select.
We made a method that returns the same result as the DataTable.Select so
that we have less impact on the rest of the code:

public static DataRow[] Select(DataTable table, string filter, string
sort)
{
if(table == null)
return null;

DataView filteredData = new DataView(table);

try
{
if(sort != null && sort != String.Empty)
filteredData.Sort = sort;

if(filter != null && filter != String.Empty)
filteredData.RowFilter = filter;

DataRow[] resultSet = new DataRow[filteredData.Count];

int i=0;

foreach(DataRowView rowView in filteredData)
{
resultSet = rowView.Row;
i++;
}

return resultSet;
}
finally
{
filteredData.Dispose();
}
}





Sahil Malik said:
Lynch,

Karma does go a long way. That is a long tangential discussion which is
hard
to explain, but easy to believe in.
Yes I think you should post a solution if you know one.

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync
----------------------------------------------------------------------------



Hi Sahil.

Yes there is a workaround! And do you know something: my company payed
me
money to find the workaroud you are talking about! And Microsoft still
didn't
solve the problem...
If we all thought the way you do some time from now there will be no
workaround for the framework. It would all just be a huge bug!
I want Microsoft to solve this one. They made us loose money so thats
the
least they can do, for now any way...

PS - By the way, did you ever trie to feed your daughter on karma?????

If you want me to I will submit the workaround(its realy simple, by the
way).

Lynch



:

Lynch,

What we do get back in return is good Karma.

Anyway, solving this issue programmatically shouldn't be very hard,
and
it's
not like workarounds donot exist.

--

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync
----------------------------------------------------------------------------
---------------

Hi Sahil:

Thanks for your interest.

It's always the same: those that want to help never get anything in
return. ;)

Still if any of you could lend me a hand, I would appreciate it.

Thanks

PS - Still hoping Microsoft solves this one.

:

Lynch, I'd love to help you out, as I am sure others will too.
Unfortunately
I donot have enough hardware to support this issue. :-( . I.e. I
donot
have
a machine with .NET 1.1 and another with .NET 1.1 and SP1.

BTW, before you point out, Microsoft does not pay me zilch for
anything,
neither they control what I say.

- Sahil Malik [MVP]
Upcoming ADO.NET 2.0 book - http://tinyurl.com/9bync

--------------------------------------------------------------------------
--



Since Microsoft removed the original thread from this
newsgroup...
here it
is
again.

Just to add some comments: we are MSDN Universal Subscribers and
didn't
get
support from Microsoft. We are being ignored for almost 5
months.
We
have
tested Microsoft .Net Framework 2.0 Beta 2 and the problem is
still
there.

This is worst than a virus and Microsoft does nothing.

By the way: did you know that Windows 2003 SP1 installs .Net
Framework
SP1?
You can't find this information in any of Microsoft's documents
about
the
SP1
of Windows 2003 but it does...

Has anyone from Microsoft the decency to answer this post and to
solve
this
problem?

Here is one of the original posts (25/02/2005):

There is a bug in .Net Framework 1.1 SP1 with the
DataTable.Select
method.

Microsoft's description of the problem may be found here

http://support.microsoft.com/default.aspx?scid=kb;en-us;891742

and you can ask for a fix to that issue. The problem is that the
fix
does
not work properly!

Some problems were solved with this fix but some persist. And
the
problem
relates to the overload of DataTable.Select with a filter and a
sort
expression: in that case the result of the method returns rows
that
do
not
meet the filter conditions.

We have a detailed description of this problem at :

http://pwp.netcabo.pt/0434926301/

and have included a new test harness (source code included) to
test
the
problems we are experiencing.

As we can't tell for sure what gets solved and what doesn't with
this
fix
we
decided to continue limiting the support for our applications
for
clients
not
running the SP1 of the .Net Framework 1.1.

The purpose of this thread is to notify microsoft for the
problem
so
that
an
effective fix is released and to alert the developer community
for
this
very
harmful bug.

The first thread I know of reporting this problem dates from
9/9/2004
and
Microsoft still hasn't fixed the problem...
Click to expand...
Click to expand...
Click to expand...
 
Yeah no kidding. No wounder no one answered my postes with a real answer to
my issue. Because they choose the political beauracratic way which is to not
answer or provide a work around that is either expensive on the hardware or
requires 200 lines of code to do one small thing.

:/
 
We are aware of the DataTable.Select bug in v1.1 SP1 and fix for it has been
release as a hotfix. Please refer to the KB article mentioned below for more
information on the bug and the associated hotfix.
http://support.microsoft.com/?id=900468.


I noticed in this thread the use of DataView to query the underlying
DataTable. This can have performance implications if used incorrectly, here
is some background that might help in making the choice between DataView Vs
DataTable.Select.

Background:

DataTable.Select tries to use an existing index (i.e. internal index) on the
columns specified in the Select expression or creates a new one if no
matching index is found. The index created during the Select operation is
kind of a temporary index which gets discarded/destroyed when any write
operations on the underlying DataTable are performed. So Select queries
issued between 2 write operations may reuse applicable indexes and result in
faster execution time by virtue of reusing pre-existing indexes.

On the other hand when a DataView is created, it too creates an index which
is a kind of permanent index that remains in existence as long as DataView
exists. This index is maintained during right operation and is not
discarded. The index maintenance overhead is much higher in v1.x than v2.0.

DataView.RowFitler Vs DataTable.Select: If one is not expecting multiple
queries on the same set of columns i.e. it's only a single query then using
DataTable.Select or creating a DataView in terms of performance is kind of
equivalent. DataTable.Select would be marginally faster as it saves on the
time required to create DataRowViews that a DataView would have to create.
Both require an index to be created.

If the query expression is a simple equality based predicate then the best
option is to use DataView.FindRows which reuses the same DataView an index
to answer the queries.


Thanks,

Kawarjit Bedi [MSFT]
Program Manager - ADO.NET Team
(e-mail address removed)

This posting is provided "AS IS" with no warranties, and confers no rights.
 
Back
Top