A
Adrian Roman
Hi,
Here is my problem very short:
Loading an oledb table data takes a fair amount of time, if there is only
one thread that loads a table with data. If there are two of them, each
loading its own table, loading takes unacceptable time.
Now very long:
I've written an oledb supplier (ATL) for a proprietary data source (it's not
a database, it a server that exposes data on the network using an xml
protocol).
So far so good. I tested the oledb supplier with various methods, including
a C++ program (not .net, mfc and ATL for oledb) that launched 10 threads to
query the same data. It worked without problems.
Now my job is to make a client program that does something with the exposed
data. Unfortunately I have to use C# to do that, and of course, .net.
Since the program should get data from more than one host from the network,
I thought that querying each of them in a separate thread (each loading its
own table, there shouldn't be multithreading issues, each thread deals with
its own data) would be a good idea. It turned out to be a bad idea. If there
is only one thread that queries a host, it works ok. If there are two
threads, loading data takes orders of magnitude more time (100 times slower,
I didn't actually measured, it's only a guess).
I used first OleDbDataAdaptor to load data into a DataTable. Seeing how the
program performs, I changed the method and used OleDbDataReader instead. I
even tried to avoid Load and load the table with my own code, walking over
records, too see what's actually happening. What I noticed is that the
GetValue method takes a lot of time, so it seems that the actual data
retrieval takes so much time. I mean retrieval from the oledb, because the
data it's already in the recordset. Just passing it between the oledb
supplier and the .net program takes so much time.
To be sure I modified the oledb supplier to log the times of the query
execution. Well, the problem it's not in there, of course. Execution takes
usually around one second for several hundreds records (that test was done
while the .net executed to queries in to threads). It takes minutes to get
the data into the .net program.
Environment: Windows 2000. Visual Studio 2005, C# for the client program,
..net 2.0.50727.
The oledb supplier is written in C++ with ATL (the one that comes with
Visual Studio 2005).
If there is anybody that knows about such a problem, please help me!
Adrian Roman
Here is my problem very short:
Loading an oledb table data takes a fair amount of time, if there is only
one thread that loads a table with data. If there are two of them, each
loading its own table, loading takes unacceptable time.
Now very long:
I've written an oledb supplier (ATL) for a proprietary data source (it's not
a database, it a server that exposes data on the network using an xml
protocol).
So far so good. I tested the oledb supplier with various methods, including
a C++ program (not .net, mfc and ATL for oledb) that launched 10 threads to
query the same data. It worked without problems.
Now my job is to make a client program that does something with the exposed
data. Unfortunately I have to use C# to do that, and of course, .net.
Since the program should get data from more than one host from the network,
I thought that querying each of them in a separate thread (each loading its
own table, there shouldn't be multithreading issues, each thread deals with
its own data) would be a good idea. It turned out to be a bad idea. If there
is only one thread that queries a host, it works ok. If there are two
threads, loading data takes orders of magnitude more time (100 times slower,
I didn't actually measured, it's only a guess).
I used first OleDbDataAdaptor to load data into a DataTable. Seeing how the
program performs, I changed the method and used OleDbDataReader instead. I
even tried to avoid Load and load the table with my own code, walking over
records, too see what's actually happening. What I noticed is that the
GetValue method takes a lot of time, so it seems that the actual data
retrieval takes so much time. I mean retrieval from the oledb, because the
data it's already in the recordset. Just passing it between the oledb
supplier and the .net program takes so much time.
To be sure I modified the oledb supplier to log the times of the query
execution. Well, the problem it's not in there, of course. Execution takes
usually around one second for several hundreds records (that test was done
while the .net executed to queries in to threads). It takes minutes to get
the data into the .net program.
Environment: Windows 2000. Visual Studio 2005, C# for the client program,
..net 2.0.50727.
The oledb supplier is written in C++ with ATL (the one that comes with
Visual Studio 2005).
If there is anybody that knows about such a problem, please help me!
Adrian Roman