Dataset, large, performance, solution

[Guest]

Dear reader

I'm building a multi-user application in .NET and was wondering what benefits the dataset has (for my application).
(1). Database contains tables with 10.000 - 20.000 records. Do I have to use a dataset to fetch all the records from the database (and load it into the dataset (memory on client machine))? Isn't this memory consuming? I can't use the DataReader because the user can modify the data

(2). When the user for examples navigates to the next record the application needs to refresh the data (in case an other user changed the next record). This means that it has to refresh all the 10.000 - 20.000 records when the user navigates
Isn't this generating a lot of network traffic and performance?

Or does someone have a better idea

Thx
Mauric

 

[Miha Markic [MVP C#]]

Hi Maurice,

You might consider paging or providing user a filter before fetching data.
Fetching large number of records is never a good idea.
You can also refresh a single record, if you need to (just construct a
select that returns the record and use Fill method).

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

Dear reader,

I'm building a multi-user application in .NET and was wondering what

benefits the dataset has (for my application).

(1). Database contains tables with 10.000 - 20.000 records. Do I have to

use a dataset to fetch all the records from the database (and load it into
the dataset (memory on client machine))? Isn't this memory consuming? I
can't use the DataReader because the user can modify the data.

(2). When the user for examples navigates to the next record the

application needs to refresh the data (in case an other user changed the
next record). This means that it has to refresh all the 10.000 - 20.000
records when the user navigates.

 

[Thomas Tomiczek [MVP]]

Inline...

--
Regards

Thomas Tomiczek
THONA Software & Consulting Ltd.
(Microsoft MVP C#/.NET)
(CTO PowerNodes Ltd.)
---

Still waiting for ObjectSpaces? Try the EntityBroker today - more versatile,
more powerfull.
And something in use NOW. for the projects you have to deliver - NOW.

Dear reader,

I'm building a multi-user application in .NET and was wondering what

benefits the dataset has (for my application).

(1). Database contains tables with 10.000 - 20.000 records. Do I have to

use a dataset to fetch all the records from the database (and load it into
the dataset (memory on client machine))? Isn't this memory consuming? I
can't use the DataReader because the user can modify the data.

Both approaches are nuts.

Ask for the data you need. ONLY. WHEN you need it.

(2). When the user for examples navigates to the next record the

application needs to refresh the data (in case an other user changed the
next record). This means that it has to refresh all the 10.000 - 20.000
records when the user navigates.

Isn't this generating a lot of network traffic and performance?

Sure.

The approach is a beginner approach. Ask for the data you need. ONLY. WHEN
you need it.

Thomas

 

[Tom Krueger]

Hello,

Yes you are correct, passing all the records in the database would consume
more resources and most likely would not added any benefit to the user.
That is unless your users need all that data.

The best practice to follow is only return the data that is necessary by
filtering in the SQL statements. If you get into a scenario that you need
to be able to scroll through the entire database, you should look into
Paging.

Refreshing the dataset every time the user moves to the next record may be a
little excessive in a WinForm app depending on your need. Doing this would
add more load to the server and network. In Web apps, it is necessary to
hit the database more often, however, in this case hopefully you are working
with datareaders or lighter business objects since everything is created and
destroyed for each request. If refreshing is necessary, you could add a
refresh button so the user has the option to refresh when they need to.

There is always a tradeoff to every architecture question, it's just a
matter of finding the decisions that work best for the majority.

Take care,

Tom Krueger
Blue Pen Solutions
http://www.BluePenSolutions.com

Dear reader,

I'm building a multi-user application in .NET and was wondering what

benefits the dataset has (for my application).

(1). Database contains tables with 10.000 - 20.000 records. Do I have to

use a dataset to fetch all the records from the database (and load it into
the dataset (memory on client machine))? Isn't this memory consuming? I
can't use the DataReader because the user can modify the data.

(2). When the user for examples navigates to the next record the

application needs to refresh the data (in case an other user changed the
next record). This means that it has to refresh all the 10.000 - 20.000
records when the user navigates.

 

[Guest]

To all,

thanks for the answers. Paging would be a solution but I thought when using paging the dataadapter still fetches all the 10.000-20.000 records but only passes a few to the dataset.

 

[Miha Markic [MVP C#]]

Hi,

Yes, you are rigtht.
That's why you should create paging condition within sql select command.

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

To all,

thanks for the answers. Paging would be a solution but I thought when

using paging the dataadapter still fetches all the 10.000-20.000 records but
only passes a few to the dataset.

 

[Miha Markic [MVP C#]]

And it doesn't fetches all records. It fetches just all records preceding
the page...

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

To all,

thanks for the answers. Paging would be a solution but I thought when

using paging the dataadapter still fetches all the 10.000-20.000 records but
only passes a few to the dataset.

 

[Nunya]

Thomas, Tom, & Miha are quite right -- minimize the data you're passing back
and forth.
You can still use datareaders if users need to modify data. Use
insert/update/delete statements; you don't have to everything in the
dataset.

You didn't mention what db you're using. If Sql Server, you might consider
using stored procedures for data access and manipulation. This allows the
server use existing query plans, possibly improving performance. It would
probably also shorten the queries you're passing to the server. Stored
procedures also allow you to use MSSQL security to restrict direct table
access.

Dear reader,

I'm building a multi-user application in .NET and was wondering what

benefits the dataset has (for my application).

(1). Database contains tables with 10.000 - 20.000 records. Do I have to

use a dataset to fetch all the records from the database (and load it into
the dataset (memory on client machine))? Isn't this memory consuming? I
can't use the DataReader because the user can modify the data.

(2). When the user for examples navigates to the next record the

application needs to refresh the data (in case an other user changed the
next record). This means that it has to refresh all the 10.000 - 20.000
records when the user navigates.

 

[Benny Tordrup]

Which in case of the last page will be all records

Benny

And it doesn't fetches all records. It fetches just all records preceding
the page...

--
Miha Markic [MVP C#] - RightHand .NET consulting & software development
miha at rthand com
www.rthand.com

To all,

thanks for the answers. Paging would be a solution but I thought when

using paging the dataadapter still fetches all the 10.000-20.000 records but
only passes a few to the dataset.