Report progess during black box process

[Bill]

I am using a dataview to sort a table that can have a million or so
rows. I am reporting to the user the progress while rows are added,
but when the code executes the Sort method there is no progress
reported until the sort finishes, which can take more than a minute
and the Task Manager indicates the application is not responding. I
dont want users to think it has hung. I was thinking about using a
timer to periodically display something, but the process is being
executed in a backgroundworker. Any suggestions?


Bill

 

[Mr. Arnold]

I am using a dataview to sort a table that can have a million or so
rows. I am reporting to the user the progress while rows are added,
but when the code executes the Sort method there is no progress
reported until the sort finishes, which can take more than a minute
and the Task Manager indicates the application is not responding. I
dont want users to think it has hung. I was thinking about using a
timer to periodically display something, but the process is being
executed in a backgroundworker. Any suggestions?

Flash a message "Please wait process may take a long time.....", set the
mouse pointer to an "hour glass" while processing, and set the mouse pointer
back to normal when the process is completed.


__________ Information from ESET NOD32 Antivirus, version of virus signature database 4061 (20090507) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

 

[Pavel Minaev]

I am using a dataview to sort  a table that can have a million or so
rows. I am reporting to the user the progress while rows are added,
but when the code executes the Sort method there is no progress
reported until the sort finishes, which can take more than a minute
and the Task Manager indicates the application is not responding. I
dont want users to think it has hung.  I was thinking about using a
timer to periodically display something, but the process is being
executed in a backgroundworker. Any suggestions?

If you read the documentation on BackgroundWorker (see ReportProgress
method and ProgressChanged event), you'll see that it specifically
includes means to marshal progress information from worker thread to
the UI thread, presumably so that the latter can display it in some
way.

Also, if you actually perform your sort entirely in BackgroundWorker,
then your application should not be reported as "not responding". If
it does that, then you aren't telling the whole story here. Or are you
doing the sort in BackgroundWorker, but block the UI thread waiting
for it to finish??

 

[Morten Wennevik [C# MVP]]

Hi Bill,

Create the DataSource in a BackGroundWorker and make sure the DataSource is
sorted before adding it to the DataGridView. Use the
BackgroundWorker.ProgressChanged event (which is thread safe) or Invoke your
own methods when you need to display changes. Remember that lots of
events/invokes are expensive and may drastically slow the whole process down.
Keep a counter and only update if((counter % 1000) == 0) or similar. You
won't be able to report progress during the actual sorting unless you
implement your own sorting algorithm as Peter points out.

If you get the rows from a query, the fastest way would be to get the rows
presorted by doing the sorting as part of the query. If you need to sort it
afterwards sort it in the BackgroundWorker. If you need to use
DataGridView.Sort, implement paging and lower the number of rows displayed.

 

[Bill]

Hi Bill,

Create the DataSource in a BackGroundWorker and make sure the DataSource is
sorted before adding it to the DataGridView. Use the
BackgroundWorker.ProgressChanged event (which is thread safe) or Invoke your
own methods when you need to display changes.  Remember that lots of
events/invokes are expensive and may drastically slow the whole process down.
 Keep a counter and only update if((counter % 1000) == 0) or similar.  You
won't be able to report progress during the actual sorting unless you
implement your own sorting algorithm as Peter points out.

If you get the rows from a query, the fastest way would be to get the rows
presorted by doing the sorting as part of the query.  If you need to sort it
afterwards sort it in the BackgroundWorker.  If you need to use
DataGridView.Sort, implement paging and lower the number of rows displayed.

--
Happy Coding!
Morten Wennevik [C# MVP]

I am using a dataview to sort  a table that can have a million or so
rows. I am reporting to the user the progress while rows are added,
but when the code executes the Sort method there is no progress
reported until the sort finishes, which can take more than a minute
and the Task Manager indicates the application is not responding. I
dont want users to think it has hung.  I was thinking about using a
timer to periodically display something, but the process is being
executed in a backgroundworker. Any suggestions?

Bill- Hide quoted text -

- Show quoted text -

I should have been clearer about the problem. I do not have a
datasource. the code adds rows to a datatable and I am using the
dataview to sort it. The problem is that once the dataview sort method
is invoked, it is essentiall a "black box" as I have no way of
knowing how long it will take or any way of reporting percentage
progress. I already am using the backgroundworker reportprogress
method and I change the counter to a "Sorting..." message, but the
sort takes a while and the task manager sometimes indicates the
application is not responding. One of the replies suggested faking it
and periodically displaying some kind of update message using the
reportprogress, and that appears to be the best strategy.

 

[Pavel Minaev]

I should have been clearer about the problem. I do not have a
datasource. the code adds rows to a datatable and I am using the
dataview to sort it. The problem is that once the dataview sort method
is invoked, it is essentiall a "black box" as  I have no way of
knowing how long it will take or any way of reporting percentage
progress. I already am using the backgroundworker reportprogress
method and I change the counter to a "Sorting..." message, but the
sort takes a while and the task manager sometimes indicates the
application is not responding.

This is still extremely strange. If your call to Sort is in the
BackgroundWorker worker thread, then you shouldn't get "application
not responding".

Otherwise, yes, the best you can do is one of those fake progress
indicators which just loop infinitely. Like Vista's busy mouse cursor.

 

[Morten Wennevik [C# MVP]]

Hi Bill,

Create the DataSource in a BackGroundWorker and make sure the DataSource is
sorted before adding it to the DataGridView. Use the
BackgroundWorker.ProgressChanged event (which is thread safe) or Invoke your
own methods when you need to display changes. Remember that lots of
events/invokes are expensive and may drastically slow the whole process down.
Keep a counter and only update if((counter % 1000) == 0) or similar. You
won't be able to report progress during the actual sorting unless you
implement your own sorting algorithm as Peter points out.

If you get the rows from a query, the fastest way would be to get the rows
presorted by doing the sorting as part of the query. If you need to sort it
afterwards sort it in the BackgroundWorker. If you need to use
DataGridView.Sort, implement paging and lower the number of rows displayed.

--
Happy Coding!
Morten Wennevik [C# MVP]

I am using a dataview to sort a table that can have a million or so
rows. I am reporting to the user the progress while rows are added,
but when the code executes the Sort method there is no progress
reported until the sort finishes, which can take more than a minute
and the Task Manager indicates the application is not responding. I
dont want users to think it has hung. I was thinking about using a
timer to periodically display something, but the process is being
executed in a backgroundworker. Any suggestions?

Bill- Hide quoted text -

- Show quoted text -

I should have been clearer about the problem. I do not have a
datasource. the code adds rows to a datatable and I am using the
dataview to sort it. The problem is that once the dataview sort method
is invoked, it is essentiall a "black box" as I have no way of
knowing how long it will take or any way of reporting percentage
progress. I already am using the backgroundworker reportprogress
method and I change the counter to a "Sorting..." message, but the
sort takes a while and the task manager sometimes indicates the
application is not responding. One of the replies suggested faking it
and periodically displaying some kind of update message using the
reportprogress, and that appears to be the best strategy.

Can't you add the DataRows presorted? If you sort the DataTable before
attaching it to a control, while still in the BackgroundWorker you should
still get a responsive client, yet you do not know the progress of the
sorting.

I suggest you create a business object representing the DataRow. Create a
list of these objects in memory and do the sorting against this list, and
then use this list as a DataSource in the DataGridView, or even better, use a
subset from this list. I still question the need for showing million items
in a list.

 

[Morten Wennevik [C# MVP]]

Hi Bill,

Create the DataSource in a BackGroundWorker and make sure the DataSource is
sorted before adding it to the DataGridView. Use the
BackgroundWorker.ProgressChanged event (which is thread safe) or Invoke your
own methods when you need to display changes. Remember that lots of
events/invokes are expensive and may drastically slow the whole process down.
Keep a counter and only update if((counter % 1000) == 0) or similar. You
won't be able to report progress during the actual sorting unless you
implement your own sorting algorithm as Peter points out.

If you get the rows from a query, the fastest way would be to get the rows
presorted by doing the sorting as part of the query. If you need to sort it
afterwards sort it in the BackgroundWorker. If you need to use
DataGridView.Sort, implement paging and lower the number of rows displayed.

--
Happy Coding!
Morten Wennevik [C# MVP]

I am using a dataview to sort a table that can have a million or so
rows. I am reporting to the user the progress while rows are added,
but when the code executes the Sort method there is no progress
reported until the sort finishes, which can take more than a minute
and the Task Manager indicates the application is not responding. I
dont want users to think it has hung. I was thinking about using a
timer to periodically display something, but the process is being
executed in a backgroundworker. Any suggestions?

Bill- Hide quoted text -

- Show quoted text -

I should have been clearer about the problem. I do not have a
datasource. the code adds rows to a datatable and I am using the
dataview to sort it. The problem is that once the dataview sort method
is invoked, it is essentiall a "black box" as I have no way of
knowing how long it will take or any way of reporting percentage
progress. I already am using the backgroundworker reportprogress
method and I change the counter to a "Sorting..." message, but the
sort takes a while and the task manager sometimes indicates the
application is not responding. One of the replies suggested faking it
and periodically displaying some kind of update message using the
reportprogress, and that appears to be the best strategy.

Can't you add the DataRows presorted? If you sort the DataTable before
attaching it to a control, while still in the BackgroundWorker you should
still get a responsive client, yet you do not know the progress of the
sorting.

I suggest you create a business object representing the DataRow. Create a
list of these objects in memory and do the sorting against this list, and
then use this list as a DataSource in the DataGridView, or even better, use a
subset from this list. I still question the need for showing million items
in a list.