populating listview is too slow

  • Thread starter Thread starter Anushya
  • Start date Start date
A

Anushya

Hi
I am populating a listview with more than 5000 items from my outlook
inbox and even the form looks hanged for 10 mins and then listing with
the details. The number of items i need to populate cannot be reduced
in any way. I need the performance to be good even in case of 2 lakhs
items..

Do anybody have any paging idea of how to do this with ListView. I am
working in c#.net?? Or else any other idea. Here i am not using any
database just reading my folders in outlook and have it in memory and
then straight way adding to listview using add item??

Can the Mail item collection be linked straight to the listview. If so
will the performance be good??

Or can i go thru some other way..
Pls let me know immediately if u have any suggestion....

Anushya
 
* (e-mail address removed) (Anushya) scripsit:
I am populating a listview with more than 5000 items from my outlook
inbox and even the form looks hanged for 10 mins and then listing with
the details. The number of items i need to populate cannot be reduced
in any way. I need the performance to be good even in case of 2 lakhs
items..

Do anybody have any paging idea of how to do this with ListView. I am
working in c#.net?? Or else any other idea. Here i am not using any
database just reading my folders in outlook and have it in memory and
then straight way adding to listview using add item??

Maybe a virtual listview (VB.NET implementation):

<http://www.ilypsys-systems.co.uk/ils1controls.zip>
 
When filling make sure you don't have sorting enabled. Set the
ListViewItemSorter = null before filling. After it's done, then you can
reapply the current sort. Also, make sure you're using "AddRange()" to add
all of the items at once.

Terry
 
ListViewItemSorter to null definitely, but AddRange simply calls Add() for
each object... just FYI
 
One reason of slow loading for ListView is that the display of ListView gets
refreshed each time an item is added (although you may not visibly notice
the display is refreshed timely when in lengthy loading process). Since
display refresh is slowest process, hence slow loading. You can set
ListView.Visible to False while adding items to ListView, you will notice
shortened loading time.
 
I was under the impression that the AddRange also calls
BeginUpdate/EndUpdate. If you're adding them one at a time via Add, you
should call Begin/EndUpdate yourself to supress paints until you're done
adding items.

It is the following paragraph from the "BeginUpdate" docs that led me to
that.

"The preferred way to add items to a tree view control is to use the
AddRange method to add an array of tree node items to a tree view. However,
if you want to add items one at a time, use the BeginUpdate method to
prevent the TreeView control from painting during the add operations. To
allow control the control to resume painting, call the EndUpdate method when
all the tree nodes have been added to the tree view."
 
yea, that makes sense, but AddRange doesnt do the Begin/End update like they
should...

public virtual void AddRange(System.Windows.Forms.TreeNode[] nodes) {
TreeNode local0;
System.Windows.Forms.TreeNode[] local1;
int local2;

if (nodes == null)
throw new ArgumentNullException("nodes");
local1 = nodes;
local2 = 0;
while (local2 < (int) local1.Length) {
local0 = local1[local2];
this.Add(local0);
local2++;
}
}

so you'd normally call BeginUpdate() then AddRange() then EndUpdate()
 
Well, that's a bummer! Here I was interpreting their docs to say that it
did.

Can I ask where you got this code snippet from? I'm having a problem with
an OnPaint override in ListView and I'm curious to see how the clipping rect
is being calculated. For some reason the first item (and only the first
item) is being excluded from the clipping region when clicking on different
items in the list. It looks like a bug to me.


Eric Newton said:
yea, that makes sense, but AddRange doesnt do the Begin/End update like they
should...

public virtual void AddRange(System.Windows.Forms.TreeNode[] nodes) {
TreeNode local0;
System.Windows.Forms.TreeNode[] local1;
int local2;

if (nodes == null)
throw new ArgumentNullException("nodes");
local1 = nodes;
local2 = 0;
while (local2 < (int) local1.Length) {
local0 = local1[local2];
this.Add(local0);
local2++;
}
}

so you'd normally call BeginUpdate() then AddRange() then EndUpdate()

Terry said:
I was under the impression that the AddRange also calls
BeginUpdate/EndUpdate. If you're adding them one at a time via Add, you
should call Begin/EndUpdate yourself to supress paints until you're done
adding items.

It is the following paragraph from the "BeginUpdate" docs that led me to
that.

"The preferred way to add items to a tree view control is to use the
AddRange method to add an array of tree node items to a tree view. However,
if you want to add items one at a time, use the BeginUpdate method to
prevent the TreeView control from painting during the add operations. To
allow control the control to resume painting, call the EndUpdate method when
all the tree nodes have been added to the tree view."
If
 
Hi Terry, Eric & Norman

Thanks for ur valuable suggestions.

I tried all ur suggestions. But still it is working slow. I used
AddRange and listView1.ListViewItemSorter = null.

Tried using virtual Listview control provided in net. It says that it
populates based on the user selected page. But still i cant add image
items. And even without adding images when i scroll thru it gives a
flickering effect.

When i scroll thru outlook mails in its inbox, its not flickering. How
can i do it?? any ideas??

Many Thanks Terry, Eric & Norman

pls let me know immediatelyyyy..
Anushya

Eric Newton said:
yea, that makes sense, but AddRange doesnt do the Begin/End update like they
should...

public virtual void AddRange(System.Windows.Forms.TreeNode[] nodes) {
TreeNode local0;
System.Windows.Forms.TreeNode[] local1;
int local2;

if (nodes == null)
throw new ArgumentNullException("nodes");
local1 = nodes;
local2 = 0;
while (local2 < (int) local1.Length) {
local0 = local1[local2];
this.Add(local0);
local2++;
}
}

so you'd normally call BeginUpdate() then AddRange() then EndUpdate()

Terry said:
I was under the impression that the AddRange also calls
BeginUpdate/EndUpdate. If you're adding them one at a time via Add, you
should call Begin/EndUpdate yourself to supress paints until you're done
adding items.

It is the following paragraph from the "BeginUpdate" docs that led me to
that.

"The preferred way to add items to a tree view control is to use the
AddRange method to add an array of tree node items to a tree view. However,
if you want to add items one at a time, use the BeginUpdate method to
prevent the TreeView control from painting during the add operations. To
allow control the control to resume painting, call the EndUpdate method when
all the tree nodes have been added to the tree view."

to
add
 
Hi Eric& Terry
i was using addrange only to add items.. Now disabled
sorting too.. Still it is slow..

then tried virtual list which adds only text by using
System.Runtime.InteropServices.Marshal.Copy(data, 0,
info.item.pszText, data.Length);

It was flickering and changed the above statement to
prevent updating.
LockWindowUpdate(base.Handle);
System.Runtime.InteropServices.Marshal.Copy(data,
0, info.item.pszText, data.Length);

LockWindowUpdate(IntPtr.Zero);
still its flickering

I want the soln to avoid flickering and also how to add
images to virtual listview.. pasting the code here.. Pls
let me know immediately...
using System;
using System.Windows.Forms;
using System.Diagnostics;

namespace Microsoft.Samples.VirtualListView
{
#region VirtualListView Delegates
public delegate void QueryItemTextHandler(int item,
int subItem, out string text);
public delegate void QueryItemImageHandler(int item,
int subItem, out int imageIndex);
public delegate void QueryItemIndentHandler(int item,
out int itemIndent);
#endregion

/// <summary>
/// Summary description for
VirtualListViewControl.
/// </summary>
public class VirtualListView : ListView {
// store the item count to prevent the call to
SendMessage(LVM_GETITEMCOUNT)
private int itemCount = 0;
public int ItemCount {
get { return itemCount; }
set {
itemCount = value;
int result;
result = WindowsFunction.SendMessage(
this.Handle,
(int)
ListViewMessages.LVM_SETITEMCOUNT,
itemCount,
0);
}
}

public VirtualListView ()
{
// virtual listviews must be Details or List
view with no sorting
View = View.Details;
Sorting = SortOrder.None;

}

protected override
System.Windows.Forms.CreateParams CreateParams {
get {
CreateParams cp = base.CreateParams;
// LVS_OWNERDATA style must be set when
the control is created
cp.Style |= (int)
ListViewStyles.LVS_OWNERDATA;
return cp;
}
}

public new System.Windows.Forms.View View {
get {
return new System.Windows.Forms.View();
}
set {
if (value == View.LargeIcon ||
value == View.SmallIcon) {
//throw new ArgumentException("Icon
views are invalid for virtual ListViews", "View");
}
base.View = value;
}
}

#region Display query callbacks
public event QueryItemTextHandler QueryItemText;
public event QueryItemImageHandler QueryItemImage;
public event QueryItemIndentHandler
QueryItemIndent;
#endregion

void OnDispInfoNotice(ref Message m, bool
useAnsi) {

LVDISPINFO info = (LVDISPINFO)m.GetLParam
(typeof(LVDISPINFO));
string lvtext = null;

if((info.item.mask & (uint)
ListViewItemMask.LVIF_TEXT) > 0) {
if (QueryItemText != null) {
QueryItemText(info.item.iItem,
info.item.iSubItem, out lvtext);
if (lvtext != null) {
try {

int maxIndex = Math.Min(info.item.cchTextMax-1,
lvtext.Length);

char[] data = new char[maxIndex+1];

lvtext.CopyTo(0, data, 0, lvtext.Length);

data[maxIndex] = '\0';

System.Runtime.InteropServices.Marshal.Copy(data,
0, info.item.pszText, data.Length);
}
catch (Exception e) {
Debug.WriteLine("Failed to
copy text name from client: " + e.ToString
(), "VirtualListView.OnDispInfoNotice");
}
}
}
}

if((info.item.mask & (uint)
ListViewItemMask.LVIF_IMAGE) > 0) {
int imageIndex = 0;
if (QueryItemImage != null) {
QueryItemImage(info.item.iItem,
info.item.iSubItem, out imageIndex);
}
info.item.iImage = imageIndex;
}

if ((info.item.mask & (uint)
ListViewItemMask.LVIF_INDENT) > 0) {
int itemIndent = 0;
if (QueryItemIndent != null) {
QueryItemIndent(info.item.iItem, out
itemIndent);
}
info.item.iIndent = itemIndent;
}
m.Result = new IntPtr(0);
}


protected override void WndProc(ref
System.Windows.Forms.Message m) {
NMHDR nm1;
bool messageProcessed = false;
switch (m.Msg) {
case (int)WindowsMessage.WM_REFLECT +
(int)WindowsMessage.WM_NOTIFY:
nm1 = (NMHDR) m.GetLParam(typeof
(NMHDR));
switch(nm1.code) {
case (int)
ListViewNotices.LVN_GETDISPINFOW:
OnDispInfoNotice(ref m,
false);
messageProcessed = true;
break;
default:
break;
}
break;
default:
break;
}
if (!messageProcessed) {
base.WndProc(ref m);
}
}
}
}


-----Original Message-----
yea, that makes sense, but AddRange doesnt do the Begin/End update like they
should...

public virtual void AddRange
(System.Windows.Forms.TreeNode[] nodes) {
TreeNode local0;
System.Windows.Forms.TreeNode[] local1;
int local2;

if (nodes == null)
throw new ArgumentNullException("nodes");
local1 = nodes;
local2 = 0;
while (local2 < (int) local1.Length) {
local0 = local1[local2];
this.Add(local0);
local2++;
}
}

so you'd normally call BeginUpdate() then AddRange() then EndUpdate()

Terry said:
I was under the impression that the AddRange also calls
BeginUpdate/EndUpdate. If you're adding them one at a time via Add, you
should call Begin/EndUpdate yourself to supress paints until you're done
adding items.

It is the following paragraph from the "BeginUpdate" docs that led me to
that.

"The preferred way to add items to a tree view control is to use the
AddRange method to add an array of tree node items to
a tree view.
However,
if you want to add items one at a time, use the BeginUpdate method to
prevent the TreeView control from painting during the add operations. To
allow control the control to resume painting, call the
EndUpdate method
when
all the tree nodes have been added to the tree view."
simply calls Add()
for it's done, then you
can using "AddRange()"
to and then listing
with populate cannot be
reduced even in case of 2
lakhs this with ListView. I
am have it in memory
and to the listview. If
so

.
 
What version of the framework are you using, I found the same problem with
1.0 but it went away with 1.1.

Anushya said:
Hi Eric& Terry
i was using addrange only to add items.. Now disabled
sorting too.. Still it is slow..

then tried virtual list which adds only text by using
System.Runtime.InteropServices.Marshal.Copy(data, 0,
info.item.pszText, data.Length);

It was flickering and changed the above statement to
prevent updating.
LockWindowUpdate(base.Handle);
System.Runtime.InteropServices.Marshal.Copy(data,
0, info.item.pszText, data.Length);

LockWindowUpdate(IntPtr.Zero);
still its flickering

I want the soln to avoid flickering and also how to add
images to virtual listview.. pasting the code here.. Pls
let me know immediately...
using System;
using System.Windows.Forms;
using System.Diagnostics;

namespace Microsoft.Samples.VirtualListView
{
#region VirtualListView Delegates
public delegate void QueryItemTextHandler(int item,
int subItem, out string text);
public delegate void QueryItemImageHandler(int item,
int subItem, out int imageIndex);
public delegate void QueryItemIndentHandler(int item,
out int itemIndent);
#endregion

/// <summary>
/// Summary description for
VirtualListViewControl.
/// </summary>
public class VirtualListView : ListView {
// store the item count to prevent the call to
SendMessage(LVM_GETITEMCOUNT)
private int itemCount = 0;
public int ItemCount {
get { return itemCount; }
set {
itemCount = value;
int result;
result = WindowsFunction.SendMessage(
this.Handle,
(int)
ListViewMessages.LVM_SETITEMCOUNT,
itemCount,
0);
}
}

public VirtualListView ()
{
// virtual listviews must be Details or List
view with no sorting
View = View.Details;
Sorting = SortOrder.None;

}

protected override
System.Windows.Forms.CreateParams CreateParams {
get {
CreateParams cp = base.CreateParams;
// LVS_OWNERDATA style must be set when
the control is created
cp.Style |= (int)
ListViewStyles.LVS_OWNERDATA;
return cp;
}
}

public new System.Windows.Forms.View View {
get {
return new System.Windows.Forms.View();
}
set {
if (value == View.LargeIcon ||
value == View.SmallIcon) {
//throw new ArgumentException("Icon
views are invalid for virtual ListViews", "View");
}
base.View = value;
}
}

#region Display query callbacks
public event QueryItemTextHandler QueryItemText;
public event QueryItemImageHandler QueryItemImage;
public event QueryItemIndentHandler
QueryItemIndent;
#endregion

void OnDispInfoNotice(ref Message m, bool
useAnsi) {

LVDISPINFO info = (LVDISPINFO)m.GetLParam
(typeof(LVDISPINFO));
string lvtext = null;

if((info.item.mask & (uint)
ListViewItemMask.LVIF_TEXT) > 0) {
if (QueryItemText != null) {
QueryItemText(info.item.iItem,
info.item.iSubItem, out lvtext);
if (lvtext != null) {
try {

int maxIndex = Math.Min(info.item.cchTextMax-1,
lvtext.Length);

char[] data = new char[maxIndex+1];

lvtext.CopyTo(0, data, 0, lvtext.Length);

data[maxIndex] = '\0';

System.Runtime.InteropServices.Marshal.Copy(data,
0, info.item.pszText, data.Length);
}
catch (Exception e) {
Debug.WriteLine("Failed to
copy text name from client: " + e.ToString
(), "VirtualListView.OnDispInfoNotice");
}
}
}
}

if((info.item.mask & (uint)
ListViewItemMask.LVIF_IMAGE) > 0) {
int imageIndex = 0;
if (QueryItemImage != null) {
QueryItemImage(info.item.iItem,
info.item.iSubItem, out imageIndex);
}
info.item.iImage = imageIndex;
}

if ((info.item.mask & (uint)
ListViewItemMask.LVIF_INDENT) > 0) {
int itemIndent = 0;
if (QueryItemIndent != null) {
QueryItemIndent(info.item.iItem, out
itemIndent);
}
info.item.iIndent = itemIndent;
}
m.Result = new IntPtr(0);
}


protected override void WndProc(ref
System.Windows.Forms.Message m) {
NMHDR nm1;
bool messageProcessed = false;
switch (m.Msg) {
case (int)WindowsMessage.WM_REFLECT +
(int)WindowsMessage.WM_NOTIFY:
nm1 = (NMHDR) m.GetLParam(typeof
(NMHDR));
switch(nm1.code) {
case (int)
ListViewNotices.LVN_GETDISPINFOW:
OnDispInfoNotice(ref m,
false);
messageProcessed = true;
break;
default:
break;
}
break;
default:
break;
}
if (!messageProcessed) {
base.WndProc(ref m);
}
}
}
}


-----Original Message-----
yea, that makes sense, but AddRange doesnt do the Begin/End update like they
should...

public virtual void AddRange
(System.Windows.Forms.TreeNode[] nodes) {
TreeNode local0;
System.Windows.Forms.TreeNode[] local1;
int local2;

if (nodes == null)
throw new ArgumentNullException("nodes");
local1 = nodes;
local2 = 0;
while (local2 < (int) local1.Length) {
local0 = local1[local2];
this.Add(local0);
local2++;
}
}

so you'd normally call BeginUpdate() then AddRange() then EndUpdate()

Terry said:
I was under the impression that the AddRange also calls
BeginUpdate/EndUpdate. If you're adding them one at a time via Add, you
should call Begin/EndUpdate yourself to supress paints until you're done
adding items.

It is the following paragraph from the "BeginUpdate" docs that led me to
that.

"The preferred way to add items to a tree view control is to use the
AddRange method to add an array of tree node items to
a tree view.
However,
if you want to add items one at a time, use the BeginUpdate method to
prevent the TreeView control from painting during the add operations. To
allow control the control to resume painting, call the
EndUpdate method
when
all the tree nodes have been added to the tree view."

ListViewItemSorter to null definitely, but AddRange
simply calls Add()
for
each object... just FYI


When filling make sure you don't have sorting enabled. Set the
ListViewItemSorter = null before filling. After
it's done, then you
can
reapply the current sort. Also, make sure you're
using "AddRange()"
to
add
all of the items at once.

Terry

Hi
I am populating a listview with more than 5000 items from my outlook
inbox and even the form looks hanged for 10 mins
and then listing
with
the details. The number of items i need to
populate cannot be
reduced
in any way. I need the performance to be good
even in case of 2
lakhs
items..

Do anybody have any paging idea of how to do
this with ListView. I
am
working in c#.net?? Or else any other idea. Here i am not using any
database just reading my folders in outlook and
have it in memory
and
then straight way adding to listview using add item??

Can the Mail item collection be linked straight
to the listview. If
so
will the performance be good??

Or can i go thru some other way..
Pls let me know immediately if u have any suggestion....

Anushya


.
 
Hi mark
1.1 only
Anushya
Anushya said:
Hi Eric& Terry
i was using addrange only to add items.. Now disabled
sorting too.. Still it is slow..

then tried virtual list which adds only text by using
System.Runtime.InteropServices.Marshal.Copy(data, 0,
info.item.pszText, data.Length);

It was flickering and changed the above statement to
prevent updating.
LockWindowUpdate(base.Handle);
System.Runtime.InteropServices.Marshal.Copy(data,
0, info.item.pszText, data.Length);

LockWindowUpdate(IntPtr.Zero);
still its flickering

I want the soln to avoid flickering and also how to add
images to virtual listview.. pasting the code here.. Pls
let me know immediately...
using System;
using System.Windows.Forms;
using System.Diagnostics;

namespace Microsoft.Samples.VirtualListView
{
#region VirtualListView Delegates
public delegate void QueryItemTextHandler(int item,
int subItem, out string text);
public delegate void QueryItemImageHandler(int item,
int subItem, out int imageIndex);
public delegate void QueryItemIndentHandler(int item,
out int itemIndent);
#endregion

/// <summary>
/// Summary description for
VirtualListViewControl.
/// </summary>
public class VirtualListView : ListView {
// store the item count to prevent the call to
SendMessage(LVM_GETITEMCOUNT)
private int itemCount = 0;
public int ItemCount {
get { return itemCount; }
set {
itemCount = value;
int result;
result = WindowsFunction.SendMessage(
this.Handle,
(int)
ListViewMessages.LVM_SETITEMCOUNT,
itemCount,
0);
}
}

public VirtualListView ()
{
// virtual listviews must be Details or List
view with no sorting
View = View.Details;
Sorting = SortOrder.None;

}

protected override
System.Windows.Forms.CreateParams CreateParams {
get {
CreateParams cp = base.CreateParams;
// LVS_OWNERDATA style must be set when
the control is created
cp.Style |= (int)
ListViewStyles.LVS_OWNERDATA;
return cp;
}
}

public new System.Windows.Forms.View View {
get {
return new System.Windows.Forms.View();
}
set {
if (value == View.LargeIcon ||
value == View.SmallIcon) {
//throw new ArgumentException("Icon
views are invalid for virtual ListViews", "View");
}
base.View = value;
}
}

#region Display query callbacks
public event QueryItemTextHandler QueryItemText;
public event QueryItemImageHandler QueryItemImage;
public event QueryItemIndentHandler
QueryItemIndent;
#endregion

void OnDispInfoNotice(ref Message m, bool
useAnsi) {

LVDISPINFO info = (LVDISPINFO)m.GetLParam
(typeof(LVDISPINFO));
string lvtext = null;

if((info.item.mask & (uint)
ListViewItemMask.LVIF_TEXT) > 0) {
if (QueryItemText != null) {
QueryItemText(info.item.iItem,
info.item.iSubItem, out lvtext);
if (lvtext != null) {
try {

int maxIndex = Math.Min(info.item.cchTextMax-1,
lvtext.Length);

char[] data = new char[maxIndex+1];

lvtext.CopyTo(0, data, 0, lvtext.Length);

data[maxIndex] = '\0';

System.Runtime.InteropServices.Marshal.Copy(data,
0, info.item.pszText, data.Length);
}
catch (Exception e) {
Debug.WriteLine("Failed to
copy text name from client: " + e.ToString
(), "VirtualListView.OnDispInfoNotice");
}
}
}
}

if((info.item.mask & (uint)
ListViewItemMask.LVIF_IMAGE) > 0) {
int imageIndex = 0;
if (QueryItemImage != null) {
QueryItemImage(info.item.iItem,
info.item.iSubItem, out imageIndex);
}
info.item.iImage = imageIndex;
}

if ((info.item.mask & (uint)
ListViewItemMask.LVIF_INDENT) > 0) {
int itemIndent = 0;
if (QueryItemIndent != null) {
QueryItemIndent(info.item.iItem, out
itemIndent);
}
info.item.iIndent = itemIndent;
}
m.Result = new IntPtr(0);
}


protected override void WndProc(ref
System.Windows.Forms.Message m) {
NMHDR nm1;
bool messageProcessed = false;
switch (m.Msg) {
case (int)WindowsMessage.WM_REFLECT +
(int)WindowsMessage.WM_NOTIFY:
nm1 = (NMHDR) m.GetLParam(typeof
(NMHDR));
switch(nm1.code) {
case (int)
ListViewNotices.LVN_GETDISPINFOW:
OnDispInfoNotice(ref m,
false);
messageProcessed = true;
break;
default:
break;
}
break;
default:
break;
}
if (!messageProcessed) {
base.WndProc(ref m);
}
}
}
}


-----Original Message-----
yea, that makes sense, but AddRange doesnt do the Begin/End update like they
should...

public virtual void AddRange
(System.Windows.Forms.TreeNode[] nodes) {
TreeNode local0;
System.Windows.Forms.TreeNode[] local1;
int local2;

if (nodes == null)
throw new ArgumentNullException("nodes");
local1 = nodes;
local2 = 0;
while (local2 < (int) local1.Length) {
local0 = local1[local2];
this.Add(local0);
local2++;
}
}

so you'd normally call BeginUpdate() then AddRange() then EndUpdate()

Terry said:
I was under the impression that the AddRange also calls
BeginUpdate/EndUpdate. If you're adding them one at a time via Add, you
should call Begin/EndUpdate yourself to supress paints until you're done
adding items.

It is the following paragraph from the "BeginUpdate" docs that led me to
that.

"The preferred way to add items to a tree view control is to use the
AddRange method to add an array of tree node items to
a tree view.
However,EndUpdate method
whensimply calls Add()
forit's done, then you
canusing "AddRange()"
to
addand then listing
withpopulate cannot be
reducedeven in case of 2
lakhsthis with ListView. I
amhave it in memory
andto the listview. If
so
 
Back
Top