loading XML into collection slow

  • Thread starter Thread starter Kurt Bauer
  • Start date Start date
K

Kurt Bauer

I have an ASP group calendar application which pulls calendar data from
Exchange via webdav into an XML string. I then loop the XML nodes to
populate a collection of appointments. Finally I use the appointment
collection to populate the calendar control. The performance getting the
XML data is fine, but loading the data into the collection is slow. My
question/problem is should I be using the collection, a dataset, or
something else to load the xml data into the calendar control?
Thanks,
Kurt Bauer
 
Kurt,

How are you loading the Xml data into the collection? If you could post code
or proovide some details I could probably provide some details. There may be
faster ways to do it. Does populating the collection involve executing a lot
of XPath expressions? Then make sure you are using an XPathDocument. Is it
simple, forward-only parsing of the XML, then you should try using an
XmlReader or XmlSerialization (which is built around XmlReaders).

HTH,
Christoph Schittko
Software Architect, .NET Mentor
MVP XML .NET
 
Thanks for the help. Using the reader has helped a little bit in the
performance, but I am still taking a big hit when I load the data into my
collection. If I write the output directly to the console it takes a split
second to write 24 appointments. As soon as I load the XML into the
collection and then loop through the collection to display an appointment,
the time jumps up to over 5 seconds.
The main reason I load the data into the collection is so I can loop through
the collection in the dayrender of the calendar control. Is there a better
way to get from the XML string to populating the calendar control that I am
missing?
Thanks,
Kurt Bauer

Here is my new code using the XML reader:
Dim oAppointment As Calendar.Appointment
Dim oAppointments As Calendar.Appointments
Dim doc As Xml.XPath.XPathDocument = New Xml.XPath.XPathDocument(New
XmlTextReader(New System.IO.StringReader(strXML)))
Dim nav As Xml.XPath.XPathNavigator = doc.CreateNavigator()
Dim oNSManager As XmlNamespaceManager = New
XmlNamespaceManager(nav.NameTable)
Dim expression As Xml.XPath.XPathExpression = nav.Compile("//a:prop")
oNSManager.AddNamespace("a", "DAV:")
expression.SetContext(oNSManager)
Dim it As Xml.XPath.XPathNodeIterator = nav.Select(expression)
If it.Count > 0 Then
oAppointments = New Calendar.Appointments
' Walk the response collection
While it.MoveNext
oAppointment = New Calendar.Appointment
Dim appointments As Xml.XPath.XPathNavigator = it.Current
Dim children As Xml.XPath.XPathNodeIterator =
appointments.SelectChildren(XPath.XPathNodeType.Element)
While children.MoveNext
Select Case children.Current.LocalName
Case "subject"
Dim intIndex As Integer
intIndex = InStr(1, children.Current.Value, ":")
If intIndex > 0 Then
oAppointment.Category = Mid(children.Current.Value, 1,
intIndex - 1)
oAppointment.Subject = Mid(children.Current.Value,
intIndex + 1)
Else
oAppointment.Subject = children.Current.Value
End If
intIndex = Nothing
Case "dtstart"
oAppointment.StartTime = children.Current.Value
Case "dtend"
oAppointment.EndTime = children.Current.Value
Case "location"
oAppointment.Location = children.Current.Value
Case "alldayevent"
oAppointment.AllDayEvent = children.Current.Value
Case "reminderoffset"
oAppointment.ReminderOffset = children.Current.Value / 60
Case "description"
oAppointment.Body = children.Current.Value
End Select
End While
oAppointments.Add(oAppointment)
oAppointment = Nothing
End While
End If

Here are my 2 collections for reference:
Public Class Appointments
Implements IEnumerable
Private mcolItems As Collection

Public Sub New()
mcolItems = New Collection
End Sub

Public ReadOnly Property Count()
Get
Count = mcolItems.Count
End Get
End Property

Sub Add(ByVal objNew As Calendar.Appointment)
mcolItems.Add(objNew)
End Sub

Public ReadOnly Property Item(ByVal vIndex As Object) As Appointment
Get
Item = mcolItems.Item(vIndex)
End Get
End Property
End Class

Public Class Appointment
Private strUrl As String
Private dtStartTime As DateTime
Private dtEndTime As DateTime
Private strSubject As String
Private strLocation As String
Private strBody As String
Private strCategory As String
Private strBusyStatus As String
Private blnAllDayEvent As Boolean
Private lngReminderOffset As Long
Private objAttendies As Calendar.Attendies

Public Property EmailUrl() As String
Get
Return strUrl
End Get
Set(ByVal Value As String)
strUrl = Value
End Set
End Property

Public Property StartTime() As DateTime
Get
Return dtStartTime
End Get
Set(ByVal Value As DateTime)
dtStartTime = Value
End Set
End Property

Public Property EndTime() As DateTime
Get
Return dtEndTime
End Get
Set(ByVal Value As DateTime)
dtEndTime = Value
End Set
End Property

Public Property Subject() As String
Get
Return strSubject
End Get
Set(ByVal Value As String)
strSubject = Value
End Set
End Property

Public Property Location() As String
Get
Return strLocation
End Get
Set(ByVal Value As String)
strLocation = Value
End Set
End Property

Public Property Body() As String
Get
Return strBody
End Get
Set(ByVal Value As String)
strBody = Value
End Set
End Property

Public Property Category() As String
Get
Return strCategory
End Get
Set(ByVal Value As String)
strCategory = Value
End Set
End Property

Public Property BusyStatus() As String
Get
Return strBusyStatus
End Get
Set(ByVal Value As String)
strBusyStatus = Value
End Set
End Property

Public Property AllDayEvent() As Boolean
Get
Return blnAllDayEvent
End Get
Set(ByVal Value As Boolean)
blnAllDayEvent = Value
End Set
End Property

Public Property ReminderOffset() As Long
Get
Return lngReminderOffset
End Get
Set(ByVal Value As Long)
lngReminderOffset = Value
End Set
End Property

Public Property Attendies() As Calendar.Attendies
Get
Return objAttendies
End Get
Set(ByVal Value As Calendar.Attendies)
objAttendies = Value
End Set
End Property

Sub New()
objAttendies = Calendar.Attendies.GetAttendies
End Sub
End Class
 
It sounds more like that there is something going on inside the Appointments
collection when you populate the data.

Do you have a profiler available to see where the bottleneck is and if it's
really the XML parsing that's at fault here?
Have you tried to populate your appointments from a simple string array how
much faster is that compared to populating it from the Xml doc?

HTH,
Christoph Schittko
Software Architect, .NET Mentor
MVP XML .NET


Kurt Bauer said:
Thanks for the help. Using the reader has helped a little bit in the
performance, but I am still taking a big hit when I load the data into my
collection. If I write the output directly to the console it takes a split
second to write 24 appointments. As soon as I load the XML into the
collection and then loop through the collection to display an appointment,
the time jumps up to over 5 seconds.
The main reason I load the data into the collection is so I can loop through
the collection in the dayrender of the calendar control. Is there a better
way to get from the XML string to populating the calendar control that I am
missing?
Thanks,
Kurt Bauer

Here is my new code using the XML reader:
Dim oAppointment As Calendar.Appointment
Dim oAppointments As Calendar.Appointments
Dim doc As Xml.XPath.XPathDocument = New Xml.XPath.XPathDocument(New
XmlTextReader(New System.IO.StringReader(strXML)))
Dim nav As Xml.XPath.XPathNavigator = doc.CreateNavigator()
Dim oNSManager As XmlNamespaceManager = New
XmlNamespaceManager(nav.NameTable)
Dim expression As Xml.XPath.XPathExpression = nav.Compile("//a:prop")
oNSManager.AddNamespace("a", "DAV:")
expression.SetContext(oNSManager)
Dim it As Xml.XPath.XPathNodeIterator = nav.Select(expression)
If it.Count > 0 Then
oAppointments = New Calendar.Appointments
' Walk the response collection
While it.MoveNext
oAppointment = New Calendar.Appointment
Dim appointments As Xml.XPath.XPathNavigator = it.Current
Dim children As Xml.XPath.XPathNodeIterator =
appointments.SelectChildren(XPath.XPathNodeType.Element)
While children.MoveNext
Select Case children.Current.LocalName
Case "subject"
Dim intIndex As Integer
intIndex = InStr(1, children.Current.Value, ":")
If intIndex > 0 Then
oAppointment.Category = Mid(children.Current.Value, 1,
intIndex - 1)
oAppointment.Subject = Mid(children.Current.Value,
intIndex + 1)
Else
oAppointment.Subject = children.Current.Value
End If
intIndex = Nothing
Case "dtstart"
oAppointment.StartTime = children.Current.Value
Case "dtend"
oAppointment.EndTime = children.Current.Value
Case "location"
oAppointment.Location = children.Current.Value
Case "alldayevent"
oAppointment.AllDayEvent = children.Current.Value
Case "reminderoffset"
oAppointment.ReminderOffset = children.Current.Value / 60
Case "description"
oAppointment.Body = children.Current.Value
End Select
End While
oAppointments.Add(oAppointment)
oAppointment = Nothing
End While
End If

Here are my 2 collections for reference:
Public Class Appointments
Implements IEnumerable
Private mcolItems As Collection

Public Sub New()
mcolItems = New Collection
End Sub

Public ReadOnly Property Count()
Get
Count = mcolItems.Count
End Get
End Property

Sub Add(ByVal objNew As Calendar.Appointment)
mcolItems.Add(objNew)
End Sub

Public ReadOnly Property Item(ByVal vIndex As Object) As Appointment
Get
Item = mcolItems.Item(vIndex)
End Get
End Property
End Class

Public Class Appointment
Private strUrl As String
Private dtStartTime As DateTime
Private dtEndTime As DateTime
Private strSubject As String
Private strLocation As String
Private strBody As String
Private strCategory As String
Private strBusyStatus As String
Private blnAllDayEvent As Boolean
Private lngReminderOffset As Long
Private objAttendies As Calendar.Attendies

Public Property EmailUrl() As String
Get
Return strUrl
End Get
Set(ByVal Value As String)
strUrl = Value
End Set
End Property

Public Property StartTime() As DateTime
Get
Return dtStartTime
End Get
Set(ByVal Value As DateTime)
dtStartTime = Value
End Set
End Property

Public Property EndTime() As DateTime
Get
Return dtEndTime
End Get
Set(ByVal Value As DateTime)
dtEndTime = Value
End Set
End Property

Public Property Subject() As String
Get
Return strSubject
End Get
Set(ByVal Value As String)
strSubject = Value
End Set
End Property

Public Property Location() As String
Get
Return strLocation
End Get
Set(ByVal Value As String)
strLocation = Value
End Set
End Property

Public Property Body() As String
Get
Return strBody
End Get
Set(ByVal Value As String)
strBody = Value
End Set
End Property

Public Property Category() As String
Get
Return strCategory
End Get
Set(ByVal Value As String)
strCategory = Value
End Set
End Property

Public Property BusyStatus() As String
Get
Return strBusyStatus
End Get
Set(ByVal Value As String)
strBusyStatus = Value
End Set
End Property

Public Property AllDayEvent() As Boolean
Get
Return blnAllDayEvent
End Get
Set(ByVal Value As Boolean)
blnAllDayEvent = Value
End Set
End Property

Public Property ReminderOffset() As Long
Get
Return lngReminderOffset
End Get
Set(ByVal Value As Long)
lngReminderOffset = Value
End Set
End Property

Public Property Attendies() As Calendar.Attendies
Get
Return objAttendies
End Get
Set(ByVal Value As Calendar.Attendies)
objAttendies = Value
End Set
End Property

Sub New()
objAttendies = Calendar.Attendies.GetAttendies
End Sub
End Class
Christoph Schittko said:
Kurt,

try the XPathNavigator/XPathNodeIterator combo rather than the XmlDocument.
This combo is optimized for read-only XPath access and performs better than
the SelectNodes couterpart.

See the code snippet below. I hope you get the idea:

Dim doc As Xml.XPath.XPathDocument = New Xml.XPath.XPathDocument(New
XmlTextReader(New System.IO.StringReader(strXML)))
Dim nav As Xml.XPath.XPathNavigator = doc.CreateNavigator()
Dim oNSManager As XmlNamespaceManager = New
XmlNamespaceManager(nav.NameTable)
Dim expression As Xml.XPath.XPathExpression = nav.Compile("//a:prop")
oNSManager.AddNamespace("a", "DAV:")
expression.SetContext(oNSManager)
Dim it As Xml.XPath.XPathNodeIterator = nav.Select(expression)
' Walk the response collection
While it.MoveNext
Dim appointments As Xml.XPath.XPathNavigator = it.Current
Dim children As Xml.XPath.XPathNodeIterator =
appointments.SelectChildren(XPath.XPathNodeType.Element)
While children.MoveNext
Select Case children.Current.LocalName
Case "subject"
Dim intIndex As Integer
intIndex = InStr(1, children.Current.Value, ":")
If intIndex > 0 Then
Console.WriteLine(String.Format("Category {0}",
Mid(children.Current.Value, 1, intIndex - 1)))
Console.WriteLine(String.Format("subject {0}",
Mid(children.Current.Value, intIndex + 1)))
Else
children.Current.MoveToFirstChild()
Console.WriteLine(String.Format("subject {0}",
children.Current.Value))
End If
' ...

HTH,
Christoph Schittko
Software Architect, .NET Mentor
MS MVP XML .NET
Click to expand...
http://msdn.microsoft.com/library/d.../en-us/dnmes2k/html/calwp_0001.asp?frame=true
dar/{D1A44985-AD7A-41E9-97BA-A6245517AB4C}.EML said:
Call:Add
my
Click to expand...
Click to expand...
 
I'm getting closer. The slow down occurs when I instantiate a new
oAppointment object.
After I loop through each child in the appointment, I add the appointment
object to my appointments collection. Then I set the appointment object to
nothing, but I then create a new appointment object for the next appointment
in the xml string.
I tried not destroying and recreating the appointment object after each loop
to the string, but when I then loop through my collection of appointments
after processing my xml string, I have the correct number of appointments in
the collection, but they are all just copies of the last appointment added
to the collection.
So is there an easy to populate the collection of appointments without
destroying and recreating my appointment object on each loop?
Thanks,
Kurt

Christoph Schittko said:
It sounds more like that there is something going on inside the Appointments
collection when you populate the data.

Do you have a profiler available to see where the bottleneck is and if it's
really the XML parsing that's at fault here?
Have you tried to populate your appointments from a simple string array how
much faster is that compared to populating it from the Xml doc?

HTH,
Christoph Schittko
Software Architect, .NET Mentor
MVP XML .NET


Kurt Bauer said:
Thanks for the help. Using the reader has helped a little bit in the
performance, but I am still taking a big hit when I load the data into my
collection. If I write the output directly to the console it takes a split
second to write 24 appointments. As soon as I load the XML into the
collection and then loop through the collection to display an appointment,
the time jumps up to over 5 seconds.
The main reason I load the data into the collection is so I can loop through
the collection in the dayrender of the calendar control. Is there a better
way to get from the XML string to populating the calendar control that I am
missing?
Thanks,
Kurt Bauer

Here is my new code using the XML reader:
Dim oAppointment As Calendar.Appointment
Dim oAppointments As Calendar.Appointments
Dim doc As Xml.XPath.XPathDocument = New Xml.XPath.XPathDocument(New
XmlTextReader(New System.IO.StringReader(strXML)))
Dim nav As Xml.XPath.XPathNavigator = doc.CreateNavigator()
Dim oNSManager As XmlNamespaceManager = New
XmlNamespaceManager(nav.NameTable)
Dim expression As Xml.XPath.XPathExpression = nav.Compile("//a:prop")
oNSManager.AddNamespace("a", "DAV:")
expression.SetContext(oNSManager)
Dim it As Xml.XPath.XPathNodeIterator = nav.Select(expression)
If it.Count > 0 Then
oAppointments = New Calendar.Appointments
' Walk the response collection
While it.MoveNext
oAppointment = New Calendar.Appointment
Dim appointments As Xml.XPath.XPathNavigator = it.Current
Dim children As Xml.XPath.XPathNodeIterator =
appointments.SelectChildren(XPath.XPathNodeType.Element)
While children.MoveNext
Select Case children.Current.LocalName
Case "subject"
Dim intIndex As Integer
intIndex = InStr(1, children.Current.Value, ":")
If intIndex > 0 Then
oAppointment.Category = Mid(children.Current.Value, 1,
intIndex - 1)
oAppointment.Subject = Mid(children.Current.Value,
intIndex + 1)
Else
oAppointment.Subject = children.Current.Value
End If
intIndex = Nothing
Case "dtstart"
oAppointment.StartTime = children.Current.Value
Case "dtend"
oAppointment.EndTime = children.Current.Value
Case "location"
oAppointment.Location = children.Current.Value
Case "alldayevent"
oAppointment.AllDayEvent = children.Current.Value
Case "reminderoffset"
oAppointment.ReminderOffset = children.Current.Value / 60
Case "description"
oAppointment.Body = children.Current.Value
End Select
End While
oAppointments.Add(oAppointment)
oAppointment = Nothing
End While
End If

Here are my 2 collections for reference:
Public Class Appointments
Implements IEnumerable
Private mcolItems As Collection

Public Sub New()
mcolItems = New Collection
End Sub

Public ReadOnly Property Count()
Get
Count = mcolItems.Count
End Get
End Property

Sub Add(ByVal objNew As Calendar.Appointment)
mcolItems.Add(objNew)
End Sub

Public ReadOnly Property Item(ByVal vIndex As Object) As Appointment
Get
Item = mcolItems.Item(vIndex)
End Get
End Property
End Class

Public Class Appointment
Private strUrl As String
Private dtStartTime As DateTime
Private dtEndTime As DateTime
Private strSubject As String
Private strLocation As String
Private strBody As String
Private strCategory As String
Private strBusyStatus As String
Private blnAllDayEvent As Boolean
Private lngReminderOffset As Long
Private objAttendies As Calendar.Attendies

Public Property EmailUrl() As String
Get
Return strUrl
End Get
Set(ByVal Value As String)
strUrl = Value
End Set
End Property

Public Property StartTime() As DateTime
Get
Return dtStartTime
End Get
Set(ByVal Value As DateTime)
dtStartTime = Value
End Set
End Property

Public Property EndTime() As DateTime
Get
Return dtEndTime
End Get
Set(ByVal Value As DateTime)
dtEndTime = Value
End Set
End Property

Public Property Subject() As String
Get
Return strSubject
End Get
Set(ByVal Value As String)
strSubject = Value
End Set
End Property

Public Property Location() As String
Get
Return strLocation
End Get
Set(ByVal Value As String)
strLocation = Value
End Set
End Property

Public Property Body() As String
Get
Return strBody
End Get
Set(ByVal Value As String)
strBody = Value
End Set
End Property

Public Property Category() As String
Get
Return strCategory
End Get
Set(ByVal Value As String)
strCategory = Value
End Set
End Property

Public Property BusyStatus() As String
Get
Return strBusyStatus
End Get
Set(ByVal Value As String)
strBusyStatus = Value
End Set
End Property

Public Property AllDayEvent() As Boolean
Get
Return blnAllDayEvent
End Get
Set(ByVal Value As Boolean)
blnAllDayEvent = Value
End Set
End Property

Public Property ReminderOffset() As Long
Get
Return lngReminderOffset
End Get
Set(ByVal Value As Long)
lngReminderOffset = Value
End Set
End Property

Public Property Attendies() As Calendar.Attendies
Get
Return objAttendies
End Get
Set(ByVal Value As Calendar.Attendies)
objAttendies = Value
End Set
End Property

Sub New()
objAttendies = Calendar.Attendies.GetAttendies
End Sub
End Class
message news:[email protected]...
Click to expand...
give
me
Click to expand...
http://msdn.microsoft.com/library/d.../en-us/dnmes2k/html/calwp_0001.asp?frame=true
Click to expand...
 
Not that I am aware of. Maybe someone in the Exchange/Outlook (you're
talking about the Exchange or Outlook calendar, correct?) related groups can
provide some help here. You are probably dealing with out-of-process COM
servers and there's not good way to avoid the performance penalty unless
there is an API especially designed for that.

HTH,
Christoph Schittko
Software Architect, .NET Mentor
MS MVP XML .NET


Kurt Bauer said:
I'm getting closer. The slow down occurs when I instantiate a new
oAppointment object.
After I loop through each child in the appointment, I add the appointment
object to my appointments collection. Then I set the appointment object to
nothing, but I then create a new appointment object for the next appointment
in the xml string.
I tried not destroying and recreating the appointment object after each loop
to the string, but when I then loop through my collection of appointments
after processing my xml string, I have the correct number of appointments in
the collection, but they are all just copies of the last appointment added
to the collection.
So is there an easy to populate the collection of appointments without
destroying and recreating my appointment object on each loop?
Thanks,
Kurt

Christoph Schittko said:
It sounds more like that there is something going on inside the Appointments
collection when you populate the data.

Do you have a profiler available to see where the bottleneck is and if it's
really the XML parsing that's at fault here?
Have you tried to populate your appointments from a simple string array how
much faster is that compared to populating it from the Xml doc?

HTH,
Christoph Schittko
Software Architect, .NET Mentor
MVP XML .NET


I
am
Click to expand...
Mid(children.Current.Value,
1,
Click to expand...
Click to expand...
 
Back
Top