XMLDocument and NetworkStream (VB.NET 2005)

  • Thread starter Thread starter Sid Price
  • Start date Start date
S

Sid Price

Hello,
I have an application that needs to receive XML documents over a TCP/IP
connection. I have tried several approaches and so far have failed to find
one that I can get to work. The server is sending the XML correctly since I
can replace my XML receiver with a byte buffer and string builder and I see
the document. Here is the code that I am trying to make work:
Dim netStream As NetworkStream = tcpClient.GetStream()

If netStream.CanRead = True Then

Dim oData As XmlDocument = New XmlDocument

oData.Load(netStream) ' read the xml doc from the server ... blocking
call

The "Load" never returns. As I said if I replace the oData.Load call with
code to read the stream into a byte buffer the data is received.

Any help or pointers would be much appreciated,

Sid.
 
There's a good chance the XML you are attempting to load is not well-formed
and thus, the object is not being created.

-Scott
 
Hi Scott,
The server is my code too and I wrote the XML to a file as part of my
testing, it appears well formed and is opened by several XML apps I have
including XMLNotepad without complaint.
Sid.
 
Hello again Scott,
I have done a little more research on my problem and although I don't really
understand what is wrong with the transmitted document when I look at the
XMLDocument object's properties I do see something odd.

The Property "InnerXML" has the following:
"<?xml version="1.0"
encoding="UTF-8"?><RouteStatus><Devices><Device><Name>RMX_24</Name><Number>1</Number></Device></Devices><Routes
/></RouteStatus>"

I think this is well formed XML, any comments?

The Property "InnerText" has:
"RMX_241"
This is odd and if I select the XMLVisualizer I see the following:
The XML page cannot be displayed
Cannot view XML input using style sheet. Please correct the error and then
click the Refresh button, or try again later.
Invalid at the top level of the document. Error processing resource
'file:///C:/Documents and Settings/Sid Price/Local Sett...

RMX_241
^
Any comments would be appreciated.
Also, at the client end I have changed the code to read the stream into a
byte buffer,
convert it to string and then use XMLDocument.LoadXML to load my document. I
do
get an exception reported that the root of the document is not correctly
formed.
If I look at the string I see some odd characters before the XML.
"???<?xml version="1.0" encoding="UTF-8"?><RouteStatus> <Devices>
<Device> <Name>RMX_24</Name> <Number>1</Number> </Device>
</Devices> <Routes /></RouteStatus>
I don't know where these extra bytes are coming from, again any pointers
would be much appreciated.
Sid
 
The XML is well-formed (easy way to test is to just copy it to a text file,
save with an .xml extension and open in IE).

So, my next question is, what error are you getting when you use this code?

-Scott
 
If I look at the string I see some odd characters before the XML.
"???<?xml version="1.0" encoding="UTF-8"?><RouteStatus> <Devices>
<Device> <Name>RMX_24</Name> <Number>1</Number> </Device>
</Devices> <Routes /></RouteStatus>
I don't know where these extra bytes are coming from, again any pointers
would be much appreciated.
Click to expand...

They're almost certainly Byte-Order Marks.

I *think* I've seen an issue like this before, where the first packet
of data only contains a few bytes. I'll see if I can reproduce it
without network streams...
 
Here is the code I am using to receive the document:
Dim bytes(tcpClient.ReceiveBufferSize) As Byte



netStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))

Dim returnData As String = System.Text.Encoding.ASCII.GetString(bytes)

Debug.WriteLine(returnData)

oData.LoadXml(returnData)

The exception from the LoadXML says "Data at the root level is invalid. Line
1, position 1."

As you will see in my previous post there are a few strange bytes at the
head of the document.

Sid.
 
Here is the code I am using to receive the document:
Dim bytes(tcpClient.ReceiveBufferSize) As Byte



netStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))

Dim returnData As String = System.Text.Encoding.ASCII.GetString(bytes)

Debug.WriteLine(returnData)

oData.LoadXml(returnData)

The exception from the LoadXML says "Data at the root level is invalid. Line
1, position 1."

As you will see in my previous post there are a few strange bytes at the
head of the document.

Sid.
 
Sid Price said:
Here is the code I am using to receive the document:
Dim bytes(tcpClient.ReceiveBufferSize) As Byte



netStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))

Dim returnData As String = System.Text.Encoding.ASCII.GetString(bytes)

Debug.WriteLine(returnData)

oData.LoadXml(returnData)

The exception from the LoadXML says "Data at the root level is invalid. Line
1, position 1."

As you will see in my previous post there are a few strange bytes at the
head of the document.
Click to expand...

That's not good code to receive an XML document:

1) It's using UTF-8, not ASCII. Don't do decoding yourself - let the
document loader do it.

2) There's no guarantee that Read will return the whole of the document
in a single call. If you're going to load it all first, you should loop
round, reading into a buffer until there's no more data. (An
alternative is to read from the NetworkStream and write to a
MemoryStream until you're done - then rewind the MemoryStream.)
 
Jon said:
That's not good code to receive an XML document:

1) It's using UTF-8, not ASCII. Don't do decoding yourself - let the
document loader do it.

2) There's no guarantee that Read will return the whole of the document
in a single call. If you're going to load it all first, you should loop
round, reading into a buffer until there's no more data. (An
alternative is to read from the NetworkStream and write to a
MemoryStream until you're done - then rewind the MemoryStream.)
Click to expand...
The code I posted is not the code I wrote originally and posted in my first
message. I wrote this code to try and figure out why my original code was
not working. Here is what I started with:
Dim netStream As NetworkStream = tcpClient.GetStream()

If netStream.CanRead = True Then

Dim oData As XmlDocument = New XmlDocument

oData.Load(netStream)

With this code the "Load" never returns.

Sid.
 
The code I posted is not the code I wrote originally and posted in my first
message. I wrote this code to try and figure out why my original code was
not working. Here is what I started with:
Dim netStream As NetworkStream = tcpClient.GetStream()

If netStream.CanRead = True Then

Dim oData As XmlDocument = New XmlDocument

oData.Load(netStream)

With this code the "Load" never returns.
Click to expand...

Okay - so what's the protocol involved? Are you expecting the
connection to be closed after the whole document has been written? If
not, I suspect Load won't work - I'd at least *expect* it to read to
the end of the stream. If the stream never ends, that's quite possibly
the problem...
 
Jon Skeet said:
Okay - so what's the protocol involved? Are you expecting the
connection to be closed after the whole document has been written? If
not, I suspect Load won't work - I'd at least *expect* it to read to
the end of the stream. If the stream never ends, that's quite possibly
the problem...
Click to expand...
Hi Jon,
That then is probably the issue, the stream indeed stays open. Can you
suggest a scheme that would enable me to send an XMLDocument to a stream and
receive it into new document? If possible can one send an EOF and have the
"Load" terminate?
You mentioned in another thread that you may have seen the strange bytes
that are at the head of the stream before, did you discover any information
about that?
Thanks so much for your help,
Sid.
 
Sid Price said:
That then is probably the issue, the stream indeed stays open. Can you
suggest a scheme that would enable me to send an XMLDocument to a stream and
receive it into new document? If possible can one send an EOF and have the
"Load" terminate?
Click to expand...

I'm not really sure what you mean. The reader would have to notice that
it's read a complete document, and stop at that point. What exactly is
the protocol you're using here?
You mentioned in another thread that you may have seen the strange bytes
that are at the head of the stream before, did you discover any information
about that?
Click to expand...

The bytes are almost certainly a UTF-8 byte-order mark - nothing
particularly odd about that.

However, I *think* I've seen an issue where if only part of a byte-
order mark is read in the first read, things can get screwy. I haven't
tried to reproduce that though, and it sounds like it isn't your issue.
 
Jon Skeet said:
I'm not really sure what you mean. The reader would have to notice that
it's read a complete document, and stop at that point. What exactly is
the protocol you're using here?
Click to expand...
I am not using any wrapper protocol around the XML document. The sender
writes the document to the stream and the reader reads it from the stream,
expecting only the XML document. So, if I understand correctly you are
saying that I need to add a protocol layer between the sender and the stream
that delimits the XML document, is that correct?
Thanks,
Sid.
 
Sid Price said:
I am not using any wrapper protocol around the XML document.
Click to expand...

No, in terms of the network connection. Are you talking to a web
server, an IRC server, a telnet server, an ssh server, etc? What's the
protocol of the network?
The sender
writes the document to the stream and the reader reads it from the stream,
expecting only the XML document. So, if I understand correctly you are
saying that I need to add a protocol layer between the sender and the stream
that delimits the XML document, is that correct?
Click to expand...

Pretty much. I find the best way is to get the sender to prefix the
actual data with how many bytes of data they're going to send. The
receiver then knows when they're "done" just from the amount of data
received, without having to parse it until it's all ready.
 
Jon Skeet said:
No, in terms of the network connection. Are you talking to a web
server, an IRC server, a telnet server, an ssh server, etc? What's the
protocol of the network?
Click to expand...

The protocol being used is TCP/IP, the server needs to send the XML to each
client that is logged in.
Pretty much. I find the best way is to get the sender to prefix the
actual data with how many bytes of data they're going to send. The
receiver then knows when they're "done" just from the amount of data
received, without having to parse it until it's all ready.
Click to expand...
That's kind of where I was thinking I needed to go. Since I am generating
the class libray for both the client and server I can prepend the byte
count, read the stream into memory and when I have a full document load it
into into an XMLDocument object for the user. Many thanks for helping think
this through,
Sid.
 
Sid Price said:
That's kind of where I was thinking I needed to go. Since I am generating
the class libray for both the client and server I can prepend the byte
count, read the stream into memory and when I have a full document load it
into into an XMLDocument object for the user. Many thanks for helping
think this through,
Click to expand...
Okay, so I have a new wrapper class that handles the sending of the document
to the stream by prepending the length of the document to the document and a
reader that uses the length to read the document into a memory stream and
then this is loaded into an XMLDocument object. I still have the same issue
of the three "rogue" bytes on the front of the document, these bytes cause
the XMLDocument object to throw an exception that the document does not
contain a root element.
Here is the sender:
'

' Send the passed doc to the passed stream

'

theDoc.Save(theData)

messageLength = theData.Length

'

' write the number of bytes in the message

' to the stream

theBuffer = Array.CreateInstance(GetType(Byte), 4)

'

theBuffer(0) = messageLength And &HFF

theBuffer(1) = (messageLength >> 8) And &HFF

theBuffer(2) = (messageLength >> 16) And &HFF

theBuffer(3) = (messageLength >> 24) And &HFF

theStream.Write(theBuffer, 0, 4)

'

' Now the data

'

theBuffer = Array.CreateInstance(GetType(Byte), theData.Length)

theBuffer = theData.ToArray()

theStream.Write(theBuffer, 0, theBuffer.Length)



And the reader

theBuffer = Array.CreateInstance(GetType(Byte), 4)

theStream.Read(theBuffer, 0, 4)

messageLength = theBuffer(0)

messageLength += (theBuffer(1) << 8)

messageLength += (theBuffer(2) << 16)

messageLength += (theBuffer(3) << 24)

Dim rxBytes As Int32 = 0

theBuffer = Array.CreateInstance(GetType(Byte), messageLength)

While (rxBytes <> messageLength)

rxBytes += theStream.Read(theBuffer, rxBytes, messageLength - rxBytes)

End While

theData.Write(theBuffer, 0, messageLength)

theDoc.Load(theData)

Seems that altough I now have better stream i/o code I still have the same
issue, any ideas?

Thanks,

Sid.
 
Sid Price said:
The protocol being used is TCP/IP, the server needs to send the XML to each
client that is logged in.
Click to expand...

Well, I'm sure there's some protocol involved above that level, even if
it's one you've created yourself (which it sounds like you are).
That's kind of where I was thinking I needed to go. Since I am generating
the class libray for both the client and server I can prepend the byte
count, read the stream into memory and when I have a full document load it
into into an XMLDocument object for the user. Many thanks for helping think
this through,
Click to expand...

Sounds like the way to go. Best of luck.
 
Sid Price said:
Okay, so I have a new wrapper class that handles the sending of the document
to the stream by prepending the length of the document to the document and a
reader that uses the length to read the document into a memory stream and
then this is loaded into an XMLDocument object. I still have the same issue
of the three "rogue" bytes on the front of the document, these bytes cause
the XMLDocument object to throw an exception that the document does not
contain a root element.
Here is the sender:
Click to expand...

<snip>

1) BinaryWriter/BinaryReader and/or BitConverter are your friends :)

2) You're not not rewinding your MemoryStream on the reading side. Use
theData.Position = 0
before passing it to theDoc.Load
 
Back
Top