Problem synschronizing serialport.read()

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

Hello

I'm trying to read a stream of 4 byte messages in which the
4th byte is the checksum of the first 3 bytes:
Message= [byte#1, byte#2, byte#3, CRC8]

For this purpose I'm using
SerialPort.Read Method (Byte[], Int32, Int32)

(buffer=The byte array to write the input to.
offset=The offset in the buffer array to begin writing.
count=The number of bytes to read. )

to read 1-4 new bytes depending on wheter or not the read
operation is synchronized with the input stream. If the
read operation is unsynchronized, the read operation is shifted
1-3 bytes with respect to the input stream, and then
4 and 4 new bytes are read which is the normal situation when
the read operation is synchronized.

Example
Assume the 4 bytes
[n+1, n+2, n+3, n+4]
is read but the (valid) messages are located at
[n+2, n+3, n+4, n+5][n+6, n+7, n+8, n+9]
Then perform a shift and continue to read 4 and 4 new bytes.

This read algorithm is executed in a worker thread created by the
ThreadPool class.

This all works just fine for a couple of thousand read operations performed
at
9600 baud. In this time maximum 3 byte shifts are performed
to synchronize the read operation, and the read operation stays synchronized.

After a while however something wrong happens:
The read operation gets unsynchronized, and never manages to stay
synchronized again.
It almost seems as if the read operations are performed too slow, even
though the
baud rate is only 9600, and Win CE is running on a 500MHz CPU in a program
that
does nothing else than displaying the data read from the serial port in a
Windows
Form.

I first thought this was the case, and then called
serialPort.DiscardInBuffer()
frequently to ensure that the receive buffer did not experience a buffer
overflow.
However the same thing happened.

What could be the reason for this?

Is there any reason reading 4 and 4 bytes using SerialPort.Read Method
(Byte[], Int32, Int32)
should be (considerably) slower than reading 1 and 1 byte?
 
I've done some more testing and made the following discoveries:

1) Everything works just fine (=the read operation stays synchronized) as
long as the data read using
"_serialport.Read(SomeByteArraySize4, 0, 4) "
are read from the serial port's receive buffer which has a default size of
4096.

This is the case when there is a time delay between "_serialport.Open()"-
which
as far as I understand causes the serial port to start reading and fills up
the internal receive buffer with the data read to get a kickstart when
_serialport.Read() eventually is called- and
"_serialport.Read(SomeByteArraySize4, 0, 4)" if
"_serialport.DiscardInBuffer()" is not called to empty the already read data.


2) Everything goes hangwire when the read loop catches up, empties the
serial port receive buffer and starts reading data 'real time' as new data
are fed to the
serial port with a rate of 9600 bps.
After studying the bytes read by "_serialport.Read(SomeByteArraySize4, 0,
4) " it
seems that even though 4 NEW bytes should be read every time, once in a
while the same bytes are read over again, causing false messages and
unnecessary byte shifts.

3) The read loop is not at all executed too slow, but on the contrary it
seems to be executed too fast. When adding a little sleep time in between
every new read so that the serial port manages to buffer a few bytes in its
receiver buffer before
"_serialport.Read(SomeByteArraySize4, 0, 4) " is executed, everything works
just fine again.

while (continue)
{
Thread.Sleep(sleepTimeInMS);
_serialport.Read(SomeByteArraySize4, 0, 4);

// check if message is valid, and perform byte shifts if not to
// synchronize read
}

Is there any reason that Serialport.Read() should go hangwire just because
it has to
perform 'real time'?
What am I missing here?
 
I'm completely confused. Why would you assume *anything* about how long it
takes for bytes to arrive, how many will be in the buffer when you call
read, how many were read when it returns, etc.? That's just *wrong*.

If you need to check the checksum of every fourth byte, then you should try
to read some bytes and there perform the checksum for every group of four
that you got.

Paul T.
 
Paul G. Tobey said:
I'm completely confused. Why would you assume *anything* about how long it
takes for bytes to arrive, how many will be in the buffer when you call
read, how many were read when it returns, etc.? That's just *wrong*.
Click to expand...

To my knowledge I'm not assuming anything about this. Could you please specify
where you mean I'm making these assumptions?
My only assumption is that _serailport.Read(someBufferSized4, 0, 4) reads 4
NEW bytes.
If you need to check the checksum of every fourth byte, then you should try
to read some bytes and there perform the checksum for every group of four
that you got.

Paul T.
Click to expand...

This is exactly what I've been trying to describe:
I read 4 bytes and compute the checksum to make sure that I'm receiving a
correct message, then I read another 4 and computes the checksum againg and
so forth. But in addition I store the last 4 bytes that was read in case the
read operation is unsynchronized (checksum test fails), so that I can shift
1-3 bytes to get synchronized again.
 
It will read four new bytes, unless your serial port driver is goofed up in
some way. Note, however, that it may return *less than four bytes*, too,
and that is completely valid. You have to make sure that you are not
assuming something about how many bytes are coming back (if you ask for
four, you might get any number between 0 and 4).

Paul T.
 
Thank you. That explains everything.
Is there a way to ensure that you actually get 4 new bytes when you ask for 4?
Or 1 new if you ask for 1 for that sake?
 
You should never, for performance reasons, be asking for very small numbers
of bytes. I'm not a .NET CF serial class expert, so I don't know how you'd
go about turning off time-outs, but that would be required to do what you
describe.

General serial operations, however, should *never* assume that any given
number of bytes will come back and code should be written to handle all of
the cases. For your case, you can easily code things to check and remove
the CRC bytes. All you have to do is keep track of how many total bytes
have been received so far.

Paul T.
 
Can the return value of Serialport.Read() be used for this purpose?
The returned int of SerialPort.Read (byte[] buffer, int offset, int count)
is according to MSDN "The number of bytes read".
Given that you ask for 4 bytes, but that only 2 new bytes are read. Will
then Serialport.Read() then return 2?
 
Hi,

Here is what I do:

I use the BytesToRead property to detemine how many bytes are available. If
the number is equal to (or greater than) the number of bytes that I want,
then I call Read specifying the number of bytes to be read. Otherwise...
Just exit.

BTW, I recommend that you always keep ReceiveThreshold at the default (1).
Any other value can lead to hard to debug problems -- though there always is
a solution, that solution may be less than satisfactory.

Dick

--
Richard Grier, MVP
Hard & Software
Author of Visual Basic Programmer's Guide to Serial Communications, Fourth
Edition,
ISBN 1-890422-28-2 (391 pages, includes CD-ROM). July 2004, Revised March
2006.
See www.hardandsoftware.net for details and contact information.
 
Forgot that I've already tested this.
The answer is NO. In this case 4 will be returned and not 2 which one might
expect. This is the reason my algorithm fails. (And this also makes me
wonder if the return value of Serialport.Read() can be used for anything
sensible at all?)

So, let me ask again, what can be done to determine the number of NEW bytes
read?

Looking forward to finally get an answer to this question.
 
The underlying OS calls *do* return the number of bytes returned to the
caller, so, if the .NET CF code is returning something that doesn't
correspond to the number of bytes, it sounds like a bug in .NET CF or in the
serial driver that you're using. You could always look at the source code
from OpenNETCF SDF 1.x for how to wrap CreateFile(), SetCommState(),
SetCommTimeouts(), ReadFile(), and WriteFile() into basically the same
interface as what the .NET CF 2.0 does, and to do it yourself.

Paul T.

GT said:
Forgot that I've already tested this.
The answer is NO. In this case 4 will be returned and not 2 which one
might
expect. This is the reason my algorithm fails. (And this also makes me
wonder if the return value of Serialport.Read() can be used for anything
sensible at all?)

So, let me ask again, what can be done to determine the number of NEW
bytes
read?

Looking forward to finally get an answer to this question.

GT said:
Can the return value of Serialport.Read() be used for this purpose?
The returned int of SerialPort.Read (byte[] buffer, int offset, int
count)
is according to MSDN "The number of bytes read".
Given that you ask for 4 bytes, but that only 2 new bytes are read. Will
then Serialport.Read() then return 2?
Click to expand...
Click to expand...
 
It has been my experience that Serialport.Read() returns the number of bytes
read and BytesToRead gives the number of bytes available to read. If I were
you, I'd pay attention to what Dick Grier has to say about it. The method he
describes works fine for me.
Jim

GT said:
Forgot that I've already tested this.
The answer is NO. In this case 4 will be returned and not 2 which one might
expect. This is the reason my algorithm fails. (And this also makes me
wonder if the return value of Serialport.Read() can be used for anything
sensible at all?)

So, let me ask again, what can be done to determine the number of NEW bytes
read?

Looking forward to finally get an answer to this question.

GT said:
Can the return value of Serialport.Read() be used for this purpose?
The returned int of SerialPort.Read (byte[] buffer, int offset, int count)
is according to MSDN "The number of bytes read".
Given that you ask for 4 bytes, but that only 2 new bytes are read. Will
then Serialport.Read() then return 2?
Click to expand...
Click to expand...
 
Ok, that I'll do.
Thank you very much guys.

Jim McGhee said:
It has been my experience that Serialport.Read() returns the number of bytes
read and BytesToRead gives the number of bytes available to read. If I were
you, I'd pay attention to what Dick Grier has to say about it. The method he
describes works fine for me.
Jim

GT said:
Forgot that I've already tested this.
The answer is NO. In this case 4 will be returned and not 2 which one might
expect. This is the reason my algorithm fails. (And this also makes me
wonder if the return value of Serialport.Read() can be used for anything
sensible at all?)

So, let me ask again, what can be done to determine the number of NEW bytes
read?

Looking forward to finally get an answer to this question.

GT said:
Can the return value of Serialport.Read() be used for this purpose?
The returned int of SerialPort.Read (byte[] buffer, int offset, int count)
is according to MSDN "The number of bytes read".
Given that you ask for 4 bytes, but that only 2 new bytes are read. Will
then Serialport.Read() then return 2?
Click to expand...
Click to expand...
Click to expand...
 
One more thing.
I've just tested the Serialport.BytesToRead property with the following code
(see below), which does nothing else than open the serial port and store
BytesToRead if
there are 4 or more bytes in the receive buffer. Nothing else is performed.

The result of this is that with the default size of the receive buffer being
4096
BytesToRead are first stored when BytesToRead equals 273.
Meaning that if you try something like this:

if (_serialPort.BytesToRead >= 4)
{
k=_serialport.Read( someBufferSized4, 0, 4);
}

then the read operation will never be performed unless 273 bytes already
have been read and stored in the serial port receive buffer.
But I want to read 4 NEW bytes immediately, not wait for another 269 to be
read and then start reading.
Any ideas of how to solve this problem?

******* Serialport.BytesToRead property test:************
_serialport.Open();

for (int i = 0; i < iNumberOfIterations; i++)
{
if (_serialPort.BytesToRead >= 4)
{
BytesToReadLogArray = _serialPort.BytesToRead;
}
}

//Result: BytesToReadLogArray=[0,....,0, 273,...,273]
***************************


GT said:
Ok, that I'll do.
Thank you very much guys.

Jim McGhee said:
It has been my experience that Serialport.Read() returns the number of bytes
read and BytesToRead gives the number of bytes available to read. If I were
you, I'd pay attention to what Dick Grier has to say about it. The method he
describes works fine for me.
Jim

GT said:
Forgot that I've already tested this.
The answer is NO. In this case 4 will be returned and not 2 which one might
expect. This is the reason my algorithm fails. (And this also makes me
wonder if the return value of Serialport.Read() can be used for anything
sensible at all?)

So, let me ask again, what can be done to determine the number of NEW bytes
read?

Looking forward to finally get an answer to this question.

:

Can the return value of Serialport.Read() be used for this purpose?
The returned int of SerialPort.Read (byte[] buffer, int offset, int count)
is according to MSDN "The number of bytes read".
Given that you ask for 4 bytes, but that only 2 new bytes are read. Will
then Serialport.Read() then return 2?
Click to expand...
Click to expand...
Click to expand...
 
This I presume is caused by the DataReceived event which fires only when a
multiple of 273 new bytes have been added to the receive buffer.
This despite the fact that Serialport.ReceivedBytesThreshold =1.

How can this be?
And is it possible to change so that the DataReceived event fires for every
4 new bytes received?
 
There's either something really wrong with the CF port code or with how
you've got it set up. The behaviors you're seeing just make no sense. You
might try getting the OpenNETCF SDF 1.4 code[1], which has a
interface-matching set of serial port classes and see what they're doing.
That gives you a chance to debug right up to the driver call and return to
see where things are going south.

[1]www.opennetcf.org/sdf
--
Chris Tacke
OpenNETCF Consulting
Managed Code in the Embedded World
www.opennetcf.com
--


GT said:
This I presume is caused by the DataReceived event which fires only when a
multiple of 273 new bytes have been added to the receive buffer.
This despite the fact that Serialport.ReceivedBytesThreshold =1.

How can this be?
And is it possible to change so that the DataReceived event fires for
every
4 new bytes received?


GT said:
One more thing.
I've just tested the Serialport.BytesToRead property with the following
code
(see below), which does nothing else than open the serial port and store
BytesToRead if
there are 4 or more bytes in the receive buffer. Nothing else is
performed.

The result of this is that with the default size of the receive buffer
being
4096
BytesToRead are first stored when BytesToRead equals 273.
Meaning that if you try something like this:

if (_serialPort.BytesToRead >= 4)
{
k=_serialport.Read( someBufferSized4, 0, 4);
}

then the read operation will never be performed unless 273 bytes already
have been read and stored in the serial port receive buffer.
But I want to read 4 NEW bytes immediately, not wait for another 269 to
be
read and then start reading.
Any ideas of how to solve this problem?

******* Serialport.BytesToRead property test:************
_serialport.Open();

for (int i = 0; i < iNumberOfIterations; i++)
{
if (_serialPort.BytesToRead >= 4)
{
BytesToReadLogArray = _serialPort.BytesToRead;
}
}

//Result: BytesToReadLogArray=[0,....,0, 273,...,273]
***************************
Click to expand...
Click to expand...
 
OK, I think I see where your problem is. The DataReceived event fires
whenever it feels like it! When it fires it might have 1 byte or 273 bytes
or any number of bytes.
When it does fire, you need to process all the bytes the buffer contains
because you have no way of knowing when it will fire again. The odds a
buffer will end on one of your 4 byte boundries is... 1 in 4 so you need to
save the last 1 to 3 bytes to be appended to the next time it fires.
Hope this helps,
Jim

GT said:
This I presume is caused by the DataReceived event which fires only when a
multiple of 273 new bytes have been added to the receive buffer.
This despite the fact that Serialport.ReceivedBytesThreshold =1.

How can this be?
And is it possible to change so that the DataReceived event fires for every
4 new bytes received?


GT said:
One more thing.
I've just tested the Serialport.BytesToRead property with the following code
(see below), which does nothing else than open the serial port and store
BytesToRead if
there are 4 or more bytes in the receive buffer. Nothing else is performed.

The result of this is that with the default size of the receive buffer being
4096
BytesToRead are first stored when BytesToRead equals 273.
Meaning that if you try something like this:

if (_serialPort.BytesToRead >= 4)
{
k=_serialport.Read( someBufferSized4, 0, 4);
}

then the read operation will never be performed unless 273 bytes already
have been read and stored in the serial port receive buffer.
But I want to read 4 NEW bytes immediately, not wait for another 269 to be
read and then start reading.
Any ideas of how to solve this problem?

******* Serialport.BytesToRead property test:************
_serialport.Open();

for (int i = 0; i < iNumberOfIterations; i++)
{
if (_serialPort.BytesToRead >= 4)
{
BytesToReadLogArray = _serialPort.BytesToRead;
}
}

//Result: BytesToReadLogArray=[0,....,0, 273,...,273]
***************************
Click to expand...
Click to expand...
 
Something odd is happening, I think. I have never seen the problem that you
describe, and I've use Read the way that I described to implement an
error-checked file transfer protocol (XMODEM), with no problem.

When data are read, they are cleared, not repeated. You can download
CFSerialIO.dll from my homepage and use the ReadArray method, instead of
Read, to see if you get any different results. CFSerialIO does work
differently than the VS2005 SerialPort Read, though. It returns ALL
available data; you do not specify the number of bytes. So, if there is
more there than you need, you have to retain any extra to pre-pend to any
data from subsequent calls. (InBufferCount returns the number of bytes that
will be returned when ReadArray is called.)

I simply cannot duplicate the results that you are seeing. Perhaps if you
post the complete code in the DataAvailable event, something may become
clear.

Dick

--
Richard Grier, MVP
Hard & Software
Author of Visual Basic Programmer's Guide to Serial Communications, Fourth
Edition,
ISBN 1-890422-28-2 (391 pages, includes CD-ROM). July 2004, Revised March
2006.
See www.hardandsoftware.net for details and contact information.
 
Hi,
then the read operation will never be performed unless 273 bytes already
have been read and stored in the serial port receive buffer.
But I want to read 4 NEW bytes immediately, not wait for another 269 to be
read and then start reading.
Any ideas of how to solve this problem?
<<

Put in some debuging code (don't bother with the Read). E.g., simply
display the number of bytes in the BytesToRead property. You will see that
the DataAvailable event is "chunky." It isn't generated for every byte.
However, you also will see that the actual "chunks" are smaller than you are
seeing.

Here is VB code that I use:

Dim Count As Integer = SerialPort.BytesToRead
Dim Buffer(0 To Count - 1) As Byte

SerialPort.Read(Buffer, 0, Count)

When I use this code, I see "chunks" of, 5 to 25 bytes -- though the actual
number of bytes cannot be really predicted -- they change with the serial
speed, and with other load imposed by my application and other applications
that may be running.

Dick

--
Richard Grier, MVP
Hard & Software
Author of Visual Basic Programmer's Guide to Serial Communications, Fourth
Edition,
ISBN 1-890422-28-2 (391 pages, includes CD-ROM). July 2004, Revised March
2006.
See www.hardandsoftware.net for details and contact information.
 
GT,

Try something like:

int total = 0;
byte[] bytes = new byte[4];

while (total < bytes.Length)
{
total += s.Read (bytes, total, bytes.Length - total);
}

You'll need to throw in error check etc, but this should work just fine
(excluding the error cases).

Hilton
 
Back
Top