Random file access -con't

  • Thread starter Thread starter Wapiti
  • Start date Start date
W

Wapiti

Could someone point me to some examples of random file access using the cf -
vb.net?

I've found plenty of examples for straight text access on msdn and
elsewhere - but I'd like to use random access - using a table structure and
be able to build a system to quickly access records (relative to text files
anyway). I'm currently looking at streamwriter and filestream - not sure
which is my best bet, if any.

I thought I saw an example a long while back while first studying the use of
..net in converting our msvc1.5 based handheld application - but can't find
it/them now. Looking too hard? Or do they not exist?

Are there any good books on building cf vb.net apps for the purposes of data
collection?

side question, it seems to me that handheld devices primary purpose would be
for data collection of some sort, being that they are mostly, now anyway,
not 'connected'. I'm surprised that there isn't more details on the
differing data storage methods. I could be very wrong here and its all
over, but I'm just not finding it.

Mike
 
Well, you could just P/Invoke to the Win32 API functions, CreateFile,
ReadFile, and WriteFile. I don't know if that's the .NET way of doing
things or not...

Paul T.
 
Mike,

Is there a particular reason you aren't considering SqlServer CE? How many
records do you expect the system to handle?
 
There could be 1 - 2 thousand records downloaded to the device (warehouse
parts). But the reason I haven't been considering sqlce is because this
application needs to have the capability of running on different models and
classes of ppc's; some limited in memory. I understand that sqlce has a
huge footprint - if I start requiring that my customers run on high
memory/processor devices they're not going to be happy.

What is the footprint of sqlce? What mem/process is -actually- required?
We've all learned that we can't trust the specs for the products. Real
life, what is needed?

This app only needs to import a flat text file, and export a flat text file,
to the host system. There is no synchronization with another database -this
hhc app is going to be interfacing with a legacy system of which I have no
control over.

Why is it that everything has to be so bleeding edge? Random files work
well for handheld applications. I've been using them for years. Now, it
seems, we need to keep buying more and more powerful devices just to perform
simple tasks, such as offsite data collection. How frustrating.

Please tell me I'm wrong - and that, as an example, I could install and run
sqlce, my cf vb.net app and data onto a default Axim device (as an example)
without a hitch. If there is a hitch, my customers will complain and then
go away - and rightly so. I really do hope I'm wrong, but my reading
suggests otherwise.
 
Can you explain?

Paul G. Tobey said:
Well, you could just P/Invoke to the Win32 API functions, CreateFile,
ReadFile, and WriteFile. I don't know if that's the .NET way of doing
things or not...

Paul T.

use would
Click to expand...
 
Thought I did. Those are the basic Win32 API functions for file access. In
combination with SetFilePointer(), you can read/write from anywhere in an
existing file...

Paul T.
 
Certainly you *can* use direct file I/O to accomplish what you want. Paul's
suggestion of CreateFile/ReadFile/WriteFile would certainly work as would
the .NetCF StreamReader and StreamWriter classes which wrap them, although
neither of these gives you random access - you'd need to code that for
yourself using these functions or classes as primitives.

1-2 thousand records, depending on the size of each record, isn't a large
amount of data for modern devices. And where the amount of data *is* large,
the usual solution is to store the data on a storage card.

I don't know off hand the exact footprint of SqlCe, but that product was
designed for exactly your type of scenario (and also for amounts of data
that are much larger.) What is the absolute minimal hardware that you need
to target?

There are also 3rd party database solutions available.

--
Ginny Caughey
..Net Compact Framework MVP
 
In Platform Builder 4.2, the SQL CE catalog component size is 1.08MB.
That's going to be rough and is probably based on x86. For an ARM device, a
multiplier of 1.3 is pretty reasonable and I've seen that used before. I'd
say a conservative guess for the executable parts of SQL CE on a Pocket PC
is about 1.5MB, then.

Paul T.
 
That sounds reasonable. Thanks, Paul!

--
Ginny Caughey
..Net Compact Framework MVP
 
Ginny Caughey said:
Certainly you *can* use direct file I/O to accomplish what you want. Paul's
suggestion of CreateFile/ReadFile/WriteFile would certainly work as would
the .NetCF StreamReader and StreamWriter classes which wrap them, although
neither of these gives you random access - you'd need to code that for
yourself using these functions or classes as primitives.
Click to expand...

I guess I'll have to do some digging to figure out what you guys are talking
about here. Even tho Paul 'already explained it' - explain to me why, if CF
doesn't support random access, would the basic win32 api functions for file
access in CF be supported?

Something doesn't sound right.

1-2 thousand records, depending on the size of each record, isn't a large
amount of data for modern devices. And where the amount of data *is* large,
the usual solution is to store the data on a storage card.
Click to expand...

Sounds reasonable. A storage card might be the best way of going anyway.

I don't know off hand the exact footprint of SqlCe, but that product was
designed for exactly your type of scenario (and also for amounts of data
that are much larger.) What is the absolute minimal hardware that you need
to target?
Click to expand...

I'm testing on an Axim right now. 'Absolute minimal' is difficult to
define. I guess I'd have to resort to the answer, it depends. But that
would be too vague so, I could set a minimum to my users. They're going to
be collecting data - that data could span 1 day to a weeks worth of data.
The records aren't large by any means, say, 10 fields per record each. its
a warehouse application - Issue, Receipt, Transfer and Inventory
adjustments.

There are also 3rd party database solutions available.
Click to expand...

I've been burned before on these, but if anyone has recommendations, I'd be
open to hearing them. Speed in development, sufficient speed in data access
and a company that will be around for the next year are the important
factors pertaining to a 3rd party solution, at this point.
 
You can call anything in any DLL anywhere on the device, but you have to
write your own declarations. That's what P/Invoke is: Platform Invoke. You
declare the functions which are found in DLLs as externs and then call them,
passing suitable parameters. If you feel like something is missing, then
you haven't been involved in .NET CF development long enough, yet.

Download the OpenNETCF SDF and look through the source code. A huge
percentage of what is done there is working around the lack of managed code
wrappers for Win32 API functions. In fact, CreateFile, WriteFile, ReadFile
and, maybe, SetFilePointer, are in there.

Paul T.
 
That helps, thanks Paul!

Paul G. Tobey said:
You can call anything in any DLL anywhere on the device, but you have to
write your own declarations. That's what P/Invoke is: Platform Invoke. You
declare the functions which are found in DLLs as externs and then call them,
passing suitable parameters. If you feel like something is missing, then
you haven't been involved in .NET CF development long enough, yet.

Download the OpenNETCF SDF and look through the source code. A huge
percentage of what is done there is working around the lack of managed code
wrappers for Win32 API functions. In fact, CreateFile, WriteFile, ReadFile
and, maybe, SetFilePointer, are in there.

Paul T.

if has
Click to expand...
 
Paul G. Tobey said:
If you feel like something is missing, then
you haven't been involved in .NET CF development long enough, yet.
Click to expand...

16 days isn't long enough? ;) so much to learn. I was so comfortable in
my vb6 environment (even in the vc1.5 for the intermec handhelds) - but i do
see the benefits of .net - just frustrating getting there on such a short
production timeline.

Thanks for the help.
 
You can still use C/C++ on Windows CE devices. eMbedded Visual C++ 3.0 and
4.0 are free...

Paul T.
 
I was never much of a C developer ... at least never enjoyed C development
.... I used MSVC v1.5 on the Intermec devices only because it was a
requirment of their SDK.
 
So, with that link in mind, what would anyone think regarding the use of xml
to store, and otherwise manage, data for a data collection application?

The data doesn't have to be synchronized with any host database, but
extracted to a ascii flat text file.

Seems though, that xml will be slow as an ascii stream - isn't it just a big
text file anyway?

I've never used xml - maybe now is the time to try??

Input appreciated.

Mike
 
The main problem with XML with large amounts of data is that it's generally
slow to parse it on mobile devices as well as taking up more space than, for
example, as CSV file. I use text files instead of XML in most cases where I
can.

--
Ginny Caughey
..Net Compact Framework MVP



Wapiti said:
So, with that link in mind, what would anyone think regarding the use of xml
to store, and otherwise manage, data for a data collection application?

The data doesn't have to be synchronized with any host database, but
extracted to a ascii flat text file.

Seems though, that xml will be slow as an ascii stream - isn't it just a big
text file anyway?

I've never used xml - maybe now is the time to try??

Input appreciated.

Mike


Noble said:
"SetFilePointer" is a cool function but it does not provide TRUE random
access like the days of old.

Take a look at this thread:
http://www.xtremevbtalk.com/archive/index.php/t-179063.html

Thanks,
Noble

Invoke.
You why,
if product
was of
data sqlce
has flat
text
Click to expand...
just
to
perform
simple tasks, such as offsite data collection. How frustrating.

Please tell me I'm wrong - and that, as an example, I could install
and
run
sqlce, my cf vb.net app and data onto a default Axim device (as an
example)
without a hitch. If there is a hitch, my customers will
Click to expand...
complain
and
Click to expand...
Click to expand...
 
So, what method do you use then, for accessing your text files? In the
Compact Framework? And how many records are you accessing - do you perform
record(column) modifications with the collection data? If so, how?

Regular text files seem like they would consume a lot of time.


Ginny Caughey said:
The main problem with XML with large amounts of data is that it's generally
slow to parse it on mobile devices as well as taking up more space than, for
example, as CSV file. I use text files instead of XML in most cases where I
can.

--
Ginny Caughey
.Net Compact Framework MVP



Wapiti said:
So, with that link in mind, what would anyone think regarding the use of xml
to store, and otherwise manage, data for a data collection application?

The data doesn't have to be synchronized with any host database, but
extracted to a ascii flat text file.

Seems though, that xml will be slow as an ascii stream - isn't it just a big
text file anyway?

I've never used xml - maybe now is the time to try??

Input appreciated.

Mike


Noble said:
"SetFilePointer" is a cool function but it does not provide TRUE random
access like the days of old.

Take a look at this thread:
http://www.xtremevbtalk.com/archive/index.php/t-179063.html

Thanks,
Noble

"Paul G. Tobey [eMVP]" <ptobey no spam AT no instrument no spam DOT com>
wrote in message You can call anything in any DLL anywhere on the device, but you
Click to expand...
Click to expand...
have
to that
for
Click to expand...
isn't
a that
you
Click to expand...
difficult
to recommendations,
I'd
Click to expand...
which
I just
Click to expand...
(as
Click to expand...
 
I use a text file format similar to CSV. I separate records by CRLF and
fields by the pipe symbol. I read in the whole file when the program loads
and parse the parts into collections of business objects primarily using
StreamReader.ReadLine, String.Split, and various members of the Convert
class. (This is indeed the slowest part, but it's many times faster than
parsing XML and it only happens once when the app loads.) Whenever I need to
add a record to a file, I add it to the memory collection and I also append
a record to the text file itself. (This is fast.) If I need to delete or
modify a record, I use a special code in a new record I append that
indicates that the record is deleted or modified. This approach works well
for reasonably small amounts of data - a few thousand records - but it
requires a desktop app that can play back the log files and update
enterprise data correctly as well as the intelligence in the device app to
correctly interpret the files. But for large amounts of data requiring a
storage card, I would certainly use SqlCe instead rather than trying to keep
all the data in memory all the time.

I think for the volume of data you're expecting that you have a number of
options. I recommend trying several and see what you like, but I don't
recommend XML unless the data volume is very small.

--
Ginny Caughey
..Net Compact Framework MVP



Wapiti said:
So, what method do you use then, for accessing your text files? In the
Compact Framework? And how many records are you accessing - do you perform
record(column) modifications with the collection data? If so, how?

Regular text files seem like they would consume a lot of time.


The main problem with XML with large amounts of data is that it's generally
slow to parse it on mobile devices as well as taking up more space than, for
example, as CSV file. I use text files instead of XML in most cases
Click to expand...
where
I
can.

--
Ginny Caughey
.Net Compact Framework MVP



Wapiti said:
So, with that link in mind, what would anyone think regarding the use
Click to expand...
of
xml
to store, and otherwise manage, data for a data collection application?

The data doesn't have to be synchronized with any host database, but
extracted to a ascii flat text file.

Seems though, that xml will be slow as an ascii stream - isn't it just
Click to expand...
a
big
text file anyway?

I've never used xml - maybe now is the time to try??

Input appreciated.

Mike


"SetFilePointer" is a cool function but it does not provide TRUE random
access like the days of old.

Take a look at this thread:
http://www.xtremevbtalk.com/archive/index.php/t-179063.html

Thanks,
Noble

"Paul G. Tobey [eMVP]" <ptobey no spam AT no instrument no spam DOT com>
wrote in message You can call anything in any DLL anywhere on the device, but you
Click to expand...
have
Click to expand...
Click to expand...
 
Back
Top