Instance object directly to disk not to memory

  • Thread starter Thread starter thomas.w
  • Start date Start date
T

thomas.w

Hi,

Is it possible to create an instance of an obect and directly assign
it to disk and not to memory.

e.g


'The code below creates a string array and loads it with 100 string
elements.

Dim o(100) As String

For i As Integer = 0 To 99

o(i) = "String No:" & i

Next

'This "o" object is now stored in memory.

Can you instance the "Dim o(100) as String" object directly to disk?

I realise there are serialization methods that can write the entire
object to disk after it has been created, but I want to have the
ability to incrementally update data in large objects without moving
it in and out of memory.
 
(e-mail address removed) wrote in @b2g2000hsg.googlegroups.com:
Hi,

Is it possible to create an instance of an obect and directly assign
it to disk and not to memory.
Click to expand...

No, you cannot instantiate to disk.
I realise there are serialization methods that can write the entire
object to disk after it has been created, but I want to have the
ability to incrementally update data in large objects without moving
it in and out of memory.
Click to expand...

If you need to incrementally edit large files on disk, take a look at
System.IO - the stream classes allow you to edit files directly on disk.
 
(e-mail address removed) wrote in @b2g2000hsg.googlegroups.com:



No, you cannot instantiate to disk.


If you need to incrementally edit large files on disk, take a look at
System.IO - the stream classes allow you to edit files directly on disk.
Click to expand...

Thanks,

I've tried to use that methodology to asume the file is like an
"object", with locations in the file being like "Properties" of an
object.

Although when I used the stream writers to edit the "object" (file),
they only allow you to write (NOT INSERT) text or buffer at a specific
position, which then overwrites the remainder of the files text.


e.g

-A file has this data

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

(total length 111)

When you open the stream and then set the postion to say 20, and
write("bbbb")

aaaaaaaaaaaaaaaaaaaabbbaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

(total length 111)
 
Can you tell us just how large your 'large objects' are?


(e-mail address removed) wrote in @b2g2000hsg.googlegroups.com:



No, you cannot instantiate to disk.


If you need to incrementally edit large files on disk, take a look at
System.IO - the stream classes allow you to edit files directly on disk.
Click to expand...

Thanks,

I've tried to use that methodology to asume the file is like an
"object", with locations in the file being like "Properties" of an
object.

Although when I used the stream writers to edit the "object" (file),
they only allow you to write (NOT INSERT) text or buffer at a specific
position, which then overwrites the remainder of the files text.


e.g

-A file has this data

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

(total length 111)

When you open the stream and then set the postion to say 20, and
write("bbbb")

aaaaaaaaaaaaaaaaaaaabbbaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

(total length 111)
 
Thomas,

Although string inherits from object is it not what we are normally calling
an instance of an object.

That would be in VB 2008 with option infer On (standard)
\\\
Dim o(99) as object
for i = 0 to 99
o = ("String No:" & i.ToString)
next
///

That you call o an object in your message does not mean anything

In Dim o(100) as String is an array of Strings which contains 101 strings (a
crazy thing from VB)

You can write this simple to disk

In VB for Net in all versions
\\\
Dim fs As System.IO.FileStream = System.IO.File.Create(YourFullDiskPath)
Dim writer As New System.IO.StreamWriter(fs)
For i As Integer = 1 To 100
writer.WriteLine("String No:" & i.ToString)
Next
writer.Close()
///

Cor
 
You can't "insert" in a file (for example new "records" are generally
inserted at the end of the file or where there is available space reserved
for this inside the file).

You could expose your file content using a n object programming model.

Not sure what you are trying to do but depending on what exactly are those
data you could perhaps use a DB Management software. IMO your ebst bet would
to to explain first what youa re trying to do (so that someone could perhaps
suggest another approach to what you are currently trying to do).

--
Patrice

<[email protected]> a écrit dans le message de (e-mail address removed)...
(e-mail address removed) wrote in @b2g2000hsg.googlegroups.com:



No, you cannot instantiate to disk.


If you need to incrementally edit large files on disk, take a look at
System.IO - the stream classes allow you to edit files directly on disk.
Click to expand...

Thanks,

I've tried to use that methodology to asume the file is like an
"object", with locations in the file being like "Properties" of an
object.

Although when I used the stream writers to edit the "object" (file),
they only allow you to write (NOT INSERT) text or buffer at a specific
position, which then overwrites the remainder of the files text.


e.g

-A file has this data

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

(total length 111)

When you open the stream and then set the postion to say 20, and
write("bbbb")

aaaaaaaaaaaaaaaaaaaabbbaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

(total length 111)
 
Although when I used the stream writers to edit the "object" (file),
they only allow you to write (NOT INSERT) text or buffer at a specific
position, which then overwrites the remainder of the files text.
Click to expand...

That's the way it's always been, as far as I know. The disk drive head
cannot push the magnetic domains along to stick other data in between.

Andrew
 
You can't "insert" in a file (for example new "records" are generally
inserted at the end of the file or where there is available space reserved
for this inside the file).

You could expose your file content using a n object programming model.

 Not sure what you are trying to do but depending on what exactly are those
data you could perhaps use a DB Management software. IMO your ebst bet would
to to explain first what youa re trying to do (so that someone could perhaps
suggest another approach to what you are currently trying to do).
Click to expand...
My main objective is to try and store numerous query outputs on the
users machine for later use using .Net forms app only.

Filling the .Net dataset (v2) with large amounts of data causes the
application to not respond, even with the BeginLoad set.

Filling an ADO recordset with the same amount of data works ok, but it
can only be serialised using the .Save method to one particular file.
Ive then tried to create a new .net object with an ado recordset as a
property, but this fails by throwing a 'This Object is not
serializable' error when serializing the .net object (I think because
the ado recordset is somehow marked Not Serializable)

I'm wanting to really create something like that of a MS Word, Excel
and Access file that stores all of their information in either .doc,
xls, or .mdb files. If you open a .mdb file with notepad you can see
some tables and their data stored in the proprietery format. Somehow
the MS access.exe effeciently writes to the .mdb file when records are
inserted, updated or deleted from a table. Im really trying to find
out how this can be done using the .net framework.

So far I've tried system.io (as per previous responses)
 
(e-mail address removed) wrote in

My main objective is to try and store numerous query outputs on the
users machine for later use using .Net forms app only.
Click to expand...

You should use a local database ... such a SQL Compact Edition 3.5. SQL CE
has the ability to synchronize data between a remote SQL server and the local
database. Or you can just bulk import the data into the local DB if all you
need is a data cache.
 
(e-mail address removed) wrote in

You should use a local database ... such a SQL Compact Edition 3.5. SQL CE
has the ability to synchronize data between a remote SQL server and the local
database. Or you can just bulk import the data into the local DB if all you
need is a data cache.
Click to expand...
Thanks but I'm really trying only to use a .net forms app with no
other application dependencies. The query data also come from non MS
DBMS's which will make syncronisation too complex. I'm pretty sure
the BULKINSERT & OPENROWSET functions are not availabie in SQL CE.
 
I think it's time to put things in perspective.

It would appear that you are looking for a magic bullet, but, at the same
time, hamstringing yourself by ruling out the most obvious mechanisms.

In addition, you are not giving us much to go on, in that you are
continually referring to these 'large objects' without defining what they
actually are.

Now you are referring to 'query outputs'.

I think you will find it helpfull to describe what these 'query outputs'
actually are and provide a sample of their layout.


You can't "insert" in a file (for example new "records" are generally
inserted at the end of the file or where there is available space reserved
for this inside the file).

You could expose your file content using a n object programming model.

Not sure what you are trying to do but depending on what exactly are those
data you could perhaps use a DB Management software. IMO your ebst bet
would
to to explain first what youa re trying to do (so that someone could
perhaps
suggest another approach to what you are currently trying to do).
Click to expand...
My main objective is to try and store numerous query outputs on the
users machine for later use using .Net forms app only.

Filling the .Net dataset (v2) with large amounts of data causes the
application to not respond, even with the BeginLoad set.

Filling an ADO recordset with the same amount of data works ok, but it
can only be serialised using the .Save method to one particular file.
Ive then tried to create a new .net object with an ado recordset as a
property, but this fails by throwing a 'This Object is not
serializable' error when serializing the .net object (I think because
the ado recordset is somehow marked Not Serializable)

I'm wanting to really create something like that of a MS Word, Excel
and Access file that stores all of their information in either .doc,
xls, or .mdb files. If you open a .mdb file with notepad you can see
some tables and their data stored in the proprietery format. Somehow
the MS access.exe effeciently writes to the .mdb file when records are
inserted, updated or deleted from a table. Im really trying to find
out how this can be done using the .net framework.

So far I've tried system.io (as per previous responses)
 
(e-mail address removed) wrote in @m34g2000hsb.googlegroups.com:
Thanks but I'm really trying only to use a .net forms app with no
other application dependencies. The query data also come from non MS
DBMS's which will make syncronisation too complex. I'm pretty sure
the BULKINSERT & OPENROWSET functions are not availabie in SQL CE.
Click to expand...

True, those functions are not available in SQL CE.

Managing multiple 200MB datasets is not trival - so perhaps building with
the base .NET libraries is a bit unrealistic. You could look at using
another embedded database such as Firebird or Vista DB for your needs.

Another option is an object oriented database to store your objects.

Otherwise you're going to have to invest significant amount of time writing
code to efficiently manage your objects on disk.

P.S. Have you considered loading your datasets when you need them? Or just
purchase more RAM for the machines?
 
You are welcome. Desaware is a company that has been around for a long time,
supplying VB add on's going way back to VB1 or 2. The founder/president, Dan
Appleman has been a long time guru and quit an interesting person. If you
get a chance to look at some of his other stuff, including any of the white
papers he has done - it is well worth it.

good luck on your project,
 
Back
Top