Record size and other stuff

[Fred H]

Hi

I'm currently working on deepening my understanding
on hard disk drives and file systems, and I have come
accross somethine that I find hard to get any relevant
hits on Google on: Record size.

Now, I know it has something to do with the size of
the chunck of data the OS/file system is asking for
from the disk when reading a file, and the other way
around for writing. My question is best presented as
a list I suppose:

- What is the definition of RS?
- How normal is it for a FS to vary it?
- On what conditions does the FS vary RS, if it does?
- Does "default RS" exist, and can I change it?
- Does the term "RS" mean fairly the same between FSs?

FS = File System and RS = Record size, but I suppose that any of
you that may actually be able to help me out, understood that
perfectly well ;-)

I know that the file system benchmarking tool IOZone checks
the file systems performance over two axis, namely file size and
record size, creating a 3D surface graph hoovering above the
file size / record size "floor", with performance along the z-axis,
so I guess that sort of answers some of my questions. But the
reason I want to know more about record size, is to better understand
the results generated by IOZone.

Slightly on another note, I'd like to ask you guys if you know about
any good web sites that deals with file systems, and their inner
structures and workings.

(The reason I'm trying to expand my horizon on the subject is that I'm
currently bench marking a computer with a circuit "inserted" on the
IDE-cable between the host and device. This circuit has some fifos,
and I'm trying to figure out how important the sizes of these fifos are.
More specificially I'm hoping to find the breaking point in the
performance/fifo size graph, if it exists in the fifo size range I'm
operating in...)

-Fred H, Norway.

 

[Bob Willard]

Hi

I'm currently working on deepening my understanding
on hard disk drives and file systems, and I have come
accross somethine that I find hard to get any relevant
hits on Google on: Record size.

Now, I know it has something to do with the size of
the chunck of data the OS/file system is asking for
from the disk when reading a file, and the other way
around for writing. My question is best presented as
a list I suppose:

- What is the definition of RS?
- How normal is it for a FS to vary it?
- On what conditions does the FS vary RS, if it does?
- Does "default RS" exist, and can I change it?
- Does the term "RS" mean fairly the same between FSs?

FS = File System and RS = Record size, but I suppose that any of
you that may actually be able to help me out, understood that
perfectly well ;-)

I know that the file system benchmarking tool IOZone checks
the file systems performance over two axis, namely file size and
record size, creating a 3D surface graph hoovering above the
file size / record size "floor", with performance along the z-axis,
so I guess that sort of answers some of my questions. But the
reason I want to know more about record size, is to better understand
the results generated by IOZone.

Slightly on another note, I'd like to ask you guys if you know about
any good web sites that deals with file systems, and their inner
structures and workings.

(The reason I'm trying to expand my horizon on the subject is that I'm
currently bench marking a computer with a circuit "inserted" on the
IDE-cable between the host and device. This circuit has some fifos,
and I'm trying to figure out how important the sizes of these fifos are.
More specificially I'm hoping to find the breaking point in the
performance/fifo size graph, if it exists in the fifo size range I'm
operating in...)

-Fred H, Norway.

I think it is typical that a record system (or record management system)
is a layer of software that resides on top of the file system, just as the
file system is layered on top of the logical volume layer which, in turn,
sits atop the physical formatted layer which rests on the unformatted
sea of mag.domains. Some apps access storage through the record system,
some through the file system, and some through the logical volume layer;
and, some utilities exist to manage logical volumes through the physical
formatted layer, and some utilities (perhaps restricted) exist to manage
formatting at the lowest layer. (Note that some common file systems do
incorporate the notion of some flavor of records, but I think of this as
a mere merging of two distinct layers into one software package.)

The IDE bus transports commands and data at, roughly, the physical but
formatted layer. The IDE bus does not see files, and certainly does not
see records. Hence, from the perspective of the IDE bus and its
connection between a host and HDs, there is no such thing as a record
or a record size or other record attributes.

Note that most file systems and record systems support caching of sectors
and/or records -- both read-ahead and write-behind caches. Hence, knowing
statistics on record sizes may be of very little value in predicting the
sizes of read/write data blocks to be seen on the IDE bus.

 

[Rod Speed]

I'm currently working on deepening my understanding
on hard disk drives and file systems, and I have come
accross somethine that I find hard to get any relevant
hits on Google on: Record size.

Now, I know it has something to do with the
size of the chunck of data the OS/file system
is asking for from the disk when reading a
file, and the other way around for writing.

Records are an application level concept,
nothing to do with OS level file structures.

In other words applications impose
record structures on the OS level files.

You sure you arent talking about clusters rather than records ?

My question is best presented as a list I suppose:

- What is the definition of RS?

There is no nice tidy definition except that its
a chunk of data handled by the application. The
concept is clearest with database applications.

- How normal is it for a FS to vary it?

It has nothing to do with the OS level file system at all.
Its a construct that apply above the level of the OS FS.

- On what conditions does the FS vary RS, if it does?

It never does.

- Does "default RS" exist,
Nope.

and can I change it?

Yes, the application can obviously change it,
most obviously with database applications.

- Does the term "RS" mean fairly the same between FSs?

Not a meaningful question.

FS = File System and RS = Record size, but I suppose
that any of you that may actually be able to help me out,
understood that perfectly well ;-)

I know that the file system benchmarking tool IOZone
checks the file systems performance over two axis,
namely file size and record size,

It also uses the term 'transfer size'
which is an entirely different animal.

creating a 3D surface graph hoovering above the file size
/ record size "floor", with performance along the z-axis,
so I guess that sort of answers some of my questions.

Nope. Even the transfer size is an application level concept,
in this case the application being the benchmark.

But the reason I want to know more about record size,
is to better understand the results generated by IOZone.

Its basically the size of the chunk being moved at one time.

Slightly on another note, I'd like to ask you guys if
you know about any good web sites that deals with
file systems, and their inner structures and workings.

(The reason I'm trying to expand my horizon on the subject
is that I'm currently bench marking a computer with a circuit
"inserted" on the IDE-cable between the host and device.

Sounds painful.

This circuit has some fifos, and I'm trying to figure
out how important the sizes of these fifos are.

That obviously varys with the application
thats doing the reading and writing.

More specificially I'm hoping to find the breaking
point in the performance/fifo size graph, if it
exists in the fifo size range I'm operating in...)

There cant be any such animal independant
of the application generating the disk activity.

Thats the reason you see a variety of benchmarks with
hard drives, very different disk activity in different situations.

 

[Fred H]

Thank you both for good answers