Need RAID?

[DrBubbles]

Hi,
I've just bought 3 x 400GB Western Digital RE2 drives for a new system
for video editing. These are fast drives, and I was wondering about two
things:

1) is it worthwhile interleave them using striped RAID to avoid a
bottleneck in re-reading video data once I've written it to disk?

2) is it worth getting a 4th drive for parity?

I'm not tooooo worried about losing everthing on the drives as it's
also on tape, but at the same time, I'd rather not waste all the time
it takes to edit 30 hours of underwater fish video down to a 1 hr
movie!.

Thanks.
Andrew

NB: Video editing software generally makes partial duplicates of the
original data once you make edits so it can access them quickly;
especially if you use several scenes from a single file. As a result,
if you start with 300GB of movie data (about 20 DV tapes on long play)
you can soon end up with 600GB or 900GB of data if you make lots of
cuts. (imagine what it's like for HDTV! could be 10x this easilly if
someone has been busy with their camera!)

 

[Arno Wagner]

Hi,
I've just bought 3 x 400GB Western Digital RE2 drives for a new system
for video editing. These are fast drives, and I was wondering about two
things:

1) is it worthwhile interleave them using striped RAID to avoid a
bottleneck in re-reading video data once I've written it to disk?

2) is it worth getting a 4th drive for parity?

I'm not tooooo worried about losing everthing on the drives as it's
also on tape, but at the same time, I'd rather not waste all the time
it takes to edit 30 hours of underwater fish video down to a 1 hr
movie!.

NB: Video editing software generally makes partial duplicates of the
original data once you make edits so it can access them quickly;
especially if you use several scenes from a single file. As a result,
if you start with 300GB of movie data (about 20 DV tapes on long play)
you can soon end up with 600GB or 900GB of data if you make lots of
cuts. (imagine what it's like for HDTV! could be 10x this easilly if
someone has been busy with their camera!)

You are confuding things: RAID5 is not just RAID0 plus one more disk.
Theroetically you could do it like that, but RAID0 does not really
make much sense with more than 2 disks today (if you find a controller
that can do it...). RAID5 beginns to make sense with 3 and more disks.

If you are concerned about loosing data, then you should use
the RAID5 variant. Not quite as fast, but with a good hardware
controller (3ware has a good name) or a good software RAID
implementatation (The one in Linux is good. Don't now whther
XP even has one.) you should still get very good disk performance.

Arno

 

[J. Clarke]

You are confuding things: RAID5 is not just RAID0 plus one more disk.
Theroetically you could do it like that, but RAID0 does not really
make much sense with more than 2 disks today (if you find a controller
that can do it...). RAID5 beginns to make sense with 3 and more disks.

If you are concerned about loosing data, then you should use
the RAID5 variant. Not quite as fast, but with a good hardware
controller (3ware has a good name) or a good software RAID
implementatation (The one in Linux is good. Don't now whther
XP even has one.) you should still get very good disk performance.

Wouldn't use either software or 3ware for video editing. One of the Netcell
3000 or 5000 controllers (PNY S-Cure, XFX Revo54, etc) would be a better
bet if one _must_ have parity--they aren't terribly expensive as such
things go and excel at long sequential transfers, which are the norm in
video editing. The 3Ware RAID controllers do well in transaction
processing but less well with long sequential writes. Any software RAID 5
will be contending with the image processing routines for CPU cycles, which
is an undesirable state of affairs.

Note that Netcell is promising to have PCI Express chips out "real soon
now", might be worth waiting for.

 

[J. Clarke]

Wouldn't use either software or 3ware for video editing. One of the
Netcell 3000 or 5000 controllers (PNY S-Cure, XFX Revo54

That should be Revo64, not _5_4.

 

[DrBubbles]

Thanks for the advice. I do want to avoid bogging the cpu down with
software RAID. The motherboard is a MSI k8n diamond which has built in
RAID, but I'm not sure if this one is considered good.
I will give this a try whilst I wait for the PCI Express chips from
Netcell if the onboard is not so good.

I am look forward to scanning through hours of video with quick updates
on screen to show where I am. This way I can find the worthwhile parts
and edit them down to something I can reasonably expect friends and
family to sit through without getting completely "fished out" (although
if it was just for me, hours of fish and coral would be fine

Many thanks,
Andrew

 

[Arno Wagner]

Arno Wagner wrote:

Wouldn't use either software or 3ware for video editing. One of the Netcell
3000 or 5000 controllers (PNY S-Cure, XFX Revo54, etc) would be a better
bet if one _must_ have parity--they aren't terribly expensive as such
things go and excel at long sequential transfers, which are the norm in
video editing. The 3Ware RAID controllers do well in transaction
processing but less well with long sequential writes. Any software RAID 5
will be contending with the image processing routines for CPU cycles, which
is an undesirable state of affairs.

Well, I don't have any real experience with hardware RAID (except for
Adaptec SATA, which is a really bad product). As for CPU cycles
for RAID5, it does not need that many. It is more I/O bound. But
a really fast RAID controller should do better, also because it
has ways around some of the I/O limits.

Arno

 

[Eric Gisin]

Thanks for the advice. I do want to avoid bogging the cpu down with
software RAID. The motherboard is a MSI k8n diamond which has built in
RAID, but I'm not sure if this one is considered good.
I will give this a try whilst I wait for the PCI Express chips from
Netcell if the onboard is not so good.

RAID 5 uses only a few % CPU. Any RAID 5 will be slow.