Best way to move HUGE data from one folder to another

  • Thread starter Thread starter Seum
  • Start date Start date
S

Seum

I have a folder with about 35GB of data and, after I started
moving file after file, from one column F: to another N:, it seemed very
slow and tedious. I could not select all the files, not even 2 together,
and I thought about renaming the two folders.

I renamed N with O, but it ended up with (O:)(N:).

Any safe and quick way to do this?

TIA
 
I have a folder with about 35GB of data and, after I started
moving file after file, from one column F: to another N:, it seemed very
slow and tedious. I could not select all the files, not even 2 together,
and I thought about renaming the two folders.

I renamed N with O, but it ended up with (O:)(N:).

Any safe and quick way to do this?
Click to expand...

We are not clear about moving files from one "column" to another.

Repost saying what methods you used to copy (e.g. Windows
Copy, XXCOPY, some Linux utility etc.) with sample times per 1 Gb.
 
Seum said:
I have a folder with about 35GB of data and, after I started
moving file after file, from one column F: to another N:, it seemed very
slow and tedious. I could not select all the files, not even 2 together,
and I thought about renaming the two folders.

I renamed N with O, but it ended up with (O:)(N:).

Any safe and quick way to do this?

TIA
Click to expand...

Robocopy, running in file synchronization mode (mirroring).

I use this, to copy one partition to another, with the intent to
preserve every aspect of the file system.

robocopy J:\ E:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb /np /tee /v /log:robocopy_j_to_e3.log

You run that from command prompt, being *very careful* to check
syntax. Why ? Because in that example, E: will be erased, before
J: is transferred. What robocopy will be doing, is trying to mirror
the contents of J:, onto E:. If E: had "extra files" on it, then
it would no longer be a mirror. Robocopy also has other operating
modes besides mirror, but mirror is what I use for the things I do.

By typing the wrong drive letter by accident, I've had the contents
of E: start to be erased, before I could stop the command. I lost
a gigabyte of files, by making a typing mistake.

As far as I know, version XP026 comes with this download. This would be
the minimum version I would consider using. I think version XP027 may
come with Windows 7.

http://technet.microsoft.com/en-us/magazine/2006.11.utilityspotlight.aspx

You can find more info on the various versions, here.

http://en.wikipedia.org/wiki/Robocopy

That is supposed to be a folder copying tool, but seems to work at the
partition level. You could also use it like this:

robocopy J:\some_folder E:\some_other_folder ...

to copy just the "some_folder".

I've never used the GUI for it, and I don't know if the
GUI helps at all, or not. I've basically been using the same
command parameters, for a long time, without changing them. I
just change the drive letters, when copying an entire drive
from one place to another.

Be careful!

Paul
 
I have a folder with about 35GB of data and, after I started
moving file after file, from one column F: to another N:, it seemed very
slow and tedious. I could not select all the files, not even 2 together,
and I thought about renaming the two folders.

I renamed N with O, but it ended up with (O:)(N:).

Any safe and quick way to do this?
Click to expand...

Go download Total Commander--the free version is fully functional,
just nagware.

It makes it very easy to select a whole directory and copy it. I've
done copies in excess of 1tb with it.
 
Don said:
We are not clear about moving files from one "column" to another.

Repost saying what methods you used to copy (e.g. Windows
Copy, XXCOPY, some Linux utility etc.) with sample times per 1 Gb.
Click to expand...


One computer with 2 HDs.
Transfer files from one HD to the other.
 
Seum said:
I have a folder with about 35GB of data and, after I started
moving file after file, from one column F: to another N:, it seemed very
slow and tedious. I could not select all the files, not even 2 together,
and I thought about renaming the two folders.

I renamed N with O, but it ended up with (O:)(N:).

Any safe and quick way to do this?

TIA
Click to expand...

I just hit it. Copy F to N. How could I have missed that. :-(
 
Paul said:
Robocopy, running in file synchronization mode (mirroring).

I use this, to copy one partition to another, with the intent to
preserve every aspect of the file system.

robocopy J:\ E:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb /np /tee /v
/log:robocopy_j_to_e3.log

You run that from command prompt, being *very careful* to check
syntax. Why ? Because in that example, E: will be erased, before
J: is transferred. What robocopy will be doing, is trying to mirror
the contents of J:, onto E:. If E: had "extra files" on it, then
it would no longer be a mirror. Robocopy also has other operating
modes besides mirror, but mirror is what I use for the things I do.

By typing the wrong drive letter by accident, I've had the contents
of E: start to be erased, before I could stop the command. I lost
a gigabyte of files, by making a typing mistake.

As far as I know, version XP026 comes with this download. This would be
the minimum version I would consider using. I think version XP027 may
come with Windows 7.

http://technet.microsoft.com/en-us/magazine/2006.11.utilityspotlight.aspx

You can find more info on the various versions, here.

http://en.wikipedia.org/wiki/Robocopy

That is supposed to be a folder copying tool, but seems to work at the
partition level. You could also use it like this:

robocopy J:\some_folder E:\some_other_folder ...

to copy just the "some_folder".

I've never used the GUI for it, and I don't know if the
GUI helps at all, or not. I've basically been using the same
command parameters, for a long time, without changing them. I
just change the drive letters, when copying an entire drive
from one place to another.

Be careful!

Paul
Click to expand...

Thanks Paul, I just started a copy process by selecting the K: column
and the N: column and copying. Oppss!! I just realized that I want to
move, not just copy. It has started and indicates that it will take over
5 hours.
 
Paul said:
Robocopy, running in file synchronization mode (mirroring).

I use this, to copy one partition to another, with the intent to
preserve every aspect of the file system.

robocopy J:\ E:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb /np /tee /v
/log:robocopy_j_to_e3.log

You run that from command prompt, being *very careful* to check
syntax. Why ? Because in that example, E: will be erased, before
J: is transferred. What robocopy will be doing, is trying to mirror
the contents of J:, onto E:. If E: had "extra files" on it, then
it would no longer be a mirror. Robocopy also has other operating
modes besides mirror, but mirror is what I use for the things I do.

By typing the wrong drive letter by accident, I've had the contents
of E: start to be erased, before I could stop the command. I lost
a gigabyte of files, by making a typing mistake.

As far as I know, version XP026 comes with this download. This would be
the minimum version I would consider using. I think version XP027 may
come with Windows 7.

http://technet.microsoft.com/en-us/magazine/2006.11.utilityspotlight.aspx

You can find more info on the various versions, here.

http://en.wikipedia.org/wiki/Robocopy

That is supposed to be a folder copying tool, but seems to work at the
partition level. You could also use it like this:

robocopy J:\some_folder E:\some_other_folder ...

to copy just the "some_folder".

I've never used the GUI for it, and I don't know if the
GUI helps at all, or not. I've basically been using the same
command parameters, for a long time, without changing them. I
just change the drive letters, when copying an entire drive
from one place to another.

Be careful!

Paul
Click to expand...

OK Paul, I'll take your word for it. My E is already empty.

Here was my first attempt:
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\jvalh>robocopy K:\ N:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb
/np /tee /v /log:robocopy_j_to_e3.log

Log File : C:\Users\jvalh\robocopy_j_to_e3.log

-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------

Started : Sat Aug 13 09:58:16 2011

Source : K:\
Dest : N:\

Files : *.*

Options : *.* /V /TEE /S /E /COPY:DATSO /DCOPY:T /PURGE /MIR /ZB /NP
/R:3 /W:2

------------------------------------------------------------------------------

ERROR : You do not have the Backup and Restore Files user rights.
***** You need these to perform Backup copies (/B or /ZB).

ERROR : Robocopy ran out of memory, exiting.
ERROR : Invalid Parameter #%d : "%s"

ERROR : Invalid Job File, Line #%d :"%s"

Started : %hs

Source %c

Dest %c
Simple Usage :: ROBOCOPY source destination /MIR

source :: Source Directory (drive:\path or
\\server\share\path).
destination :: Destination Dir (drive:\path or
\\server\share\path).
/MIR :: Mirror a complete directory tree.

For more usage information run ROBOCOPY /?


**** /MIR can DELETE files as well as copy them !

C:\Users\jvalh>
================================================
More problems here:

1 2GB memory is not enough for (greedy!) Robocopy.
2 I don't have the authority. This one has problems.
The computer has two users - a big 'un and a small 'un. Big 'un
has all the authority but the user interface is totally
different e.g. no emails at all in T'Bird. Q: Was the big 'un
sleeping?

I hope this doesn't keep you awake at night Paul :-)
 
Seum said:
Paul said:
Robocopy, running in file synchronization mode (mirroring).

I use this, to copy one partition to another, with the intent to
preserve every aspect of the file system.

robocopy J:\ E:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb /np /tee /v
/log:robocopy_j_to_e3.log

You run that from command prompt, being *very careful* to check
syntax. Why ? Because in that example, E: will be erased, before
J: is transferred. What robocopy will be doing, is trying to mirror
the contents of J:, onto E:. If E: had "extra files" on it, then
it would no longer be a mirror. Robocopy also has other operating
modes besides mirror, but mirror is what I use for the things I do.

By typing the wrong drive letter by accident, I've had the contents
of E: start to be erased, before I could stop the command. I lost
a gigabyte of files, by making a typing mistake.

As far as I know, version XP026 comes with this download. This would be
the minimum version I would consider using. I think version XP027 may
come with Windows 7.

http://technet.microsoft.com/en-us/magazine/2006.11.utilityspotlight.aspx

You can find more info on the various versions, here.

http://en.wikipedia.org/wiki/Robocopy

That is supposed to be a folder copying tool, but seems to work at the
partition level. You could also use it like this:

robocopy J:\some_folder E:\some_other_folder ...

to copy just the "some_folder".

I've never used the GUI for it, and I don't know if the
GUI helps at all, or not. I've basically been using the same
command parameters, for a long time, without changing them. I
just change the drive letters, when copying an entire drive
from one place to another.

Be careful!

Paul
Click to expand...

OK Paul, I'll take your word for it. My E is already empty.

Here was my first attempt:
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\jvalh>robocopy K:\ N:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb
/np /tee /v /log:robocopy_j_to_e3.log

Log File : C:\Users\jvalh\robocopy_j_to_e3.log

-------------------------------------------------------------------------------

ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------


Started : Sat Aug 13 09:58:16 2011

Source : K:\
Dest : N:\

Files : *.*

Options : *.* /V /TEE /S /E /COPY:DATSO /DCOPY:T /PURGE /MIR /ZB /NP
/R:3 /W:2

------------------------------------------------------------------------------


ERROR : You do not have the Backup and Restore Files user rights.
***** You need these to perform Backup copies (/B or /ZB).

ERROR : Robocopy ran out of memory, exiting.
ERROR : Invalid Parameter #%d : "%s"

ERROR : Invalid Job File, Line #%d :"%s"

Started : %hs

Source %c

Dest %c
Simple Usage :: ROBOCOPY source destination /MIR

source :: Source Directory (drive:\path or
\\server\share\path).
destination :: Destination Dir (drive:\path or
\\server\share\path).
/MIR :: Mirror a complete directory tree.

For more usage information run ROBOCOPY /?


**** /MIR can DELETE files as well as copy them !

C:\Users\jvalh>
================================================
More problems here:

1 2GB memory is not enough for (greedy!) Robocopy.
2 I don't have the authority. This one has problems.
The computer has two users - a big 'un and a small 'un. Big 'un
has all the authority but the user interface is totally
different e.g. no emails at all in T'Bird. Q: Was the big 'un
sleeping?

I hope this doesn't keep you awake at night Paul :-)
Click to expand...

All I can tell you is, it works for me, running as Admin.

Some of those parameters in the command, are to preserve
ownership and permissions on the files, so that the
copy is as exact as possible.

If a "little user", tries to transfer files owned by a
"big user", what would you expect the outcome to be ? The
command has to be run with sufficient permissions, to
wrangle the file ownership properly. You wouldn't expect
a limited user, to be trashing the Administrator's files...

The main advantage of robocopy, is asynchronous operation.
It can be reading one disk, and writing the other disk, at
the same time. It makes maximum usage of any hardware
performance you've got. That's why I like it, versus
some kinda copy and paste. If you transfer thousands of
small files, it works as slow as any other transfer method,
because the transfer speed in that case, is affected by
the rate you can move the disk head around.

Paul
 
One computer with 2 HDs.
Transfer files from one HD to the other.
Click to expand...

I've tried several different programs, including Xcopy, XXcopy and
others, but for use with Win 7, Robocopy seems to work best. In
addition to the fact that it's problem-free, it's advantages are that
it's fast, it's easy to write macros for, and you already have it on
your computer.

If you're still on XP, then probably XXCopy is best.
 
Paul said:
Seum said:
Paul said:
Seum wrote:
I have a folder with about 35GB of data and, after I started
moving file after file, from one column F: to another N:, it seemed
very slow and tedious. I could not select all the files, not even 2
together, and I thought about renaming the two folders.

I renamed N with O, but it ended up with (O:)(N:).

Any safe and quick way to do this?

TIA

Robocopy, running in file synchronization mode (mirroring).

I use this, to copy one partition to another, with the intent to
preserve every aspect of the file system.

robocopy J:\ E:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb /np /tee /v
/log:robocopy_j_to_e3.log

You run that from command prompt, being *very careful* to check
syntax. Why ? Because in that example, E: will be erased, before
J: is transferred. What robocopy will be doing, is trying to mirror
the contents of J:, onto E:. If E: had "extra files" on it, then
it would no longer be a mirror. Robocopy also has other operating
modes besides mirror, but mirror is what I use for the things I do.

By typing the wrong drive letter by accident, I've had the contents
of E: start to be erased, before I could stop the command. I lost
a gigabyte of files, by making a typing mistake.

As far as I know, version XP026 comes with this download. This would be
the minimum version I would consider using. I think version XP027 may
come with Windows 7.

http://technet.microsoft.com/en-us/magazine/2006.11.utilityspotlight.aspx


You can find more info on the various versions, here.

http://en.wikipedia.org/wiki/Robocopy

That is supposed to be a folder copying tool, but seems to work at the
partition level. You could also use it like this:

robocopy J:\some_folder E:\some_other_folder ...

to copy just the "some_folder".

I've never used the GUI for it, and I don't know if the
GUI helps at all, or not. I've basically been using the same
command parameters, for a long time, without changing them. I
just change the drive letters, when copying an entire drive
from one place to another.

Be careful!

Paul
Click to expand...

OK Paul, I'll take your word for it. My E is already empty.

Here was my first attempt:
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\jvalh>robocopy K:\ N:\ /mir /copy:datso /dcopy:t /r:3 /w:2
/zb /np /tee /v /log:robocopy_j_to_e3.log

Log File : C:\Users\jvalh\robocopy_j_to_e3.log

-------------------------------------------------------------------------------

ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------


Started : Sat Aug 13 09:58:16 2011

Source : K:\
Dest : N:\

Files : *.*

Options : *.* /V /TEE /S /E /COPY:DATSO /DCOPY:T /PURGE /MIR /ZB /NP
/R:3 /W:2

------------------------------------------------------------------------------


ERROR : You do not have the Backup and Restore Files user rights.
***** You need these to perform Backup copies (/B or /ZB).

ERROR : Robocopy ran out of memory, exiting.
ERROR : Invalid Parameter #%d : "%s"

ERROR : Invalid Job File, Line #%d :"%s"

Started : %hs

Source %c

Dest %c
Simple Usage :: ROBOCOPY source destination /MIR

source :: Source Directory (drive:\path or
\\server\share\path).
destination :: Destination Dir (drive:\path or
\\server\share\path).
/MIR :: Mirror a complete directory tree.

For more usage information run ROBOCOPY /?


**** /MIR can DELETE files as well as copy them !

C:\Users\jvalh>
================================================
More problems here:

1 2GB memory is not enough for (greedy!) Robocopy.
2 I don't have the authority. This one has problems.
The computer has two users - a big 'un and a small 'un. Big 'un
has all the authority but the user interface is totally
different e.g. no emails at all in T'Bird. Q: Was the big 'un
sleeping?

I hope this doesn't keep you awake at night Paul :-)
Click to expand...

All I can tell you is, it works for me, running as Admin.

Some of those parameters in the command, are to preserve
ownership and permissions on the files, so that the
copy is as exact as possible.

If a "little user", tries to transfer files owned by a
"big user", what would you expect the outcome to be ? The
command has to be run with sufficient permissions, to
wrangle the file ownership properly. You wouldn't expect
a limited user, to be trashing the Administrator's files...

The main advantage of robocopy, is asynchronous operation.
It can be reading one disk, and writing the other disk, at
the same time. It makes maximum usage of any hardware
performance you've got. That's why I like it, versus
some kinda copy and paste. If you transfer thousands of
small files, it works as slow as any other transfer method,
because the transfer speed in that case, is affected by
the rate you can move the disk head around.

Paul
Click to expand...


Hello again Paul,

I have managed to clear the Seagate drive completely. I can't format it
using Win 7, so I will try to do it in the "new" computer with my Win2K CD.

Now I am sensing a desert because I have not needed to get down this far
into situations like this for a long time.

The things I think I need to do are:

Install the empty Seagate drive in the "new" computer.

Insert:
A floppy disk with:

NTLDR
NTDETECT.COM
and
BOOT.ini
[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(0)partition(1)\WINNT
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINNT="Windows 2000 Server" /fastdetect

Any others missing?
----------------------------

Other programs I downloaded:

ACPI_104321537.zip
BIOS-Bupdater_V114.zip
E5888_M4A88TD-V-EVO-USB-V2.zip
M4A88TD-SERIES-MEM-QVL.zip
M4A88TD-V-EVO-USB3-BIOS-1015.zip

The BIOS needs to be set back timewise to accomodate Win2K.

The others I'll have to study.

Hope you're enjoying your weekend :-)
 
Loren said:
Go download Total Commander--the free version is fully functional,
just nagware.

It makes it very easy to select a whole directory and copy it. I've
done copies in excess of 1tb with it.
Click to expand...

Thanks Loren

That is a good idea.
 
Paul said:
Seum said:
Paul said:
Seum wrote:
I have a folder with about 35GB of data and, after I started
moving file after file, from one column F: to another N:, it seemed
very slow and tedious. I could not select all the files, not even 2
together, and I thought about renaming the two folders.

I renamed N with O, but it ended up with (O:)(N:).

Any safe and quick way to do this?

TIA

Robocopy, running in file synchronization mode (mirroring).

I use this, to copy one partition to another, with the intent to
preserve every aspect of the file system.

robocopy J:\ E:\ /mir /copy:datso /dcopy:t /r:3 /w:2 /zb /np /tee /v
/log:robocopy_j_to_e3.log

You run that from command prompt, being *very careful* to check
syntax. Why ? Because in that example, E: will be erased, before
J: is transferred. What robocopy will be doing, is trying to mirror
the contents of J:, onto E:. If E: had "extra files" on it, then
it would no longer be a mirror. Robocopy also has other operating
modes besides mirror, but mirror is what I use for the things I do.

By typing the wrong drive letter by accident, I've had the contents
of E: start to be erased, before I could stop the command. I lost
a gigabyte of files, by making a typing mistake.

As far as I know, version XP026 comes with this download. This would be
the minimum version I would consider using. I think version XP027 may
come with Windows 7.

http://technet.microsoft.com/en-us/magazine/2006.11.utilityspotlight.aspx


You can find more info on the various versions, here.

http://en.wikipedia.org/wiki/Robocopy

That is supposed to be a folder copying tool, but seems to work at the
partition level. You could also use it like this:

robocopy J:\some_folder E:\some_other_folder ...

to copy just the "some_folder".

I've never used the GUI for it, and I don't know if the
GUI helps at all, or not. I've basically been using the same
command parameters, for a long time, without changing them. I
just change the drive letters, when copying an entire drive
from one place to another.

Be careful!

Paul
Click to expand...

OK Paul, I'll take your word for it. My E is already empty.

Here was my first attempt:
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\jvalh>robocopy K:\ N:\ /mir /copy:datso /dcopy:t /r:3 /w:2
/zb /np /tee /v /log:robocopy_j_to_e3.log

Log File : C:\Users\jvalh\robocopy_j_to_e3.log

-------------------------------------------------------------------------------

ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------


Started : Sat Aug 13 09:58:16 2011

Source : K:\
Dest : N:\

Files : *.*

Options : *.* /V /TEE /S /E /COPY:DATSO /DCOPY:T /PURGE /MIR /ZB /NP
/R:3 /W:2

------------------------------------------------------------------------------


ERROR : You do not have the Backup and Restore Files user rights.
***** You need these to perform Backup copies (/B or /ZB).

ERROR : Robocopy ran out of memory, exiting.
ERROR : Invalid Parameter #%d : "%s"

ERROR : Invalid Job File, Line #%d :"%s"

Started : %hs

Source %c

Dest %c
Simple Usage :: ROBOCOPY source destination /MIR

source :: Source Directory (drive:\path or
\\server\share\path).
destination :: Destination Dir (drive:\path or
\\server\share\path).
/MIR :: Mirror a complete directory tree.

For more usage information run ROBOCOPY /?


**** /MIR can DELETE files as well as copy them !

C:\Users\jvalh>
================================================
More problems here:

1 2GB memory is not enough for (greedy!) Robocopy.
2 I don't have the authority. This one has problems.
The computer has two users - a big 'un and a small 'un. Big 'un
has all the authority but the user interface is totally
different e.g. no emails at all in T'Bird. Q: Was the big 'un
sleeping?

I hope this doesn't keep you awake at night Paul :-)
Click to expand...

All I can tell you is, it works for me, running as Admin.

Some of those parameters in the command, are to preserve
ownership and permissions on the files, so that the
copy is as exact as possible.
Click to expand...

I understand that.
If a "little user", tries to transfer files owned by a
"big user", what would you expect the outcome to be ? The
command has to be run with sufficient permissions, to
wrangle the file ownership properly. You wouldn't expect
a limited user, to be trashing the Administrator's files...
Click to expand...

Again agreed :-)
The main advantage of robocopy, is asynchronous operation.
It can be reading one disk, and writing the other disk, at
the same time. It makes maximum usage of any hardware
performance you've got. That's why I like it, versus
some kinda copy and paste. If you transfer thousands of
small files, it works as slow as any other transfer method,
because the transfer speed in that case, is affected by
the rate you can move the disk head around.
Click to expand...

I would have used robocopy if I had enough memory. I had thought about
buying another 2GB chip but, since I never before needed even 2GB, I
decided to use it. Next order will include that extra 2GB.
Paul
Click to expand...

Thanks again :-)

I have already posted info about the Seagate being cleared completely
and it is now in the "new" box.
 
I have a folder with about 35GB of data and, after I started
moving file after file, from one column F: to another N:, it seemed very
slow and tedious. I could not select all the files, not even 2 together,
and I thought about renaming the two folders.

I renamed N with O, but it ended up with (O:)(N:).

Any safe and quick way to do this?
Click to expand...


May be process handlers linked to N:, in which case it won't rename.
Should be getting up to 50Kbytes a sec. between hard drives, 10-12KBs
between HD and DVD, 20-30 with USB flashmem. I don't know, 15 or 20
seconds a GB or 5 minutes or so to copy 30G. It's downhill without
being optimized for that, roughly half between same-drive partitions.
They'll also thrash and churn without defragmented files. All part of
laying out a system for least resistance beforehand.
 
Back
Top