System.IO.File.Copy + NTFS Streams + special ACLs

  • Thread starter Thread starter Peter Gibbons
  • Start date Start date
P

Peter Gibbons

Hello,

when I try to copy a file with System.IO.File.Copy I get a
"System.IO.IOException: The operation completed successfully".

File summary information was added to the file with then Windows XP
Explorer file property page. It writes the information to an alternate
NTFS Data Stream.

Additionally the user only has "Create file" permission in the
destination directory because it is used for archiving purposes.

Is it possible to copy only the primary data stream of files with .Net
built in functions to work around this?


Regards,

Peter
 
Peter said:
Hello,

when I try to copy a file with System.IO.File.Copy I get a
"System.IO.IOException: The operation completed successfully".
Click to expand...

How are you copying the file? That sounds like a OVERLAPPED I/O
(IOCP) completion status. You are using some async copy method.
File summary information was added to the file with then Windows XP
Explorer file property page. It writes the information to an alternate
NTFS Data Stream.

Additionally the user only has "Create file" permission in the
destination directory because it is used for archiving purposes.

Is it possible to copy only the primary data stream of files with .Net
built in functions to work around this?
Click to expand...

Good question.

Since streams are supports at the RTL level, I wonder if using a "."
at the end of the file name will do this? I would ask in the
microsoft.public.win32.programmer.kernel fora because I think you can
probably do this via the device file name.

One way for sure:

Create your own "COPYFILE" function opening the main file name,
creating a target and a loop to read/write X KBytes blocks. That for
sure will only give you the main stream.

That is what I do in our p-code compiler where the client has HTML
tagged code (ie, <SCRIPT language="WCBASIC"> and it saves the source
in a server side meta file (xxxx.wcx:wcc) but the main stream
(xxxx.wcx) is the compiled code. When the RTE opens, copies to cache
and runs it. It only sees the compiled stream (xxxx.wcx), not the source.

--
 
Peter said:
Hello,

when I try to copy a file with System.IO.File.Copy I get a
"System.IO.IOException: The operation completed successfully".
Click to expand...

How are you copying the file? That sounds like a OVERLAPPED I/O
(IOCP) completion status. You are using some async copy method.
File summary information was added to the file with then Windows XP
Explorer file property page. It writes the information to an alternate
NTFS Data Stream.

Additionally the user only has "Create file" permission in the
destination directory because it is used for archiving purposes.

Is it possible to copy only the primary data stream of files with .Net
built in functions to work around this?
Click to expand...

Good question.

Since streams are supports at the RTL level, I wonder if using a "."
at the end of the file name will do this? I would ask in the
microsoft.public.win32.programmer.kernel fora because I think you can
probably do this via the device file name.

One way for sure:

Create your own "COPYFILE" function opening the main file name,
creating a target and a loop to read/write X KBytes blocks. That for
sure will only give you the main stream.

That is what I do in our p-code compiler where the client has HTML
tagged code (ie, <SCRIPT language="WCBASIC"> and it saves the source
in a server side meta file (xxxx.wcx:wcc) but the main stream
(xxxx.wcx) is the compiled code. When the RTE opens, copies to cache
and runs it. It only sees the compiled stream (xxxx.wcx), not the source.

--
 
Hello,
How are you copying the file? That sounds like a OVERLAPPED I/O (IOCP)
completion status. You are using some async copy method.
Click to expand...

I use System.IO.File.Copy. It doesn't support overlapped IO. The thrown
exception is documented as "An I/O error has occurred." It's weird and
misleading that it is presented as "The operation completed successfully".
Good question.

Since streams are supports at the RTL level, I wonder if using a "." at
the end of the file name will do this?
Click to expand...

Adding a . doesn't make a difference in this case.
I would ask in the
microsoft.public.win32.programmer.kernel fora because I think you can
probably do this via the device file name.
Click to expand...

I know it can be done with unmanages code / pinvoke but I'd like to
avoid it.
One way for sure:

Create your own "COPYFILE" function opening the main file name, creating
a target and a loop to read/write X KBytes blocks. That for sure will
only give you the main stream.

That is what I do in our p-code compiler where the client has HTML
tagged code (ie, <SCRIPT language="WCBASIC"> and it saves the source in
a server side meta file (xxxx.wcx:wcc) but the main stream (xxxx.wcx) is
the compiled code. When the RTE opens, copies to cache and runs it. It
only sees the compiled stream (xxxx.wcx), not the source.

--
Click to expand...

Regards,

Peter
 
Hello,
How are you copying the file? That sounds like a OVERLAPPED I/O (IOCP)
completion status. You are using some async copy method.
Click to expand...

I use System.IO.File.Copy. It doesn't support overlapped IO. The thrown
exception is documented as "An I/O error has occurred." It's weird and
misleading that it is presented as "The operation completed successfully".
Good question.

Since streams are supports at the RTL level, I wonder if using a "." at
the end of the file name will do this?
Click to expand...

Adding a . doesn't make a difference in this case.
I would ask in the
microsoft.public.win32.programmer.kernel fora because I think you can
probably do this via the device file name.
Click to expand...

I know it can be done with unmanages code / pinvoke but I'd like to
avoid it.
One way for sure:

Create your own "COPYFILE" function opening the main file name, creating
a target and a loop to read/write X KBytes blocks. That for sure will
only give you the main stream.

That is what I do in our p-code compiler where the client has HTML
tagged code (ie, <SCRIPT language="WCBASIC"> and it saves the source in
a server side meta file (xxxx.wcx:wcc) but the main stream (xxxx.wcx) is
the compiled code. When the RTE opens, copies to cache and runs it. It
only sees the compiled stream (xxxx.wcx), not the source.

--
Click to expand...

Regards,

Peter
 
Peter said:
I use System.IO.File.Copy. It doesn't support overlapped IO. The thrown
exception is documented as "An I/O error has occurred." It's weird and
misleading that it is presented as "The operation completed successfully".
Click to expand...

+1. Does seem odd.
Adding a . doesn't make a difference in this case.
Click to expand...

Right.

Rolling your own "CopyFile" function will work:

Sub CopyFile(byval src as string, tar as string)
Dim sr As New FileStream(src,FileMode.Open,FileAccess.Read)
Dim sw As New FileStream(tar,FileMode.Create,FileAccess.Write)
Dim b(4*1024) As Byte
Do
dim n as integer = sr.Read(b, 0, b.Length)
if n <= 0 then exit do
sw.write(b,0,n)
Loop
sw.Close()
sr.Close()
end sub

usage:

try
CopyFile("afs1.txt","afs2.txt")
catch ex as exception
WriteLine(ex.message)
WriteLine(ex.stacktrace)
end try

To create the test, I used notepad to create the AFS1.TXT file and the
meta stream

NotePad afs1.txt

I typed junk lines and saved.

NotePad afs1.txt:meta. <--- NOTE THE ENDING DOT!

I typed junk lines and saved.

You can confirm that a straight DOS copy will copy the streams:

copy afs1.txt junk.txt
notepad junk.txt:meta. <--- NOTE THE ENDING DOT!

You will see the meta data.

Then I ran the above applet to copy the file to afs2.txt, and tested
to see if the the meta stream was copied as well:

NotePad afs2.txt:meta. <--- NOTE THE ENDING DOT!

Notepad asked if you want to create it. :-)

Hope this provides you the solution you seek.

--
 
Peter said:
I use System.IO.File.Copy. It doesn't support overlapped IO. The thrown
exception is documented as "An I/O error has occurred." It's weird and
misleading that it is presented as "The operation completed successfully".
Click to expand...

+1. Does seem odd.
Adding a . doesn't make a difference in this case.
Click to expand...

Right.

Rolling your own "CopyFile" function will work:

Sub CopyFile(byval src as string, tar as string)
Dim sr As New FileStream(src,FileMode.Open,FileAccess.Read)
Dim sw As New FileStream(tar,FileMode.Create,FileAccess.Write)
Dim b(4*1024) As Byte
Do
dim n as integer = sr.Read(b, 0, b.Length)
if n <= 0 then exit do
sw.write(b,0,n)
Loop
sw.Close()
sr.Close()
end sub

usage:

try
CopyFile("afs1.txt","afs2.txt")
catch ex as exception
WriteLine(ex.message)
WriteLine(ex.stacktrace)
end try

To create the test, I used notepad to create the AFS1.TXT file and the
meta stream

NotePad afs1.txt

I typed junk lines and saved.

NotePad afs1.txt:meta. <--- NOTE THE ENDING DOT!

I typed junk lines and saved.

You can confirm that a straight DOS copy will copy the streams:

copy afs1.txt junk.txt
notepad junk.txt:meta. <--- NOTE THE ENDING DOT!

You will see the meta data.

Then I ran the above applet to copy the file to afs2.txt, and tested
to see if the the meta stream was copied as well:

NotePad afs2.txt:meta. <--- NOTE THE ENDING DOT!

Notepad asked if you want to create it. :-)

Hope this provides you the solution you seek.

--
 
Hello Mike,

thanks for the example code. I changed it a little to my needs.
I'd like to preserve the LastWriteTime like the normal copy functions
but I get a "System.UnauthorizedAccessException" due to the NTFS
Permissions because the user only has "Create file" permission in the
destination directory.


Sub CopyFileWithoutStreams(ByVal sourceFileName As String, ByVal
destFileName As String, Optional ByVal buffersize As Integer = &HFFFFUI)
Dim sr As New System.IO.FileStream(sourceFileName,
IO.FileMode.Open, IO.FileAccess.Read, IO.FileShare.Read, buffersize,
IO.FileOptions.SequentialScan)
Dim sw As New System.IO.FileStream(destFileName,
IO.FileMode.CreateNew, IO.FileAccess.Write, IO.FileShare.None,
buffersize, IO.FileOptions.SequentialScan Or IO.FileOptions.WriteThrough)
Dim b(buffersize) As Byte
Dim n As Integer
Do
n = sr.Read(b, 0, buffersize)
If n <= 0 Then Exit Do
sw.Write(b, 0, n)
Loop
sw.Close()
sr.Close()
System.IO.File.SetLastWriteTimeUtc(destFileName,
System.IO.File.GetLastWriteTimeUtc(sourceFileName))
End Sub

Regards,

Peter
 
Hello Mike,

thanks for the example code. I changed it a little to my needs.
I'd like to preserve the LastWriteTime like the normal copy functions
but I get a "System.UnauthorizedAccessException" due to the NTFS
Permissions because the user only has "Create file" permission in the
destination directory.


Sub CopyFileWithoutStreams(ByVal sourceFileName As String, ByVal
destFileName As String, Optional ByVal buffersize As Integer = &HFFFFUI)
Dim sr As New System.IO.FileStream(sourceFileName,
IO.FileMode.Open, IO.FileAccess.Read, IO.FileShare.Read, buffersize,
IO.FileOptions.SequentialScan)
Dim sw As New System.IO.FileStream(destFileName,
IO.FileMode.CreateNew, IO.FileAccess.Write, IO.FileShare.None,
buffersize, IO.FileOptions.SequentialScan Or IO.FileOptions.WriteThrough)
Dim b(buffersize) As Byte
Dim n As Integer
Do
n = sr.Read(b, 0, buffersize)
If n <= 0 Then Exit Do
sw.Write(b, 0, n)
Loop
sw.Close()
sr.Close()
System.IO.File.SetLastWriteTimeUtc(destFileName,
System.IO.File.GetLastWriteTimeUtc(sourceFileName))
End Sub

Regards,

Peter
 
Ok, the user doesn't have "Modify" access? or Write Attributes?

So how are you going to handle that? Impersonate a higher level account?

If this is a system application, not just a end-user application, it
can make sense to raise the process credentials so you can do your
system work.
 
Ok, the user doesn't have "Modify" access? or Write Attributes?

So how are you going to handle that? Impersonate a higher level account?

If this is a system application, not just a end-user application, it
can make sense to raise the process credentials so you can do your
system work.
 
Hello Mike,

the applikation runs as a service under a least privilege account that
only has the "create file" permission in the archive directory, not
more, not less. Elevation isn't possible and shouldn't be neccessary.
Files that don't have NTFS alternate data streams can be copied without
problems. The LastWriteTime is also kept.

Regards,

Peter
 
Hello Mike,

the applikation runs as a service under a least privilege account that
only has the "create file" permission in the archive directory, not
more, not less. Elevation isn't possible and shouldn't be neccessary.
Files that don't have NTFS alternate data streams can be copied without
problems. The LastWriteTime is also kept.

Regards,

Peter
 
Back
Top