Using FileSystemWatcher with large files

  • Thread starter Thread starter Josh Usovsky
  • Start date Start date
J

Josh Usovsky

I'm setting up a watched folder using FileSystemWatcher.
When I drop a small file into the watched folder, I can
respond to a .Created event and process the file with
other code. However, if I try copying a large file into
the watched folder, the .Created event is fired off
immediately and not when the file is finished copying. I
have tried waiting for a certain number of .Changed
events to be fired after the initial .Created event, and
that method works greate except for the fact that some
files trigger 3 .Changed events, and others 5, 6, and 7.
I can't find a pattern to determine why there are a
differing number of events being fired for the various
files. Does anyone else have experience in this or have
any suggestions for determining when each file is
finished being written ? I'd rather not use a checking
loop or error handling for program flow, but I might have
to at this point.

Thanks in advance,
--JU
 
Josh,

Whenever I use a FileSystemWatcher in this way I process the created files
in new threads. Each thread is added to a hashtable that is referenced by
the file name thus allowing me to see if i have already created a processor
for the file whe subsequent events are fired.

Within the processor thread, I open the file exclusively and continuously
loop until the open is sucessful.

Martin.
 
I've had similar problems using FileSystemWatcher to process large files
which are FTP'd to us each night. The event firing mechanism in
FileSystemWatcher is just flakey. While not ideal, using a polling loop is
the method that I use and it seems to be the only way around this problem.
When FileSystemWatcher fires the event, immediately try to open the file in
exclusive mode. If an exception is thrown, set the timer and then try to
open the file again when the timer fires. Once the file is available, stop
the timer and process the file.

A simpler method is to respond to the FileSystemWatcher event by
entering a loop which includes a Thread.Sleep() call after each failure to
open the file. The downside of this approach is that you're causing a
threadpool thread to block during the Sleep() call. Nevertheless, 95% of the
time the Sleep() loop is the way to go. It's just simpler.

David
 
The event firing mechanism in FileSystemWatcher is just
flakey.
Click to expand...

The event firing system is Windows, and it's not
guaranteed to be one event per file operation.

If you read the documentation, you'll find where it says
"Common file system operations might raise more than one
event" and "when a file is moved from one directory to
another, several OnChanged and some OnCreated and OnDeleted
events might be raised".

Mark
 
If you read the documentation, you'll find where it says
"Common file system operations **might** raise more than one
event" and "when a file is moved from one directory to
another, **several** OnChanged and some OnCreated and OnDeleted
events **might** be raised".
Click to expand...

It's words like "might" and "several" that bother me. I wish things were more precise. Not to say that FileSystemWatcher is useless by any means -- it's just not as consistent as I'd prefer.

David
 
It's words like "might" and "several" that bother me. I wish
things were more precise. Not to say that FileSystemWatcher is
useless by any means -- it's just not as consistent as I'd prefer.
Click to expand...

Agreed, it'd be nice if things were more concrete. Just don't "blame the
messenger", like they say. The FSW here is just a messenger. Things happen
this way because of the way filesystem access works in windows.

Mark
 
Back
Top