FailFast and MiniDump creation

[Chris Mullins]

I'm looking into adding automated mini-dump creation to my companies .NET
software products.

For the past few years, we've been using ADPlus scripts (thank you
BugSlayer!) to setup and catch 2nd Chance Exceptions, generating .NET 1.1
dumps, and then using WinDbg and Son of Strike to track down the culprit
(usually System.DirectoryServices. Ugh.). The problem is that this is a
cumbersome process for clients to go through (many of whom are
non-technical), and generally relies on knowing in advance that an error is
going to happen soon.

Our products are now all migrated to .NET 2.0, which has the new method
"Environment.FailFast". The documentation on this little gem is quite
scarce, and I'm trying to determine the best practices for using it. I'm
trying to figure out the best ways to do a bunch of things:

- Is there a way to catch 100% of 2nd chance exceptions, so that I can call
FailFast? Is simply handling the UnhandledException on the AppDomain class
enough to guarantee this? I think it is, as we only have a single AppDomain.

- I would like to catch any unhandled exceptions on background threads,
which often seem to be eaten by the CLR.

- I want to control the directory the dumps are written to - but I don't see
any way to do this. Suggestions?

- Ideally I need to email notification of these dumps to someone, so they
can transfer them to us (or have this automatically in some way). I looked
into the Windows Error Reporting stuff, but it seems too heavy duty. We have
few crashes, but each of them should be looked into. I'm debating a 2nd
Service that watches for new crashdumps, then transfers them. There has to
be a better way to do this...

- When the service is "hung" and manually terminated, I would like a
crashdump to be generated. We can do this today with an ADPlus script, but I
would really like to eliminate this.

Is there an MSDN article or a CLR Team Member blog that I missed that offers
some guidance around doing this?

 

[Oleg Starodumov]

- Is there a way to catch 100% of 2nd chance exceptions, so that I can call
FailFast? Is simply handling the UnhandledException on the AppDomain class
enough to guarantee this? I think it is, as we only have a single AppDomain.

IMO the most reliable way is to use a native filter for unhandled exceptions
(see SetUnhandledExceptionFilter) and use it to generate the dump (MiniDumpWriteDump).
Here you can find a sample tool/library that does it:
http://www.debuginfo.com/tools/clrdump.html

AppDomain.UnhandledException depends on Registry settings, so you
can be sure that it will be called only if you have control over the system.
Also it will not help if an unhandled exception occurs on an unmanaged thread.

- I would like to catch any unhandled exceptions on background threads,
which often seem to be eaten by the CLR.

In .NET 2.0 they should not be eaten anymore (unless 1.x behavior is explicitly
restored via .config file; special cases are possible, though).

- I want to control the directory the dumps are written to - but I don't see
any way to do this. Suggestions?

Generate the dump yourself, then you are in control.

- When the service is "hung" and manually terminated, I would like a
crashdump to be generated. We can do this today with an ADPlus script, but I
would really like to eliminate this.

I usually use a monitor application (e.g. in tray) that allows the user
to create a minidump when the application is misbehaving (or just ask
to use clrdump.exe for that purpose). In some cases it is also possible
to detect the problem automatically and create the dump.

Regards,
Oleg
[VC++ MVP http://www.debuginfo.com/]

 

[Naveen]

Why dont you just change the value of the DbgManagedDebugger in
registry to get crash dumps incase if the application crashes.
cdb -p %ld -e %ld -c ".dump /ma /u c:\dumps\crash.dmp;q"

But you might not get a "hang dump" using this. You can configure the
cdb to notify some one by including !net_send with in the command in
the registry.

 

[Chris Mullins]

[Generating MiniDumps]

IMO the most reliable way is to use a native filter for unhandled
exceptions
(see SetUnhandledExceptionFilter) and use it to generate the
dump (MiniDumpWriteDump). Here you can find a sample tool/library
that does it:
http://www.debuginfo.com/tools/clrdump.html

So after spending way too much time reading your site and articles, I'm sold
on using the clrdump.dll to setup an exception filter and generate a dump.

I'm pretty certain too, after reading that I want, "MiniDumpWithDataSegs" as
the value to the options.

I have a few more questions though:
- In the general case, I don't want to screw up managed debugging, so I'm
only going to register the filer if it's being built in release mode. I'm
also going to add a config option to our app to allow it to be explicity
turned on and off.

- I need to distribute the binary files for clrdump and dbghelp along with
our software. Because we run on a number of platforms, I need to include the
appropaite binary files for x86, x64, and Itanium. I see the Microsoft site
for debugging tools has these now. Will the same dll p/invoke signatures
work on all the platforms do you know? I'm just going to throw the
appropiate DLL's into the application's directory, and let it go from there.

- If I set the unhandled filter, do I need to remove it once the mini-dump
is written?

Thanks alot for your information and help!

 

[Chris Mullins]

Here you can find a sample tool/library that does it:
http://www.debuginfo.com/tools/clrdump.html

I'm trying to make this work, but am having hit-or-miss results.

I'm setting up the filter by calling:
string fileName = GetDumpFileName();
int ret = RegisterFilter(fileName, 1);
if (ret == 0)
throw new System.ComponentModel.Win32Exception();

Once the filter is setup, I cause a stack overflow by calling:
void Recurse()
{
Recurse();
}

The application quickly terminates, but I don't get a dump file. The
filename that I pass in is a full Win32 path ("C:\temp\dump001.bin").

If I explicitly throw an exception (and don't catch it), then I do get a
dump file.

I'm running without a debugger when I do this.

 

[Chris Mullins]

I'm trying to make this work, but am having hit-or-miss results.

There's some signifigant other weirdness going on that I just don't yet
understand.

With the exception filter registered (no debugger involved), if I throw any
unhandled exceptions (ArgumentException, etc) in the app I get the "An
unhandled exception has occured in your application" dialog, that lets me
continue or quit. Regardless of which option I choose, no dump file is
written out.

If I cause the StackOverflowException, I do not get the dialog and the
application just exits. No dump file written.

If I run in the debugger, and explicitly throw an unhandled exception, I DO
NOT get the dialog, and I do get a crashdump written.

I'm running VS.NET 2005 on WinXP. I'm running all 32 bit software.

 

[Oleg Starodumov]

I have a few more questions though:

I will try to answer them all in one post, please tell me if I missed something.

- I need to distribute the binary files for clrdump and dbghelp along with
our software. Because we run on a number of platforms, I need to include the
appropaite binary files for x86, x64, and Itanium. I see the Microsoft site
for debugging tools has these now. Will the same dll p/invoke signatures
work on all the platforms do you know? I'm just going to throw the
appropiate DLL's into the application's directory, and let it go from there.

I have never tested clrdump on 64-bit platforms (don't have equipment).
I would expect it to work, but if it doesn't, please tell me and I will try to fix it
(use the address for bug reports on the web site, please

- If I set the unhandled filter, do I need to remove it once the mini-dump
is written?

No.

I'm trying to make this work, but am having hit-or-miss results.

I'm setting up the filter by calling:
string fileName = GetDumpFileName();
int ret = RegisterFilter(fileName, 1);
if (ret == 0)
throw new System.ComponentModel.Win32Exception();

Once the filter is setup, I cause a stack overflow by calling:
void Recurse()
{
Recurse();
}

The application quickly terminates, but I don't get a dump file. The
filename that I pass in is a full Win32 path ("C:\temp\dump001.bin").

That is expected. Stack overflow exception usually cannot be handled
in-process, because exception handlers have to execute on the same stack
which has just been exhausted. In most cases the application terminates
silently. The only reliable way to catch stack overflows is to use
a debugger (e.g. ADPlus would do).

If I explicitly throw an exception (and don't catch it), then I do get a
dump file.

Because in this case there is enough stack space for exception handling
code to run.

There's some signifigant other weirdness going on that I just don't yet
understand.

With the exception filter registered (no debugger involved), if I throw any
unhandled exceptions (ArgumentException, etc) in the app I get the "An
unhandled exception has occured in your application" dialog, that lets me
continue or quit. Regardless of which option I choose, no dump file is
written out.

It means that the exception is not in fact unhandled; WinForms catch
it and report via that dialog.

To prevent WinForms from catching exceptions, call this function at application startup
(before any forms/windows have been created):
Application.SetUnhandledExceptionMode(UnhandledExceptionMode.ThrowException, false).

Oleg