New operator returns null in some cases

[Niall]

I have an unmanaged C++ ray tracer which I am working to put a C# front end
on. It runs fine as just the unmanaged code. I have made a MC++ wrapper DLL
to expose the required types to the C# project. Things seem to be getting
through from the unmaged side to the managed side - the image is being
rendered correctly. However, at random points, calls to new are returning
null. Hence, when the managed type gets destroyed and calls delete on its
pointer for the encapsulated unmanaged type, there is an exception.

This really confuses me, I've never seen new return null before. The machine
isn't out of memory, there's hundreds of megs left. Does anyone have any
ideas what could be causing this behaviour?

Niall

 

[Carl Daniel [VC++ MVP]]

I have an unmanaged C++ ray tracer which I am working to put a C#
front end on. It runs fine as just the unmanaged code. I have made a
MC++ wrapper DLL to expose the required types to the C# project.
Things seem to be getting through from the unmaged side to the
managed side - the image is being rendered correctly. However, at
random points, calls to new are returning null. Hence, when the
managed type gets destroyed and calls delete on its pointer for the
encapsulated unmanaged type, there is an exception.

This really confuses me, I've never seen new return null before. The
machine isn't out of memory, there's hundreds of megs left. Does
anyone have any ideas what could be causing this behaviour?

VC++ has two different implementations of void* :perator new(size_t) - one
that throws (C++ standard compliant) and one that returns null (VC6
compatible). Which one gets linked into your application can be hard to
determine sometimes. One thing that you could do is explicitly link
%VSDir%\vc7\lib\thrownew.obj into your program and see if your nulls turn
into exception throws. If so, then you may be fragmenting heap space to the
point that there's no contiguous free space large enough for some
allocation, hence the failure. If your nulls don't turn into exceptions
with thrownew.obj linked in, then something else is going on - most likely
heap corruption of some kind.

-cd

 

[Niall]

Thanks for your reply, Carl. I tried your suggestion of using the throw
version of new. I get a NullReference exception on the throw line, is this
the type of exception you would expect, or is there a specific exception for
failure of the new operator? It only happens after some time (after the
first run through the render).

I don't know too much about how the native C++ heap runs. The class it's
unable to instantiate is a pretty small class (just three floats and a few
methods). There would certainly be a lot of these objects instantiated and
destroyed during the process of running the program, the class is used to
represent colours. However, I would have thought that it would be possible
to put a new object in the same spot as one of the old ones, because the
objects are all of the same size. Also, there is still plenty of memory
free, and I haven't run into this problem when running the unmanaged code on
its own.

The heap corruption you mention, are there other specific causes you know
of? I'm not ruling out some bad memory handling by myself (though for the
most part, most of the app's variables are stack based), though I would
expect that would cause problems trying to use memory or delete it, not on
the new...

Any help you can give will be greatly appreciated

Niall

 

[Carl Daniel [VC++ MVP]]

Thanks for your reply, Carl. I tried your suggestion of using the
throw version of new. I get a NullReference exception on the throw
line, is this the type of exception you would expect, or is there a
specific exception for failure of the new operator? It only happens
after some time (after the first run through the render).

NullReference is a managed (CLR) exception, and not what I'd expect new to
be throwing (it should throw std::bad_alloc). Are you sure this is
unmanaged code where you're seeing the error?

I don't know too much about how the native C++ heap runs. The class
it's unable to instantiate is a pretty small class (just three floats
and a few methods). There would certainly be a lot of these objects
instantiated and destroyed during the process of running the program,
the class is used to represent colours. However, I would have thought
that it would be possible to put a new object in the same spot as one
of the old ones, because the objects are all of the same size. Also,
there is still plenty of memory free, and I haven't run into this
problem when running the unmanaged code on its own.

Sounds very much like a heap corruption problem to me.

The heap corruption you mention, are there other specific causes you
know of? I'm not ruling out some bad memory handling by myself
(though for the most part, most of the app's variables are stack
based), though I would expect that would cause problems trying to use
memory or delete it, not on the new...

Unfortunately, heap corruption can be very hard to find. Examine all of
your access to heap-allocated objects carefully, looking for overindexed
pointers, acess of deleted objects, etc.

-cd

 

[Ken Alverson]

Thanks for your reply, Carl. I tried your suggestion of using the throw
version of new. I get a NullReference exception on the throw line, is this
the type of exception you would expect, or is there a specific exception for
failure of the new operator? It only happens after some time (after the
first run through the render).

The version of new that throws should throw a std::bad_alloc.
NullReferenceException actually isn't a C++ exception at all, it's a managed
exception, and as far as I can tell it should never be thrown from the
standard C++ libraries if everything is kosher. The only reason I can think
that it would be thrown is in cases of undefined behavior (and, of course,
your code is compiled as managed).

My gut instict says heap corruption. Try to see if you are overrunning a
buffer somewhere.

Ken

 

[Niall]

Carl and Ken:

Hopefully you will see this, as I have let the post get old. The
NullReferenceException is somewhat new. Sometimes, it seems to do new fine,
and I have watched it in the debugger, new has returned a null reference,
but with no exception. Then, of course, when the destructor comes around,
deleting the unmanaged object, which is null, causes an exception. However,
at other times, like is now happening, there is a NullReferenceException
when calling new, which is, as you say a managed exception.

The NullReferenceException is occurring on the line that is creating a new
instance of the underlying unmanaged C++ object from another DLL that is
being wrapped by the MC++ classes. The constructor being called in the
unmanaged DLL is trivial, there is nothing in it except setting up a few
floating point variables, so nothing that should cause an exception.

The strange thing is that these managed wrapper objects can work fine for a
while of the run, then cause the exception. This kind of unpredicatable
behaviour leads me to believe, as you suggest, that there is some dodgy
memory access occurring. Unfortunately, the unmanaged code runs fine on its
own with no memory issues. Because of its highly functional nature, almost
all memory is stack based, which I found much easier than trying to work out
when I was really finished with an object and deleting it. So there isn't
much opportunity for heap corruption, I think.

The one area I do suspect is the integration between the MC++ dll and the
native C++ dll. This is something I haven't done before, so I've had to
learn a fair bit. One thing I'm not sure of is the whole static
initialisation issue with the DLLs. I went through the steps to make the
MC++ dll use /NOENTRY, found on one of the MSDN pages. One of the steps is
the manual initialisation of the C ruintime. Do I have to call some method
in my native C++ dll which will manually setup values in the few static
variables I'm using? This is about the only guess I have...

Once again, any help would be greatly appreciated

Niall