Any libmtmalloc, hoard-like library from MSFT on Win32?

  • Thread starter Thread starter kiranbhogadi
  • Start date Start date
K

kiranbhogadi

I'd like to know if there is a hoard-like drop-in library from MSFT to
replace the default C-lib implementation of malloc/free provided by
Visual Studio.

We have a program - multithreaded, runs on multi processors and very
dynamic-memory intensive -- which we determined for sure has heap
contention issues causing scalability problems. (For a very similar
application, on Solaris, we used the vendor-supplied libumem or
libmtmalloc library to override the default C-lib implementation of
malloc/free and it got rid of the scalability issue.)

I tried looking for info on the a 'similar' library (from MSFT) under
Win32 but couldn't find any such lib. Due to license-related,
corporate policy etc issues, I may not be able to use 'Hoard' on
Win32.

(I have read the posts where VC++ team members said that the NT heap
manager is pretty competitive for 98% of the applications but I am
positive that the main issue with my application's scalability is due
to heap contention in addition to some other heap mgmt issues ...)

As if to complicate things even further, the program is using a mix of
managed c++ and unmanaged code. This brings up a sub-question:
if I use a drop-in replacement for the default malloc/free that
affects the unmanaged code only (e.g. say using Hoard), does that
affect the CLR's management of 'managed' memory also? Do you see any
potential incompatibilities?

Just as an experiment, I did try using Hoard on Win32 with my managed
+unmanaged c++ program and I am encountering errors with loading the
unmanaged DLL. I am not sure if it's due to something I am doing when
linking in the Hoard lib or is it because of a clash between the CLR
and the newly dropped in malloc/free from Hoard. I am thinking the
former but then if I use Hoard with a pure C++ (unmanaged code) on
Win32, it works without any errors.

(As mentioned before, I may not be able to use Hoard anyways - even if
I get it to work with my managed+unmanaged c++ program - due to
licensing issues or corporate policy etc. But if it works, and I see
the scalability improvement at least I can prove for sure that this is
related to some heap management issue for this particular
'type' [i.e., multithreaded, runs on multi processors and very dynamic-
memory intensive ] of application ...)

Any info on this is appreciated.
 
I'd like to know if there is a hoard-like drop-in library from MSFT to
replace the default C-lib implementation of malloc/free provided by
Visual Studio.

I'm not fully sure what 'hoard' is, but there are dropin
replacements for VS's allocators. See things like
http://www.microquill.com/ . Never used it, but it does seem to claim
to be useful.

Nathan Mates
 
Hi,

Any luck on this effort? I have been able to test hoard and others on managed code and can see the benefit there. However I am wondering if it could be leveraged in any way in a managed code environment.

For example, how do I swap out the builtin for the CIL JIT compiler. This may be the element I am looking for.

Thanks in advance for any guidance you can provide.

Joseph



kiranbhogad wrote:

Any libmtmalloc, hoard-like library from MSFT on Win32?
25-Jun-07

I'd like to know if there is a hoard-like drop-in library from MSFT to
replace the default C-lib implementation of malloc/free provided by
Visual Studio.

We have a program - multithreaded, runs on multi processors and very
dynamic-memory intensive -- which we determined for sure has heap
contention issues causing scalability problems. (For a very similar
application, on Solaris, we used the vendor-supplied libumem or
libmtmalloc library to override the default C-lib implementation of
malloc/free and it got rid of the scalability issue.)

I tried looking for info on the a 'similar' library (from MSFT) under
Win32 but couldn't find any such lib. Due to license-related,
corporate policy etc issues, I may not be able to use 'Hoard' on
Win32.

(I have read the posts where VC++ team members said that the NT heap
manager is pretty competitive for 98% of the applications but I am
positive that the main issue with my application's scalability is due
to heap contention in addition to some other heap mgmt issues ...)

As if to complicate things even further, the program is using a mix of
managed c++ and unmanaged code. This brings up a sub-question:
if I use a drop-in replacement for the default malloc/free that
affects the unmanaged code only (e.g. say using Hoard), does that
affect the CLR's management of 'managed' memory also? Do you see any
potential incompatibilities?

Just as an experiment, I did try using Hoard on Win32 with my managed
+unmanaged c++ program and I am encountering errors with loading the
unmanaged DLL. I am not sure if it's due to something I am doing when
linking in the Hoard lib or is it because of a clash between the CLR
and the newly dropped in malloc/free from Hoard. I am thinking the
former but then if I use Hoard with a pure C++ (unmanaged code) on
Win32, it works without any errors.

(As mentioned before, I may not be able to use Hoard anyways - even if
I get it to work with my managed+unmanaged c++ program - due to
licensing issues or corporate policy etc. But if it works, and I see
the scalability improvement at least I can prove for sure that this is
related to some heap management issue for this particular
'type' [i.e., multithreaded, runs on multi processors and very dynamic-
memory intensive ] of application ...)

Any info on this is appreciated.

Previous Posts In This Thread:

Any libmtmalloc, hoard-like library from MSFT on Win32?
I'd like to know if there is a hoard-like drop-in library from MSFT to
replace the default C-lib implementation of malloc/free provided by
Visual Studio.

We have a program - multithreaded, runs on multi processors and very
dynamic-memory intensive -- which we determined for sure has heap
contention issues causing scalability problems. (For a very similar
application, on Solaris, we used the vendor-supplied libumem or
libmtmalloc library to override the default C-lib implementation of
malloc/free and it got rid of the scalability issue.)

I tried looking for info on the a 'similar' library (from MSFT) under
Win32 but couldn't find any such lib. Due to license-related,
corporate policy etc issues, I may not be able to use 'Hoard' on
Win32.

(I have read the posts where VC++ team members said that the NT heap
manager is pretty competitive for 98% of the applications but I am
positive that the main issue with my application's scalability is due
to heap contention in addition to some other heap mgmt issues ...)

As if to complicate things even further, the program is using a mix of
managed c++ and unmanaged code. This brings up a sub-question:
if I use a drop-in replacement for the default malloc/free that
affects the unmanaged code only (e.g. say using Hoard), does that
affect the CLR's management of 'managed' memory also? Do you see any
potential incompatibilities?

Just as an experiment, I did try using Hoard on Win32 with my managed
+unmanaged c++ program and I am encountering errors with loading the
unmanaged DLL. I am not sure if it's due to something I am doing when
linking in the Hoard lib or is it because of a clash between the CLR
and the newly dropped in malloc/free from Hoard. I am thinking the
former but then if I use Hoard with a pure C++ (unmanaged code) on
Win32, it works without any errors.

(As mentioned before, I may not be able to use Hoard anyways - even if
I get it to work with my managed+unmanaged c++ program - due to
licensing issues or corporate policy etc. But if it works, and I see
the scalability improvement at least I can prove for sure that this is
related to some heap management issue for this particular
'type' [i.e., multithreaded, runs on multi processors and very dynamic-
memory intensive ] of application ...)

Any info on this is appreciated.

Re: Any libmtmalloc, hoard-like library from MSFT on Win32?
I'm not fully sure what 'hoard' is, but there are dropin
replacements for VS's allocators. See things like
http://www.microquill.com/ . Never used it, but it does seem to claim
to be useful.

Nathan Mates
--
<*> Nathan Mates - personal webpage http://www.visi.com/~nathan/

Hoard
Hoard = a (my) scalable memory allocator. See www.hoard.org. It's open source, with a GPL license (separate licensing is available for commercial users).

Regards,
-- Emery

--
Emery Berger
Assistant Professor
Dept. of Computer Science
University of Massachusetts Amherst
http://www.cs.umass.edu/~emery


Submitted via EggHeadCafe - Software Developer Portal of Choice
Slipstreaming and Unattended Windows Media Creation
http://www.eggheadcafe.com/tutorial...e-4f37a6bdfdc5/slipstreaming-and-unatten.aspx
 
Back
Top