Here's a copy of something I'm writing that may be of assistance in
general. If possible I'd like to get a repro of your problem. I'm trying
to understand what the various problems are with linking, and the more
customer issues I can help resolve, the better.
Thanks,
Kang Su Gatlin
Visual C++ Program Manager
----
Informative Short Note on the Microsoft VC2003 Linker
Introduction
The linker is the unsung hero of the build process. Everyone talks about
the compiler, devenv, even nmake, but the linker is where it all comes
together. The linker only seems to get brought up when it's not working
the way users expect. In 2003 Microsoft introduced Visual C++ 2003, and
with that a new linker. From perusing the various message boards I've seen
some complaints regarding the speed of linking with this new linker. In
this whitepaper I'll try to help people make their linking faster, and
understanding why it is slower.
Microsoft Visual C++ 6 was released in the late 90s and one of the most
popular and widely liked C++ development environments ever crafted. Visual
C++ 2003 improved on the code generation and other cool features such as
security, but there were some features that we heard our fair share of
complaints about. As you could guess, in this article I'll talk about the
linker, and how you might be able to figure out why your link times seem
slower, for some scenarios, in VC2003.
So you may find yourself asking, "This sounds cool and all, but what
actually is a linker?" Well here's my one paragraph description of what a
linker is : The linker (link.exe in Visual C++) is a program that takes
as input a collection of object files (.obj) and libraries (.lib), puts
them together, and creates an executable (or a .dll). The main things that
the linker does while linking is that it puts data into the correct
sections of the final image, fixes up relocation information, and folds
types and functions that are redundant. It has a hefty job, and as
languages like C++ become more popular , and programs get bigger, the job
of the linker is taking longer to do.
Preliminaries
Before diving into the below discussions, let me first open by talking
about a couple of hidden switches that may help people diagnose linker
problems:
/TIME - This switch will give timing information for various parts of the
link phase. It is often useful to get an idea of where the time is
actually going on and where. Admittedly, much more useful to Microsoft in
helping diagnose your problems.
/TEST - Test gives additional information regarding what might be going
wrong during linking. When things are working correctly, you tend to get
relatively little information out of it.
Additionally if you ever need to contact Microsoft about a problem with the
linker, what we often like to have available is a reproduction that we
could use internally. We call this a "link repro".
To construct a link repro, do the following:
1. mkdir <some_dir>.
2. add /linkrepro:<some_dir> to the linker command line.
3. run the offending link command.
4. copy any of the compiler generated .pdb files that are necessary into
the directory.
5. zip up the directory and that is your link repro.
Incremental Linking
We start with incremental linking, as it is one of the big features that
can substantially reduce the link time. Occasionally we will get reports
of slowdowns from incremental linking when going to VC2003 from VC6. It
turns out that there have been very few changes to the behavior of the
linker, but rather what has often changed is the project, with respect to
the project system, during migration.
Incremental linking works by only linking in new changed functions and then
adding a stub from the old location that points to the new linked in
function (yes, an extra indirection).
A good place to read up on the feature, if you've never used it, is at:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore98/HT
ML/_core_.2f.incremental.asp.
Most of the complaints associated with incremental linking are when the
linker decides NOT to incrementally link. There are several situations
when the linker will do a full link rather than an incremental link. Here
are a couple of things to watch out for:
Library Changes
If a library is changed, added, or removed then a full-link will take
place. This is similar to a new set of .obj files being added or removed.
Exports Changed
If an export is added or removed from the program then a full link will
take place.
IDL Issues
In VS2003 the linker does not properly ignore the IDL data when the
/IGNOREIDL switch is used. This results in the marking of all modules with
IDL symbols as changed, and this causes as eventual exhaustion of padding -
and from there an end to incremental linking. This has been fixed in
VS2005.
To see a complete list of reasons that full-links will be invoked rather
than incremental links see linker warnings LNK6001-6026.
Slow Link Times
Some customers have noted slower link times as they move from VC6 to
VC2003. In this section we list some of the potential problems that we
know about and various tricks and tips.
Directives
The linker's handling of directives left something to be desired in VC2003,
and for this reason code with lots of directives can suffer. In particular
__declspec(export), as it's likely the most common linker directive.
Besides that, don't export functions that you don't need to export.
Besides it becoming a code maintenance nightmare, it also increases your
link times.
Use Pre-Compiled Header Files Effectively
One of the big problems we see is not using pre-compiled header (PCH) files
(or somehow the PCH files are no longer get generated correctly). Often
times a good way to determine that this is the culprit is to look at the
size of your object files. If they're all inordinately large object files
(and especially if they're all nearly the same size)
Don't Put Libraries in Libraries
The first question one should ask themselves is "why do I want to do this?"
After enough going back and forth and thinking you will likely come to the
conclusion that there is no good reason. The linker is more efficient with
more smaller libraries, rather than a big conglomeration of libraries.
Things Are Bigger
Moving to VC2003 from VC6 will often use new headers from the CRT, STL,
Platform SDK, etc… Many of these files have become larger since VC6, and
just due to the mere fact that the linker has to deal with more causes link
times to increase.
Optimization vs Link Time
If you're in the "compile, link, test" phase of development then it may
make sense to turn off /OPT:REF and /OPT:ICF in release builds. The
resulting code may be larger, but it will build more quickly.
VS2005 has a Faster Linker
There has been various performance fixes, including redoing one of its main
hash functions, which will result in faster link times! I know, people
don't like to hear "in the next version it's better", but I just want to
let you be aware that things do get better.
--------------------
| From: "TOM" <
[email protected]>
| References: <
[email protected]>
<
[email protected]>
<
[email protected]>
| Subject: Re: REPOST: extremely long link times
| Date: Tue, 27 Jul 2004 21:28:19 -0500
| Lines: 112
| X-Priority: 3
| X-MSMail-Priority: Normal
| X-Newsreader: Microsoft Outlook Express 6.00.2800.1437
| X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441
| Message-ID: <
[email protected]>
| Newsgroups: microsoft.public.dotnet.languages.vc
| NNTP-Posting-Host: 4.7.8.109
| Path:
cpmsftngxa10.phx.gbl!TK2MSFTFEED01.phx.gbl!TK2MSFTNGP08.phx.gbl!tk2msftngp13
..phx.gbl
| Xref: cpmsftngxa10.phx.gbl microsoft.public.dotnet.languages.vc:39020
| X-Tomcat-NG: microsoft.public.dotnet.languages.vc
|
| Steve:
|
| Some of the MS applications try to access the Internet whether or not
| there is
| anything that needs to be found. One of the office applications stalls
until
| the TCP
| time-out occurs (1-2 minutes before starting).
|
| Additionally, I found I have to clear the IE6 browser history
| periodically, and if I
| have any URL shortcuts on the desktop that refer to an FTP site, I have to
| remove those
| as well, or after awhile all file menus stall for about 2 minutes.
|
| -- Tom
|
|
|
|
| "Steve McLellan" <sjm.NOSPAM AT fixerlabs DOT com> wrote in message
| | > Hi,
| >
| > 1. Nope, and it's linking that's the problem - compiling's generally
very
| > fast. In any case, it's not getting anywhere near maxing the processor
so
| > it's not the workload slowing it down.
| >
| > 2. It is only recompiling the minimum but incremental linking apparently
| > doesn't work under /clr.
| >
| > 3. I am now wondering whether it'd be better to split everything up into
| > separate DLLs, but I don't want to for a couple of reasons. And besides,
| > what puzzles me is why the linker seems to be sitting on its hands for a
| > minute or two, without seeming to do any work.
| >
| > Cheers,
| >
| > Steve
| >
| > | > >
| > > I'm a fairly newbie to C++ and the like, so feel free to ignore...
| > >
| > > Some questions / comments:
| > >
| > > 1. Are you using alot of #Define (or anything like that. Can't
| remember
| > what they are actually called)?? I've had the displeasure of hacking
| > somebodies old code that had ALOT of this stuff, and to compile it took
| > close to 4 min. I went ahead and took out as much as I could (W/o
killing
| > the app), and it was a noticable change.
| > >
| > > 2. You may want to consider using the option "Enable Minimal Rebuild"
| > under the projects configureation properties -> C/C++ -> Code
Generation.
| > If set to YES, the compiler will only create new OBJ for those files
that
| > have changed. It will keep the old OBJs and then link them. If you
only
| > change 1 file between, only 1 OBJ will be recreated (instead of them
all).
| > I think (not sure), but any files that include the file you change
(Using
| > #include) may have to be recompiled...
| > >
| > > 3. If you're just testing / debugging, why don't you make temp dll's
| for
| > any part of the code that is "Done" or not being checked. For
instance, I
| > have a Map.dll that containes all my stuff related to geographical maps.
| I
| > then include it as a referance, use #using, and that stuff never gets
| > compiled. I then create a "include all" project that has all the code
| files
| > in it. This does leave me with a lot of Test projects all over the
place,
| > but to test it does make it fairly fast.
| > >
| > >
| > > Can't think of anything else. Let me know if this helps...
| > >
| > >
| > > GE
| > >
| > > "Steve McLellan" wrote:
| > >
| > > > Hi,
| > > >
| > > > Sorry to repost, but this is becoming aggravating, and causing me a
| lot
| > of
| > > > wasted time. I've got a reasonably large mixed C++ project, and
after
| a
| > > > number of builds (but not a constant number) linking (and sometimes
| > > > compiling) becomes immensely slow, and task manager shows that
| link.exe
| > (or
| > > > cl.exe) is barely using any processor time, but an awful lot of RAM
| > (around
| > > > 150-200MB). I'm going to keep an eye on page faults since I can't
| think
| > of
| > > > anything else that could be causing this - the machine doesn't seem
to
| > be
| > > > thrashing and is fairly responsive during building. Can anyone
| recommend
| > > > anything?
| > > >
| > > > Thanks,
| > > >
| > > > Steve
| > > >
| > > >
| > > >
| >
| >
|
|
|
