Inlining, linking, and layout

Everything so far in this part has been about one translation unit. Real programs are hundreds, compiled separately and stitched together at the end, and the seam is where the optimiser stops being able to help you.

This chapter is about the seam: what inline really does (not what its name says), what the linker can recover, and the two costs nobody measures — build time and binary size.

#include <iostream>

inline int marked(int n) {
    return n * 5 + 1;
}

int unmarked(int n) {
    return n * 5 + 1;
}

int caller(int n) {
    return marked(n) + unmarked(n);
}

int main() {
    int value = 0;
    std::cin >> value;
    std::cout << caller(value) << '\n';
}

Open the Assembly view. _Z6calleri contains no call at all: both functions were inlined into it. Now search for their definitions. _Z8unmarkedi is there, a standalone copy of the function. _Z6markedi is not.

The keyword did not decide which call got inlined. It decided which function needed a copy left behind.

What inline actually means

inline is a rule about the one definition rule, not about code generation. It says: this function may be defined identically in more than one translation unit, and the linker should keep one copy rather than rejecting the program for having several.

That is the entire semantic, and it exists for one reason: to let you put a function body in a header. Without inline, a header defining int square(int n) { return n * n; } and included by three .cpp files produces three definitions and a link error.

It also implies “you need not emit a standalone copy if nobody in this translation unit needs one”, which is what the sample above shows.

Several things are implicitly inline, which is why you rarely write it:

C++17 also added inline variables, which finally solved header-only global state: inline int counter = 0; in a header gives one counter for the whole program.

The real precondition: the compiler must see the body

A compiler cannot inline a function whose body it has never read. That single fact explains most of what follows.

• A function defined in a header is visible everywhere it is used, so it can be inlined everywhere. This is why header-only libraries perform well, and why they cost so much to compile.
• A function defined in another .cpp file is, at the point of the call, just a symbol name. The compiler emits a call and moves on.

The static keyword — or better, an anonymous namespace — makes the opposite promise, and buys something for it:

#include <iostream>

namespace {
int scale(int n) { return n * 7; }      // internal linkage
}

static int offset(int n) { return n + 3; }   // also internal linkage

int exported(int n) { return n * 13; }       // external linkage

int main() {
    int value = 0;
    std::cin >> value;
    std::cout << scale(value) + offset(value) + exported(value) << '\n';
}

In the assembly, _Z8exportedi is emitted as a symbol. scale and offset are not there at all — inlined into main, with no copy left behind, because nothing outside this file could possibly have called them.

That is the practical rule: anything not part of a file’s interface belongs in an anonymous namespace. It shrinks the symbol table, it removes the function from anyone else’s link, and it lets the optimiser reason about all the callers because it can see all of them.

Link-time optimisation

LTO gets the seam back. With -flto, the compiler emits its internal representation into the object files instead of finished machine code, and the linker runs the optimiser again with the whole program in view.

Two files. lib.cpp defines int helper(int x) { return x * 3 + 1; }; main.cpp declares it in a header and calls it in a loop:

int compute(int n) {
    int total = 0;
    for (int i = 0; i < n; ++i) total += helper(i);
    return total;
}
int main() { std::printf("%d\n", compute(1000)); }

Compiled normally, compute is 26 instructions with a call helper inside the loop, and helper appears in the finished binary as a symbol. Compiled with -flto on every compile and on the link, the final binary’s main is:

mov    $0x16e16c,%edx
call   printf

0x16e16c is 1,499,500 — the answer. The loop, the call, compute and helper were all inlined and folded into a single constant, and helper is not in the binary at all.

The costs are real:

• Link time. The optimiser now runs over the whole program, at the point in the build where nothing can be parallelised by file. On a large project this can turn a two-second incremental link into a minute. -flto=auto parallelises it; it is still the slowest step.
• Memory. Whole-program optimisation needs the whole program in memory.
• Debuggability. Stack traces get stranger, because functions genuinely no longer exist.
• Bugs surface. LTO applies cross-file the same assumptions the compiler was always allowed to make. Code with a latent ODR violation or a strict-aliasing bug often works without LTO and breaks with it. The bug was already there.

Use it for release builds, not for the edit-compile-test loop.

Binary size

Templates instantiate per type, and each instantiation is a separate function in the binary.

#include <iostream>
#include <string>
#include <vector>

template <class T>
std::string describe(const std::vector<T>& values) {
    std::string out = "[";
    for (const T& x : values) {
        out += std::to_string(x);
        if (x > T{0}) out += "+";
        else if (x < T{0}) out += "-";
        else out += "0";
        out += ' ';
    }
    out += "] (";
    out += std::to_string(values.size());
    out += " items)";
    return out;
}

int main() {
    std::cout << describe(std::vector<int>{1}).size()
              << describe(std::vector<long>{2}).size()
              << describe(std::vector<double>{3}).size()
              << describe(std::vector<unsigned>{4}).size()
              << describe(std::vector<float>{5}).size()
              << describe(std::vector<long long>{6}).size() << '\n';
}

Six instantiations, six function bodies in the object file — each marked as a weak symbol so that duplicates from other translation units are folded at link time. The C++ code says “one function”. The binary contains six.

That is usually a fine trade, and it is the direct cost of the speed measured in the previous chapter. When it is not fine, the levers are:

Measured on the six-instantiation program above: 35,136 bytes at -O2, 24,192 at -Os, 24,016 with section garbage collection, and 18,656 after strip — about half the original, and strip alone accounts for a fifth of it. That last one is free: it removes the symbol table, which the program does not need to run, only you need to read a stack trace.

Build time

The cost nobody profiles, paid by every developer on every build. The mechanism is textual inclusion: #include pastes a file in, transitively.

Four programs, all of which just print "hi":

Twenty-nine times slower to compile a program that does nothing, because of headers it does not use. Multiply by a thousand files and by every rebuild.

What actually helps, in order of leverage:

1. Do not include what you do not use. A forward declaration — class Widget; — is enough for a pointer, a reference, or a function declaration that takes one. You need the full definition only to create one, to hold one by value, or to call a member.
2. Move implementation out of headers. Anything not a template and not tiny belongs in a .cpp. The cost is losing cross-TU inlining — which LTO gives back, in the release build where it matters.
3. PIMPL. Hold a pointer to an implementation class declared but not defined in the header. Then the header depends on nothing the implementation depends on, and changing the implementation does not rebuild every user.
4. Precompiled headers and unity builds. Effective and blunt; they treat the symptom.
5. Modules. The actual fix — a module is compiled once and imported as structured data rather than pasted in as text. Chapter 9.2.

Check yourself

Practice