what is the "standard" practice for template definition location

[Dan Moos]

I just figured out the hard way that I can't just put my template definitions in .cpp file as I would with non-template functions.

Further research showed me some work-arounds, such as including said .cpp along with the .h file in units that need the template, or instantiating dummy instances of the template using the types anticipated being needed.

These all sound like ugly kludges to me! Another mentioned method was to put the definitions in a .inl file, and include it in the .h file that has the declarations. That was the closest to a clean solution I found.

What is industry standard practice, if there is one?

Follow up...Why CAN"T one treat template definitions like regular functions/methods? I know the stock answer is that the function code doesn't exist until an instantiation with specific type happens, but why is such a hurdle for the compiler? Lets say it encounters a template instantiation with a type of 'int' needed. Is it really that hard for the compiler to find the template definition, and just inline the custom code? Is there some crazy hurdle I'm missing?

[helios]

I generally just put the code directly in the header. I only do something else if it's so long it's unreadable.

Why CAN"T one treat template definitions like regular functions/methods? I know the stock answer is that the function code doesn't exist until an instantiation with specific type happens, but why is such a hurdle for the compiler?

Consider this function:

template <typename T1, typename T2>
T1 add(const T1 &a, const T2 &b){
    return a + b;
}

Since this definition is in for example a.cpp, and the call may be in b.cpp, the compiler must compile this function without knowing the types involved, but how? How does it generate a call to T1::operator+() without knowing what T1 is? The answer is that it just can't. To implement templates like this the compilation process needs to be more complicated than the old C process. Rather than code generation happening solely in the compiler and the linker just connecting generated code, the linker must now know how to compile arbitrary functions, just like the compiler. Alternatively, the compilation process must be entirely reworked so that compilation and linking are not separate processes anymore. This, I believe, is how parametric polymorphism works in newer languages such as Haskell.

Actually, C++98 allowed template definitions and declarations to be split, as extern templates, but the feature was later deprecated because only one compiler ever implemented it.

[Dan Moos]

I guess what I've never understood is why it was set in stone so early in the lifetime of both C and C++ that the linker and compiler need to be separate programs that basically have no knowledge or communication with each other. I realize they are different processes, but it seems having them be in the same program would make things like this trivial.

compiler part of program does what it can.

linker then tries to resolve everything, but can re-invoke the compiler as needed

problem solved! I just never understood why it came to be as it is now.

[helios]

Because C was invented in the 70s, to be compiled by machines with memory sizes measured in megabytes, if you're lucky.
C++ was designed to be almost a strict superset of C (C programs that use C++-only keywords as variable names will fail to be parsed by a C++ compiler), which is both a blessing and a curse.

[JLBorges]

> I realize they are different processes, but it seems having them be in the same program
> would make things like this trivial.

Yes.
It would also make creating programs with different parts (in particular libraries) written in different programming languages a lot more difficult.

Primarily for purposes of optimisation, most current implementations support the idea of link time code generation for modules written in C++ (and C).

[keskiverto]

linker then tries to resolve everything, but can re-invoke the compiler as needed

Ok.
You had translation units without template implementations.
They did produce object code that has function calls.
Will a call of templated function differ from regular function call,
i.e. can you tell which should link to other object files/libraries
and which need template instantiation?
What if some translation unit already has some explicitly instantiated templates? Can it?

Which files would the linker seek in order to compile templates?


It is still hazy to me how the linker dances around the ODR with the current template model.


You either have template definitions the header, or include them. The header file (declaration) should include the template definitions, for you cannot expect the user to include all necessary files.

IDE/cmake/qmake/etc can define a "project" from existing files. Perhaps auto-include all code files from a tree. Each source file leads implicitly to a translation unit. Therefore, you should not name your template definitions ".cpp", or any other suffix that the system takes as a source file.

You probably want to differentiate "regular" headers from the template definitions too, yet keep the name as a "header type" so that your IDE can do whatever fancy things IDEs do with C++ files.

[helios]

The build process right now looks roughly like this:

(Per file [compiler])
1. Preprocess.
2. Build AST.
3. Resolve names and types.
4. Resolve semantics (e.g. is "T foo(bar);" a construction or a function declaration?).
5. Generate code.
(Per executable [linker])
6. Merge everything.
7. Remove duplicate definitions from inline functions and templates.
8. Point cross-file calls to correct locations.
9. If LTCG is used, figure out cross-file inlining and do some more optimization.
10. Build final executable.

Assuming the build process was remade from the ground up, I'd say the easiest (or at least the most straightforward) method to implement extern templates would be to change the above to this:

(Per file [compiler])
1. Preprocess.
2. Build AST.
(Per executable [compiler])
3. Merge everything.
4. Resolve names and types.
5. Remove duplicate definitions from inline functions and templates.
6. Resolve semantics.
7. Generate code.
8. Build final executable.

Note that old steps 8 and 9 are no longer necessary because code generation happens after all files are merged, so the compiler not only already knows where each function will end up in memory and thus doesn't need to generate placeholder calls, but also able to see all inlining candidates, even if they came from other files.
Not that this implies this process would be faster.

Basically the idea is to merge things at the earliest opportunity, before doing any processing that would require information from other files.
If the process was reworked this much, though, header files would probably be redundant.