question on building a C++ library

[bostjanv]

Hello,

I have about 80 short C++ programs (without main) which I would like to
combine into a library. Each has the following structure:

+++start of program
#include "includefnm.h"

...

+++end of program

includefnm.h has some templates so that (I presume) when I compile the
programs individually some code gets copied into the .o file (about
20K). Therefore, when I use the ar program to build a library, I get a
really huge file.

QUESTION: is there a way to copy the code from "includefnm.h" only
once (e.g. with some linker instructions); or is the only solution to
combine by hand the individual programs into one single program with
only one include?

Regards,
bostjanv

[S G H]

If you have template-based classes and functions, you cannot put their code in a library.
As a generic rule, building a library usually makes your program bigger than leaving your code split in headers/sources.
This is because of some optimizations your compiler may take advantage of, since it knows which functions are needed and which functions can be omitted at link-time (not possible with shared libraries, probably possible with static libraries? not sure)
The memory-saving of libraries is usually a RAM-based save, and usually only for shared libraries.

Tips when trying to make your application smaller:
1. Avoid templates (or use as few templates as possible).
Consider a class that requires a template parameter.
For each parameterlist combination, an entirely new class is generated (!!!).
2. Use virtual inheritance instead of templates where possible.
Using virtual inheritance you can make that common code reusable.
3. Reusable code.
Make tiny functions. Try to keep them under 20 Lines of Code. They can call other tiny functions.
4. No inline.
Inline = bigger program = faster program.
5. Do not make it smaller.
Unless you're working with an embedded environment, there's usually no need to make your program smaller.

[bostjanv]

Thanks a lot for the advice. Regards, bostjanv

[dhayden]

I thing ESSGeEich is mistaken about libraries. On Solaris, you want to add your files to the library with

CC -xar -o libname yourObjectFile

. This will include the instantiated template functions, but only if needed: if a copy already exists in the library then it will be left out. There is probably a similar option for your compiler if it's different.

[Avilius]

Disclaimer: What I'm about to write is probably irrelevant to you, and to most library writers.

I'm assuming that this code you plan on putting into a library isn't really performance critical, but when building code that is called often, the overhead that virtual functions might not be ideal. 90 percent of the time it shouldn't matter, but when working on low-level systems such as 3D Game Engine renderers where you're sending a large quantity of data to a renderer that sorts out the data and sends it to the GPU in the most cost-effective way as possible. An example of a library that has poor performance due to this (and countless other things) is the rendering engine Orge, which is notorious for having poor performance under heavy loads.

Dynamic memory allocations (using new, or containers of any type (generally)), should probably be kept to a minimum.
__________________________________________

As I said, usually this will not be relevant to you, or your library (depending on the uses for it), but if you do plan to create such a library, I feel you should know this.

[kbw]

2. Use virtual inheritance instead of templates where possible.
Using virtual inheritance you can make that common code reusable.

How do you do that?

[S G H]

@kbw: In one of the books I have recently read, inheritance was used to sacrifice typesafety in a vector-like class, by deriving from a void*-based class as follows:

class BaseVector {
    void* m_data;
    std::uint32_t m_size;
    std::uint32_t m_alloc;
protected:
    void push_back(void*);
    void* at(std::uint32_t) const;
};

template <typename T>
class Vector : public BaseVector {
    // ... use BaseVector, casting the void* to T* where required
};

This obviously makes you remake push_back and at in Vector to make it public, but the code inside the function now is so short it can be easily inlined.

(Perhaps, "virtual" inheritance wasn't really "virtual", at least not in this example)

[kbw]

Interesting.

But as the inheritance is a mechanism to ensure the right heap is used, and not an interface, it should probably be private inheritance.

[S G H]

Members are private, functions protected, so it shouldn't matter a lot, but yes, it should be.

[bostjanv]

Hello,
Thanks for all the answers. I've probably received so much information that
I will need some time to digest it all. However, I wonder whether I can
make some immediate progress. So let me reiterate my immediate goal:

I would like to reduce the size of the C++ static library I am building.
Using the most direct method of building a static library (produce .o
files, then use ar to collect them into a library) I get a file of
size about 2.3M. On the other hand, a related library with much greater
capabilities written in C has size about 250K. Thus, I have a substantial
"C++ penalty", most probably due to the use of templates.

Regarding that I have received basically two proposals:

(1) (EssGeEich) remove templates from the library; and
(2) (dhayden) use something like
CC -xar -o libname yourObjectFile
in Solaris

Even though (1) is probably the soundest advice, it would require the
rewriting of a lot of code; therefore I would like to investigate the
possibilities of (2).

Since I am using gcc, I looked through the gcc manual, and found that
(one way) to reduce the amount of code generated by the use of templates
is to use the -frepo gcc option. I did that; but I could not determine
what are the further steps in building a static library, once the .rpo
files are generated. The use of ar, as was done previously, reduced the
size of the resulting library only by an insignificant amount. Any
further information would be appreciated.
Regards,
bostjanv

[dhayden]

Let's take a step back. Why do you want to reduce the size of the library? What really matters is the size of the executable and I don't think that will change.

[bostjanv]

Actually, yours is a good question. I suppose that what I want is a feeling
that I'm using the best possible solution; i.e, one that can't be improved
upon significantly. So, if I've reached that point I will simply take it
as a learned lesson in C++.

[TheIdeasMan]

If I may ask a related question? Hopefully the OP won't mind.

What happens if I write a program that only uses a sin() function, I link to the math library: Does the rest of all the stuff in the math library get optimised out? I am hoping the answer is yes.

If this question turns out to be epic, I will start my own topic.

Cheers

[modoran]

Usually the linker just throw away the rest of functions in that library.

[dhayden]

What happens if I write a program that only uses a sin() function, I link to the math library: Does the rest of all the stuff in the math library get optimised out?

It depends on the compilation environment I suppose but I think the basic unit in a library is the individual object file. So if you use sin() then your program will include everything whatever file sin() is defined in, but not the rest of the library (unless of course sin() refers to other symbols that are defined in other files in the library).

The same is true with your own programs: if you include an object file, the program will include everything in that object file. This is an important point since the effect can snowball in a large system: You use some class for a simple purpose. But the class is defined in one file and includes a complicated bit of code. That code in turn refers to some other bit of complicated code. Next thing you know, your 10 line program has included code for an entire web server.

For this reason sometimes it's worthwhile to break up the implementation of a class into individual files. I wrote an extreme case where a tool actually breaks the implementation files into one method per file so the user program will contain only what is actually needed. This is for code running on a hand-held calculator (the HP-50g).

[ne555]

> I would like to reduce the size
¿have you already did this http://stackoverflow.com/a/15314861 ?


> a tool actually breaks the implementation files into one method per file
> so the user program will contain only what is actually needed.
http://www.cplusplus.com/forum/general/114172/

[TheIdeasMan]

thanks to modoran, dhayden and ne555 for your informative answers :+)

And I have a HP-50-g ! Cheers