Forcing programming structure via virtual functions.

Hello,

I'm developing an extensible software framework, and I'm at a loss about implementing some features of the framework.

To summarize, the framework I'm developing has three layers. Bottom level is called foundation, and users of the framework will not touch there.

There are two upper levels called "Toolboxes" and "Flows". Toolboxes have the "Tools" to solve a specific kind of problem, and "Flows" define the order of utilization of the Tools from one or more toolboxes.

All toolboxes have the same minimum set of tools with same function signatures, but the functions' internals are completely different, hence I want to force a minimum template for these toolboxes to follow.

I've done the same thing with the Flows using classes and virtual prototype functions. So, the user have to implement that minimum set of functions.

Since toolboxes are not very suitable to packed as classes, I wanted to ask that whether are there any other ways to obtain this functionality and force a programming model in C++

Thanks in advance.
What is a toolbox at the language level?
Can you provide a minimal working example?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
┌──────────────┐
│              │          Well defined classes,derived from a base
│     Flows    │ ──────▶  class w/ virtual functions. Uses
│              │          functions from toolboxes.
└──────────────┘
┌──────────────┐
│              │          Idea: Libraries all adhering to a certain
│   Toolboxes  │ ──────▶  template. Providing concrete functions and
│              │          using foundation functions.
└──────────────┘
┌──────────────┐
│              │          Set of implemented and included headers.
│  Foundation  │ ──────▶  Provides utilities and basic functionality
│              │          of the framework.
└──────────────┘


Toolboxes are essentially .hpp files at the language level. They are plain old headers in the current iteration. However I want to force the programmers to adhere to a minimum template programmatically to keep code structure sound and provide them compiler-level guidance while extending the code.

Since the framework is scientific, excerpting meaningful code snippets is very hard. If you need more info, please tell me so, I will try to put together a small snippet together with the same structure.
Last edited on
C++ lets you inspect and verify types (at compile-time) and objects (at run-time). It doesn't offer a direct way to inspect the contents of a header file. It is possible - if usually inadvisable - to try and workaround that limitation.

In general, the solution is to perform whatever checking you require at the library boundary. But my impression is that currently the foundation layer is the library boundary, and the (only) other layer is ad-hoc user code, which you probably shouldn't interfere with.

Your description reminds me of libraries that implement rewriting systems.

A rewriting system is comprised of rewrite rules. An example rewrite rule performs the simplification from 0+x to x. In turn, a rewriting system defines a sequence of rewrite rules.
https://en.wikipedia.org/wiki/Rewriting
In terms of your model, a rewrite rule would be a tool, and a rewriting system would be the flow.

An important feature of these designs are that they make the tool a first-class object, and the flow is a library component that can perform some verification (even if its behavior is defined by the user). When tools are provided to the flow, they're not called ad-hoc by user code, but by the library, in well-defined circumstances.
Since toolboxes are not very suitable to packed as classes, I wanted to ask that whether are there any other ways to obtain this functionality and force a programming model in C++


You'll have to explain your meaning here, because I don't even want to say what that sounds like.

"Force a programming model"? What does that mean.

"Not very suitable to packed (I assume package) as classes". Certainly there are plenty of examples of non-class implementations, the algorithms library of the STL is an example. Since we have no real idea of your intent, this statement is entirely opaque.

However I want to force the programmers to adhere to a minimum template programmatically to keep code structure sound and provide them compiler-level guidance while extending the code.


I could write a chapter on the use of the word "force" here, twice actually including the original post. There are plenty of ways to establish pre-requisites, impose standards, etc.

Do you have experience using various libraries, like maybe Eigen, several of the Boost libraries, ESBTL, etc.?

Perhaps you want to investigate policy based templates. Virtual functions are used for runtime behavior selection based on a blind base type. Policies are template parameters to templates which mutate behavior at compile time based on user selection. Alexandrescu has an old book (2001) which discusses the technique. It is somewhat related, though does not exhibit, the curiously repeating template pattern, which you might also find applicable.



Have you written libraries before? Have they been peer reviewed?

If you sense a kind of uncooperative resistance from the various responses, I want to convey that we've had a rash of posts from users with 1 or 2 posts who are actually some kind of troll, sometimes hawking their own sites with links and possibly nefarious purposes.

Of course, I've encountered several with just a few posts with both genuine questions and most friendly intentions (I think I met a bio-chemist here, for example).


Last edited on
it sounds like you want something like

myflow.toolboxes[double_toolbox].mathfunction(stuff);
myflow.toolboxes[graphics_toolbox].drawfunction(otherstuff);

is this sort of what you are trying to say, or is it different? It makes no difference the types are (templates or function pointer type constructs will handle the details).
Last edited on
@mbozzi

Thanks for the answer. It helped a lot. You're actually right about not interfering with the users however, since the framework is scientific and designed for high performance, the toolboxes' and flows' implementation plays a critical role in achieving that performance.

Foundation of the framework provides some mechanisms and facilities for scaling the code (threading policies, helper functions for good data access practices, etc.). My idea was to pave the way for the future developers to implement their ideas right from get go, and interface with the foundation correctly.

Maybe also implementing tools as classes (with Java interfaces like skeleton base) makes sense. Also Niccolo's answers are helpful.

BTW, Flows can be also implemented by other developers, but they're monolithic pieces of well defined logic after they're perfected. So your description is spot-on.

This issue requires a bit more wrangling in the brain, but it's starting to make sense now.

Thanks again.
@Niccolo

Thanks for your answer.

I didn't develop software at that scale before. My second biggest project was built on top of JaDE framework with Java. I have written libraries for personal projects, but they didn't need to interface with other applications and developers.

This scientific framework is my Ph.D. thesis, and the idea is peer reviewed. I'm currently evolving it to increase code quality, performance and maintainability. Since it's a numeric-methods heavy code, all the matrix work is handled by Eigen actually. I'd like to provide a public repo, however since we didn't decide on the future of the code yet, I cannot do that. Sorry.

As I said in my previous answer, I wanted to make the future developers' life easier by showing them the right way about implementing things in the framework. This idea may be dumb/impractical/pointless. Even if it's pointless, I'm OK with that. At least I'll learn something by having a dumb idea. :)

Policy based templates sounds interesting. I'll certainly investigate that. I'll also try to find the book you mentioned.

I welcome the honesty about the language of your response. This is helpful in two ways. First, I'm a relatively shy and conflict-avoiding person. If I sense that I'm not wanted in somewhere, I just pack and leave. Secondly, if the person I'm conversing with is direct about its intentions, I can talk more freely and directly address his/her concerns.

Last, but not the least, I tend to ask questions about techniques, approaches, and best practices about a programming language. That kind of questions are harder to explain clearly with words most of the time, and triggers the anti-troll or similar defenses in most environments (this may also mean that I'm an unintentional dork, but it's another subject).

Thanks for all the help and the fish.

Regards.
@jonin

Actually Flows have a core function called evaluate(), and it calls the tools with required data as necessary. The toolboxes are #include d as normal libraries. Flows are devised as a class which are derived from a virtual base, so the implementation work is pretty similar to implementing Java Interfaces.

I want to create the same workflow/environment for the toolboxes, sans the classes, if possible. So evaluate (); function looks like the following in the simplest terms.

P.S.: Toolboxes define their own namespaces since they have the same functions to avoid conflicts and confusion.

1
2
3
4
5
6
7
void evalulate()
{
  t1::foo(&input0, &output0);
  t2::bar(&output0, &input1, &input3);
  t1::baz(&output0, &input5, &output1);
  /* and so on... */
}
Last edited on
I wanted to make the future developers' life easier by showing them the right way about implementing things in the framework. This idea may be dumb/impractical/pointless. Even if it's pointless, I'm OK with that. At least I'll learn something by having a dumb idea. :)


I wouldn't want to suggest pointless, or even misguided. Quite to the contrary. My own focus has been as a developer/engineer, so using C++ to write ambitious targets is a long practice for me. I was thinking, and perhaps erroneously interpreting, the nature of the descriptive term "force". Without any context to that point I wasn't sure what the implications, but from a software writer's perspective we thrive and many insist upon flexibility. Sometimes the various synonyms of "force" translate into brittle designs, but I sense now that may not be a problem for you.

Virtual classes are among the oldest paradigms in C++, and sometimes that shows its age. It has been successful, but was way overused in the early years (I remember, I was there when the language was new). It does present an interface which guide derived classes into compliance.

You may be a year early, though. Concepts, which are the C++20 expansion of templates, may be extremely expressive for your purposes. The problem is that the standards were just declared feature complete, and it will be until 2020 before compilers offer significant support. Only GCC offers limited support at this time, as an experimental option.

In the meantime, policy based templates can be summarized thus:

1
2
3
template <typename T> class A : public T
{
};


This is loosely related to the CRTP I mentioned, but is not exactly CRTP. Here, the template parameter becomes the base of A, which then infuses selected options into the resulting class. If one created class B, which was similarly declared to derive from its template parameter, this notion can nest.

While a virtual base class provides an interface which derived classes are expected to consume, class A would impose requirements upon T by virtue of the interface it expects (and would have code to use).

It is not customary for T to be a user class, but it could be. This provides compile time selection of behaviors without imposing the minor performance implications of virtual functions.

Alexandrescu uses this in Loki (also a project in Github as I recall, freely available). He builds a smart pointer with this technique, such that various types of pointers are assembled by policies (the T in the example above) and typedef'd to suit particular purposes.

I'm delighted to discover a genuine new member with real questions. We do see a large number of "if/then/else" questions and other elementary inquiry, some of them are worthwhile because of the member's personality. I've met a few here and on another site where pages of exchanges formed a kind of semester of education on some theory of the language or computer science.

I would be fascinated to learn more about where you're headed. I had the pleasure of assisting a PhD candidate in 2005, in those last days before his thesis had to be finished for the deadline. Though his code was beautifully done, it was written for a theoretically perfect computer with unlimited resources. It was crashing as a result. His doctorate was in computer science, but he almost missed the deadline. What he needed was an engineer to consult with, and I was privileged to participate.

I hope you have more time to work on this.

I wouldn't want to suggest pointless, or even misguided. Quite to the contrary. My own focus has been as a developer/engineer, so using C++ to write ambitious targets is a long practice for me.

Thank you, not being alone is a big motivation point :)

Without any context to that point I wasn't sure what the implications, but from a software writer's perspective we thrive and many insist upon flexibility. Sometimes the various synonyms of "force" translate into brittle designs, but I sense now that may not be a problem for you.


I've written code with extreme flexibility in the past, and that decision both saved my day and hurt my application in various ways. As a result I decided to implement my code with a rational balance of flexibility and well-defined design. Limiting all flexibility will result in brittle designs inevitably, but IMHO, striking a balance between flexibility and rigidity will result in a robust application with good maintainability and long development life.

Concepts, which are the C++20 expansion of templates, may be extremely expressive for your purposes. The problem is that the standards were just declared feature complete, and it will be until 2020 before compilers offer significant support. Only GCC offers limited support at this time, as an experimental option.

I don't have to rush, I have time (for now). I'll take a look into this, thanks.

I'm delighted to discover a genuine new member with real questions.

Thank you. I'm happy that I've succeeded in formulating my problem into a sensible, addressable question.

I would be fascinated to learn more about where you're headed.

The resulting code will be a scientific framework to solve some specific problems. I'm aiming to make the code easily extensible, maintainable and usable in bigger projects. Currently I'm exploring the ways to convert the code into a possible product, so I cannot give too much information. All I can say is I've successfully completed my Ph.D. in 2017 and the code I've produced can torture a modern system's all cores with relative ease. However for now, it saturates the memory bandwidth (the query/sec limit, not the transfer speed limit) faster than the cores, and I'll have to address it after finishing this refactoring run. :)

I hope you have more time to work on this.

I hope so. Actually, I really need to spend time on this code.

BTW, after some careful consideration, I decided to implement the middle layer as plain old headers for now. I have two reasons for this:
- This is a big refactoring run. I need to see how the end result will be in relatively short time.
- I want to see the end result and give its final shape without being limited for now. I've made some design errors in the past, and I don't want to shot myself in the foot again.

My idea is, after finalizing everything and making sure that it's the final design for a reasonable amount of time, I can fill it into a class and fix the design relatively easy. This decision doesn't mean that I've abandoned my aim, but I decided to act slowly and carefully on that front. I don't want to plan a further refactoring run to fix a broken design. It consumes too much time. :)

I'll keep this forum updated about the progress, if you wish. :)
I'm certainly interested, so please to drop a line now and then.

Registered users can post here. Sign in or register to post.