Setting variable typename after initialization

[AcarX]

Hi guys, so here's what i got:

a Table class with std::vector<Row> variable
a Row class with std::vector<Data> variable
a Data template class

Now as you might have noticed std::vector<Data> won't work without adding <anytypename> after Data. What i wanna do is to read table data type from file and create a table with according data type such as unsigned long int/string/string. Is it possible ? Thanks.

[Gamer2015]

yeah it's possible but the ways to do it are very ugly.
The problem is that in C and in C++ the type of each variable has to be known at compile time.

Do you know the layout of the table when running your application?
Do you know what you'll be doing with the data and is it hard to do it by casting all the time?
How many external libraries do you use?

[AcarX]

Thanks for the reply. I'm not using any external libraries as far as i know(unless winsock2 is considered external). What i'll be doing with data is to send it over to another server(in this case client) via tcp socket so i'll have to cast it to char array(?) before sending what's requested.

[Gamer2015]

How to you know what datatype is used in the file?
Is the datatype of each column stored in the file?

Is that all you want to do?

[andywestken]

What i wanna do is to read table data type from file

What format is the file? How do you know the types at this point?

... and create a table with according data type such as unsigned long int/string/string. Is it possible ?

Two possible ways occured to me when I read your first post:

1. Polymorphism -- so you have a std::vector<Data*> rather than std::vector<Data> and then Data is the base class for StringData, DoubleDate, IntData, etc.

2. Variant -- see, e.g.

Boost.Variant
http://www.boost.org/doc/libs/1_58_0/doc/html/variant.html

But then I saw your later comment about wanting to cast it (each "Data" separately? or a vector?) to char*.

What i'll be doing with data is to send it over to another server(in this case client) via tcp socket so i'll have to cast it to char array(?)

This will be a potential problem for your strings!?

Rather than just casting to a char array you might need to serialize the data (esp. thinking about the strings.)

before sending what's requested.

Are individual data items being requested??

Andy

[AcarX]

@Gamer2015
Here's what my current account table file looks like:

tname:Accounts
datatypes:luint:string:string
columnnames:user_id:username:password
segmentinfo:a:5:s:6:t:7
1:account1:password2
2:ssdsad:password3
0:tsdasdad:passwor1

So as you can see when i read from file c++ won't know what kind of type it is until i manually check if it's equal to one of the custom types i declared such as if(read_data_type.compare("string") == 0)

@andywestken
At the moment it's just a simple text file. what i'm doing is first reading tablenames from dbname.database and then loading table data. About individual data, i'm thinking about having a similar system to what sql has. A request from client comes and is assigned to one of the workers(threads) on idle. Once the query executed main application sends the result to client. I tried using std::vector<boost::any> and it seemed to be working but i'm opened to better solutions. I had this idea to create a pointer for each data type i may use then when data types are read from file use new() to allocate memory to them but i'm not sure if that's a safe thing to do.

[Gamer2015]

Is the amount of datatypes limited?

I mean... you can allways do something like this?

std::string tname;
std::vector<std::string> types;
std::vector<std::string> colnames;

std::ifstream file("test.txt");
std::string line;
std::string entry;

// get table name
file.ignore(std::numeric_limits<int>::max(), ':');
file >> tname;

// get types
file.ignore(std::numeric_limits<int>::max(), ':');
std::getline(file, line);
std::stringstream ss(std::move(line)); // std::move gives little performance boost
while(std::getline(ss, entry, ':'))
    types.push_back(entry);

// get colnames
file.ignore(std::numeric_limits<int>::max(), ':');
std::getline(file, line);
std::stringstream ss(std::move(line));
while(std::getline(ss, entry, ':'))
    colnames.push_back(entry);

// whatever segmentinfo means
std::getline(file, line);

// process lines
while(std::getline(file, line)) {
    std::stringstream ss(std::move(line));
    int column = 0;

    // process entries
    while(std::getline(ss, entry, ':')) 
    {
        if(types[column] == "string") {
            // send string
        }
        else if(types[column] == "luint") {
            // send long unsigned 
        }
        else if( /* ... */ ) { /* ... */ }

        // ...

        ++column;
    }
}

you could take the part with the types and pack it in a function
or even better: you could declare a class called Type that has a member named type and a void* to hold any data and then you could make that class streamable.

That being said the problem still remains, you have to do that stuff somewhere, i think boost::any won't help you with your task ...

[AcarX]

Well your code is almost identical to what i have. That being said my problem was not being able to have a generic data type that would cast whichever data type column has. Also by using boost::any i have to use boost::any_cast each time i want to use it somewhere. Well looks like i have to give up on this idea i'll just use string for all kinds of data which may increase memory usage but i guess typename isn't too big of a deal as long as i know what the data will be used for.

[Gamer2015]

Well your code is almost identical to what i have. That being said my problem was not being able to have a generic data type that would cast whichever data type column has.

Yeah I thought so...
I don't have any ideas besides that
The only thing that comes to my mind is to hide the implementation but in C/C++ you have to know the types at compile time so you have to do something like that somewhere...

The only other thing that comes to my mind is you could make it 2 programs, 1 that parses the text and then recompiles the other one and the other one is a template that's easy to manipulate to fit your needs...
Then you could reread the file in the second program...

Yeah, I know, I'm out of ideas....

if possible you should use a scripting language, they are the right tools for your task but I guess that's not an option?

i'll just use string for all kinds of data which may increase memory usage but i guess typename isn't too big of a deal as long as i know what the data will be used for.

The other problem is the other side has to know what kind of data you send, right?
You'll have some functions like WriteInt16 or WriteFloat or ReadInt16 or WriteFloat.

So as

[AcarX]

My setup in my is like this: World Server/Game Server(s)/Login Server/Database Server. Database server will be communicating with all servers and i'll know what type of data i'm asking for when i'm sending a query. I never used scripting languages with c++ but if they're really helpful i don't mind using them.

[MiiNiPaa]

Example of storing stuff using inheritance: no casts needed, as soon as you have list of types in a line, you do not need to manually say what type to read. Output/input is trivial, like normal IO (look at the OUTPUT part of code)

Code is dirty, as it was written in haste and has room for improvement.
http://coliru.stacked-crooked.com/a/feaf9f12d51e8055

[AcarX]

@MiNiPaa Thanks a bunch. Working like a charm :)

[Gamer2015]

Did that really help you?

I mean, you still don't know what's in the Holder, right?
You need to know what type is in there, otherwise you can't use it anyway...
You could just treat all data as std::string and I think you'd get the same result...

I don't see the advantage here, could you please explain me?

#include <cassert>
#include <iostream>
#include <iterator>
#include <locale>
#include <memory>
#include <sstream>
#include <string>
#include <vector>
#include <stdexcept>

using row_t = std::vector<std::string>;

row_t parse(const std::vector<std::string>& types, std::istream& data)
{
    row_t row;
    for(auto& s: types) {
        std::string p;
        data >> p;
        row.push_back(std::move(p));
    }
    return row;
}

struct colon_space : std::ctype<char>
{
    static const mask* make_table()
    {
        // make a copy of the "C" locale table
        static std::vector<mask> v(classic_table(), classic_table() + table_size);
        v[':'] |=  space;  // colon  will be classified as whitespace
        return &v[0];
    }
    colon_space(std::size_t refs = 0) : ctype(make_table(), false, refs) {}
};


int main()
{
    std::istringstream input(R"(tname:Accounts
                                datatypes:luint:string:string
                                columnnames:user_id:username:password
                                segmentinfo:a:5:s:6:t:7
                                1:account1:password2
                                2:ssdsad:password3
                                0:tsdasdad:passwor1)" );
    std::string temp;

    //PARSING HEADER
    std::getline(input, temp); //skip First line
    std::getline(input, temp); //read Second line
    std::istringstream row;
    row.imbue(std::locale(row.getloc(), new colon_space)); //Threat : as space.
    row.str(temp);
    row >> temp; assert(temp == "datatypes");
    std::vector<std::string> types {
        std::istream_iterator<std::string>(row),
        std::istream_iterator<std::string>()
    }; //read data types
    std::getline(input, temp); //skip Third line
    std::getline(input, temp); //skip fourth line

    //ACTUALLY READ DATA
    std::vector<row_t> data;
    while(std::getline(input, temp)) {
        row.clear();
        row.str(temp);
        data.push_back(parse(types, row));
    }

    //OUTPUT
    for(const auto& r: data) {
        for(const auto& e: r)
            std::cout << e << ' ';
        std::cout << '\n';
    }
}

[MiiNiPaa]

I don't see the advantage here, could you please explain me?

By changing read/write functions or adding new interface you might delegate responsibility to handle different data types to itself.
For example integers might return 4 char string containing 2complement number representatin in network order, strings might return in pascal string format (1 byte of size and size bytes of content), etc.
You won't see benefit in text IO, but when sending/receiving data over network/binary formats it is quite useful.

[Gamer2015]

By changing read/write functions or adding new interface you might delegate responsibility to handle different data types to itself.

you convert it to a string when sending it anyway so it does not really make a difference, right?

std::ostream& operator<<(std::ostream& out, const Holder& h)
{ return out << h.to_string(); }

Well, I do understand what you mean, so you'd implement a h.write function (just like your read function) and declare it like this, right?

std::ostream& operator<<(std::ostream& out, const Holder& h)
{ return h.write(out); }

template <typename T>
std::ostream& ConcreteHolder<T>::write(std::ostream& os)
{
    return on << value;
}

[MiiNiPaa]

you convert it to a string when sending it anyway so it does not really make a difference, right?

It does make a difference. Sending "256" (text representation of value) and "\0\0\0x01\0" (bytes of number in netwok order) would be different, don't you think. It is how we convert specific data type into string which matters.

so you'd implement a h.write

I would change to_string actually. And then somebody will refactor its name to "write" later.

[AcarX]

Well i already decided that i wouldn't be using this on database server but when i want to create an entity system for client side i could use it. I'm still reading articles about typename hiding which seems to be an importing topic when it comes to entity systems.

Also you could create your own castings by simply adding virtual int to_int() const = 0; to Holder class and virtual int to_int() const override; to ConcreteHolder then overriding function for ConcreteHolder like:

template <typename T>
int ConcreteHolder<T>::to_int() const
{
if(IsInt(to_string())){
std::stoi(to_string());
}
}