Boost serialization

Pages: 1234
I would like to save objects from a parent class in a text file and then be able to load the objects. The problem is that all the objects are pointers so only the addresses of the objects could be saved but not the objects themselves.

For example:
1
2
3
4
5
6
7
8
9
10
11
12
Car {

int hp;
Gear *x;    // this object would in my case be a pointer

}

Gear {

double v;

}


So I would like to, for example, save the object that x is pointing to and not the address of the pointer. I am hoping to find a way to have different levels of saving content. Say, if I want to save everything or just want to save specific parameters. Then loading either everything or just the specific parameters. In the end I want to save if in a .txt file.

Here is my code for this case. The code isn't working unfortunately. Any ideas why?


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
#include <iostream>
#include <fstream>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>


class Gear {
  public:
    template<typename Archive>
    void serialize(Archive& ar, unsigned int version) { ar & v; }

    void setV (const double& _v) { v = _v; }
    double getV () { return v; }

    void status () { std::cout << "v = " << v << std::endl; }

  private:
    double v;
};

class Car {
  public:
    template<typename Archive>
    void serialize(Archive& ar, unsigned int version) {
      ar & hp;
      ar & x;
    }
    void setHP (const int& _hp) { hp = _hp; }
    void setGear (Gear* _Gear) { x = _Gear; }
    void status () { std::cout << "hp = " << hp << " Gear with v = " << x->getV() << std::endl; }

  private:
    int hp;
    Gear *x;
};

int main() {
  // Define new Gear:
  Gear* g = new Gear();
  g->setV(2.5);
  g->status();

  // Expectation is Car sets up the Gear.
  Car c;
  c.setHP(80);
  c.setGear(g);
  c.status();

  return 0;
}

Last edited on
You cannot save pointer. You need of cours dereference x: ar & *x;

if you want to allow null pointer for x you may add a bool:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
template<typenme Archive>
void serialize(Archive& ar, unsigned int version) {
  ar & hp;
  bool valid = false;
if(ar.is_loading())
{
  ar & valid;
  if(valid)
    x = new Gear{};
}
else
{
  valid = (x != nullptr);
  ar & valid;
}
if(valid)
  ar & *x;
}
> You cannot save pointer. You need of cours dereference
nope, boost_serialization will do its magic.
you do save the pointer, and when loading it will detect the class an do the memory allocation for you.
https://beta.boost.org/doc/libs/1_32_0/libs/serialization/doc/tutorial.html (see Pointers section)

> The code isn't working unfortunately. Any ideas why?
you need to be a little more descriptive
the code that you've posted doesn't compile:
- typos
- bad copy-paste
- trying to access private members
and then you have uninitialised data

so not sure what are you trying to show there
we shouldn't have to bother to fix all that, but if that's actually your code and you don't understand the compiler messages, then post them verbatim.
Hey sorry, you're right. Since I posted, I worked and updated the code. I edited the first post with the new code.
ok, ¿so what's the problem with the new code?
I don't see you using serialization anywhere.
1
2
3
4
5
6
7
8
9
10
//line 48
	std::stringstream ss;
	boost::archive::text_oarchive oa{ss};
	oa << c;

	Car b;
	boost::archive::text_iarchive ia{ss};
	ia >> b;
	b.status();
	//now `b' have the same status as `c' 
note that b.x and c.x are different pointers (and both leak)
Yea I'm still struggling on how I can get serialization to work. When I put your code in I get the following errors:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
user@debian:~/Desktop$ g++ -std=c++11 boost.cpp -lboost_system -lboost_archive
boost.cpp: In function ‘int main()’:
boost.cpp:48:21: error: aggregate ‘std::stringstream ss’ has incomplete type and cannot be defined
   std::stringstream ss;
                     ^
boost.cpp:49:38: error: no matching function for call to ‘boost::archive::text_oarchive::text_oarchive(<brace-enclosed initializer list>)’
   boost::archive::text_oarchive oa{ss};
                                      ^
boost.cpp:49:38: note: candidate is:
In file included from boost.cpp:3:0:
/usr/include/boost/archive/text_oarchive.hpp:98:5: note: boost::archive::text_oarchive::text_oarchive(std::ostream&, unsigned int)
     text_oarchive(std::ostream & os_, unsigned int flags = 0) :
     ^
/usr/include/boost/archive/text_oarchive.hpp:98:5: note:   no known conversion for argument 1 from ‘<type error>’ to ‘std::ostream& {aka std::basic_ostream<char>&}’
boost.cpp:53:38: error: no matching function for call to ‘boost::archive::text_iarchive::text_iarchive(<brace-enclosed initializer list>)’
   boost::archive::text_iarchive ia{ss};
                                      ^
boost.cpp:53:38: note: candidate is:
In file included from boost.cpp:4:0:
/usr/include/boost/archive/text_iarchive.hpp:139:5: note: boost::archive::text_iarchive::text_iarchive(std::istream&, unsigned int)
     text_iarchive(std::istream & is_, unsigned int flags = 0) :
     ^
/usr/include/boost/archive/text_iarchive.hpp:139:5: note:   no known conversion for argument 1 from ‘<type error>’ to ‘std::istream& {aka std::basic_istream<char>&}’
#include <sstream>
Of course. Many thanks for the help. It compiles now. Now my goal is to make sure I understand what exactly your code does. Do you recommend anything else besides https://beta.boost.org/doc/libs/1_32_0/libs/serialization/doc/tutorial.html?
I wanted to save the info to a .txt file so I used if/ofstream:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
#include <iostream>
#include <fstream>
#include <boost/archive/text_oarchive.hpp>
#include <boost/archive/text_iarchive.hpp>
#include <sstream>

class Gear {
  public:
    template<typename Archive>
    void serialize(Archive& ar, unsigned int version) { ar & v; }

    void setV (const double& _v) { v = _v; }
    double getV () { return v; }

    void status () { std::cout << "v = " << v << std::endl; }

  private:
    double v;
};

class Car {
  public:
    template<typename Archive>
    void serialize(Archive& ar, unsigned int version) {
      ar & hp;
      ar & x;
    }
    void setHP (const int& _hp) { hp = _hp; }
    void setGear (Gear* _Gear) { x = _Gear; }
    void status () { std::cout << "hp = " << hp << " Gear with v = " << x->getV() << std::endl; }

  private:
    int hp;
    Gear *x;
};

int main() {
  // Define new Gear:
  Gear* g = new Gear();
  g->setV(2.5);
  g->status();

  // Expectation is Car sets up the Gear.
  Car c;
  c.setHP(80);
  c.setGear(g);
  c.status();
 

  std::ofstream outputStream;
  outputStream.open("Car.txt");
  boost::archive::text_oarchive outputArchive(outputStream);
  outputArchive << c;
	outputStream.close();

  Car b;
  std::ifstream inputStream;
  inputStream.open("Car.txt", std::ifstream::in);
  boost::archive::text_iarchive inputArchive(inputStream);
  inputArchive >> b;
  b.status();
  inputStream.close();
  return 0;


}



And in the file I get:



1
2
22 serialization::archive 10 0 0 80 1 1 0
0 2.5


What does all of this mean? 2.5 makes sense but the others I'm not sure of.
What does all of this mean? 2.5 makes sense but the others I'm not sure of.
I guess it is not such a big surprise that it uses some header information for the data in order to do its magic.

Do you recommend anything else besides
https://www.boost.org/doc/libs/1_72_0/libs/serialization/doc/index.html
perhaps the xml version is clearer
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="17">
<c class_id="0" tracking_level="0" version="0">
	<hp>80</hp>
	<x class_id="1" tracking_level="1" version="0" object_id="_0">
		<v>2.50000000000000000e+00</v>
	</x>
</c>
</boost_serialization>

note that you've saved a Car object that references a Gear through a pointer.
so you have an identifier for the class (0: car, 1: gear)
iirc, the tracking_level tells you do serialise some pointer of that class, so it may handle the case were several cars point to the same gear
for example, another car may be serialized as
<another_car class_id="0">
   <hp>42</hp>
   <x class_id_reference="1" object_id="_0"></x>

version, because you may want backwards compatibility, your class may have changed, but you still want to read those old files.
(all this is just my guess, when in doubt, rtfm)


sadly, all this means that your "text" file is no more
you can't just open it on any editor and just touch it.
Thanks for the write up. I'm a bit lost. I don't have any experience with xmls and I read the manual and it doesn't touch on this (or at least I have yet to find where it discusses this stuff).

What do you mean no more? Why can't I just open it with nano Car.txt?

Thanks in advance.
what I mean is that you can't simply change the values because you may screw up the boost representation
it's quite hard to realise that these bold numbers are the ones holding your data.
1
2
22 serialization::archive 10 0 0 80 1 1 0
0 2.5

comparing that with the xml, it's easier to identify the '<hp>80</hp>' in all the other noise.
(or just understand what the hell you are storing)


changing the format input/output should be as simply as
1
2
3
4
//#include <boost/archive/text_iarchive.hpp>
//boost::archive::text_iarchive inputArchive(inputStream);
#include <boost/archive/xml_iarchive.hpp>
boost::archive::xml_iarchive inputArchive(inputStream);
sadly, that's not the case
you need to also use a BOOST_SERIALIZATION_NVP macro in all the serialization operations.
see the List of Examples on the tutorial of https://www.boost.org/doc/libs/1_72_0/libs/serialization/doc/index.html
Last edited on
Many thanks once again. I am a newbie at all of this so sorry if I am asking questions that may seem trivial to you (to me they are not). I just added BOOST_SERIALIZATION_NVP to the serialization operations as shown in the list of examples but I am now getting an output of both what you suggested with sstream and my output using f/iostream. That is, from the new addition to the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
std::ofstream outputStream;
outputStream.open("Car.txt");
boost::archive::text_oarchive outputArchive(outputStream);
outputArchive << BOOST_SERIALIZATION_NVP(c);
outputStream.close();

Car b;
std::ifstream inputStream;
inputStream.open("Car.txt", std::ifstream::in);
boost::archive::text_iarchive inputArchive(inputStream);
inputArchive >> BOOST_SERIALIZATION_NVP(b);
b.status();
inputStream.close();
return 0;


When I compile g++ -std=c++11 boost_serial.cpp -lboost_serialization I get on the terminal:

1
2
3
v = 2.5
hp = 80 Gear with v = 2.5
hp = 80 Gear with v = 2.5


And in the Car.txt file:

1
2
22 serialization::archive 10 0 0 80 1 1 0
0 2.5

Hey ne555, in the end I guess it doesn't really matter how the content in the Car.txt file looks like. The serialization works and that's what I need. Quick question, never used xmls before but want to learn. What command did you use to get these xmls:


1
2
3
4
5
6
7
8
9
10
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="17">
<c class_id="0" tracking_level="0" version="0">
	<hp>80</hp>
	<x class_id="1" tracking_level="1" version="0" object_id="_0">
		<v>2.50000000000000000e+00</v>
	</x>
</c>
</boost_serialization>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
#include <iostream>
#include <fstream>
#include <boost/archive/xml_oarchive.hpp> //includes
#include <boost/archive/xml_iarchive.hpp>
#include <sstream>

class Gear {
  public:
    template<typename Archive>
    void serialize(Archive& ar, unsigned int version) {
    ar & BOOST_SERIALIZATION_NVP(v); //use of the macro
  }
    void setV (const double& _v) { v = _v; }
    double getV () { return v; }

    void status () { std::cout << "v = " << v << std::endl; }

  private:
    double v;
};

class Car {
  public:
    template<typename Archive>
    void serialize(Archive& ar, unsigned int version) {
      ar & BOOST_SERIALIZATION_NVP(hp); //use of the macro
      ar & BOOST_SERIALIZATION_NVP(x);
    }
    void setHP (const int& _hp) { hp = _hp; }
    void setGear (Gear* _Gear) { x = _Gear; }
    void status () { std::cout << "hp = " << hp << " Gear with v = " << x->getV() << std::endl; }

  private:
    int hp;
    Gear *x;
};

int main() {
  // Define new Gear:
  Gear* g = new Gear();
  g->setV(2.5);
  g->status();

  // Expectation is Car sets up the Gear.
  Car c;
  c.setHP(80);
  c.setGear(g);
  c.status();
 
  {
    std::ofstream outputStream;
    outputStream.open("Car.xml");
    boost::archive::xml_oarchive outputArchive(outputStream);
    outputArchive << BOOST_SERIALIZATION_NVP(c); //use of the macro
    //outputStream.close() was making boost throw an exception
  }

  Car b;
  std::ifstream inputStream;
  inputStream.open("Car.xml", std::ifstream::in);
  boost::archive::xml_iarchive inputArchive(inputStream);
  inputArchive >> BOOST_SERIALIZATION_NVP(b); //use of the macro
  b.status();
  return 0;
}

Ahh got it. I thought you used a xml parser or something. Many thanks! I'm now trying to serialize armadillo matrices (or anything from armadillo) but I'm getting a few errors.

Code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#include <iostream>
#include <fstream>
#include <boost/archive/tmpdir.hpp>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <armadillo>


int main() {

arma::mat A = arma::randu<arma::mat>(4,5);


std::ofstream outputStream;
outputStream.open("bin.dat");
std::ostringstream oss;
boost::archive::binary_oarchive oa(outputStream);
oa & A;
outputStream.close(); getting error

arma::mat B;
std::ifstream inputStream;
inputStream.open("bin.dat", std::ifstream::in);
boost::archive::binary_iarchive ia(inputStream);
ia & B;
return 0;
}



Errors:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
In file included from /usr/include/boost/serialization/split_member.hpp:23:0,
                 from /usr/include/boost/serialization/nvp.hpp:33,
                 from /usr/include/boost/serialization/array.hpp:19,
                 from /usr/include/boost/archive/basic_binary_oprimitive.hpp:50,
                 from /usr/include/boost/archive/binary_oarchive_impl.hpp:22,
                 from /usr/include/boost/archive/binary_oarchive.hpp:21,
                 from arma_boost.cpp:4:
/usr/include/boost/serialization/access.hpp: In instantiation of ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_oarchive; T = arma::Mat<double>]’:
/usr/include/boost/serialization/serialization.hpp:69:69:   required from ‘void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_oarchive; T = arma::Mat<double>]’
/usr/include/boost/serialization/serialization.hpp:128:27:   required from ‘void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_oarchive; T = arma::Mat<double>]’
/usr/include/boost/archive/detail/oserializer.hpp:152:5:   required from ‘void boost::archive::detail::oserializer<Archive, T>::save_object_data(boost::archive::detail::basic_oarchive&, const void*) const [with Archive = boost::archive::binary_oarchive; T = arma::Mat<double>]’
/usr/include/boost/archive/detail/oserializer.hpp:101:1:   required from ‘class boost::archive::detail::oserializer<boost::archive::binary_oarchive, arma::Mat<double> >’
/usr/include/boost/archive/detail/oserializer.hpp:253:13:   required from ‘static void boost::archive::detail::save_non_pointer_type<Archive>::save_standard::invoke(Archive&, const T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
/usr/include/boost/archive/detail/oserializer.hpp:308:28:   [ skipping 2 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/usr/include/boost/archive/detail/common_oarchive.hpp:69:40:   required from ‘void boost::archive::detail::common_oarchive<Archive>::save_override(T&, int) [with T = const arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
/usr/include/boost/archive/basic_binary_oarchive.hpp:75:7:   required from ‘void boost::archive::basic_binary_oarchive<Archive>::save_override(const T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
/usr/include/boost/archive/binary_oarchive_impl.hpp:51:9:   required from ‘void boost::archive::binary_oarchive_impl<Archive, Elem, Tr>::save_override(T&, int) [with T = const arma::Mat<double>; Archive = boost::archive::binary_oarchive; Elem = char; Tr = std::char_traits<char>]’
/usr/include/boost/archive/detail/interface_oarchive.hpp:63:9:   required from ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator<<(T&) [with T = const arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
/usr/include/boost/archive/detail/interface_oarchive.hpp:71:35:   required from ‘Archive& boost::archive::detail::interface_oarchive<Archive>::operator&(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_oarchive]’
arma_boost.cpp:19:6:   required from here
/usr/include/boost/serialization/access.hpp:118:9: error: ‘class arma::Mat<double>’ has no member named ‘serialize’
         t.serialize(ar, file_version);
         ^
/usr/include/boost/serialization/access.hpp: In instantiation of ‘static void boost::serialization::access::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive; T = arma::Mat<double>]’:
/usr/include/boost/serialization/serialization.hpp:69:69:   required from ‘void boost::serialization::serialize(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive; T = arma::Mat<double>]’
/usr/include/boost/serialization/serialization.hpp:128:27:   required from ‘void boost::serialization::serialize_adl(Archive&, T&, unsigned int) [with Archive = boost::archive::binary_iarchive; T = arma::Mat<double>]’
/usr/include/boost/archive/detail/iserializer.hpp:192:5:   required from ‘void boost::archive::detail::iserializer<Archive, T>::load_object_data(boost::archive::detail::basic_iarchive&, void*, unsigned int) const [with Archive = boost::archive::binary_iarchive; T = arma::Mat<double>]’
/usr/include/boost/archive/detail/iserializer.hpp:120:1:   required from ‘class boost::archive::detail::iserializer<boost::archive::binary_iarchive, arma::Mat<double> >’
/usr/include/boost/archive/detail/iserializer.hpp:387:13:   required from ‘static void boost::archive::detail::load_non_pointer_type<Archive>::load_standard::invoke(Archive&, const T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/detail/iserializer.hpp:439:28:   [ skipping 2 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/usr/include/boost/archive/detail/common_iarchive.hpp:66:40:   required from ‘void boost::archive::detail::common_iarchive<Archive>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/basic_binary_iarchive.hpp:70:7:   required from ‘void boost::archive::basic_binary_iarchive<Archive>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/binary_iarchive_impl.hpp:50:9:   required from ‘void boost::archive::binary_iarchive_impl<Archive, Elem, Tr>::load_override(T&, int) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive; Elem = char; Tr = std::char_traits<char>]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:60:9:   required from ‘Archive& boost::archive::detail::interface_iarchive<Archive>::operator>>(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
/usr/include/boost/archive/detail/interface_iarchive.hpp:67:32:   required from ‘Archive& boost::archive::detail::interface_iarchive<Archive>::operator&(T&) [with T = arma::Mat<double>; Archive = boost::archive::binary_iarchive]’
arma_boost.cpp:26:6:   required from here
/usr/include/boost/serialization/access.hpp:118:9: error: ‘class arma::Mat<double>’ has no member named ‘serialize’



Anyone know what might be the problem?
just as you did Car::serialize, boost is expecting a mat::serialize() method
I would not recommend to modify armadillo, but perhaps you could make a global function
(see non-intrusive version on the boost::serialization documentation)
1
2
3
4
5
namespace boost {
namespace serialization {

template<class Archive>
void serialize(Archive & ar, arma::mat &m, const unsigned int version)


and to do the actual serialization you may use
http://arma.sourceforge.net/docs.html#save_load_mat
Thank you ne555. What would be the purpose of the void serialize() when one can save and load the matrices from armadillo directly? Also, I've tried to use the save and load from armadillo but am not getting a matrix for B:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
namespace boost {
namespace serialization {

template<class Archive>
void serialize(Archive & ar, arma::mat &m, const unsigned int version)
{

    // ar & m.zeros; 	    	
    // ar & m.col; 	    	
    // ar & m.at; 	    	
    // ar & m.rows; 	    	
    // ar & m.*; 	


}

  }
  }
int main() {

    arma::mat A = arma::randu<arma::mat>(4,5);
    arma::mat B = arma::randu<arma::mat>(4,5);
   
    std::cout << A<< std::endl;

    A.save("arma_save",arma::raw_binary);

    B = A.load("arma_save",arma::raw_binary);

    std::cout <<B<<std::endl;

    // std::ofstream outputStream;
    // outputStream.open("bin.dat");
    // // std::ostringstream oss;
    // boost::archive::binary_oarchive oa(outputStream);
    // oa & A;
    // // outputStream.close(); getting error

    // arma::mat B;
    // std::ifstream inputStream;
    // inputStream.open("bin.dat", std::ifstream::in);
    // boost::archive::binary_iarchive ia(inputStream);
    // ia & B;
    // std::cout << B<< std::endl;

    return 0;


}


Result:

1
2
3
4
5
6
   0.7868   0.0193   0.5206   0.1400   0.4998
   0.2505   0.4049   0.3447   0.5439   0.4194
   0.7107   0.2513   0.2742   0.5219   0.7443
   0.9467   0.0227   0.5610   0.8571   0.2492

   1.0000


Last edited on
when in doubt, rtfm
http://arma.sourceforge.net/docs.html#save_load_mat
«On success, .save() and .load() return a bool set to true»
so you did B=1;

I suppose that you simply want B.load("arma_save",arma::raw_binary);


> What would be the purpose of the void serialize() when one can save and load
> the matrices from armadillo directly?
brainfart on my part.
if .save() and .load() work for you, then use that.
Pages: 1234