Efficient way to write huge boost dynamic_bitset vector to a file and read it back

[SyncMr]

I have a huge vector of boost dynamic_bitset. I want to write the dynamic_bitset vector to a file and later read the file back into a dynamic_bitset vector. Is the memory for dynamic_bitset allocated as a contiguous block of memory (so that I can write the entire vector at once without traversing) ?

The size of the bitset vector is in order of millions. So I am looking for an efficient way to write them to a file instead of iterating through the elements.

I converted the dynamic_bitset to a string and then wrote the string to a file. Later read the string from the file and converted it back to dynamic_bitset.

Below is the code I wrote in C++ using Visual Studio:

    #include "stdafx.h"
    #include <iostream>
    #include <fstream>
    #include <string>
    #include <boost/dynamic_bitset.hpp>
    using namespace std;
    
    int main(int argc, char* argv[])
    {
      // Initializing a bitset vector to 0s
      boost::dynamic_bitset<> bit_vector(10000000); 
      bit_vector[0] = 1;   bit_vector[1] = 1;  bit_vector[4] = 1;  bit_vector[7] = 1;  bit_vector[9] = 1;
      cout<<"Input :"<<bit_vector<<endl; //Prints in reverse order
    
      //Converting dynamic_bitset to a string
      string buffer;
      to_string(bit_vector, buffer);
    
      //Writing the string to a file
      ofstream out("file", ios::out | ios::binary);  
      char *c_buffer = (char*)buffer.c_str();   
      out.write(c_buffer, strlen(c_buffer));
      out.close();
    
      //Find length of the string and reading from the file
      int len = strlen(c_buffer);
      char* c_bit_vector = new char(len+1);
      ifstream in;
      in.open("file", ios::binary);
      in.read(c_bit_vector, len);
      c_bit_vector[len] = 0;
      in.close();
    
      //Converting string back to dynamic_bitset
      string str2 = c_bit_vector;
      boost::dynamic_bitset<> output_bit_vector( str2 );
      cout<<"Output:"<<output_bit_vector<<endl;
    
      system("PAUSE");
      return 0;
    }

But even this method, storing it as a string, takes a long time to write to the file. And when I try to read back from the file into the string, I get an "unhandled access violation exception". Is there a more efficient way to implement the same? Thanks for your help.

[Cubbi]

to_string() is pretty much the least efficient way to pull data out of a dynamic_bitset.

Take a look at to_block_range http://www.boost.org/doc/libs/release/libs/dynamic_bitset/dynamic_bitset.html#to_block_range

[kev82]

What's the distribution of data going to be like? How likely are there to be contiguous blocks of zero or one. First thought that enters my mind is to transform it into a binary tree, and and save/load that.

[SyncMr]

@Cubbi: thanks for the link. will have a look at it.
@kev82: The distribution is very random. So it is not possible to predict contiguous blocks of zero or one.