Problem with adding Symbols

[gregarion]

Hey guys , i would like to ask a question regarding some ways on how i can include certain characters into a string.

using namespace std;
 
int main()

{


string line;
string total;
  ifstream myfile ("tr.txt");
  if (myfile.is_open())
  {
    while ( myfile.good() )
    {
      getline (myfile,line);
      total = total + line;
    }
    myfile.close();
  }




typedef boost::tokenizer< boost::char_separator<char> > tokenizer_type ;

const char* const dropped_delimiters = "',/!@#$%^&*()-_=+|]}[{:;'?/>.<,\\" ; // nothing

const char* const kept_delimiters = " \t" ; // space, tab

boost::char_separator<char> separator( dropped_delimiters, kept_delimiters ) ;

tokenizer_type toker( total, separator ) ;

int cnt = 0 ;

for( tokenizer_type::iterator beg = toker.begin() ; beg != toker.end(); ++beg )

std::cout << "token " << ++cnt << ": '" << *beg << "'\n";

}

Basically , what the code above does is to read a text file containing a text. and then the tokenizer will tokenize it based on the dropped delimeters which i specify. So for example, if my text contains a sentence such as "Hey how , is your ! day?" , the tokenizer will seperate each word from each other thus individual tokens such as "Hey" , "how" will be formed.

[bartoli]

Those caracters should need a \ before them : http://msdn.microsoft.com/en-us/library/h21280bw%28v=vs.80%29.aspx
For the others, if they are in the first 128 ASCII characters, you should not have any problems. If not, you might have encoding related problems to solve too