Fastest and Efficient way of file read and write

[nikhilb258]

What is the fastest way to read and write file in C++ ?

Data is plain text (human readable) not binary, with some control characters.

[keskiverto]

A call to OS-function that creates a copy of the file with the new name at the filesystem level?


The "real" answer depends on what are you reading from the file and what you have to do with it.

[coder777]

What is the fastest way to read and write file in C++ ?

Certainly using the system functions.

If using streams: setting the mode to binary might increase the speed since no conversion (regarding end of line) is done.

[AbstractionAnon]

What is the fastest way to read and write file in C++ ?

You don't indicate the type of data. It's difficult to answer your question without knowing the type of data.

If it's character oriented data (\n delimited), use istream and ostream, use >> or getline and << as appropriate
If it's fixed size record oriented data or binary data, use istream::read and ostream::write or operating system calls.

[Thomas1965]

If speed is really important and the file is in binary mode then forget the streams and use the good old stdio files - they are much faster.

[nikhilb258]

Data is plain text (human readable) not binary, with some control characters.

[Thomas1965]

Sorry I was not clear enough. Even when the data is plain text you can read and write it in binary format. Since you asked about the fastest way you should prefer the binary format. Here is a little speed comparison:

#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include <stdio.h>
#include <time.h>

class StopWatch
{
public:
  void start ();
  void stop ();
  void reset ();
  double seconds ();
  unsigned int milli_seconds ();
private:
  time_t m_Start,
    m_Stop;
};

void StopWatch::start ()
{
  m_Start = clock ();
}

void StopWatch::stop ()
{
  m_Stop = clock ();
}

void StopWatch::reset ()
{
  m_Start = m_Stop = 0;
}

double StopWatch::seconds ()
{
  return double (m_Stop - m_Start) / CLOCKS_PER_SEC;
}

unsigned int StopWatch::milli_seconds ()
{
  return (double (m_Stop - m_Start) / CLOCKS_PER_SEC) * 1000;
}

typedef std::vector<std::string> StringVector;

void writeText (StringVector& vec);
void writeBinary (StringVector& vec);

int main ()
{
  StringVector v (1000000, "Hello ");
  StopWatch sw;

  sw.start ();
  writeText (v);
  sw.stop ();
  std::cout << "\n\nTime for writing text file: " << sw.milli_seconds () << " ms";

  sw.start ();
  writeBinary (v);
  sw.stop ();
  std::cout << "\n\nTime for writing binary file: " << sw.milli_seconds () << " ms";
  std::cout << "\n\n";
  system ("pause");
  return 0;
}

void writeText (StringVector& vec)
{
  FILE* dest = fopen ("TextFile.txt", "wt");
  if (dest == 0)
  {
    perror ("writeText ");
    return;
  }
  for (const std::string& s : vec)
  {
    fprintf (dest, "%s", s.c_str ());
  }
  fclose (dest);
}

void writeBinary (StringVector& vec)
{
  FILE* dest = fopen ("TextFile.bin", "wb");
  if (dest == 0)
  {
    perror ("writeBinary ");
    return;
  }
  for (const std::string& s : vec)
  {
    fwrite (s.c_str (), strlen (s.c_str ()), 1, dest);
  }
  fclose (dest);
}

Output:

Time for writing text file: 733 ms

Time for writing binary file: 546 ms

[SamuelAdams]

Controlling the size of your writes will help a lot more than you might think.

In other words, your best throughput will be to fill a memory buffer and write full and large blocks of data.

Your device probably supports 4k writes as the smallest size. Every time you write a file you have overhead which will impact performance. So the larger sized files you can write means less overhead, hopefully better performance.