C++ Help printing to multiple files because output is too large

[dzhax]

Hello Everyone! First post, wish me luck!

I am a beginner with C++ and I have managed to make a program that permutates a string with repetition. I ran it to permutate "abcdefghijklmnopqrstuvwxyz1234567890" with a limit of 5 characters. This took a little over 5 hours for my pc to process this and I ended up with a .txt 403MB in size. Needless to say I am unable to open this .txt in notepad without Notepad.exe not responding and me having to end the process.

So what I want to do is modify my code to break up the output in to several files rather than one. Possibly all permutations starting with a in one file, b in another, etc.

Here is my current code:

#include <iostream>
#include <string>
#include <sstream>
#include <fstream>

using namespace std;

void swap(char *fir, char *sec)
{
    char temp = *fir;
    *fir = *sec;
    *sec = temp;
}

void permutation(char * arr, int size, char* result, int depth, int limit)
{
  ofstream myfile ("permutation.txt", fstream::app);
  if(depth == limit)
  {
    for(int a=0; a<limit; a++){
	  myfile << result[a] << "";
      cout << result[a] << "";
	}
	myfile << "\n";
	cout << endl;
  }
  else
  {
    for(int i=0; i<size; i++)
    {
      result[depth] = arr[i];
      permutation(arr, size, result, depth + 1, limit);
    }
  }
  myfile.close();
}

int main()
{
	ofstream myfile ("permutation.txt");
	myfile << "";
	myfile.close();
	string answer;
	char *rArray;
	string startProcess = "N";
	std::cout << "Welcome to permutation v1" << endl;
	std::cout << "-------------------------" << endl;
	std::cout << "Please enter how long the string should be: ";
	std::getline (std::cin,answer);
	int result = atoi(answer.c_str());
	rArray = new char[result];
	std::cout << "\n\nThank You!\n" << endl;
	std::cout << "Please wait, generating possible character array for length of " << result << "." << endl;
	std::cout << "Would you like to proceed? Y = yes & N = no: ";
	std::getline (std::cin,startProcess);
    char str[] = "abcdefghijklmnopqrstuvwxyz1234567890";
	if(startProcess == "Y")
	{
	  permutation(str, sizeof(str)-1, rArray, 0, result);	
	}
	else
	{
	  std::cout << "\n\nOperation Terminated. No permutations being generated..." << endl;
	}
	cin.get();
	return EXIT_SUCCESS;
}

As you can see it currently appends permutation.txt with all output. I would like it to make files like this permut_5char_a.txt, permut_5char_b.txt, etc. Being new to C++ I am lucky I got what I have so far (and no I did not code this from scratch, I have spoken to a multitude of people to get where I am).

If anyone would be willing to help me with this I would greatly appreciate it. My main issue is I don't know where to catch it changing from a to b to c etc, otherwise I could probably make it work.

[writetonsharma]

I have added some code.

#include <iostream>
#include <string>
#include <sstream>
#include <fstream>

using namespace std;

void swap(char *fir, char *sec)
{
	char temp = *fir;
	*fir = *sec;
	*sec = temp;
}

ofstream *pFilelist[27];

void createFileStreams(char* basename)
{
	int i;
	char fullname[255];
	for(i = 0; i < 26;i++)
	{	
		sprintf(fullname,"%s%s%c%s", basename, "_", (i + 'a'), ".txt");
		//ofstream *pStream = new ofstream();
		pFilelist[i] = new ofstream();
		pFilelist[i]->open(fullname);
	}
	sprintf(fullname, "%s%s%s", basename, "_misc", ".txt");		//filename starting with numbers
	pFilelist[i] = new ofstream();
	pFilelist[i]->open(fullname);
}

ofstream* getFileStream(const char *pString)
{
	if(isalpha(*pString))
	{
		return pFilelist[*pString - 'a'];
	}
	else
	{
		return pFilelist[26];		// filename starting with numbers
	}
}

void deleteFileStreams()
{
	//close the streams and delete the memory when the program closes
}

void permutation(char * arr, int size, char* result, int depth, int limit)
{
	//ofstream myfile ("permutation.txt", fstream::app);
	if(depth == limit)
	{
		std::string str;
		for(int a=0; a<limit; a++){
			//myfile << result[a] << "";
			//cout << result[a] << "";
			str += result[a];
		}
		ofstream* stream = getFileStream(str.c_str());
		*stream << str;
		//myfile << str;
		cout << str << "";
		*stream << "\n";
		cout << endl;
	}
	else
	{
		for(int i=0; i<size; i++)
		{
			result[depth] = arr[i];
			permutation(arr, size, result, depth + 1, limit);
		}
	}
	//myfile.close();
}

int main()
{
	//ofstream myfile ("permutation.txt");
	//myfile << "";
	//myfile.close();

	createFileStreams("permutation");

	string answer;
	char *rArray;
	string startProcess = "N";
	std::cout << "Welcome to permutation v1" << endl;
	std::cout << "-------------------------" << endl;
	std::cout << "Please enter how long the string should be: ";
	std::getline (std::cin,answer);
	int result = atoi(answer.c_str());
	rArray = new char[result];
	std::cout << "\n\nThank You!\n" << endl;
	std::cout << "Please wait, generating possible character array for length of " << result << "." << endl;
	std::cout << "Would you like to proceed? Y = yes & N = no: ";
	std::getline (std::cin,startProcess);
	char str[] = "abcdefghijklmnopqrstuvwxyz1234567890";
	if(startProcess == "Y")
	{
		permutation(str, sizeof(str)-1, rArray, 0, result);	
	}
	else
	{
		std::cout << "\n\nOperation Terminated. No permutations being generated..." << endl;
	}
	cin.get();
	return EXIT_SUCCESS;
}

PS: the size of file is not an issue. Use some good editor like notepad++ which will open a big files happily. I also expect that you will write the cleanup function yourself.

[IndieExe]

I would like it to make files like this permut_5char_a.txt, permut_5char_b.txt, etc.

#include <string>
#include <cmath>
#include <iostream>
using namespace std;

string _Convert(unsigned n){
    string c;
    c.resize(  1+ (log(n)) / 3.2 );//3.2 = log(25)
    for(unsigned l = c.size();l;c[--l] = (n%25) + 'a' ,n/=25);
    return c;
}

int main (){
    unsigned n = 0;
    //Keep track of n.
    //For each new file, generate the filename.
    string filename = "File_" + _Convert(++n) + ".txt";
    cout<<filename<<endl;
    //Then create the file and add content.
    return 0;
}

n=0

File_b.txt

n = 24

File_ba.txt

[SamuelAdams]

It will finish faster if you get rid of line 22 and 25
cout << result[a] << "";

[dzhax]

PS: the size of file is not an issue. Use some good editor like notepad++ which will open a big files happily.

When I tried to open the 403MB file in Notepad++ i get the following error: File is to big to be opened by Notepad++.

I also expect that you will write the cleanup function yourself.

Mind you im still a newb, what do you mean by a cleanup function?

[writetonsharma]

Not possible. Notepad++ can open files as big as in gigs. May be your need to upgrade or something else is the issue.

Did you try to run the program and look at what I have added ? If you trying to write a complex program, I expect that you should be capable enough that you can free the memory allocated. Which means, whatever memory is allocated by new should be freed by a delete call.

In case you cannot do so, you need to take a beginners book and learn basics. This is the only way to learn and move forward. Everytime, you are not going to get a working program from someone.

[dzhax]

is something like this what you mean?

void deleteFileStreams()
{
	//close the streams and delete the memory when the program closes
	int i;
	for(i = 0; i < 26;i++)
	{
		pFilelist[i]->close();
		delete[] pFilelist[i];
	}
}

[writetonsharma]

memory allocated with new [] should be deleted using delete []. When using new, it should be only delete.

The code you wrote is fine, just a small thing you are missing. Have a closer look, you are leaking some memory (not freeing all the memory).

[dzhax]

oh yea missing the misc file. would changing 26 in the for() to 27 fix this?

also to manage the output a little better how would I write these files to the folder "5" if i'm generating a list 5 character long. This way it's a little more organized. I tried "D:\permutation\5\permutation.txt" earlier but it didn't create the file and was not giving a compile error so I changed it to what it is in the original source posted.

[writetonsharma]

cool.. That will do it.

Remember to use delete only and not delete [].

[dzhax]

any insight on the different folders?

[writetonsharma]

there are functin calls to create folders. I think createdirectory() on windows.

[IndieExe]

Why don't you just keep one instance of ofstream to operate on
all required files?