Read 2d array from txt file

tempester86 (4)
Hello.
I have a problem.
I have a huge txt file - 10.4 MB (13316 lines by 75 columns) that I have to read into an array.
Values are separated by spaces, end of line is '\n', not space.
I have tried multiple code variations with no success. Now I just got to this function.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
 
void Readm(int N, int M)				/*Test function. Reads matrix*/
{
	if(!datafile) //test the file open.
    {
                cout << "Error opening output file" << endl;
                system("pause");
    }
    while(!datafile.eof())
    {
		for(int R=0;R<N;R++)
			for(int C=0;C<M;C++)
				getline(datafile,A[R][C],' ');
	}
}


At one point in the process I added a verifier at the end of the function to see if it did it's job.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
 
void Readm(int N, int M)				/*Test function. Reads matrix*/
{
	if(!datafile) //test the file open.
    {
                cout << "Error opening output file" << endl;
                system("pause");
    }
    while(!datafile.eof())
    {
		for(int R=0;R<N;R++)
			for(int C=0;C<M;C++)
				getline(datafile,A[R][C],' ');
	}
cout << A[1][1];
}


It did not write out anything.
Noting that I wrote separate functions for readfromfile, sizeofline, linesinfile, columns [last two in case the program needed to be used for files with different numbers of lines/columns). I am just stuck on this one. I thought it could be because of the size of the file. It does take abou 10 seconds just to calculate the number of lines in the file. But when I try to use the read in matrix function, the program just goes nuts, or enters a weird loop that makes no sense.
I made alot of test functions to verify each step of the way. I will also attach the full code here.



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
 
#include "stdafx.h"
#include <iostream>
#include <time.h>
#include <fstream>
#include <istream>
#include <iomanip>
#include <sstream>
#include <string>
#include <stdio.h>
using namespace std;

/*Declaring unviversal variables*/

ifstream datafile ("randaex1.txt");				/*Original data file*/
ifstream testfile ("test1.txt");
ofstream outputfile;							/*Output data file*/
string line,a,b,auxiliar;
stringstream ss;
int number_of_lines,col,matrix[100][15000];
string A[4][5]; /*= {{1, 2, 3, 4, 5},
               {6, 7, 8, 9, 10},
               {11, 12, 13, 14, 15},
               {16, 17, 18, 19, 20}};*/



int readfromfile()								/*Test function. Reads the first line of the file into variable "line"*/
{
	if (datafile.is_open())
	{
		getline (datafile,line);
		datafile.close();
	}
	else
		cout << "Unable to open file";
	return 0;
}

int writetofile ()								/*Test function. Writes "line" to the output file.*/
{
	outputfile.open ("example.txt");
	outputfile << line << "\n";
	outputfile.close();
	cout << "\nThe line\n \n" << line << "\n\nwas copied to example.txt" << endl;/*Gives feedback of the copied line to user.*/
	return 0;
}

int sizeofline()							/*Test function. Calculates the size of a line*/
{
	cout << "\nThe size of line is " << line.size() << " characters.\n";
	return line.size();
}

int linesinfile()							/*Calculates the number of lines in datafile(the number of rows in matrix)*/
{
	number_of_lines=0;
	std::string liney;
    std::ifstream myfile("randaex1.txt");

    while (std::getline(myfile, liney))
	{
        ++number_of_lines;
	}

    std::cout << "Number of lines in text file: " << number_of_lines;

	system("pause");
	return number_of_lines;
}


int columns()							/*Calculates the number of columns in datafile*/
{
	
	col=0;
	for (int i=0;i<=line.size();i++)
		if (line[i]==' ')
			col++;
	col = col++;
	cout << col;
	return col;
}

void Readm(int N, int M)				/*Test function. Reads matrix*/
{
	if(!datafile) //Always test the file open.
    {
                cout << "Error opening output file" << endl;
                system("pause");
    }
    while(!datafile.eof())
    {
		for(int R=0;R<N;R++)
			for(int C=0;C<M;C++)
				getline(datafile,A[R][C],' ');
	}
	cout << A[1][1];
}


void Displaym(int N, int M)				/*Displays matrix*/
{
	for(int R=0;R<N;R++)
	{
		for(int C=0;C<M;C++)
			cout << A[R][C];
		cout << endl;
   }
}

int main()
{
	readfromfile();
	writetofile();
	sizeofline();
	getchar();
	linesinfile();
	getchar();
	columns();
	getchar(); 
	/*int cc=columns();
	int rr=linesinfile();*/
	Readm(13316,75);
	/*Displaym(13316,75);*/
	return 0;
}



I hope someone can help me.
Thank you.
infoartenovo (15)
What I don't understand is: why do you read such a big file in a 4 by 5 array of char:)?
It seems a case of buffer overflow: After you overcome the size of your array you are writing on the stack outside the limits of your program.
Probably, you wanted to read the matrix variable and, by mistake, you used the A variable.
Hope it'll help. Ciao
Last edited on
tempester86 (4)
and what is the limit of an array?
infoartenovo (15)
Anyway, the problem is not the limit of an array in C, but the fact that you are using the wrong array in the wrong place.
Your A array is a 4x5 array, but you try to access and write in memory locations that are outside your array (in the main you call Readm(13316,75)).
Moreover, even if you substitute the A array with the matrix array in the Readm, you have to check its use in the function. In fact, being matrix a 100x15000 array you can't call the Readm with (13316,75).
But you can safely call Readm with (75,13316).
Hope it helps.
Ciao.
Last edited on
tempester86 (4)
Thank you.
I fixed the problem with the array reading. I got the code so far that I let a small problem slide.
Thanks for your help. :)

Now all I have to do is include a function that will put every value in the string array into a double array, so i can work with the actual numbers in the text file.
Does C++ have a special function for that or do I have to figure out a way to convert string to double. I am not sure about the process as I know string is 8 bit I think and double, well, it shouldn't be just 8.

Any ideas would be great.
thanks again, guys!
tempester86 (4)
Hello.
I need some more help on the program. I get this weird error and it aborts.
It's because of the stod function i am using to convert the string array into a double array.
If I use matrix[0][0] = stod(A[0][0]); it works, but it aborts if i try to use the for sequence.
I am attaching the code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
 
// Randaprog.cpp : Defines the entry point for the console application.
#include "stdafx.h"
#include <iostream>
#include <conio.h>
#include <time.h>
#include <fstream>
#include <cstdlib>
#include <istream>
#include <iomanip>
#include <sstream>
#include <string>
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
using namespace std;
#define SCREEN_HEIGHT 50

ifstream datafile; 				/*Original data file*/
ofstream outputfile;
string infilename,outfilename,A[13316][75],line;
int option,menuin,menuout,menualg,number_of_lines,col;
double matrix[13316][75];

void clrscr()
{
	int i;
	for ( i = 0; i < SCREEN_HEIGHT; i++ )
		putchar ( '\n' );
}

int readfromfile()								/*Reads the first line of the file into variable "line"*/
{
	if (datafile.is_open())
	{
		getline (datafile,line);
		datafile.close();
	}
	return 0;
}

int writetofile ()								/*Test function. Writes "line" to the output file.*/
{
	cout << "\nThe line\n \n" << line << "\n\nwas copied variable line" << endl;/*Gives feedback of the copied line to user.*/
	return 0;
}

string Inputfilename()
{
	clrscr();
	cout << "What is the name of your data input file?\n";
	cin >> infilename;
	datafile.open(infilename);
	while(!datafile)
		{
			cout << "Filename invalid. Try again!\n";
			cin >> infilename;
			datafile.open(infilename);
		}
	if (datafile)			/*Verificare existenta fisier*/
		{
			cout << "Filename saved. Press any key...\n";
			getchar();
		}
	return infilename;
}

string Outputfilename()
{
	clrscr();
	cout << "What is the name of your data output file?\n";
	cin >> outfilename;
	outputfile.open(outfilename);
	while(!outputfile)
		{
			cout << "Filename invalid. Try again!\n";
			cin >> outfilename;
			outputfile.open(outfilename);
		}
	if (outputfile)		/*Verificare existenta fisier*/
		{
			cout << "Filename saved. Press any key...\n";
			getchar();
		}
	return outfilename;
}

int linesinfile()							/*Calculates the number of lines in datafile(the number of rows in matrix)*/
{
	datafile.open(infilename);
	cout << "Computing number of rows...\n";
	number_of_lines=0;
	std::string liney;
    while (std::getline(datafile, liney))
	{
        ++number_of_lines;
	}
	datafile.close();
	std::cout << "Number of rows is: " << number_of_lines <<". Press any key...\n";
	getchar();
	return number_of_lines;
}

int columns()							/*Calculates the number of columns in datafile*/
{
	datafile.open(infilename);
	cout << "Computing number of columns...\n";
	col=0;
	for (int i=0;i<=line.size();i++)
		if (line[i]==' ')
			col++;
	col = col++;
	datafile.close();
	cout << "Number of columns is: " << col <<". Press any key...\n";
	getchar();
	return col;
}

int Algorythm(int N, int M)
{
	clrscr();
	datafile.open(infilename);
	if(!datafile) //Always test the file open.
    {
                cout << "Error opening output file" << endl;
                system("pause");
    }
	cout << "Reading file to a string matrix...\n";
	while(!datafile.eof())
    	for(int R=0;R<N;R++)
			for(int C=0;C<M;C++)
				getline(datafile,A[R][C],' ');		
	for(int R=0;R<N;R++)
		for(int C=0;C<M;C++)
			matrix[R][C] = stod(A[R][C]);		
	getchar();
	/*cout << "Writing the matrix as double in the output\n";
	outputfile.open(outfilename);
	for(int R=0;R<N;R++)
			for(int C=0;C<M;C++)
			{
				matrix[R][C] = stod( A[R][C] );
				outputfile << matrix[R][C] << ' ';
			}
	outputfile.close();*/
	return 0;
}

int main()
{
	cout << "This program reads the 0.3 step data from an input file and generates an output file with 3 step data.\nPress any key to start...\n";
	Inputfilename();
	Outputfilename();
	readfromfile();
	columns();
	linesinfile();
	Algorythm(number_of_lines,col);
	getchar();
	return 0;
}
MikeyBoy (175)
I'd recommend using a debugger, so that you can see what values R, C and A[R][C] have at the point where your program crashes. That might give you a clue as to why it crashes.

Edit: I also recommend using some named constants for your array sizes, rather than magic number. That will help keep things consistent when you're making changes, and if you name them sensibly, it will give other people some understanding of what the numbers are supposed to represent and why they are set to those values.
Last edited on
Registered users can post here. Sign in or register to post.

C++

Forum