random line from file

[JesusH1]

Hello all,

Intro

I'm trying to read in a file and randomly grab a line. The file "Spiral_Periods_5.dat" has 6 columns and millions of lines and the size is around 100 gigabytes:

37.3253288 34.6592178 2.68544000E+10 1.14858511E-03 3.11963275E+16 2.68544205E+10
.....
.....

Eventually I want to save the randomly selected line to a file if the last column is less than 2000.

Attempt

I'm having all kinds of trouble seeking to a random line, yet alone parsing the line. I'm trying to seekg to a random spot but I don't know how to make it get the beginning of the line.

#include <iostream>     // std::cout
#include <fstream>      // std::ifstream
#include <cstdlib>
#include <string>
using namespace std;

int main () {
	srand(time(0)); //seed for randomness
	ifstream in ("Spiral_Periods_5.dat");
	if (in) {
		// get length of file:
		in.seekg (0, in.end);
		int length = in.tellg();
		in.seekg (0, in.beg);
		// alocate memory:
		//cout << length<< endl;
		string line;
		getline (in,line);
		int line_length = line.length();
		//cout << line_length << endl;
		int randomPlace = ((rand()%length+1));
		//cout << randomPlace << endl;
		//in.seekg (0, in.beg);
		in.seekg (randomPlace);
		getline (in, line);
		cout << line << endl;


	}
	return 0;
}

I just can't seem to calculate the correct multiples in line 21 for the seekg to move to the beginning of a line.

Thanks for all the help :)

[JLBorges]

> I just can't seem to calculate the correct multiples in line 21 for the seekg to move to the beginning of a line.

To read a line, use std::getline() http://www.cplusplus.com/reference/string/string/getline/

To parse the line and get the value of the last column into a double, use std::istringstream

Divide and conquer: break up the code into small functions, each of which is easy to write, read and understand.

For instance (untested):

#include <string>
#include <fstream>
#include <sstream>
#include <ctime>
#include <random>

std::string get_last_col( const std::string& line )
{
    auto pos = line.find_last_of( " \t\n" ) ;
    return line.substr( pos == std::string::npos ? 0 : pos ) ;
}

bool is_eligible( const std::string& col, double ubound = 2000.0 )
{
    std::istringstream stm(col) ;
    double d ;
    return stm >> d >> std::ws && d < ubound && stm.eof() ;
}

std::istream& get_eligible_line( std::istream& stm, std::string& line )
{
    std::string temp ;
    while( std::getline( stm, temp ) )
    {
        if( is_eligible( get_last_col(temp) ) )
        {
           line = temp ;
           return stm ;
        }
    }

    return stm ;
}

std::string random_eligible_line( const char* path2file )
{
    static std::mt19937 twister( std::time(0) ) ;

    std::string selected ;
    std::ifstream file(path2file) ;

    std::string line ;
    std::streamsize nlines = 0 ;
    while( get_eligible_line( file, line ) )
    {
        ++nlines ;
        // select this line with probability 1/nlines
        if( std::uniform_int_distribution<std::streamsize>( 0, nlines )(twister) == 0 )
            selected = line ;
    }

    return selected ;
}