Passing in Multiple Text files

[hollow60]

I have been working on code for quite some time and am able to successfully read in a text document and take certain words and information that I need. The issue is that I need to read in close to 100 plus documents and was wondering how I could read in the multiple documents. I thought about creating a structure of arrays and have each text document be an element and walk through taking each document but I am not sure how this works. I can post my code if necessary but its 400 lines so I'm not sure what to exactly post. [b]Any help would be much appreciated.

[TheKingOfTyrants]

I think the best answer depends on how exactly you are/how exactly you want to store data taken from the files? Does the information extracted from the file have to say associated with that particular file or can it all be heaped together? If not, you could make a function that takes the file name as a parameter and either returns the necessary information or stores it in a variable passed by reference. Then you would only require an array which stores the file names, iterate through said array, calling the function each time.

[hollow60]

I am reading in and parsing words from mltiple files, those said words plus their word count and position are stored in an array of structures. The information extracted needs to both be associated with the txt doc it was taken from and be heaped together (both have been done in the project already using one text document). Originally we thought of making the ifstream action, parsing, and storing, all parts of a function that could be iterated in the main. However the ifstream instance kept throwing and we decided, for now, to keep file IO's in the main program. with your same logic i should be able to do what your saying in the main program too. If i created a FileofFileNames.txt and read that in, stored it in an array and then incremented through the text document names i should be able to iterate through multiple ifstream reads, this is not the case though and it will not work. Any examples specific to what im talking about would be much appreciated, im trying to conceptually figure this out and incorporate it myself. I would rather not post code i just need a source to view and learn from. I have searched many threads similar to this topic but nothing seems to fit.

[TheKingOfTyrants]

So something like:

//FileofFileNames.txt
file1.txt
file2.txt
file3.text

#include <fstream>
#include <string>
using namespace std;
int main (void)
{
    ifstream list ("FileOfFilenames.txt", ifstream::in);
    while ( !list.eof() )
    {
        string fileName  << list;
        
        ifstream file (fileName, ifstream::in);
        //function to iterate through file
    }
}

wouldn't work?

[hollow60]

Thanks for the example. I think this may actually work. I just have to plug it in and mess around with it. Our original program is using c-strings but we may be able to switch over to strings. I really appreciate and hope we can incorporate it in some way for this is a big hurdle in our overall goal.

One issue were having which we had before was the fact that you are not allowed to have more than one ifstream open at a time, so at some point we have to open one and then close the other and then open it again. In this instance i believe it requires for ifstream list and ifstream file open at the same time. Do you have anything that could counteract that issue?

[TheKingOfTyrants]

You could open list, store all the file names in an array/vector, and then close it. From there you could just iterate through the array.

[hollow60]

The most recent thing I've came acrossed that seems quite possible is the idea of threads. Im not familiar with them at all but I was told we could use multi threads to open up multiple text documents and then somehow walk through that looking at each document. Are you familar with them at all or know how this may work?

[hollow60]

anyone out there have any information for me that could lead me in the right direction?

[JLBorges]

#include <iostream>
#include <fstream>
#include <string>
#include <map>
#include <vector>
#include <future>

// first == path to file second == count of occurrences of unique tokens
using result_type = std::pair< std::string, std::map< std::string, std::size_t > > ;

// get token-count of one file
result_type token_count( const std::string& path )
{
    result_type result { path, {} } ;
    std::ifstream file( path ) ;
    std::string token ;
    while( file >> token ) ++result.second[token] ;
    return result ;
}

// print first n tokens from one file
void print( const result_type& result, std::size_t n = 10 )
{
    std::cout << "file: " << result.first << '\n' ;
    std::size_t cnt = 0 ;
    for( const auto& pair : result.second )
    {
        if( cnt == n ) break ;
        std::cout << '\t' << ++cnt << ". " << pair.first << " - " << pair.second << '\n' ;
    }
}

int main()
{
    std::vector< std::string > paths_to_files { "a.txt", "b.txt", "c.txt" } ;
    std::ofstream( paths_to_files[0] ) << "one two three four five six seven one two three four five one two three\n" ; 
    std::ofstream( paths_to_files[1] ) << "to be or not to be that is the question be Whether 'tis it is nobler in the mind to be\n" ; 
    std::ofstream( paths_to_files[2] ) << "Happy families are all alike are all alike all alike alike every unhappy family is unhappy in its own way\n" ; 

    // http://en.cppreference.com/w/cpp/thread/future
    std::vector< std::future<result_type> > results ;
    // http://en.cppreference.com/w/cpp/thread/async.html
    for( const std::string& path : paths_to_files )
        results.push_back( std::async( std::launch::async, token_count, path ) ) ;

    for( auto& future : results )
    {
        // http://en.cppreference.com/w/cpp/thread/future/get.html
        const auto& result = future.get() ;
        print( result ) ;
    }
}

http://coliru.stacked-crooked.com/a/d535c0a03541f186