Removing redundancies from a text file

[Shahabonweb]

I have a text file in which there are numbers seperated by spaces stored in the following format:

1 2 3
1 3 4
1 2 3
2 3 4
1 3 5
1 3 5

Each line represents a sequence of 3 numbers. I want to create a file in which there are all the sequences represented above without duplication. So the new file should be like this:

1 2 3
1 3 4
2 3 4
1 3 5

What I have been trying is that I open the original file, read an array of 3 numbers, then open a new file and search within it whether that array is already in it or not, if it isn't, then write that array in it. However something goes wrong in that code, because of repeated opening and closing it. Can anyone help?
I can post the code if someone wants to take a look at it.

Thanks.

[JLBorges]

#include <fstream>
#include <tuple>
#include <set>

void make_unique( const char* srce_file, const char* dest_file )
{
    std::ifstream input(srce_file) ;
    std::ofstream output(dest_file) ;
    std::set< std::tuple<int,int,int> > triplets ;

    int i, j, k ;
    while( input >> i >> j >> k )
        if( triplets.insert( std::make_tuple(i,j,k) ).second ) // emplace(i,j,k)
            output << i << ' ' << j << ' ' << '\n' ;
}

[Shahabonweb]

Thanks for your reply but it isnt working. It simply states that there is no such file or directory named tuple. I do not have visual studio by the way but am programming in Code::Blocks.

[Cubbi]

@JLBorges: you forgot to output k

@thelastman If your compiler is old, you could do the same thing with pairs:

    std::set< std::pair<int, std::pair<int, int> > > triplets ;
...
        if(triplets.insert(make_pair(i, std::make_pair(j,k))).second )

[Maddier93]

Hi, I am new to these forums

@thelastman
How about using strings instead of containers?

[Shahabonweb]

Thanks every1. I have a question though. How do I modify the program to work for sequences of arbitrary length and not for length 3 only?

[JLBorges]

> How do I modify the program to work for sequences of arbitrary length and not for length 3 only?

C++11: std::tuple<> is of arbitrary length

C++98: boost::tuple<> http://www.boost.org/doc/libs/1_49_0/libs/tuple/doc/tuple_users_guide.html
or something home grown - for instance, for monomorphic sequences:

#include <fstream>
#include <algorithm>
#include <iterator>
#include <set>

// T is less_than_comparable, default_constructible, copy_assignable and streamable
template < typename T, std::size_t N > struct n_tuple
{
    inline bool operator< ( const n_tuple& that ) const
    { return std::lexicographical_compare( array, array+N, that.array, that.array+N ) ; }

    inline friend std::istream& operator >> ( std::istream& stm, n_tuple<T,N>& tup )
    {
        // quick and dirty
        for( std::size_t i = 0 ; i < N ; ++i ) stm >> tup.array[i] ;
        return stm ;
    }

    inline friend std::ostream& operator << ( std::ostream& stm, const n_tuple<T,N>& tup )
    {
        std::copy( tup.array, tup.array+N, std::ostream_iterator<T>( stm, " " ) ) ;
        return stm ;
    }

    T array[N] ;
};

template < typename T, std::size_t N >
void make_unique( const char* srce_file, const char* dest_file )
{
    std::ifstream input(srce_file) ;
    std::ofstream output(dest_file) ;

    std::set< n_tuple<T,N> > tuples ;
    n_tuple<T,N> tup ;
    while( input >> tup ) if( tuples.insert(tup).second ) output << tup << '\n' ;
}