Boost: those darn exceptions...

[IWishIKnew]

for(recursive_directory_iterator end, dir(p); dir != end; ++dir)
{
    subs.push_back(dir->path().string());
}

Great, but then boost hits an exception, and just completely gives up without trying... I have tried to figure out how to skip these problematic directories, but I can not seem to. Is there anyone who can help me out?

The exceptions it meets are "access denied", so it doesn't take long for it to just stop working. I need to be able to skip over these folders/files, without skipping over the end of it, and I can't find the answer anywhere.

[naraku9333]

Have you tryd catching the exception?

[IWishIKnew]

I can't catch it, because the exception throws during the for() condition.

This is exactly what it throws:

Exception thrown: boost::filesystem::directory_iterator::construct: Access is de
nied: "C:\$Recycle.Bin\S-1-5-18"

That's surely not the end, there are plenty more files to get, so that (and any other) directories like it need to be skipped, whithout hitting the actual end of the iterator.

[naraku9333]

Looks like this is a known issue https://svn.boost.org/trac/boost/ticket/6821
Try the method used in this thread (third code block) http://thisthread.blogspot.com/2011/07/using-recursivedirectoryiterator.html

[IWishIKnew]

no, it is not just that one directory.

Here are all of them:

boost::filesystem::directory_iterator::construct: Access is denied: "C:\Config.Msi"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Program Files\WindowsApps"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Network\Downloader"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Search\Data"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Windows\AppRepository"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Windows\LocationProvider"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Windows\Power Efficiency Diagnostics"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Windows\SystemData"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Windows Defender"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Windows NT\MSFax"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\System Volume Information"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\AppCompat\Programs"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\AUInstallAgent"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\LiveKernelReports"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\Logs\HomeGroup"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\Logs\SystemRestore"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\Logs\WindowsBackup"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\Minidump"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\ModemLogs"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\PLA\Reports"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\PLA\Rules"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\PLA\Templates"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\Prefetch"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\Resources\Themes\aero\VSCache"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\security\audit"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\security\cap"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\ServiceProfiles\LocalService"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\ServiceProfiles\NetworkService"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\Com\dmp"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\config"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\LogFiles\Fax\Incoming"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\LogFiles\Fax\Outgoing"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\LogFiles\Firewall"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\LogFiles\WMI"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\MsDtc"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\networklist"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\sru"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\Tasks"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\SysWOW64\Com\dmp"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\SysWOW64\config"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\SysWOW64\MsDtc"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\SysWOW64\networklist"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\SysWOW64\sru"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\SysWOW64\Tasks"

This was done using my own recursive directory iterator using boost's single-level directory iterator. I made it keep on going regardless of whether or not exeptions were thrown, and I made it write the exceptions to a file. This is not a single-directory problem, this is an OS issue.

It's actually nothing to do with the x64 bit versions, because my old 32 bit laptop returned these same types of exceptions en-mass. I would hypothisize that this has somthing to do with the properties of the folders/files, and that windows only gives restricted access to them.

Since this was run with non-admin priveliges, it never hits some of the lower-level files/folders.

Here are exceptions thrown when run with administrative privelidges:

boost::filesystem::directory_iterator::construct: Access is denied: "C:\Windows\System32\LogFiles\WMI\RtBackup"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Windows\LocationProvider"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\ProgramData\Microsoft\Windows\SystemData"
boost::filesystem::directory_iterator::construct: Access is denied: "C:\System Volume Information"

Using my same recursive iterator. I guess boost's recursive iterator likes to go deeper or somthing if the recycle bin is stopping it.

Either way, the simple fix is to allow the skipping of these directories, and I need to find out how.

[naraku9333]

Either way, the simple fix is to allow the skipping of these directories, and I need to find out how.

I believe the second link I posted does that. This block

void plainListTree(boost::filesystem::path path) // 1.
{
    dump(path, 0);
 
    boost::filesystem::recursive_directory_iterator it = createRIterator(path);
    boost::filesystem::recursive_directory_iterator end;
 
    while(it != end) // 2.
    {
        dump(*it, it.level()); // 3.
 
        if(boost::filesystem::is_directory(*it) && boost::filesystem::is_symlink(*it)) // 4.
            it.no_push();
 
        try
        {
            ++it; // 5.
        }
        catch(std::exception& ex)
        {
            std::cout << ex.what() << std::endl;
            it.no_push(); // 6.
            try { ++it; } catch(...) { std::cout << "!!" << std::endl; return; } // 7.
        }
    }
}

And his explanation

1. We don't need the user to start a recursion, and we don't need to keep track of the current level of recursion, since it is kept internally by the iterator.
2. Using a while loop does not look very natural in this context, but incrementing a recursive_directory_iterator could result in an exception, and we want it to be managed inside the loop, since we don't want the looping to be interrupted while an error happens before we terminate scanning all the elements.
3. Interestingly and usefully, recursive_directory_iterator stores in its status the current recursion level.
4. Actually, this functionality was in checkAndDump(), now we divorced it from dump() and let it live on its own. Another change is regarding what we actually do when we detect that we have at hand a directory that is also a symbolic link: we call no_push() on the iterator, that basically says not to go into the directory, but skip to the next element. That is exactely the behaviour we want in this context.
5. As said above, moving a recursive_directory_iterator could result in an exception (for instance, if the next element is a directory and we have no read access on it).
6. We couldn't access the next item in the collection, so we assume it refers to a directory that we can't access, and we ask the iterator class not to navigate in that directory but skip to the next element.
7. OK, this line should be written in a more expanded way, and we should try to recover in a more graceful way in case of troubles. But let just assume our previous assumption ("bad" directory) was right so that the catch clause is almost a paranoid check.

[IWishIKnew]

sorry, missed the link, lol. I'm quite wrapped up in college, so sorry about that.

I will look into that.

[IWishIKnew]

It is not compileing......

fsysclass.cpp: In function 'std::vector<std::basic_string<char> > sub_stuff_suppliment(const string&)':
fsysclass.cpp:309:65: error: 'createRIterator' was not declared in this scope
     boost::filesystem::directory_iterator it = createRIterator(p);
                                                                 ^
fsysclass.cpp:315:16: error: 'class boost::filesystem::directory_iterator' has no member named 'no_push'
             it.no_push();
                ^
fsysclass.cpp:325:16: error: 'class boost::filesystem::directory_iterator' has no member named 'no_push'
             it.no_push();
                ^

-,-

[cire]

Did you declare/define the createRIterator function and make it visible to the sub_stuff_suppliment function? (And, btw, that's not how supplement is spelled.)

Why are you using boost::filesystem::directory_iterator (which doesn't have a no_push member) as opposed to the boost::filesystem::recursive_directory_iterator you need (and which does have a no_push member?)

[IWishIKnew]

no, because I DON'T KNOW WHAT THE HELL IS IN THAT FUNCTION.

If I did not know this emplementation, why on earth would I know that?

for(boost::filesystem::recusive_directory_iterator end, dir(p); dir != end; ++dir)
{
    /blablabla
}

works, so I just assumed this would work:

boost::filesystem::recursive_directory_iterator it(p);

and it DOES. But then we go down, and aparently:

it.no_push() is "wrong".

[IWishIKnew]

oh, nvm. It's a directory iterator, not a recursive directory iterator.

Can it still hit symlink directories though? if so, how would I do it if no_push() is not a member of directory_iterator?

[cire]

no, because I DON'T KNOW WHAT THE HELL IS IN THAT FUNCTION.

If I did not know this emplementation, why on earth would I know that?

Because it was fully implemented in the link whose code you butchered.

Can it still hit symlink directories though?

You could try it and find out, or perhaps use the google. Perhaps you'll look up the iterator constructors in the boost documentation.