Theading - Please confirm

[donnyb]

I think I have an understanding of this. I need to have any variable or object on the heap for each thread, so that they do not conflic when processing multi-threads and then I do not have to mutex my varablies.
the question remains however if I pass a function or class the variable that I need worked on can the function or varable be on the stack, so as not to incure ovehead. So in short can I use the same processes for each thread.
thank you
Donald

[donnyb]

For clairification I would keep objects or var on heap that I want to change the values and the process would be shared with all the threads that do the work on the varables or objects.
Thanks,
donald

[Athar]

You can do everything that you are used to, except that you have to use locks if at least two threads write and read the same object.

[Galik]

I'm not sure I really understand the question but simply allocating your objects from the free-store is not enough to make them thread safe.

You still need to use mutex to access any object from more than one thread of you are going to be modifying it.

[kbw]

It sounds like you want them on the stack, not the heap. Each thread has it's own stack.

There's also thread local storage. The coming C++ standard with provide support in the language, but in the mean time, you'll have to use your OS native calls.

My preferred method is to use a thread class (not a free floating function). So all the data can be wrapped up around the thread and used independently with some guarantee of independence. Pete Becker wrote a simple thread class for C++ around '94, which is simple, clever and clear; it's gone on to be the basis of the Java thread class.

[donnyb]

Here is the thing I have a massive bottle neck on my program. I have figured out how to store the data once i get it. The problem is getting the information. Here is the code I am using it is to get information from the net from a number of url's.

void *UrlGetInfo(char* VarUrlToGet, int threadnumber)
{
    const char *p = VarUrlToGet; // get const char * representation
    //Reset string varable for getting data
    memory = NULL;
    UrlConnectionHtmlBody_size = 0;
    CURL *curl_handle;
    CURLcode res;
    curl_global_init(CURL_GLOBAL_ALL);

    /* init the curl session */
    curl_handle = curl_easy_init();

    /* set URL to get */
    curl_easy_setopt(curl_handle, CURLOPT_URL, p);

    /* no progress meter please */
    curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 1L);

    /* send all data to this function  */
    curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);

    /*
    * Notice here that if you want the actual data sent anywhere else but
    * stdout, you should consider using the CURLOPT_WRITEDATA option.  */

    /* get it! */
    res = curl_easy_perform(curl_handle);
    if(CURLE_OK == res)
    {
        //set the information for the body to the UrlInfo
        // this where I store the information into my deque
        // pointer Redirect Site
        char *ra;
        char *ip;
        long HttpResponse;
        /* get the CURLINFO_HTTP_CONNECTCODE*/
        res = curl_easy_getinfo(curl_handle, CURLINFO_RESPONSE_CODE, &HttpResponse);
        // this where I store the information into my deque
        /* ask for the ReDirectAddress*/
        res = curl_easy_getinfo(curl_handle, CURLINFO_REDIRECT_URL, &ra);
        if((CURLE_OK == res) && ra)
        {
            // this where I store the information into my deque
        };
        // Get the IP address for the web site
        res = curl_easy_getinfo(curl_handle, CURLINFO_PRIMARY_IP, &ip);
        if((CURLE_OK == res) && ip)
        {
            // this where I store the information into my deque

        };
    }
    free (memory);
    /* cleanup curl stuff */
    curl_easy_cleanup(curl_handle);
};

I use pthread.h and am calling the UrlGetInfo from a number of threads. I know that I have the creation of the threads right and the calling of the function right. I did my homework and to the best of my knowledge the UrlGetInfo functions and the the things in it are thread safe, except one. When I come to call the writedata functions is where the problems come in. Following your advice I am going to try and mutex the function and/or the varaibles in it. But I am kinda of running blind here.
Once I get the information I am going to store it into a deque, so I can rejoin the thread and then still use the information. The Main problem I am trying to solve here is that when I call these urls if there is a slow address resolution it holds up my computer, so I need to be able to get more then one at a time to stop the bottle neck.
Please feel free to comment.
Thanks,
Don

[donnyb]

If I use the following code to call GetUrlInfo I am creating a new copy of the function and its code I hope?

static void *StartUrlProcessing(void *VarUrlId)
{
    UrlConnection* UrlConn = new UrlConnection;
    unsigned long int a;
    a = (unsigned long int)VarUrlId;
    URL.UrlDomIndexKey[a] = a;
    if (URL.UrlDomIndexKey[a] == 1)
    {
        UrlConn->UrlGetInfo(URL.UrlDomIndexKey[a],"www.google.com");
    }
    if (URL.UrlDomIndexKey[a] == 2)
    {
        UrlConn->UrlGetInfo(URL.UrlDomIndexKey[a],"www.google.ca");
    }
    if (URL.UrlDomIndexKey[a] == 3)
    {
        UrlConn->UrlGetInfo(URL.UrlDomIndexKey[a],"www.dmoz.org");
    }
    return NULL;

}
// then in the main function
int i;
    pthread_t UrlThread[PSettings.Option_t];
    int UrlID = PSettings.CurrentUrlID;
    char *a;
    for (i = 0; i <= PSettings.Option_t; i ++)
    {
        a = (char*)UrlID;
        UrlID ++;
        ThreadError = pthread_create(&UrlThread[i],NULL,StartUrlProcessing,(void *)a);
        if( 0 != ThreadError)
        {
            fprintf(stderr, "Cound't run thread number %d, errorno %d\n",i,ThreadError);
        }
        else
        {
            fprintf(stderr, "Thread %d, gets UrlID Number %lu\n",i,PSettings.CurrentUrlID+i);
        }
    }
    // Put in Destructor to shut down program
    /* now wait for all threads to terminate */
    for (i = 0;i <= PSettings.Option_t; i ++)
    {
        ThreadError = pthread_join(UrlThread[i], NULL);
        fprintf(stderr, "Thread %d terminated\n", i);
    };

Ideally that is what I would like to do with the write function create a whole new copy of it and its varaibles.
I think I am a bit lost here. Will look up the information you provided me above and I did order a book online as they are very hard to find on the subject.
The Psettings.optionT just specifies the number of threads from an other class and it holds program information that does not apply.
Thanks,
Don

[Galik]

You can't create a whole new copy of the write_data function (or any function). What you need to do is make sure that it is thread-safe.

That means you only use the local stack to store temporary data and you make sure that the parameters you pass in point to unique data for each thread.

IIRC your write_data function received a void* right? Make sure that points to a struct (or class) object that was created specifically for that thread.

[kbw]

If you think of your thread function as something that get's a URL asynchronously, then you can make it completely self contained. So you pass it a structure with all the info it needs (passed in and returned). You really should use a thread class, but failing that, define a struct that you pass in.

e.g.

struct UrlInfo
{
    std::string url, reply;
    int threadid;
    //...
};

void* UrlGetInfo(UrlInfo* pInfo) { ... }

The point is, the function should fill in the reply in UrlInfo, rather than trying to use the result.

The caller should start the thread, and use the data when the data's available.

[bigearsbilly]

This might not help but I would suggest a simpler approach.
I've done something similar encrypting files.

have an input container (vector/deque) and push all the urls onto it.
have a similar output container for the results.

then each thread is very simple...
pop a value off the input deque, process and push onto the output deque.
obviously you need mutexes around the container operations.

I also suggest a wrapper class for threads and mutexes and condition variables.
It's a good investment as you can reuse it.