delete[]?

[Jeff Whittle]

I have a function which allocates a significant amount of dynamic memory

which is no longer required after it returns.

I followed the approach described in

http://www.cplusplus.com/doc/tutorial/dynamic/.

That is I allocated arrays such as:

int *xxx;
xxx = new int[n];

where n is large, and different for each invocation of the function.

I also put:

delete[] xxx;

just before returning from the function, but it doesn't have any effect.

Repeated invocations of the function result in an ever-increasing memory

Working Set.

What am I doing wrong?

Thanks for any help.

[tath]

Many things can be at fault here. Just show some code so it will be easier for us to find it.

[Jeff Whittle]

It's a big routine and I tried to show the essence of it. It is about 600 lines of code and won't fit within the 8192 character limit.

The deletes are at the end. The essentials are:

Array definitions:

double *objective;
objective = new (nothrow) double[nVars + 1];

double *rhs;
rhs = new (nothrow) double[nCons + 1];

char *sense;
sense = new (nothrow) char[nCons + 1];

int *matbeg;
matbeg = new (nothrow) int[nVars + 1];

int *matlen;
matlen = new (nothrow) int[nVars + 1];

int *matind;
matind = new (nothrow) int[nNonZ + 1];

double *matval;
matval = new (nothrow) double[nNonZ + 1];

double *lb;
lb = new (nothrow) double[nVars + 1];

double *ub;
ub = new (nothrow) double[nVars + 1];

double *x;
x = new (nothrow) double[nVars + 1];

These arrays are populated with appropriate values and passed to a third party DLL written in C.

After it has done its thing and we have retrieved the results we have:

delete[] objective;
delete[] rhs;
delete[] sense;
delete[] matbeg;
delete[] matlen;
delete[] matind;
delete[] matval;
delete[] lb;
delete[] ub;
delete[] x;

return;

[TarikNeaj]

You cant just put the delete[] wherever you want. You have to do it in the exact place it needs to be. With this little info I cant tell.

[Jeff Whittle]

What are the rules that determine where you can put the deletes/
All the code I showed is in the top level. There are no branches round it.

[D80DSL3A]

What you show looks OK on the face of it.
Perhaps the DLL is changing the pointer values to null before returning? Are these pointers passed by reference?
Try adding some output in your function to display the before and after values of these pointers as a check. I'd check right after the calls to new and just before delete[], in case any intervening code (including your own) is monkeying with the values.

[Jeff Whittle]

Thanks fun2code.

I have now checked (very carefully) the first and last pointers after creating them, after passing them to the DLL and after trying to delete them.

Once created, the pointers do not change. The values they point to are initially rubbish, then the expected values for [0] after they have been populated, then (a different) rubbish after they have been "deleted". This latter is despite the fact that the Private Working Set does not change.

I have now also discovered that a vector of vectors that I create and clear in the routine is not being deleted either.

When I get out of the routine, the Working Set goes up slightly (8k), which I find surprising.

The plot thickens!

Jeff Whittle

[JLBorges]

The free store may be getting fragmented.
Try one large allocation instead of ten small odd-sized allocations.

Something along these lines (untested):

#include <vector>
#include <cstddef>
#include <iostream>
#include <memory>

template <typename T> std::size_t rounded_size_max_aligned( std::size_t N )
{
    const std::size_t SZ = N * sizeof(T) + sizeof(std::max_align_t) - 1 ; // - 1 ;
    return SZ / sizeof(std::max_align_t) ;
}

void foo( std::size_t nCons, std::size_t nVars, std::size_t nNonZ )
{
    const std::size_t sz_objective = rounded_size_max_aligned<double>(nVars+1) ;
    const std::size_t sz_rhs = rounded_size_max_aligned<double>(nCons+1) ;
    const std::size_t sz_sense = rounded_size_max_aligned<char>(nCons+1) ;
    const std::size_t sz_matbeg = rounded_size_max_aligned<int>(nVars+1) ;
    const std::size_t sz_matlen = rounded_size_max_aligned<int>(nVars+1) ;
    const std::size_t sz_matind = rounded_size_max_aligned<double>(nNonZ+1) ;
    const std::size_t sz_matval = rounded_size_max_aligned<double>(nNonZ+1) ;
    const std::size_t sz_lb = rounded_size_max_aligned<double>(nVars+1) ;
    const std::size_t sz_ub = rounded_size_max_aligned<double>(nVars+1) ;
    const std::size_t sz_x = rounded_size_max_aligned<double>(nVars+1) ;

    const std::size_t N = sz_objective + sz_rhs + sz_sense + sz_matbeg + sz_matlen + sz_matind + sz_lb + sz_ub + sz_x ;
    const std::size_t NALLOC = (N+255)/256 * 256 ;
    std::clog << "allocating " << NALLOC * sizeof(std::max_align_t) << " bytes\n" ;

    std::vector<std::max_align_t> buffer(NALLOC) ;
    auto ptr = std::addressof( buffer.front() ) ;

    double* objective = reinterpret_cast<double*>(ptr) ; ptr += sz_objective ;
    double* rhs = reinterpret_cast<double*>(ptr) ; ptr += sz_rhs ;
    char* sense = reinterpret_cast<char*>(ptr) ; ptr += sz_sense ;
    int* matbeg = reinterpret_cast<int*>(ptr) ; ptr += sz_matbeg ;
    int* matlen = reinterpret_cast<int*>(ptr) ; ptr += sz_matlen ;
    double* matind = reinterpret_cast<double*>(ptr) ; ptr += sz_matind ;
    double* matval = reinterpret_cast<double*>(ptr) ; ptr += sz_matval ;
    double* lb = reinterpret_cast<double*>(ptr) ; ptr += sz_lb ;
    double* ub = reinterpret_cast<double*>(ptr) ; ptr += sz_ub ;
    double* x = reinterpret_cast<double*>(ptr) ;


    // ...
}

[Jeff Whittle]

Thank you JBL for the trouble you have gone to.

We tried your approach but it didn't make any obvious difference.

We have been investigating hard and now know a little more about the C++ memory handling system.

Basically C++ doesn't do garbage collection immediately, or even particularly soon. When we reached about 95% of physical memory it started to nibble away at the Working Set, initially it just kep it stable but then, a few minutes later, there were bigger drops.

Meanwhile the Commit increased inexorably, and eventually the program just stopped progressing much at all. Not only was there little CPU activity but there were few page faults either.

I think we have real memory leaks.

It is fustrating because this program was originally written in VB.NET which has memory compaction as well as garbage collection. That version runs indefinitely just fine.

Thanks again,

Jeff Whittle

[JLBorges]

> I think we have real memory leaks.

Have you used a (dynamic analysis) tool that can detect memory leaks?
For instance, Valgrind: http://valgrind.org/
Or enabling memory leak detection in the Microsoft CRT: https://msdn.microsoft.com/en-us/library/x98tx3cf.aspx

Also check for out of bounds write access to allocated memory blocks.

[Jeff Whittle]

Hi JBL,

Been away, so slow reply.

We have been doing a lot of work on this, and now think we may have solvd it. However, a lot of testing is still required because the application has to handle a very wide range of data and we need to be sure that it works in all cases (I only presented a small sub-set of our problem.)

Thank you for your input. I'll get back to you when we know more.

Jeff Whittle