why size of allocated memory to array is wrong ?

[mohsenti]

Hi,
I write simple program to allocate memory to char array.

#include <cstdlib>
#include <iostream>

using namespace std;

int main(int argc, char **argv) {
    cout << sizeof(char **) << endl;
    for(int i=0;i<409113;i++)
        char **row = new char*[4];
    cin.get();
}

base on size of "char **" total size of array must be 8*4*409113 = 13 MB but size of allocated memory in task manager is 29 MB!
if I comment "char **row = new char*[4];" , size of allocated memory in task manager is 1.6 MB.
What's wrong ?!

[z05DSL3A]

base on size of "char **" total size...

How 'big' do you think a pointer is?

[mohsenti]

a pointer size is 64bit in 64bit PCs (8 byte).

8*4*409113 = 13 MB

[z05DSL3A]

I don't think the code is doing what you are expecting, it is not doing what I was expecting from the text (didn't look to close at first).

[Chervil]

I'm not sure that the individual blocks allocated by new will be contiguous.

Though the elements of the array will occupy consecutive locations, the next array may not necessarily follow immediately after that.

Compare this, see how much is used in task manager:

    char **row = new char*[4*409113];

[coder777]

It is not wrong. There will be at least as much memory allocated as it is needed to hold the data. But it is very much possible that more memory is used for internal purpose (header data).

[Moschops]

You're looking in Task Manager.

Task manager does not tell you how much memory is allocated for those arrays.

[mohsenti]

You're looking in Task Manager.

Task manager does not tell you how much memory is allocated for those arrays.

Yes but in simple application like this, Task Manager tell array memory.

I test program with "valgrind", show memory allocated and it is 13 MB!

[Moschops]

Yes but in simple application like this, Task Manager tell array memory.

No it doesn't.

[helios]

Valgrind can show exactly how much memory has been requested by the code because it is designed to do just that. Task Manager will only show how much memory has been requested by the process. A process may request more memory that the programmer's code has requested. Code itself resides in memory and needs to be placed somewhere. The stack also resides in memory and needs to be placed somewhere. All these things use some memory not directly related to computing the result of the program.

[JLBorges]

> base on size of "char **" total size of array must be 8*4*409113 = 13 MB

Yes. The amount of allocated memory will be at least 13 MB


> I test program with "valgrind", show memory allocated and it is 13 MB!

Try some odd sized allocations: say 37 bytes each, or char **row = new char*[/*4*/ 5];

#include <iostream>
#include <new>
#include <malloc.h>

struct A
{
    char buffer[5] {} ;
    #ifdef NOT_TRIVIALLY_DESTRUCTIBLE
        ~A() {} // not trivially destructible
    #endif // NOT_TRIVIALLY_DESTRUCTIBLE

    static void* operator new( std::size_t sz ) { return ::operator new(sz) ; }
    static void operator delete( void* pv ) { ::operator delete(pv) ; }

    static void* operator new[]( std::size_t sz )
    {
        void* pointer = ::operator new(sz) ;
        std::cout << "#alloc request for " << sz << " bytes; return pointer " << pointer << '\n' ;
        return pointer ;
    }

    static void operator delete[]( void* pv )
    {
        std::cout << "#deallocate memory at " << pv << '\n' ;
        ::operator delete(pv) ;
    }
};

int main()
{
   #ifdef NOT_TRIVIALLY_DESTRUCTIBLE
        std::cout << "# *** NOT_TRIVIALLY_DESTRUCTIBLE ***\n\n" ;
   #endif // NOT_TRIVIALLY_DESTRUCTIBLE
    
    std::size_t N = 7 ;
    std::cout << "#require at least " << N * sizeof(A) << " bytes\n" ;
    A* array = new A[N] ;

    std::cout << "#array allocated at address: " << array << '\n' ;
    std::cout.flush() ; malloc_stats() ;
    
    delete[] array ;
    std::cout.flush() ; malloc_stats() ;
}

clang++ -std=c++14 -stdlib=libc++ -O2 -Wall -Wextra -pedantic-errors main.cpp && ./a.out 2>&1 | grep 'in use\|#'
echo -e '\n--------------------\n'
clang++ -DNOT_TRIVIALLY_DESTRUCTIBLE -std=c++14 -stdlib=libc++ -O2 -Wall -Wextra -pedantic-errors main.cpp && ./a.out 2>&1 | grep 'in use\|#'

#require at least 35 bytes
#alloc request for 35 bytes; return pointer 0x959010
#array allocated at address: 0x959010
in use bytes     =         48
in use bytes     =         48
#deallocate memory at 0x959010
in use bytes     =          0
in use bytes     =          0

--------------------

# *** NOT_TRIVIALLY_DESTRUCTIBLE ***
#require at least 35 bytes
#alloc request for 43 bytes; return pointer 0x227b010
#array allocated at address: 0x227b018
in use bytes     =         64
in use bytes     =         64
#deallocate memory at 0x227b010
in use bytes     =          0
in use bytes     =          0

http://coliru.stacked-crooked.com/a/af8d6021c7a65973

In this implementation, we can see that 48 bytes are allocated to service an allocation request for 35 bytes.

When the object is not trivially destructible, there is an overhead for array allocations (space for a magic cookie is also allocated). The array size is still 35 bytes, the allocation request is for 43 bytes (an eight byte magic cookie, possibly holds a pointer), and 64 bytes are allocated in all.

[nikcpp]

A 16-byte allocation granularity is perfectly normal assuming the compiler is optimising cache access based on paragraph alignment.