Exception thrown when allocating big array with new

[kalkas]

Hi

I'm writing a program with Visual Studio Express 2008 using Windows Forms, that is
C++/CLI I think - not sure, and because I want to use OpenGL I am using a Native Window object or something like that( I am not pretty sure for all the word cause it's kind complicating for me but I think using managed with unmanaged code has something to do with my problem, hope you'll understand).

Now, I have a bit of code where I dynamically allocate memory for an array with the keyword new. All running good until I try to allocate memory around 512*512*373 chars it is throwing an exception. I know I have enough memory free. I have read similar problems others have and all the solution suggested to them was like "the bound are wrong" or "the step is wrong" etc. I asure you everything is correct, besides it's working with smaller numbers. I have found something that I did not understand and has to do with the CoTaskMemAlloc() function.

Thoughts?

[Disch]

Have you tried catching the exception and printing it? That's likely to tell you exactly what the problem is.

[kalkas]

External component has thrown an exception.

First, while you have enough memory, what is another reason for new to throw an exception? Note that only when the numbers increase the exception is thrown.

[kalkas]

Meanwhile, I tried CoTaskMemAlloc(), it did allocate the memory but when I try to write to that array the program crash with this error:

"An unhandled exception of type 'System.AccessViolationException' occurred in VolumeRenderer.exe

Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt."

the piece of code is this:

if(_depth == 8){
		for(int i=1; i<_x-1; i++){
			for(int j=1; j<_y-1; j++){
				for(int k=1; k<_z-1; k++){
					//_normals is the pointer to the array allocated
					//with CoTaskMemAlloc()
					_normals[i*_y*_z*3 + j*_z*3 + k*3 + 2] = (_v[(i+1)*_y*_z + (j+1)*_z + (k+1)]/3 + 
															  _v[(i+1)*_y*_z + (j+1)*_z + (k+0)]/2 +
															  _v[(i+1)*_y*_z + (j+1)*_z + (k-1)]/3 + 
															  _v[(i+1)*_y*_z + (j+0)*_z + (k+1)]/2 +
															  _v[(i+1)*_y*_z + (j+0)*_z + (k+0)]/1 +
															  _v[(i+1)*_y*_z + (j+0)*_z + (k-1)]/2 +
															  _v[(i+1)*_y*_z + (j-1)*_z + (k+1)]/3 +
															  _v[(i+1)*_y*_z + (j-1)*_z + (k+0)]/2 +
															  _v[(i+1)*_y*_z + (j-1)*_z + (k-1)]/3 -
															  _v[(i-1)*_y*_z + (j+1)*_z + (k+1)]/3 - 
															  _v[(i-1)*_y*_z + (j+1)*_z + (k+0)]/2 -
															  _v[(i-1)*_y*_z + (j+1)*_z + (k-1)]/3 - 
															  _v[(i-1)*_y*_z + (j+0)*_z + (k+1)]/2 -
															  _v[(i-1)*_y*_z + (j+0)*_z + (k+0)]/1 -
															  _v[(i-1)*_y*_z + (j+0)*_z + (k-1)]/2 -
															  _v[(i-1)*_y*_z + (j-1)*_z + (k+1)]/3 -
															  _v[(i-1)*_y*_z + (j-1)*_z + (k+0)]/2 -
															  _v[(i-1)*_y*_z + (j-1)*_z + (k-1)]/3)*8.6f;

[kbw]

You need to check those index values. It's not obvious that they're correct.

[kalkas]

Ok, as I say before, I don't need that kind of answer. "The index is wrong". If the index is wrong explain with the allocation fails. What does the index has to do with the allocation? Besides, it's working when I try to allocate less memory.

Here is where it fails the first time:

try{
		_normals = new float[_x*_y*_z*3];
	}catch(System::Exception ^ e){
		_normals = (float*)CoTaskMemAlloc(_x*_y*_z*3);
		if(_normals != NULL){
			Windows::Forms::MessageBox::Show("Komple");
		}
		else{
			Windows::Forms::MessageBox::Show("Normals\n" + e->Message);
			_file->Close();

			return 0;
		}

The second is an experiment with the CoTaskMemAlloc() where I also need an explanation.

[cire]

CoTaskMemAlloc returns NULL on failure. Where is the check for that?

First, while you have enough memory, what is another reason for new to throw an exception?

new throws an exception when the memory requested is unable to be allocated. The memory you are requesting must be contiguous, so if there is not enough contiguous (unfragmented) memory available to the operating system/program... you're out of luck.

How much memory are you trying to allocate?

[kbw]

Ok, as I say before, I don't need that kind of answer

If you have all the answers and don't post your actual code, then why are you posting a question?

[kalkas]

@cire : Ok now we are starting to talking.(I updated my code above to see that I do check if the returned value is NULL)
I'm trying to allocate 512*512*373*3 floats. When this happening(the "not enough contiguous memory") how can someone handle it? Is it a dead-end?
I read somewhere, and here is where CoTaskMemAlloc() falls into, that you use CoTaskMemAlloc() when a proccess tries to allocate memory that crosses process boundaries. I don't really understand that but I get that every proccess has a limit on how much memory will use(if this true, how do you increase that limit?). Note that I noticed when I terminate the program, Visual Studio posts that 2 programs has terminated, the "MyProgramName.exe : Managed" and the "MyProgramName.exe : Native"

The program '[4712] MyProgramName.exe: Managed' has exited with code 0 (0x0).
The program '[4712] MyProgramName.exe: Native' has exited with code 0 (0x0).

and I sure this has something to do with the mixing of managed and unmanaged code.

@kbw : I was going to ingnore you but I won't because it's rude. I don't need that kind of answers means I have all the answers of that kind not all the answers of all kind.

[cire]

I'm trying to allocate 512*512*373*3 floats.

On my system that's 1173356544 bytes, which is over a gigabyte of memory.

When this happening(the "not enough contiguous memory") how can someone handle it? Is it a dead-end?

You could kill tasks to make it less likely memory is fragmented between multiple programs prior to running your program. You could try to work with less memory. You might pre-allocate the array at a global scope and avoid the heap.

If it is something to do with mixing of managed and unmanaged code, I have no insight to give.

[kalkas]

You probably right, 1,1GB contiguous memory with 1,6 free memory must be hard to find, but I've tried it with 832*832*494 unsigned chars which are around 350MB. This time I had to allocate to arrays of this, the allocation with new fails on the first array and when I try to allocate them with CoTaskMemAlloc(), it allocates the first but fails on second. What, it can't even find 700MB? But I guess this is it. And in my situation I really need contiguous memory, cause in case you wonder about the arrays size, I'm working with volume data and those are the datasets that fail, and OpenGL only takes arrays which they allocate as one-dimension contiguous array. Anyway, if someone has anything more to add, please go ahead.

Thanks for your time.