Crashing on creating large-ish array

[Codeeater]

Hi there,

I've been programming with C# and Basic for quite some time. But now I am trying to learn C and C++ mainly for the speed as C#.net is really nowhere near fast enough for image processing.

So I am using Dev-C++ and I am trying to load in a bitmap.

#include <cstdlib>
#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{

   FILE* myF = fopen("z:\\mybitmap.bmp", "rb");
  
  fseek(myF, 0, SEEK_END); //find the end of the file
  
  long Size = ftell(myF);//get the length
  
  printf("File Size: %d \n", Size);//print the length
  
  rewind(myF);//return to start of file
  
  unsigned char data[Size]; //define an array to store the data

  fread(&data, 1, Size, myF);//read the data into said array
    
//print data in decimal and hex
  int count = 0;
  for(count = 0; count< Size; count++)
  {
      printf("%d \t %d \t %x \n", count, data[count], data[count]);           
  }  
    
    system("PAUSE");
    return EXIT_SUCCESS;
}

This works fine when the file is small, but when the file gets larger (around 2MB) it has problems. The program just ends. It starts up and immediately ends. With the message "...has encountered problems..".

I have tried commenting out lines and have found that the issue resides in the definition of the array.

So, say I try to define an array of size 2million

#include <cstdlib>
#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{
    printf("hello \n");  
    unsigned char data[2000000];
    
    system("PAUSE");
    return EXIT_SUCCESS;
}

...It starts up and prints "hello" and then waits for a key input, perfectly normal. But when I try to create an array of size 2.5million or larger, the error occurs. And it doesn't even print "hello" before it crashes. The console window displays, but then the error message is given straight away.

I am running a 32Bit Windows XP machine.

Can anyone please help me out?

Best wishes,

Codeeater

[naraku9333]

The max value a long can hold is 2147483647. Try an unsigned long or unsigned long long.

[freddie1]

I haven't tried any of your code Codeeater (I guess it would be hard as I don't have that file of yours), but my guess is you may be running out of stack space. That is a limited quantity you know. The reason I'm guessing that is that you are saying your issues are occurring much, much under the max value of a long in 32 bit address space. Actually, you are starting to have issues around the 2 or 3 MB number, and that is I believe a typical max size for the stack. And the description of the error you are giving is about what you would see if you ran out of stack space.

What I'd recommend is allocating memory from the heap or free store and using pointers. Its not much harder than using arrays. Although if your previous background is in languages that don't allow direct memory access like C or C++ or PowerBASIC does, then you'll have to familiarize yourself with that.

[freddie1]

Don't listen to freddie1. A lot of the time he doesn't know what he's talking about! For example, I used Code::Block 10.05 (MinGW 4.5 series I think) with this program (yours, essentially), and it worked fine with no crashes ...

#include <cstdlib>
#include <iostream>
#include <cstdio>
using namespace std;

int main(int argc, char *argv[])
{
    printf("hello \n");
    unsigned char data[400000000];
    data[39999999]='a';
    printf("data[3999999]=%c\n",data[39999999]);
    system("PAUSE");
    return EXIT_SUCCESS;
}

Take note that I used a 40 MB array. I wish somebody would explain to me why that works. I'd never, ever, do anything like that. But it ran fine for me. By the way, that little program compiled to a 457K program for me. This ....

#include <cstdio>

int main()
{
 char data[40000000];

 printf("hello \n");
 data[39999999]='a';
 printf("data[3999999]=%c\n",data[39999999]);
 getchar();

 return 0;
}

...compiled to 6K. That's the difference between C++isms, and Cisms.

[freddie1]

So, I can't seem to duplicate your crashes. However, I'd still never do what you did. I'd use pointers to dynamically allocated memory instead. That would leave your stack space alone, and work memory in the heap instead. Just my opinion.

[Codeeater]

@naraku9333
Why would the maximum value of a long be an issue? 2.5million is less than 2147483647 for one thing...

@freddie1
I was thinking it would be something along those lines and to be honest most of my code (that I haven't posted) manipulates pointers to access the rest of the array instead of using the brackets, so it should cause much of an issue.

Fixed it! Thank you very much!

Best wishes,

Codeeater

[naraku9333]

My mistake, didn't read closely enough.

[freddie1]

Array notation is very similiar to pointer notation, or can be if you you base pointer offset notation. Here is an example. Note I allocated a 100MB address space ...

#include <cstdio>

int main()
{
 unsigned char* pChar=NULL;
 char cLetter=0;

 pChar=new unsigned char[100000000];
 if(pChar)
 {
    printf("pChar = %d\n\n",(int)pChar);
    printf("i\tcLetter\tpChar[i]\n");
    printf("======================\n");
    for(int i=0; i<26; i++)
    {
        cLetter=i+65;
        pChar[i]=cLetter;
        printf("%d\t%d\t%c\n",i,cLetter,pChar[i]);
    }
    delete [] pChar;
 }
 else
   printf("Memory Allocation Failed!\n");
 getchar();

 return 0;
}

/*
Output:

pChar = 4456480

i       cLetter pChar[i]
======================
0       65      A
1       66      B
2       67      C
3       68      D
4       69      E
5       70      F
6       71      G
7       72      H
8       73      I
9       74      J
10      75      K
11      76      L
12      77      M
13      78      N
14      79      O
15      80      P
16      81      Q
17      82      R
18      83      S
19      84      T
20      85      U
21      86      V
22      87      W
23      88      X
24      89      Y
25      90      Z
*/

[Cubbi]

long Size = ftell(myF);//get the length
unsigned char data[Size]; //define an array to store the data 

Note that this only works in C, it's an error to define an array with non-constant size in C++, although some compilers allow that with non-strict settings.

Since you're apparently learning C++ (based on the #include's), read your file into a string or into a vector<unsigned char> instead.

[freddie1]

Note that this only works in C, it's an error to define an array with non-constant size in C++, although some compilers allow that with non-strict settings.

That's why I was somewhat taken aback when I saw that Cubbi. I wasn't even aware that would compile. You seem to know your stuff. I'm wondering if you have an explanation (or anybody else, for that matter), as to how that is even working. In my code above I tried 40 MB for Size and it compiled and ran clean with no errors. I can't imagine how that could be, as I can't believe that amount of memory can be allocated on the stack. What I'm wondering now is if the compiler is indeed allocating it on the heap, like a basic dialect language would.

[Cubbi]

I can't imagine how that could be, as I can't believe that amount of memory can be allocated on the stack.

Why not? There is no limit other than what's imposed by the OS, and that is changeable at runtime (ulimit -s on linux, and looks like it's more involved on windows: http://stackoverflow.com/questions/156510/increase-stack-size-on-windows-gcc )