Deallocating memory in chunks rather than whole

[curiousengineer]

In my program I will be reading in data from an external device. This data will need to be parsed. Basically, I will allocate a huge amount of memory like dozens of bytes that will be used to store the data when it comes in the first time. The incoming data basically consists of collection of structs. Each struct is identified by its first ID byte followed by a constant length value. I am not sure which and how many structs are going to be in the incoming data and thus will parse this data to seperate the structs each time the data comes in.

I have decided that I could have pointer to these structs or array of pointers to store address of multiple structs of the same type. As I parse the data, I will set the pointer value equal to the address where that struct exists in the huge allocated memory chunk into this struct pointer. This way I can access the data corresponding to each struct in a very simple way later in the program.

The problem is freeing this huge chunk. Is it possible to free this chunk in parts by using the struct pointers where each struct is only part of the whole allocated memory?

[TheIdeasMan]

Basically, I will allocate a huge amount of memory like dozens of bytes that will be used to store the data when it comes in the first time.

Dozens of bytes is nothing - did you mean dozens of Gigabytes?

If you use an STL container, then you shouldn't need to worry about memory management at all - that's part of the good thing about them. However if you are approaching the max amount of RAM that your computer has, then you may have a problem.

Also, by using the STL, you could probably avoid using raw pointers

Can you say more about your actual problem? If you provide some details, we could help with the design.

Edit:

Unless you mean that you have a very low memory device. If so, you should have said so - as I guess there are relatively few people who do that.

[curiousengineer]

Yes, I am dealing with a device with RAM in kB range and the allocated memory will be less than 255 bytes. However, it is still a lot in this environment. I am dealing with C language so cannot use STL.

As far as I am aware, if I allocate a chunk of memory and then set pointers to parts of that memory, I cannot deallocate part, if I do free(start address of memory chunk) then all of the memory shall be deallocated.

[TheIdeasMan]

As far as I am aware, if I allocate a chunk of memory and then set pointers to parts of that memory, I cannot deallocate part, if I do free(start address of memory chunk) then all of the memory shall be deallocated.

Yes, that is my understanding too.

Ok, so is the data dependent on data following it? Could you allocate memory for data for 1 struct, process it, then free it?

I have seen before where someone had 32KB of ram, and had software to process an entire General Ledger. It would read 1 line from the file, and process it, then free. Processing meant writing results to a file, which might then be analysed by another program: for example calculate the balance of an account by summing all the entries in it, and it worked line by line as well. So it could handle files with millions of lines of data. The UNIX utility wc (wordcount ) is like that too.

If the data being read in is dependent on other data, then you will just have to work out how to have the minimum amount in memory.

Hope this helps :+)