Loading and managind data in memory efficiently

I'm trying to write a map editor for a game, and I'm unsure as how to deal with all the data. Here's what I have and what I need:

This is a voxel game, so the data is stored in region files which each contains 1024 16x16 chunks.

Region files have a pkzip files in them that is the compressed chunk data. Each chunk is a binary data stream containing various data, ranging from different flags indicating what the next bytes are, up to char arrays of various lengths.

This is what I'm doing right now.

I read the region file in memory, and save each of the pkzips as a vector<unsigned char>. I know all the lengths in advance, so I can allocate the exact amount of memory for each pkzip.

When a chunk would be requested, a plan is to unzip it (I have a library) and construct a class that would hold all the data needed with in.

Here is where it gets interesting. The utility I have for unzipping the pkzips will read the stream from the memory and store it into a unsigned char array. Thus, all the data of the class is in the memory already.

I have two choices here. I can either copy each variable using memcpy to the class, or I can construct a class using pointers to different points in the array, saving on CPU time and memory. I know that this is unsafe, but I know the exact structure of the array, and I can add a check at the end of the whole process to make sure it's all adding up.

The class itself will shrink and enlarge as the user edits the map requiring some additional allocations, but it should be relatively infrequent.

My main concern is the default memory manager. If you remember, I allocated a full chunk of memory as an array of some length. I now have a bunch of pointers all pointing to different parts of that array. If I were to delete one of that pointers, would that release that memory, or would I have to delete the original pointer? What about smart pointers?

Is all of this a good approach to this problem, or should I be handling it differently? As the user moves around the map, a bunch of chunks will be loaded and unloaded in a short period of time, and I want to ensure this is done as fast as possible. The pointer shenanigans here is to minimize the number of calls to memcpy which can be times consuming, opposed to save on memory - I can always delete that original array after all data has been extracted.

What would be the proper way of going about this?
Last edited on
Topic archived. No new replies allowed.