Best way to handle threaded object management

[wrymn]

Hi,

I have this game engine where I wanted to thread object/chunk loading, so main thread doesn`t get lag spikes when new objects/2D tiles are loading.

What I wanted to do is:
- have a list of all main_objects (the objects are 2D tiles which represent terrain as grass or water)
- have an std::unordered_map, which consists of threadId and objects to be loaded/merged into main_objects later

1. Now, so I do not get iterator errors, I wanted to do that while the thread is loading new objects, it loads them into a std::unordered_map with threadId key and vector of objects.
2. When thread is finished loading, it will merge the objects from map into the main_objects vector, so objects are ready to be used and rendered.

However, this is giving me hard times, I am getting some iterator errors etc.. These are some code snippets of main logic behind this. Please, if somebody could take a look where I am doing a mistake.

//I wanted to make some lock here so we don`t iterate while threads are merging objects. Didn`t help though.
for (auto it = main_objects.begin(); it != main_objects.end(); it++)
{
      (*it)->Update();
      //Some other stuff here, as render data updates etc.
}

//The thread function which merges it`s loaded objects into main_objects so they can be used.
void Scene::MergeObjects(int threadId)
{
	objectMergeLock.lock();

	for (auto it = objectLoadList[threadId].begin(); it != objectLoadList[threadId].end();)
	{
		main_objects.push_back((*it));
		it = objectLoadList[threadId].erase(it);
	}
	objectLoadList.erase(threadId);

	objectMergeLock.unlock();
}

One of the errors is: vector iterators are incompatible.

[helios]

I have no idea what I'm looking at, without knowing the types of the objects involved.

[wrymn]

Yes sorry, updates the description.

The objects I am loading in my game are 2D Tiles, which have position, texture etc. in the world.
As you move camera in top-down style, the world is procedurally generated, and as you move camera, the tiles are loading and unloading, so there aren`t too many tiles loaded at one time.
let`s say 20x20 area of tiles (they can be grass, water, etc.) is one chunk.

I am always loading and unloading chunks of tiles as camera moves. This however, has to be done in separate thread, or the game will drop to 1fps when these chunks (2D tiles) are loading.

[helios]

Yes sorry, updates the description.

I don't see them.
What is main_objects? What is objectLoadList? I don't mean conceptually, I mean their actual types.

PS: Is it strictly necessary to do this with threads? If you know how fast the character can possibly move, you can load little bits of data each frame at a reasonable rate so that there's always something to display. Synchronization is just such a pain in the neck...