Saving data in complex types to file

[TheBeardedQuack]

I've been struggling over this for a while but I think I've finally come up with a solution to what I need this for. I'm making a "raw data" file and a "definition" file which states what each little piece of data is, where to find it, what class its from, if it's an array, etc... The raw data is almost like a binary dump, but as long as both files are created by the same function call and are unaltered externally they should work fine together.

Has anyone else had trouble needing to save data from types into file to load later, but these types are virtual or contain pointers or arrays so you can't just write as is... And what did you do to write it in the end?
Does what I've come up with sound like a decent idea?

[TwilightSpectre]

I have normally just done it as a normal binary file. For example, the data in the files would have a set order, and for objects of varying lengths (for example, arrays) I would first write a number declaring the size of the array and then read that many elements from the file. Pointers I simply write in the contents of the data pointed to.

As for virtual types or whatever, rather than just doing a binary dump it is easier to simply write the contents of the data within the class to the file, and get the object to read the data back in again.

[TheBeardedQuack]

Well that was my very first idea but then I thought if one version of the object changes and has a different order, or for some reason reads / writes in a different order then eveything is out.

Also what did you do if you had a user defined class that also has a member which is another user defined class? I tried adding tags to my data to indicate that from A to B in the file to actually just pass that chunk into a new instance of that user defined class and let it read it for itself... This is recursive but got rather complicated later on.

[TwilightSpectre]

I normally give my files versions as well, to alleviate that first problem: I might implement a way for the program to load from the earlier version differently, or just give up and throw some sort of error.

As for the 'user defined class' problem, I went for the idea that each class is responsible for itself. This means, say I have defined that the file will load each member in order, when it gets to the part related to the class I simply pass the file stream to the class, which loads the data it requires and passes it back, which is a recursive approach. It keeps everything nice and decoupled as well, due to each class being in charge of loading its own data, well at least for the classes that doing so makes sense.

This also works very well for if, say, I am storing a pointer of some sort for a polymorphic class. At some point before hand, I would have some bit of data defining what type of class the member should be, so I allocate that class (with new), and then I can pass the file stream to the class through its inherited public interface to load from the file as it needs.

[TheBeardedQuack]

Yeah, this is EXACTLY what my first idea was but I kept confusing myself with certain things and also I just can't help but feel it was destined to blow up in my face... Then I tried something similar but added tags into the data stream but that just confused me even more and was ridiculously complex.

I think my problem is that I'm intending to release this as a library and this approach requires anyone using it to do everything in the right order as well.

[TwilightSpectre]

True...

Have you ever considered that maybe a binary file isn't the best approach? Often I much prefer to export my data as an XML file or the like, simply because its generic structured approach makes it very easy to parse and you don't get any problems. Unless you are storing ridiculous amounts of data, or data that can't be expressed in UTF-8, its probably just easier to use an XML file. You can normally compress them, too, if you really want to save space.

[CodeGazer]

I would agree with going to the XML route. I generally find saving data to XML files much easier to work with then the binary approach. There is plenty of well tested and easy to use libraries already availible also. My favorite is personally TinyXML2 which is quite fast in it's parsing and very light weight and easy to use.

Here is a quick example code which shows TinyXML2 grabbing data from XML files to create objects (In this case components for a game entity). While it doesn't show actual saving data to XML files it still might help give the feel of dealing with XML files.

bool TransformComponent::init(tinyxml2::XMLElement* data)
{
	assert(data);

	tinyxml2::XMLElement* positionElement = data->FirstChildElement("Position");
	if (positionElement)
	{
		// If the query fails the default position is (0, 0)
		float posX = 0.0f;
		float posY = 0.0f;
		positionElement->QueryFloatAttribute("x", &posX);
		positionElement->QueryFloatAttribute("y", &posY);

		setPosition(posX, posY);
		m_OldPosition = sf::Vector2f(posX, posY);
	}

	tinyxml2::XMLElement* rotationElement = data->FirstChildElement("RotationDegree");
	if (rotationElement)
	{
		float degrees;
		if (rotationElement->QueryFloatText(&degrees) != tinyxml2::XMLError::XML_NO_ERROR)
			degrees = 0.0f;

		setRotation(degrees);
	}

	tinyxml2::XMLElement* scaleElement = data->FirstChildElement("ScaleFactor");
	if (scaleElement)
	{
		float factorX = 1.0f;
		float factorY = 1.0f;
		scaleElement->QueryFloatAttribute("factorX", &factorX);
		scaleElement->QueryFloatAttribute("factorY", &factorY);

		setScale(factorX, factorY);
	}

	return true;
}

And this might be what the XML file might look like (I bolded the part that the code above parses).

<?xml version="1.0" encoding="utf-8"?>
<PlayerEntity>
  <TransformComponent>
    <Position x="100" y="100"/>
    <RotationDegree>-90</RotationDegree>
    <ScaleFactor factorX="1" factorY="1"/>
  </TransformComponent>
  <SquareRenderComponent>
    <Texture path="Assets/Textures/Player.png"/>
  </SquareRenderComponent>
</PlayerEntity>

There is also the option of using JSON to hold you data also. Personally I find it really comes down to whatever you feel most comfortable with.

[helios]

Check out Google's Protocol Buffers. You have a design time executable that takes a data definition file and outputs C++ sources to use in your program, which implement de/serialization of C++ objects. The buffers containing the serialized objects are platform- and language-agnostic.