2D table with templated types

[Stewbond]

I'm working on a logging utility for a real-time application. The user of this class should be able to specify addresses and names of parameters to save during initialization, then those values will be recorded in a circular buffer during runtime, and saved to disk when a specific trigger occurs.

The output to disk should be a 2D matrix with all data converted to human-readable ascii (and separated by tabs). The horizontal axis contains the different parameters to save. The vertical axis contains time. My issue is how to define something like this with columns.

Desired interface:

class DyingSeconds
{
public:
  // logname = name of the file where the log will be saved
  // maxlines = how many rows of data to store before the buffer starts over.
  DyingSeconds(std::string logname, int maxlines);

  // Registers a column during initialization phase
  // name = what will appear in the header.  Do not use tabs
  // valueAddr address of the value to be recorded
  template<typename T> void Register(std::string name, T* valueAddr);

  // Records all registered values and stores it in a circular buffer.
  void Record();
  
  // When a trigger is encountered, this saves the data to disk
  void Save();
};

Here's the thing:
1) I don't want to allocate any dynamic memory during the Record() function. That's called during real-time and dynamic memory allocation is forbidden during real-time operation. That includes allocating memory inside of the STL.
2) Each column could be a different primitive type. So a simple 2D array doesn't help.
3) The class needs to be designed so that the Record() function has a minimal execution time.

What I've tried is to store this data as a vector of strings. Each string represents a line in the log (all data in ascii format for a specific time).

My problems with this method are:
1) Convrting data to characters places a computation in Record() that I'd rather do during the Trigger()
2) The method I use below constantly creates and destroys strings. Strings use dynamic memory allocation and I'm trying to avoid that.

Do you have ideas on how to store a 2D table where the type of each column can vary?


Simplified code below describing the implementation described above.

class DyingValueContainer
{
public:
	virtual std::string ToString() = 0;
};

template <typename T>
class DyingValue : public DyingValueContainer
{
private:
	T* m_value;
public:
	DyingValue(T* valueAddr) : 
		m_value(valueAddr) {	}
	
	std::string ToString() const
	{
		std::stringstream iss;
		iss << &m_value;
		return iss.str();
	}
};


class DyingSeconds
{
public:
  DyingSeconds(std::string logname, int maxlines) : m_logname(logname), m_maxlines(maxlines), m_lastline( -1 )
  { 
		m_table.insert(m_table.begin(), m_maxlines, std::string() );
  }
  
  ~DyingSeconds()
  {
	  for (std::vector<DyingValueContainer*>::iterator it = m_header.begin(); it != m_header.end(); ++it)
		  delete *it;
  }

  // Register will define the columns of our log file.  Name will appear at the header, and the values will be in each time-step
  template <typename T>
  void Register(std::string name, T* valueAddr)
  {
	  m_header.push_back( new DyingValue(valueAddr) );
  }
  
  // To be called at regular intervals by the scheduler during realtime loop
  // Reads the value of each registered variable, and stores them
  void Record()
  {
	  if (++m_lastline >= m_maxlines) m_lastline = 0;
	  m_table[m_lastline].clear();
	  for (std::vector<DyingValueContainer*>::iterator it = m_header.begin(); it != m_header.end(); ++it)
		  m_table[m_lastline] += (*it)->ToString() + '\t';
  }
  
  // When called, it will dump the stored memory
  void Save()
  {
  }

private: 
  const std::string m_logname
  const int m_maxlines;
  int m_lastline;
  
  std::vector<DyingValueContainer*> m_header;
  std::vector<std::string> m_table;
};

[Gamer2015]

So you know the amount of rows when creating the table, right?
The amount of colums may varry, right?

Either way you need dynamic memory allocation.
To reduce the time needed in Record you have to determine when to allocate new memory.


Most STL containers have a method called "resize", so you could call this function before recording data.
To reduce the time needed to allocate memory you could allocate them in blocks.

std::vector<std::vector<Data>> m_table; // m_table[x][y]
std::vector<Data> m_column;

DyingSeconds(std::string logname, int maxrows) : m_column(maxrows) // your stuff
{ 
    // ... your stuff 
AddColums(32); // have 32 colums in the table at the start
}
void AddColums(size_t _count) // rename to your desire
{
for(int i = 0; i < _count; ++i)
    m_table.push_back(m_column);
}
void Record()
{
    m_table[column][row] = data;
}

std::strings use dynamic memory allocation internally too, now you have 2 possibilities:
set the string size to be long enough (64 chars + 0 termination?)
or make a char array ... (might erase data but allways the same size)

I hope I answered any of your questons

[Stewbond]

Thanks Gamer2015.

I think I've figured out a good solution which lets me avoid using strings until the Trigger() and does not allocate any dynamic memory during the Record(). Everything is allocated in Register().

class DyingColumnContainer
{
	const std::string m_name;
public:
	DyingColumnContainer( std::string name ) : m_name( name ) {}
	
	virtual std::string DataString( int row ) = 0;
	virtual void Record( int row ) = 0;
	
	std::string GetName() const { return m_name; }
};

template <typename T>
class DyingColumn : public DyingColumnContainer
{
private:
	T* m_valueAddr;
	std::vector<T> m_data;
public:
	DyingColumn( T* valueAddr, std::string name, int maxlines ) 
		: 
		DyingColumnContainer(name), 
		m_valueAddr(valueAddr) 
	{	
		m_data.insert( m_data.begin(), maxlines, T() );
	}
	
	void Record(int row)
	{
		m_data[row] = *m_valueAddr;
	}
	
	std::string DataString(int row) const
	{
		std::stringstream iss;
		iss << m_data[row];
		return iss.str();
	}
};


class DyingSeconds
{
public:
	DyingSeconds(std::string logname, int maxlines) : m_logname(logname), m_maxlines(maxlines), m_lastline( -1 )
	{   }

	~DyingSeconds()
	{
		for (std::vector<DyingColumnContainer*>::iterator it = m_table.begin(); it != m_table.end(); ++it)
			delete *it;
	}

	// Register will define the columns of our log file.  Name will appear at the header, and the values will be in each time-step
	template <typename T>
	void Register(std::string name, T* valueAddr)
	{
		m_table.push_back( new DyingColumn(name, valueAddr, m_maxlines) );
	}
  
	// To be called at regular intervals by the scheduler during realtime loop
	// Reads the value of each registered variable, and stores them
	void Record()
	{
		if (++m_lastline >= m_maxlines) 
			m_lastline = 0;

		for (std::vector<DyingColumnContainer*>::iterator it = m_table.begin(); it != m_table.end(); ++it)
			(*it)->Record(m_lastline);
	}
  
	// When called, it will dump the stored memory
	void Trigger()
	{
		std::ofstream fout(logname);

		// print header
		for (std::vector<DyingValueContainer*>::iterator it = m_table.begin(); it != m_table.end(); ++it)
			fout << (*it)->GetName() << '\t';

		fout << std::endl;

		// print data
		for (int i = m_lastline + 1;  i < m_maxlines; ++i)
		{
			for (std::vector<DyingValueContainer*>::iterator it = m_table.begin(); it != m_table.end(); ++it)
				fout << (*it)->DataString(i) << '\t';

			fout << m_table[i] << std::endl;
		}

		for (int i = 0; i <= m_lastline; ++i)
		{
			for (std::vector<DyingValueContainer*>::iterator it = m_table.begin(); it != m_table.end(); ++it)
				fout << (*it)->DataString(i) << '\t';

			fout << m_table[i] << std::endl;
		}

		fout.close();
	}

private: 
	const std::string m_logname
	const int m_maxlines;
	int m_lastline;

	std::vector<DyingColumnContainer*> m_table;
};

[Gamer2015]

If you have a solution to your problem please mark this question as answered
(scroll to top and click on the "Mark as ✓")