How to tell wheater vector realloc fails

[JuliusCaesar]

Hey,
I'm quite new to C++. I have used C in the embedded world and C#/Java in PC applications.

I have a question regarding the vector class. I now that a vector is a dynamic array (i.e continuous memory) in opposite to list).

  vector<uint32_t> test_vector;
for(uint32_t i = 0;i<uint32_max;i++)
{
test_vector.push_back(i);
}

I am well aware that the actual content of the vector goes onto the heap.
But what happens if there is no much heap for a reallocation any more? Right now I'm getting a hard fault on my STM32.

My question is: how can i detect weather a reallocation failed before the hard fault occurs?

thx for your help :)

[keskiverto]

http://www.cplusplus.com/reference/vector/vector/push_back/
writes:

If a reallocation happens, the storage is allocated using the container's allocator, which may throw exceptions on failure (for the default allocator, bad_alloc is thrown if the allocation request does not succeed).

Exception (bad_alloc) is thrown. If you don't catch and handle exceptions, then you do get "hard fault".

Seek material on exception safety

[Ganado]

In addition to what keskiverto said about catching exceptions, I'm not sure if your code snippet is what you're actually doing, but note that it can be very beneficial to pre-allocate the size in advance when possible.

vector<uint32_t> test_vector(uint32_max);
for (uint32_t i = 0; i < uint32_max; i++)
    test_vector[i] = i;

(That's like, 4 GB of memory right?)

[Peter87]

Julius, do you have exceptions available? I've heard they are often turned off in embedded environments for various reasons. It's not clear to me how you would be able to detect these kind of errors without exceptions.

[JuliusCaesar]

hey,
I'm using a custom termination handler for unhandled exceptions. Therefore it's wired that this handler does not catch this hardfault.

void CFatal_Error::Set_Termination_Handler()
	{
		set_terminate(Termination_Handler);
	}

	void CFatal_Error::Termination_Handler()
	{
		// stop right here...
		configASSERT(0);
	}

I did try to use a try and catch. But that did not work either.
My target is a Stm32 F4.


Pre Allocation is a good hint, i'll definatly keep that in mind :)

I'm using the following compiler flags

Building file: ../Src/User/debug/fatal_error.cpp
Invoking: MCU G++ Compiler
C:\Users\Julian\Giessomat\uC\Giessomat\Debug
arm-none-eabi-g++ -mcpu=cortex-m4 -mthumb -mfloat-abi=hard -mfpu=fpv4-sp-d16 '-D__weak=__attribute__((weak))' '-D__packed=__attribute__((__packed__))' -DUSE_HAL_DRIVER -DSTM32F429xx -I"xxx/Giessomat/Inc" -I"xxx/Giessomat/Inc/User" -I"xxx/Giessomat/Inc/User/debug" -I"xxx/Giessomat/Inc/User/Testcases" -I"xxx/Giessomat/Inc/User/display" -I"xxx/Giessomat/Inc/User/SD" -I"xxx/Giessomat/Inc/User/SD/XML" -I"xxx/Giessomat/Drivers/STM32F4xx_HAL_Driver/Inc" -I"xxx/Giessomat/Drivers/STM32F4xx_HAL_Driver/Inc/Legacy" -I"xxx/Giessomat/Middlewares/Third_Party/FreeRTOS/Source/portable/GCC/ARM_CM4F" -I"xxx/Giessomat/Middlewares/ST/STM32_USB_Device_Library/Core/Inc" -I"xxx/Giessomat/Middlewares/ST/STemWin/Config" -I"xxx/Giessomat/Middlewares/ST/STemWin/inc" -I"xxx/Giessomat/Middlewares/ST/STM32_USB_Device_Library/Class/CustomHID/Inc" -I"xxx/Giessomat/Drivers/CMSIS/Device/ST/STM32F4xx/Include" -I"xxx/Giessomat/Middlewares/Third_Party/FatFs/src" -I"xxx/Giessomat/Middlewares/Third_Party/FreeRTOS/Source/include" -I"xxx/Giessomat/Middlewares/Third_Party/FreeRTOS/Source/CMSIS_RTOS" -I"xxx/Giessomat/Drivers/CMSIS/Include"  -O2 -g3 -pedantic -Wall -Wextra -Wconversion -fmessage-length=0 -std=c++17 -ffunction-sections -c -fno-rtti -MMD -MP -MF"Src/User/debug/fatal_error.d" -MT"Src/User/debug/fatal_error.o" -o "Src/User/debug/fatal_error.o" "../Src/User/debug/fatal_error.cpp"
Finished building: ../Src/User/debug/fatal_error.cpp

and the following linker flags

Building target: Giessomat.elf
Invoking: MCU G++ Linker
arm-none-eabi-g++ -mcpu=cortex-m4 -mthumb -mfloat-abi=hard -mfpu=fpv4-sp-d16 -L"xxx\Middlewares\ST\STemWin\Lib"  -specs=nosys.specs -specs=nano.specs -u_printf_float -T"../STM32F429ZITx_FLASH.ld" -Wl,-Map=output.map -Wl,--gc-sections -fno-rtti -o "Giessomat.elf" @"objects.list"  -lSTemWin540_CM4_OS_GCC -lm
Finished building target: Giessomat.elf
 

i do not see any fno exceptions. Therefore I'd guess exceptinos are enabled right?

[JuliusCaesar]

I did try using emplace back, but the same problem occurs.

vector<int> test_vector;
try
{
			
        for (int i = 0; i < 50000; i++)
	{
		test_vector.emplace_back(i);
		if (0 == (i % 10))
		{
			//wait 1 ms for every 10th element
			 osDelay(1);
		}
	}
}
catch (const exception& ex)
{
          //stop right here
	configASSERT(0);
}

http://www.cplusplus.com/reference/vector/vector/emplace_back/ says:

for the default allocator, bad_alloc is thrown if the allocation request does not succeed).

Why do i not get into the exception handler? It seems like i'm missing something really basic here?

[Thomas1965]

void CFatal_Error::Set_Termination_Handler()
	{
		set_terminate(Termination_Handler);
	}

	void CFatal_Error::Termination_Handler()
	{
		// stop right here...
		configASSERT(0);
	}

I wonder why it compiles. VS 2017 complains that
E0167 argument of type "void (CFatal_Error::*)()" is incompatible with parameter of type "terminate_handler"

[Peter87]

Some systems don't allocate physical memory until the memory is actually being used. This can lead to a dynamic allocation seemingly succeeding but then the program crashes (without throwing an exception) when it tries to use the memory. I'm not sure this is a problem on your system but at least it can be a problem on Linux.

http://www.gotw.ca/publications/mill16.htm
(scroll down to Moral #2: There's Often Little Point in Checking for New Failure Anyway)

[JuliusCaesar]

uhm...
if that's really the case on my STM32 i canot use the vector class because i never know if i'm running out of memory :(

@Thomas:
Really? Because my G++ compiler does not complain

http://www.cplusplus.com/reference/exception/set_terminate/

From this i'd say that my termination handler has to be a void function with no arguments, right?

[jlb]

How much memory does your system contain? What is the exact version of the STM32 processor that you are using? Is there any documentation for your processor board publicly available?

Also what operating system are you using to develop applications for this microprocessor? Do you have a link for the compiler documentation in regards to C++ supported features?

Often when dealing with embedded or memory constrained systems dynamic memory is a no-no. The data sheet for that processor is talking about memory in the Kilobyte ranges so I would be hesitant about using things like std::vector with this processor unless there is a good amount of "external" memory on your development board.

[JuliusCaesar]

I'm using an STM32f429 which has 256 kB RAM.
I'm using GCC/G++ Compiler and Linker.
An info on how to enable exceptions can be found here:
https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_exceptions.html

But i decided to go the "GUI" way :)
(I do not know how to post images, therefore a link ;))

Compiler options
http://de.tinypic.com/r/2r619ae/9

Linker options
http://de.tinypic.com/r/2ikpkkk/9

However i did some tests today. looks like i'm still not having exceptions enabled despite the 2 selected checkboxes in the GUI.

try
	{
		//Test Exception
		throw invalid_argument("test exception");
	}
	catch (const exception& ex)
	{
		configASSERT(0);
	}

This cuases the program to stop in the termination handler. It looks like the exception is not properly caught. Or am i missing something obvious?

greets

[mbozzi]

Try passing -fexceptions (or --exceptions) to the compiler.

This is not necessarily the same as not passing -fno-exceptions.

[JuliusCaesar]

hm thx for the tip but that did not work either

according to https://gcc.gnu.org/onlinedocs/gcc-2.95.2/gcc_2.html#SEC5

-fexceptions
Enable exception handling. Generates extra code needed to propagate exceptions. For some targets, this implies generation of frame unwind information for all functions. This can produce significant data size overhead, although it does not affect execution. If you do not specify this option, it is enabled by default for languages like C++ which normally require exception handling, and disabled for languages like C that do not normally require it. However, when compiling C code that needs to interoperate properly with exception handlers written in C++, you may need to enable this option. You may also wish to disable this option is you are compiling older C++ programs that don't use exception handling.

In C++ Exceptions are enabled by default. I'm using C (for RTOS Funktions) and C++ in the same project.

Just to be safe i added fexcetpions to both the gcc compiler and the g++ compiler. And as an extra failsafe i added fexceptions to the linker as well. But that did not work :(

GCC Flags

Building file: ../Src/sd_diskio.c
Invoking: MCU GCC Compiler
xxx\\Debug
arm-none-eabi-gcc -mcpu=cortex-m4 -mthumb -mfloat-abi=hard -mfpu=fpv4-sp-d16 -std=c11 '-D__weak=__attribute__((weak))' '-D__packed=__attribute__((__packed__))' -DUSE_HAL_DRIVER -DSTM32F429xx -I"xxx//Inc" -I"xxx//Inc/User" -I"xxx//Inc/User/debug" -I"xxx//Inc/User/Testcases" -I"xxx//Inc/User/display" -I"xxx//Inc/User/display/img" -I"xxx//Inc/User/SD" -I"xxx//Inc/User/SD/XML" -I"xxx//Drivers/STM32F4xx_HAL_Driver/Inc" -I"xxx//Drivers/STM32F4xx_HAL_Driver/Inc/Legacy" -I"xxx//Middlewares/Third_Party/FreeRTOS/Source/portable/GCC/ARM_CM4F" -I"xxx//Middlewares/ST/STM32_USB_Device_Library/Core/Inc" -I"xxx//Middlewares/ST/STemWin/Config" -I"xxx//Middlewares/ST/STemWin/inc" -I"xxx//Middlewares/ST/STM32_USB_Device_Library/Class/CustomHID/Inc" -I"xxx//Drivers/CMSIS/Device/ST/STM32F4xx/Include" -I"xxx//Middlewares/Third_Party/FatFs/src" -I"xxx//Middlewares/Third_Party/FreeRTOS/Source/include" -I"xxx//Middlewares/Third_Party/FreeRTOS/Source/CMSIS_RTOS" -I"xxx//Drivers/CMSIS/Include"  -Og -g3 -pedantic -Wall -Wextra -fmessage-length=0 -fexceptions -ffunction-sections -c -fmessage-length=0 -MMD -MP -MF"Src/sd_diskio.d" -MT"Src/sd_diskio.o" -o "Src/sd_diskio.o" "../Src/sd_diskio.c"
../Src/sd_diskio.c: In function 'SD_CheckStatus':

G++ Compiler

Building file: ../Src/User/debug/fatal_error.cpp
Invoking: MCU G++ Compiler
xxx\\Debug
arm-none-eabi-g++ -mcpu=cortex-m4 -mthumb -mfloat-abi=hard -mfpu=fpv4-sp-d16 '-D__weak=__attribute__((weak))' '-D__packed=__attribute__((__packed__))' -DUSE_HAL_DRIVER -DSTM32F429xx -I"xxx//Inc" -I"xxx//Inc/User" -I"xxx//Inc/User/debug" -I"xxx//Inc/User/Testcases" -I"xxx//Inc/User/display" -I"xxx//Inc/User/display/img" -I"xxx//Inc/User/SD" -I"xxx//Inc/User/SD/XML" -I"xxx//Drivers/STM32F4xx_HAL_Driver/Inc" -I"xxx//Drivers/STM32F4xx_HAL_Driver/Inc/Legacy" -I"xxx//Middlewares/Third_Party/FreeRTOS/Source/portable/GCC/ARM_CM4F" -I"xxx//Middlewares/ST/STM32_USB_Device_Library/Core/Inc" -I"xxx//Middlewares/ST/STemWin/Config" -I"xxx//Middlewares/ST/STemWin/inc" -I"xxx//Middlewares/ST/STM32_USB_Device_Library/Class/CustomHID/Inc" -I"xxx//Drivers/CMSIS/Device/ST/STM32F4xx/Include" -I"xxx//Middlewares/Third_Party/FatFs/src" -I"xxx//Middlewares/Third_Party/FreeRTOS/Source/include" -I"xxx//Middlewares/Third_Party/FreeRTOS/Source/CMSIS_RTOS" -I"xxx//Drivers/CMSIS/Include"  -O2 -g3 -pedantic -Wall -Wextra -Wconversion -fmessage-length=0 -std=c++17 -fexceptions -ffunction-sections -c -fno-rtti -MMD -MP -MF"Src/User/debug/fatal_error.d" -MT"Src/User/debug/fatal_error.o" -o "Src/User/debug/fatal_error.o" "../Src/User/debug/fatal_error.cpp"
Finished building: ../Src/User/debug/fatal_error.cpp

G++ Linker

Building target: Giessomat.elf
Invoking: MCU G++ Linker
arm-none-eabi-g++ -mcpu=cortex-m4 -mthumb -mfloat-abi=hard -mfpu=fpv4-sp-d16 -L"xxx\\Middlewares\ST\STemWin\Lib"  -specs=nosys.specs -specs=nano.specs -u_printf_float -fexceptions -T"../STM32F429ZITx_FLASH.ld" -Wl,-Map=output.map -Wl,--gc-sections -fno-rtti -o "Giessomat.elf" @"objects.list"  -lSTemWin540_CM4_OS_GCC -lm
Finished building target: Giessomat.elf

i can't see any reason why there aren't any exceptions:O

[Thomas1965]

Have you had a look at the manual ?
It's possible that the standard library on embedded devices behaves differently.

[JuliusCaesar]

hey guys just to notify you.
Turns out that on my STM32 new/malloc do not work reliable (against all specifications).
New always ends in a hard fault of the allocated structure is bigger than the heap, malloc does not allways return a nullpointer once allocation fails.

Luckily for me (i'm using freertos) freertos comes with it's own malloc/free routine. It basically s a malloc call but with a prior boundary check with malloc obviously does not.

So i had to somehow use freertos malloc routines.

i found 2 solutions:
1) use a custom allocator like this one
http://coliru.stacked-crooked.com/a/cfd0c5c5021596ad

2) replace all new/delete calls in every translation unit

void * operator new(decltype(sizeof(0)) n) noexcept(false)
{
	uint32_t free_heap = xPortGetFreeHeapSize();
	if (free_heap < SAFTY_BUFFER)
	{
		//Fehler es gibt nicht mehr genügend Speicher
		throw std::bad_alloc();
	}
	auto* p_tmp = pvPortMalloc(n);
	if (nullptr == p_tmp)
	{
		throw std::bad_alloc();
	}

	return p_tmp;

}

void operator delete(void * p) noexcept(true)
{
	vPortFree(p);
}

void operator delete(void * p, size_t n) noexcept(true)
{
	UNUSED(n);
	vPortFree(p);
}

(just to share my solution)

[jlb]

Really in such a memory constrained system you should be avoiding new/delete or malloc/free as much as possible, this includes not using things like std::vector and std::string.