Strtok reset point

[paulm]

I have a piece of code that uses strtok_r to split an incoming string of decimals, separated by comma's, into an array of doubles. But I've noticed "strange" behaviour with it: although I copy the pointer of the original string, the original string is altered after calling the function. See the code below, the pointer 'data' is not even used in the strtok_r.

Normally my input string is freshly generated before use, so no problem. On occasion though, I want to insert a standard string that I defined once. I noticed that after the first call, the original string only contains the last decimal value.

Any idea why my copy is not working, or how I can reset the pointer? Thanks!

void mySplitFunction(char * data) {
	
	double setpoints[8];
	uint8_t count = 0;

	// Split string on comma (,)
	// Convert to double
	char * tokenStr;
	char * rest = data;
	double tmp;
	while ((tokenStr = strtok_r(rest, ",", &rest)) != NULL) {
		tmp = atof(tokenStr);
		setpoints[count] = tmp;
		count++;
	}
}

char testString[] = "0.0,0.0";

void main() {
	printf(testString); 	// returns "0.0,0.0"
	mySplitFunction(testString);
	printf(testString);	// returns "0.0"
}

[Manga]

I think line 9 is to blame.
I want to test.

[Manga]

Look here...

#include <iostream>



int main() {
	
	char testString[] = "0.0,0.0";
	char * test1 = testString;

	test1[0] = 'p';

	std::cout << testString;

	std::cin.get();
	return 0;
}

Test1 is = to testString. When I change test1, testString is also modified because they are one and the same.

[Manga]

I am not going to be popular with this code but this is one very stupid way to make a copy of an array...

#include <iostream>


int main() {
	
	char testString[] = "0.0,0.0";
	char * test1 = new char[sizeof(testString)];

	for (int i = 0; i < sizeof(testString); i++) {
		test1[i] = testString[i];
	}

	test1[0] = 'p';
	std::cout << test1 << std::endl;
	std::cout << testString;

	delete[] test1;
	std::cin.get();
	return 0;
}

[tpb]

Two points.

Firstly, strtok_r modifies the first parameter. It writes '\0' over the delimiters it finds. So when you try to print the string out after strtok is done with it, only the first field (not the last as you seem to think) will be printed.

Secondly, you're not really using it correctly, All but the first call are supposed to have NULL as the first argument.

At any rate, if all you need to do is read the floating point values that are separated by commas, you don't need to use strtok at all.

It looks like you are using C, so here's a C solution.

#include <stdio.h>

// Reads up to max doubles separated by commas from the given string.
int readDoubles(const char *s, double *points, int max) {
    int size = 0;
    double f;
    int pos = 0;

    if (max <= 0 || sscanf(s, "%lf%n", &f, &pos) != 1)
        return 0;
    points[size++] = f;

    while (sscanf(s += pos, " ,%lf%n", &f, &pos) == 1) {
        if (size >= max) break;
        points[size++] = f;
    }

    return size;
}

int main() {
    double points[8];

    int size = readDoubles(" 1.2 , 3.4,5.6,  7.8  ", points,
                           sizeof points/sizeof *points);

    for (int i = 0; i < size; i++)
        printf("%f\n", points[i]);

    return 0;
}

[paulm]

Thanks for the suggestions.

@tpb
I like the code. Unfortunately I'm running it on an Arduino and for memory reasons, float and doubles are not supported for sprintf() or sscanf().

@Manga
Maybe not pretty, but given the restrictions it might be my only option.
My other option is stick to strtok_r and copy the string before I call the function. That's maybe a bit more elegant.

[paulm]

Solved. I though about it a few more times, but since I only have to call this function occasionally I went for the crude way. I stuck with the old function and as workaround I copy the required string to a buffer (that I had allocated anyway) and feed the buffer into the function. That buffer is cleared on every iteration anyway.

void mySplitFunction(char * data) {
	
	// unaltered, see original post
}

const uint8_t BUFFERSIZE = 128;
const char testString[] = "0.0,0.0";

char dataBuffer[BUFFERSIZE];

void main() {	
	
	strncpy(dataBuffer, testString, strlen(testString));
	dataBuffer[strlen(testString)] = '\0'; // Don't forget to NULL-terminate!
	
	mySplitFunction(testString);
	
}