Problem saving an array

[urko18]

Hello everybody,

I am doing a program in arduino and I have a question / problem:

My program controls a robotic arm with 4 dynamixel motors using an ArbotiX-M. I move manually the arm and the program saves the positions of the movements of the motors in an array. Afterwards, the program plays the sequence and the arm moves repeating the sequence that has been saved.

My problem is that the maximum number of positions that I can save are 400, (I mean, 400 positions for each engine, 1600 positions),aand if I put to store more positions the program goes crazy.

So, I would like to know if I am exceeding the limit of any memory, because when compiling what it says is:

"The sketch uses 8,656 bytes (13%) of the program's storage space, the maximum is 65,536 bytes.
Global variables use 806 bytes (19%) of dynamic memory, leaving 3,290 bytes for local variables. The maximum is 4,096 bytes. "

If so, is there a solution ???

many thanks,

Urko.

[jonnin]

its been almost a decade on ardiuno for me so bear with me...

can you save to a (binary!) file and replay from that? I can't recall if these systems have a flash disk or something small that might be usable for a small file..

what is the array data type for the positions? Can you downgrade it to a char? Or a short? Float instead of double? If you can convert on the fly from a char to a double via a computation for example, 1600 byte positions will certainly fit in 4k bytes.

another issue is simply the human factor. A human is relatively imprecise and clumsy; it *may* and very likely is possible to iterate over the saved data to eliminate and clean up the motion to optimize the paths and positions of the system.

I can probably find a way to help you do this. These sort of limitations are the things I was constrained to for most of my career under dos etc.

[urko18]

thanks for answering jonnin !!
Sorry but these days in my country are holidays and I have not been available.

The data of the positions that I take, I get them like this:

Positions vector 1: [3826, 406, 231, 2391; 3826, 406, 232, 2391; 3825, 406, 232,
2391;.......]

In that small part of the array you can see the first three positions of each of the four engines (of the encoders), that is, 12 data, the array in total has 400 positions of each with which they do 1600 data.

I do not know if I can pass / save them to binary or not, and I do not know how it would be done ...
And i also do not know how i can convert the fly from a char to a double via a computation.

to say the truth I do not know a lot of arduino, i an new in this kind of program.

Thank you!!

Urko.

[jonnin]

what are the max/min positions?

lets just look at the first engine.
3826, no change, 3825(is 1 a significant valid change?)? Or can you just store 1 value for 3 here? 1 'feels like' joystick jitter or margin of error stuff on the input device. I would even say +- 5 might be jitter, but I don't *know* that?

second engine... 406,406,406 ... no change. store it once.

third engine.... 231,232,232 .. again, is 1 significant or is this one point 3 times?

4th .... 2931,2931,2931...

this is almost 3 repetitions of the same 4 values... presumably over time.
one way to condense this would be to add a time component to your storage, for how many time slices to 'hold position here'. If you did that and broke all 4 engine vectors into individual pairs of (position, time to hold here) ... would it drastically reduce your data storage needs?

Not knowing fully your problem yet, this is just an off the cuff example of something to look at.

a different, or possibly use both, idea might be to store the differential between the now position and the last position. those might be byte sized... how fast can it change values, can it change faster than 120 or so units per saved time slice?

you might also want to eliminate in-flight values, and only store target positions. No one cares where it was between 2 points, right?

[urko18]

Thank you jonnin!

The minimum position is 0 and the maximum is 4095. (encoders positions)

The thing is that I sent you the positions when the robot was stopped, sorry. In fact, when the robot moves, the positions vary quite a lot.

As for the idea of ​​obtaining the difference between the current position and the previous one, it is not clear for me how I would do it and with what intention. The program what it does is to keep a sequence of movements, and then, through some comparisons between the positions, it causes the robot to move, so that the positions are necessary for the correct operation of the program. That's why I'm telling you that I do not know how to calculate the difference between positions.

Regarding the last thing you mentioned, I don´tt think it could be done. The program, first, collects all the movements that I do (the 400 positions of each engine), then, saves the vector, and after that uses it for comparisons . This is why I do not understand how I could do eliminate-in-flight values. I would have to change the structure of the program and that is something I can not do.

Anyway, thank you very much for giving me these advices, and if you have more, you are welcome :):) , because at the moment I continue trying it without results.

[jonnin]

I didnt explain it well.
lets say the thing can only move +- 100 units in 1 time slice. If that were the case, 100 fits in a single byte. So if you had int initialpositions[4] you could then store the 2000 or whatever ongoing positions in char deltapositions[2000] using 2000 bytes of memory instead of 2000* 2 or 2000*4 bytes. A significant savings if it is doable.

4096 is 3 bytes. Check your compiler to see if you have a 3-byte integer. If you don't, you may have to cook one up. You can do that with fairly crude approaches like
typedef an array of 3 characters to a name like i3.

then you can brute force it...
int x = 4022; //assume x is a 4 byte int;
i3 example;
memcpy(example, x, 3);

and the reverse to get the value back out.

I am HOPING you can use the delta approach with either a 1 or 2 byte integer and save the space without the clunkyness of the above memory mess which is a little inefficient and a little ugly.

to eliminate the in-flight values you would have to record the data (possibly, on an emulator on a PC or something else that lets you record it with more memory) and post process it to eliminate the useless values. But it may not be possible.

if the current code requires the full vector in memory all at once, you are in trouble. You need to increase the memory on the system or eliminate memory somewhere else to make space for it, or possibly split the problem into 2 boards each controlling 2 motors, or the like. I was assuming you had sufficient control / access to the code to modify the data structures to be more efficient somehow.

can you feed the program a partial vector, like 1/3 or 1/4 of the values, then the next chunk, then again... ?? If you can do that, I think we can make the delta approach work.

[urko18]

To say the truth that I just started programming in arduino and that's why it's difficult for me to follow your advice, sorry if i do stupid questions.

I will post to you parts of my code, in order to understand us better:

  int positionn1[m]; //Matrix of movements
  int positionn2[m];
  int positionn3[m];
  int positionn4[m];

those are the vectors of each engine, (m=400 positions).

Then, what i do is to "create" 400 columns or elements, and i asign the positions to of each engine to the differents vectors and finally i dysplay the all vectors in one (Positions vector)

  Serial.print("Positions vector ");
  Serial.print(": [");

  for (i = 0; i < m; i++) // structure to create columns
  {
    regData1 = dxlGetPosition(SERVO_ID[0]); //read and save the actual position
    regData2 = dxlGetPosition(SERVO_ID[1]);
    regData3 = dxlGetPosition(SERVO_ID[2]);
    regData4 = dxlGetPosition(SERVO_ID[3]);

    positionn1[i] = regData1; //Creation movements vector
    positionn2[i] = regData2;
    positionn3[i] = regData3;
    positionn4[i] = regData4;

    delay(20);

    Serial.print(positionn1[i]); //Display the vector
    Serial.print(", ");
    Serial.print(positionn2[i]);
    Serial.print(", ");
    Serial.print(positionn3[i]);
    Serial.print(", ");
    Serial.print(positionn4[i]);
    Serial.print("; ");
  }
  Serial.print("]\n");

during this morning and tried to change the data type int to char, but it does not work, (the char data type only works between -128 and 127 does not?) I was trying to store in 1 byte each element and no in two (like with the int data type).

I don´t know if what you tell me in the first paragraph also refers to that, to try to change the data so that they occupy less, sorry, but i don´t understand it the way you would do it.

When you refer to my compiler, you are saying my ArbotiX-M? I have looked but I have not seen anything, and, honestly, i don´t know how i can cook one up.

Can you explain me those advices more detailed please?

I think thatrecord the data in other device is not a god idea because of problems with the formats.

Yes, the current code requires the full vector in memory all at once, and i an trying to eliminate memory, i have reed that changing "print and println" into "print(F)" i can store memory.

I already tried one day to split the vector into other shorter ones, but it would not let me, in the end the sum of the elements had to give those 2000 elements.

Thank you jonnin!!

Urko.

[jonnin]

right, char is just 1 byte, so 0-255 or -127 to 127 (signed or unsigned)

I meant to say 4096 is 2 bytes worth. I messed that up in a fit of brain death or something lol. So at the very least you should be able to use SHORT instead of INT for your type, barring the issues below. Forget I ever said 3 bytes. We could use a bitfield, but that is annoying and a last resort. It hinges on the other problems anyway... we can revisit that idea later. Very precisely (and right this time) 4096 requires 12 bits. a bitfield can make a 12 bit integer type, but I don't know if it can be packed into less than 2 bytes, most compilers round up to nearest byte making that pointless to save space.

if the motor can only move 100 units or so between time slices, you can save the starting points and use 1 byte differentials afterward. This saves 1600*3 bytes (!!!) (assuming int is 4 bytes). I need to know the rate of change for those motors and the amount of time between positions. This could work, but it depends on the issues... its very compressed, but it depends on the rates and also the below problems.

this is very simple. if it is at position 1000 (start position) and it can only move 100 per time slice, then it can go to 1100 or 900 or whatever, but that fits in 1 byte. Then from 1100 to 1200, another byte, etc. It depends on the motor rates if this is viable, as well as the routines that process the position vectors and whether you can modify those or not. If you can't, its a problem.

We have 2 problems before we can use any of these ideas.
Problem 1 is that you can't split the data into chunks to feed the motor program. That means having a big wad of data existing all the time, which is the root issue.
Problem 2 is that by using another format, any routine that accepts the position array needs to either understand the new format (by editing the code) or you have to expand it to uncompressed format (only useful if it can take a partial group of data instead of the whole thing).

before we try to cook up a full solution, we need to know if we can get around those 2 issues. If we can't, we have to look at more options.

so I need to know if you can edit the code that accepts the positon vector or not and I need to know why you can't call the program in pieces with a smaller set of data each time. It should be possible to do that, each run being as if a new program started and did a new set of positions from initial. Should be doable, somehow.

-------------------------------
The compiler that converts your c++ code to an executable program on the Arduino. I forget its name honestly. It seems like I had a plugin for visual studio or I wrote mine in VS and someone else compiled it maybe. Its just been too long. We had a bunch of different controllers and chips, its a blur now. You clearly have one, we may or may not need to know something more about it -- currently my 2 questions are not related to it.

[urko18]

Many thank another time jonnin!!

I don´t know the rate of change for those motors (you say the maximun change that can have the position of the motor (the encoder) in one sampling time?). In the other side, the amount of time between positions is of 20ms (i put that value):

    regData1 = dxlGetPosition(SERVO_ID[0]); //read and save the actual position
    regData2 = dxlGetPosition(SERVO_ID[1]);
    regData3 = dxlGetPosition(SERVO_ID[2]);
    regData4 = dxlGetPosition(SERVO_ID[3]);

    positionn1[i] = regData1; //Creation movements vector
    positionn2[i] = regData2;
    positionn3[i] = regData3;
    positionn4[i] = regData4;

    delay(20);

The thing is that if i increase the sampling time the robot takes the data slower and, why you are losing resolution, maybe the robot will reproduce the sequence at stoppages like trembling, because because he is not taking all the positions.

Other information is that the comunication between the ArbotiX-M and the computer is this:

  dxlInit(1000000); //start dynamixel library at 1mbps to communicate with the servos
  Serial.begin(9600); //start serial at 9600 for reporting data. 

What I understand from what you tell me is that, for example, knowing that I am in the position 3000 of the encoder, it is impossible that in a one sampling time the position of the encoder could be farther than 2900-3100 for example, but not I understand how I can do that in order to save bytes.

Respect to the Problem 1 (I can't split the data into chunks to feed the motor program), yes, i think that this is the main problem... and i don´t know how i can solucionate. One solution that i have seen in some webs could be to obtain a external ram module, but i can´t do that.

Perharps, what the program does is take all the positions you want, but the moment you put in the program more than 400 positions in each engine, the program does not go. that is, it continues to collect data but without doing anything else. Then, it occurred to me, but I do not know how and if I could do that, to put more than 400 positions, so the program will be colecting data (more than 2000 positions). My idea is to be able to tell it to save the first 2000 dates. And once the sequence is done, throw the sequence that would take the next 2000 ... I dont know. The problem is that the program takes data without stopping, I mean it does not stop, I do not know if you understand me. Other problem could be that between one sequence and other the robot would stop few time, and this is not a god idea.

About the second problem, i don´t know how can i do that, but i thing that passing to other format it will lose time isn´t? Maybe i can pass al the vector to a excel for example? and be reportin the datas from 2000 in 2000 to the program?

And yes you can send the split data in other vectors, but always with the limit of those positions or bytes. That is, I can make two vectors of the positions, but then each motor can only have 200 positions. There would be two vectors each of 1000 data.

Sorry but I don´t understand the last paragraph, do you want me to tell to you de name of the compilator???

Finally, if you are curious and if you want to see how it is and how the robot moves, I leave this link of youtube of a video that i had uploaded.

https://www.youtube.com/watch?v=pJGe6f-Kouw&feature=youtu.be

Urko.

[jonnin]

Ok. Lets take this slowly and figure something out. Be sure to make a copy of your current code before making any major changes.

print these values for me:

sizeof(int); //expect:4
sizeof(short); //expect: 2

and tell me if your program executes correctly with this change & any necessary changes to function parameters to pass these in instead of int array:

short positionn1[m]; //Matrix of movements
short positionn2[m];
short positionn3[m];
short positionn4[m];

if it does, you should be able to at least double the size of M with no impact. if sizeof(int) is 8, it may be 4 times M.

[urko18]

Hi jonnin,

i I have done the tests that you have told me but the program responds the same, when printing the values of the size of the int and the short it says that the two are 2. (I think I read something like that in a web or forum)
So the program works correctly as long as the vector of the positions is less than 1600, but if that value is exceeded the program also becomes crazy.

So with this option nothing changes, I'm still trying to find the possible solution or solutions ...

The memory used for this vector is the SRAM, is not it? which is what after having done the program is erased again. I have gotten into the microcontroller datasheet of my ArbotiX-M (ATmega644P / V), and it has a capacity of: 4KBytes Internal SRAM.

So i am eating all the memory with the vector and the print-s codes. Sorry, the vector is of 1600 positions, 400 for each of the 4 engines, not of I 2000,that i have told you wrong in the previous post.

Thank you so much!

[jonnin]

is the 1600 or 400 hard coded somewhere? There may just be a hard limit on # of points in the code!

Right that is why I was trying to get either the # of points reduced or the amount of bytes per point reduced. You have to do one or the other if you want it to fit in that memory.

[urko18]

I'm sorry jonnin, but I've been ocupated and also have coincided Christmas holidays. The truth is that I have given up on trying to make more points or positions in the program, and I honestly I believe that the only way would be to buy a devide that would allow me to have more SRAM memory, but for the moment i will not buy anything.

Anyway, thank you very much for your time and your answers!

Urko