writing your own application layer protocol

[adam2016]

Hi guys,

I am following this blog on how to write your own protocol - https://mayaposch.wordpress.com/2011/10/03/design-your-own-protocol-in-five-minutes/

personally I don't think she explains enough and doesn't actually show the finished product I'm sure there is more code than that

but anyway

for (int i = sizeof(size); i > 0 ; --i) {
data.append((size >> (i * 8)) & 0xFF);
}

my question is what is happening here with the bitwise operators

what is she appending to the end of the array?

why is she right shifting the result of i * 8 and then bitwise anding it with 0xFF?

thanks

[helios]

The code serializes an integer in big endian.

Example: std::uint32_t size = 0x11223344
sizeof(size) == 4

First iteration: i == 3
data.append((size >> (i * 8)) & 0xFF);
data.append((0x11223344 >> (i * 8)) & 0xFF);
data.append((0x11223344 >> (3 * 8)) & 0xFF);
data.append((0x11223344 >> 24) & 0xFF);
data.append(0x11 & 0xFF);
data.append(0x11);

Second iteration: i == 2
data.append((size >> (i * 8)) & 0xFF);
data.append((0x11223344 >> (2 * 8)) & 0xFF);
data.append((0x11223344 >> 16) & 0xFF);
data.append(0x1122 & 0xFF);
data.append(0x22);

Third iteration: i == 1
data.append((size >> (i * 8)) & 0xFF);
data.append((0x11223344 >> 8) & 0xFF);
data.append(0x112233 & 0xFF);
data.append(0x33);

Final iteration: i == 0
data.append((size >> (i * 8)) & 0xFF);
data.append((0x11223344 >> 0) & 0xFF);
data.append(0x11223344 & 0xFF);
data.append(0x44);

Ultimate behavior:
input = 0x11223344
data.append(0x11);
data.append(0x22);
data.append(0x33);
data.append(0x44);

[adam2016]

great explanation Helios thanks :)

one follow up what is meant by *serialising* an int in big endian?

[helios]

Serialization is the process of converting in-memory objects or data structures to a format that can be written to disk or sent over a network (generally an array of bytes).
Deserialization is the reverse process.

[adam2016]

great explanation :)

just to clear up a little confusion

First iteration: i == 3

shouldn't the first iteration be 4?

[deleted account xyzzy]

shouldn't the first iteration be 4?

How do you access the last element in an array of four elements, arr[4]?

Have ANY iteration be 4 and you will walk outside the boundaries of the array.

[helios]

It depends on where you consider the iteration to begin. The values of i I posted are taken inside the body of the loop. While the for header executes i takes different values at different times during the same iteration.

[adam2016]

How do you access the last element in an array of four elements, arr[4]?

Have ANY iteration be 4 and you will walk outside the boundaries of the array.

but sizeof(size) == 4

for (int i = sizeof(size); i > 0 ; --i) {
data.append((size >> (i * 8)) & 0xFF);
}

so wouldn't i = 4 on the first iteration?

[helios]

Oh, my bad. I didn't look closely enough. That loop is incorrect.
It should have been written like this:

for (int i = sizeof(size); i--;)
    data.append((size >> (i * 8)) & 0xFF);

The original code would have written [0x00, 0x11, 0x22, 0x33] to data, instead of [0x11, 0x22, 0x33, 0x44].

[deleted account xyzzy]

so wouldn't i = 4 on the first iteration?

The final iteration would indicate no.

Final iteration: i == 0

Four iterations, 3 to 0.

i == 4 would mean there should be FIVE iterations when the final is 0.

[adam2016]

thanks helios that makes sense,

@Furry Guy wouldn't it not be from 4 to 1? not 3 to 0?

[deleted account xyzzy]

wouldn't it not be from 4 to 1? not 3 to 0?

You should ask helios that question, it is his code that has 3 to 0 for the iterations.

[keskiverto]

Look at the condition and increment parts of the loop:

for ( int i = sizeof(size) ; i-- ; )
    data.append((size >> (i * 8)) & 0xFF);

Converted to while loop:

int X = sizeof(size);
while ( X-- ) {
    data.append((size >> (X * 8)) & 0xFF);
}

In the condition the tested values are: 4 3 2 1 0 (The 0 ends the loop.)
However, inside the body of the loop the X has values: 3 2 1 0


The OP version:

for ( int i = sizeof(size); i > 0 ; --i ) {
  data.append((size >> (i * 8)) & 0xFF);
}

as while loop:

int X = sizeof(size);
while ( X > 0 ) {
    data.append((size >> (X * 8)) & 0xFF);
    --X;
}

In the condition the tested values are: 4 3 2 1 0 (The 0 ends the loop.)
And, inside the body of the loop the X has values: 4 3 2 1 (which does indeed look wrong)