They are called bitwise operations because they operate on the bits of a (integral) value on the left side of the operator. Generally for bitwise operations the left hand is the value and the right side is the mask, masking is a big reason they are useful but I'll get to that in a moment.
As I said the operations work at bit level, so for the value 6 the binary representation is
and if we AND (&) it with the value 2
each respective bit in 6 is AND'd with 2. What I mean is, bit 0 (for the sake of argument lets say the leftmost bit) is AND'd with bit 0 of 2 and the result is stored in bit 0 of (in this case) a temporary value. In my example of 6 & 2 the result is
or 2 so we know that bit is set because I used a mask that was a power of 2. That is where, in my opinion, AND and OR are most useful, you can use you one variable say a short to store 16 flags (true/false, on/off, etc...). If you use WinAPI you will see this a lot (in wparam and lparam I believe).
This stuff isn't the easiest thing to understand at first, until you get a solid understanding of how binary works. Try writing it out on paper, take 2 numbers convert them to binary (choose small numbers) and AND them , then OR them to get a feel for it. Windows calculator (if you use Windows) has a programmer mode that allows you to see how the bits are set for decimal values, you can use that to check your answers.