|What does it mean in a sense of classification?|
Not sure how to give a suscinct answer, so I'll attempt an example: dogs are comonly classified (i.e. grouped, I suppose) by breed, into Cocker Spanials, Red Setters, Dalmations, Corgis, etc.
In the same way, windows can be classified by class as BUTTON, EDIT, etc (control window classes), #32770 (the "atom" for the Dialog window class, etc. Or an application defined windows. The window's class name or "atom" works in the same was as the breed of dog.
(An atom is a unique ID for the windows class. You get one as the return of RegisterClassEx if it's successful, elsewise zero. It can be used to identify the class rather than the name. And some classes, like the one for the standard systme dialog window, have no name.)
|Doesn't windows redraw the window itself?|
Not in the case of normal windows. It's up to the app to draw (or paint) itself in response to a WM_PAINT message from the system. And erase it's background when sent a WM_ERASEBKGND message (these are the common messages; there others, for example, to do with custom drawing of controls.)
Control (button, scrollbar, etc) and dialogs are drawn by the system.
|Why is it only valid for overlapped windows?|
Window Features / Overlapped Windows