Um, sorry to burst your bubble, but nothing to do anything with graphics is "simple". Look in to OpenGL (I love it) or DirectX. Or just skip to a higher level library, like SFML (I'm using it... It's pretty good). What do you need/want it to do? Tell me your dream...
SFML is pretty nice... It has an entire section of their library dedicated to 2D. I don't know what you mean by a windows form, but you can make your own collision detection system... Image rendering is made easy though..
If you want something that's as simple as possible, you can just try to talk to the Windows API directly. It's hardly easy, but you may find it simpler than any of my next suggestions.
Otherwise, you can use a GUI toolkit such as wxWidgets or Qt. I wouldn't recommend it because there's a lot of setup involved in preparing to use them in a program. They're also huge. Qt takes several hours to build, and the core DLL it produces weights 5 MiB or so. It's probably not worth if you only need very simple functionality.
GDI+ is windows native and efficient enough for this work.
Open GL and DirectX are not for this kind of work. for creating buttons etc (controls generally), you need to do it yourself in GL and DX, or use something like GLUI.