For Windows you need to compile your .c or .cpp file(s) to a .pyd file, which is a DLL with a different extension. But you need to implement the correct entrypoints, which I've only ever done using the Boost.Python library. It was pretty easy, if I recall correctly.
If you don't use Boost, or something similar, then you'll need to implement the required entrypoints yourself, which will be more work.
The Python web site does state that:
You will still need the C compiler that was used to build Python; typically Microsoft Visual C++.
(I used Visual C++)
The Windows version is only relevevant if your extension relies on newer system features.