Software libraries aren't bound directly to hardware. They are bound to some abstract interface defined in software.
This introduces a layer of indirection; indirection from the software to the physical device. This is required to support different kinds of devices that perform the same function.
The software that talks to the physical device is called a device driver. It's the operating system that provides that software interface (API) to the application and an interface for the device driver (SPI) and maps the function calls in the application to hardware instructions on the device. This also allows software emulation of devices.
An alternative arrangement in DOS all those years ago was to write software that talks to a device from one popular vendor. Then all the other vendors would make their devices support that interface. That's what happened with IBM PC text, EGA and VGA for display, Sound Blaster for audio, Hayes for modems, and so on.
Once there was a thriving market for different kinds of hardware devices, MS' next OS, Windows, supported the devices so that application software didn't have to.
This wasn't a new idea, just a new idea for micro-computers. Unix faced the same problem two decades earlier and used this device driver solution to support different physical devices.
|lets say we use a library like SDL or SMFL we can create a window and access the raw pixels, but if I was trying to access these pixels directly that would be impossible ( well certainly improbable ) as the operating system will not allow us to access that portion of memory, yet some how libraries like SDL and SMFL can?|
Running all these software abstractions takes time. It was a real problem back in the DOS/WIN16 days as it was very noticeable. MS addressed this problem by supporting what was initially called the Game API, later called Direct X, probably called something else now ... It allowed the OS to support applications (games) to bypass all that generic abstraction stuff and go straight to the hardware thru a new API.