- Forum
- General C++ Programming
- Image editor - efficient pixel data
Image editor - efficient pixel data
I'm making a simple image editor for fun. It has a feature that's analogous to having multiple frames/layers in Photoshop, memory-wise. I'm not expecting any Photoshop-tier efficiency just yet, however I'm curious if anyone happens to know some best practices for memory use.
Let's say I have a 1920x1080 image represented by an std::vector<unsigned char>, each pixel having 4 bytes, and I have 60 frames. So that's 1920 * 1080 * 4 * 60 = 497,664,000 bytes, about 0.5 GB.
I suppose that isn't too much, all things considered, but then all that data is copied over to the GPU memory as textures. And of course, every time you edit, blocks of data have to be transferred to the GPU to update the image. This can be done in 2^n by 2^n blocks at a time to prevent unnecessary data transfers, but if you edit multiple blocks at a time, you can still see noticeable lag when data is being written to the std::vector, and then copied to the GPU. Not the best for smooth drawing.
I don't have code to show at the moment, and that's because I don't think that's the point here, since I would like this to be more about general concepts and best practices for dealing with this large amount of data used for real-time-friendly processing. My first implementation of this used sf::Texture updates (from SFML), and I'm planning on doing it directly in modern OpenGL soon, with glBuffer___ calls.
tl;dr would anyone that is familiar with video/image/data editing/processing know what tricks are used to give a smoother experience for the end user?
Let's say I have a 1920x1080 image represented by an std::vector<unsigned char>, each pixel having 4 bytes, and I have 60 frames. So that's 1920 * 1080 * 4 * 60 = 497,664,000 bytes, about 0.5 GB.
I suppose that isn't too much, all things considered, but then all that data is copied over to the GPU memory as textures. And of course, every time you edit, blocks of data have to be transferred to the GPU to update the image. This can be done in 2^n by 2^n blocks at a time to prevent unnecessary data transfers, but if you edit multiple blocks at a time, you can still see noticeable lag when data is being written to the std::vector, and then copied to the GPU. Not the best for smooth drawing.
I don't have code to show at the moment, and that's because I don't think that's the point here, since I would like this to be more about general concepts and best practices for dealing with this large amount of data used for real-time-friendly processing. My first implementation of this used sf::Texture updates (from SFML), and I'm planning on doing it directly in modern OpenGL soon, with glBuffer___ calls.
tl;dr would anyone that is familiar with video/image/data editing/processing know what tricks are used to give a smoother experience for the end user?
Half a gigabyte per second is not exactly blistering bandwidth requirements for a RAM to VRAM transfer. I don't really see what the problem is, or why you'd need to optimize by transferring only chunks.
Hmm, well if I don't transfer by chunk, then it would transfer the whole image to the GPU at once (1920x1080x4) every frame update, and that definitely causes lag. Doing it by chunk lets it copy a bit more locally if the brush size is small, but the advantage of this goes away when the brush size is large. Maybe SFML is doing unnecessary copying under the hood, but either way it evidentally is a lot for the graphics, and I certainly don't have this problem on Photoshop or MS Paint, so there's clearly some fundamental technique I seem to be missing.
I guess I should have asked this after trying to implement it myself in OpenGL (and then I'll go ask it on an OpenGL forum...) or looking deeper into SFML source. So I guess this is an inappropriate forum.
But anyone does have advice or links to image editing techniques or tricks, I'm always open to hear it.
I guess I should have asked this after trying to implement it myself in OpenGL (and then I'll go ask it on an OpenGL forum...) or looking deeper into SFML source. So I guess this is an inappropriate forum.
But anyone does have advice or links to image editing techniques or tricks, I'm always open to hear it.
Last edited on
Naw, man. Fully uploading an HD texture every frame at 60 fps is definitely doable. Think about it. If it was not possible then it would not be possible to play 60 fps HD video, and even phones can do that nowadays
If you're dropping frames only from sending the data to the GPU then there's something wrong. Doing a RAM->VRAM transfer of a contiguous buffer is a single command to the GPU. The GPU just performs a DMA and the CPU can go do other stuff while the data is copying to the GPU.
Make sure SFML is not doing any pixel format conversions (e.g. reordering RGBA channels) and that there's no other silliness going on.
If you're dropping frames only from sending the data to the GPU then there's something wrong. Doing a RAM->VRAM transfer of a contiguous buffer is a single command to the GPU. The GPU just performs a DMA and the CPU can go do other stuff while the data is copying to the GPU.
Make sure SFML is not doing any pixel format conversions (e.g. reordering RGBA channels) and that there's no other silliness going on.
Last edited on
Yeah I definitely see what you mean about HD textures at 60 fps. I need to re-evaluate my code carefully then. Thank you.
Edit: But now that I think about it closely, most things that are doing 60 fps are games, and games usually pre-load all the textures needed for a scene, instead of constantly updating the texture pixels themselves 60 times a second. But no point wondering about until I dig through the code.
Edit: But now that I think about it closely, most things that are doing 60 fps are games, and games usually pre-load all the textures needed for a scene, instead of constantly updating the texture pixels themselves 60 times a second. But no point wondering about until I dig through the code.
Last edited on
If you're using a modern OS then everything you look at all times is doing 60 fps.
Not all games load their assets ahead of time, nor do they do it for all assets. For example if in the virtual world you have a TV playing a long video, that will probably be streamed to VRAM frame by frame. Other things that can cause less frequent uploads are LOD, and unloading of assets that are hidden (behind the camera or occluded).
Not all games load their assets ahead of time, nor do they do it for all assets. For example if in the virtual world you have a TV playing a long video, that will probably be streamed to VRAM frame by frame. Other things that can cause less frequent uploads are LOD, and unloading of assets that are hidden (behind the camera or occluded).
and your graphics library will use your graphics card. That is, copying to the GPU yourself with memory access is not going to beat bitblt (directx); at best if you write really slick code you might TIE some of the functions but overall yours will lag behind the highly optimized libraries which use the drivers for the card directly etc.
You're referring to this? https://en.wikipedia.org/wiki/Bit_blit
Seems like it's some bit-hacking to allow multiple copies of textures in different parts of the screen. Is there more it than that?
And what optimized libraries are you referring to, specifically? Sounds like you're saying to ignore standards APIs like OpenGL/DirectX/Vulkan/etc and do some bit-level non-portable (per driver) manipulation yourself? Seems a bit fishy, and hard to maintain. Or maybe I'm totally misunderstanding.
Seems like it's some bit-hacking to allow multiple copies of textures in different parts of the screen. Is there more it than that?
And what optimized libraries are you referring to, specifically? Sounds like you're saying to ignore standards APIs like OpenGL/DirectX/Vulkan/etc and do some bit-level non-portable (per driver) manipulation yourself? Seems a bit fishy, and hard to maintain. Or maybe I'm totally misunderstanding.
Last edited on
https://msdn.microsoft.com/en-us/library/windows/desktop/dd183370(v=vs.85).aspx
What are you even talking about? There's no dichotomy here. The only way to do bulk transfers to VRAM is to call the driver to have it instruct the GPU to initiate a DMA. There's no way to "copy to the GPU yourself".
| copying to the GPU yourself with memory access is not going to beat bitblt |
no, I was saying that you should use the libraries.
If those are producing screen lag, you did something wrong; the support redrawing the screen 80+ times / sec for most current cards and settings, without lag or problems.
The way I remember doing it is asking the library to give you 2 buffers; one is what is being displayed now, the other is where you 'draw'. Then you flip them over and over. If this has changed, then I am too far past it to comment. The trick of course is to be done drawing in time to swap, and 1/60 or less of a second is a good amount of time but not if you have a slow process in there.
At one time you could write directly to the draw buffer. If you cannot anymore, forget I said anything.
If those are producing screen lag, you did something wrong; the support redrawing the screen 80+ times / sec for most current cards and settings, without lag or problems.
The way I remember doing it is asking the library to give you 2 buffers; one is what is being displayed now, the other is where you 'draw'. Then you flip them over and over. If this has changed, then I am too far past it to comment. The trick of course is to be done drawing in time to swap, and 1/60 or less of a second is a good amount of time but not if you have a slow process in there.
At one time you could write directly to the draw buffer. If you cannot anymore, forget I said anything.
Last edited on
You're describing software-based rendering. This is still possible (indirectly) but nothing works this way anymore. To do this you draw to a buffer an then send that data to a hardware texture, which you can then map onto a full-screen polygon. There's no longer any way to draw directly to the framebuffer in modern graphics architectures.
Topic archived. No new replies allowed.