Using GDI to analyze image

[h4ever]

Hi,
I am not C++ programmer, however came here to ask if somebody of you have experience with programming of graphical programs which can analyse either what you see on screen or what is in bitmap. I program in AHK language and am starting to use GDI there. It should be similar how to use it there and here in C++, but I stucked some problems in AHK so want to ask you what are your experience with it or if is it possible to solve the problem more efficiently with some other method I don't know about.

What I wanted to do is a application which can analyze pixels from bitmap or screen to find out some shapes or areas of certain color, select that area and get it to image. I believe these must be some better way to do it because for example Photohop is pretty fast to perform similar actions. So I just dont know correct way. What I do is that I run a loop in AHK like this:

obj := findRectXYLimits(0,400,65) ; define Integrals for search
While ( c < 2073601)
while (y<1080)
{
y++
x:=0 ; reset x coordinate
While ( x < 1920)
{
x++
; PixelGetColor, c, %x%, %y% ; get pixel from screen or:
DllCall("gdiplus\GdipBitmapGetPixel", "uint", pImage, "int", x, "int", y, "uint*", ARGB); get pixel from image
c:=DEC2HEX(ARGB); convert to HEX format
count := findRectXYLimits( obj, x, y, c); compare coords with Integrals and save results.
}
}

So this is nested loop for my max. screen resolution. On my computer which is 2.4GHz it takes 1 minute and 27 seconds to finish. If I skip the function calls it takes only 6 seconds. If I add GdipBitmapGetPixel call so it take 28 seconds. So 22 seconds to recieve the pixel color information. If I add convertion from decimal to hex format I increases to 45 seconds which is delay 16 seconds for this function. Then if I had my little function to compare coordinates with integrals, I got total delay of one minute and 27 seconds. Which is pretty much.

So I ask if is it possible tu use GDI in more efficient way to detect whether pixels of certain color is in the image. Is it possible for example to define my own function and pass it to some GDI function to read and compare pixels from graphics? Also am interested if you know how to get better performance in C++. Maybe I could use C++ program to get what I want.

[andywestken]

It's the first I've heard of AHK (AutoHotKey?) Thing is, as it's a scripting languages, I'm not convinced it's cut out for performance...

1. To start with, what is the point of the DEC2HEX call. Does AHK store numbers as strings? If so, no wonder it's so slow.

If it actually uses some sort of numeric format to store numbers, could you just pass ARGB directly to findRectXYLimits instead?

(decimal and hexadecimal only make sense as display formats, so apply to strings. The internal representation of numbers is always binary.)

2. The online docs for DllCall do suggest in the Performance section that you bind to a function using GetProcAddress if you're going to use it repeatedly. So this might be worth checking out.
http://www.autohotkey.com/docs/commands/DllCall.htm

3. You could look at using GdipBitmapLockBits, and then processing the bitmap data directly. This approach does require you to understand how the data is stored, so will make your code rather more involved.

(It is used in C++ and C# for performance reasons, e.g.

LockBits vs Get Pixel Set Pixel – Performance
http://www.mfranc.com/programming/operacje-na-bitmapkach-net-1/

As you can see the GDI+ Lockbits are almost seven times faster than GetPixel SetPixel method

Using LockBits in GDI+
http://supercomputingblog.com/graphics/using-lockbits-in-gdi/ )

4.You could write a custom DLL using C++ and get that to do the hard work for you AHK program.

Andy

[m4ster r0shi]

[...] is it possible tu use GDI in more efficient way to detect whether pixels of certain color is in the image.

Are you sure this is all you want? Consider using your GPU -> http://ideone.com/jeg3UO <- This counts the number of pixels of a certain color in a given image. I tested it with a 1600 x 1200 image, it took less than a second to complete and it's not optimized at all - you could, for example, perform more reductions on the GPU.

Useful link -> http://www.lfd.uci.edu/~gohlke/pythonlibs/

EDIT:

What I wanted to do is a application which can analyze pixels from bitmap or screen
to find out some shapes or areas of certain color, select that area and get it to image.

I just noticed this. Can you be more specific? A concrete example of what you want to do would help a lot.

[h4ever]

Thanks for reply

ahk uses strings and numbers like that:

str:="0" ; is string
str2=string ; is string
num:=str ; is number as result

DEC2HEX(DEC, RARGB=1) {
    SetFormat, IntegerFast, hex
    RGB += DEC ;Converts the decimal to hexidecimal
	if(RARGB=="true")
		RGB := RGB & 0x00ffffff
    return RGB
}

should convert decimal number to hex. However I needed to remove alpha, so not exactly what I wanted. Just for testing and displaying of color value.

Great recommendation for the dllcall performance

I gonna learn C++ , it is similar with php which I know and GDI+ uses it in many examples.

[h4ever]

m4ster r0shi: also great, however C++ looks closer to me and I started to learn it already. I want to perform the search with screen resolution max x=1920 and max y=1080 (the maximum dimensions of my Display in pixles).

With relation to Drawing programs like Gimp or Photoshop. E.g. Photoshop has such function where you can select certain color and you define fuzz. You can select as many colors and add them into one group, which is selected.

It looks like this:
http://image-24.ru/wp-content/uploads/2010/07/color_range.jpg

It does not mind if your image is 2000*2000 or 4000*4000 dimensions, it will simply select the area very fast, maybe in 0,001 s. So I thought I could use some very fast algorithm to find out the colors I look for and then filter them out once more - for example If I am looking for pixels which are in groups having certain amount of pixel as neighbours. E.g. you could want to select white groups/areas which has more than 50 pixels but less then 500 pixels...

http://eshop.mechulka.cz/fotky20610/fotos/_vyrd12_284metraz-puntik-cernobily2.jpg

[m4ster r0shi]

It does not mind if your image is 2000*2000 or 4000*4000 dimensions, it will simply select the area very fast, maybe in 0,001 s.

I'm not familiar with photoshop's color range tool, but I found a good explanation here -> [1]. I noticed that the original image doesn't change as you (de)select areas, so, have you considered that the size of the original image doesn't matter simply because the selection is being made on the preview image, which is of significantly smaller size? In any case, this is not difficult to implement, even on the original (big) image. It's not difficult to implement, because whether a pixel is in the selection or not does not depend in any way on its neighbours. All you'd need to render this is a fragment shader [2] to which you'd pass a selection color and a fuzziness value as parameters.

for example If I am looking for pixels which are in groups having certain amount of pixel as neighbours. E.g. you could want to select white groups/areas which has more than 50 pixels but less then 500 pixels...

This is a very different thing from what the color range tool does. Now, whether a pixel is in the selection or not depends on its neighbours. There probably isn't a generic way to deal with this kind of problems. Different neighbour interactions will call for different solution approaches. For the specific interaction you mention here, maybe some kind of parallel clustering algorithm could help.

C++ looks closer to me and I started to learn it already

That's ok, you can harness the power of your GPU using OpenCL [3] with
C++ too. Mind though that the equivalent python code will be much smaller.

I agree with Andy that you should process the bitmap data directly. It can only do you good
to avoid unnecessary function calls. But if you really want performance gain, you'll have to
dig into GPU programming sooner or later. And the sooner you do that the better, IMO.

[1] http://www.photoshopessentials.com/basics/selections/color-range/
[2] http://en.wikipedia.org/wiki/Shader#Pixel_shaders
[3] http://en.wikipedia.org/wiki/OpenCL

[h4ever]

m4ster r0shi:
I also notices that the image can be resized to fasten some basic calculation, however when you submit it also will select extremely fast. I meant that I could select the colors been inside the fuzziness and then to apply the alghoritm to recognize the groups... The GPU you refered is Python right? Does it use OpenGL or it has own library?

[m4ster r0shi]

I also notices that the image can be resized to fasten some basic
calculation, however when you submit it also will select extremely fast.

Okay, as I explained above, this is not surprising.

The GPU you refered is Python right? Does it use OpenGL or it has own library?

Let's have a quick terminology lesson first.

GPU stands for Graphics Processing Unit. It's your graphics card.

OpenGL stands for Open Graphics Library.
OpenCL stands for Open Computing Language.

The last two are relevant, but not the same thing. OpenGL is used for graphics, while OpenCL is general purpose. However, they both allow you to write programs that execute on your GPU. OpenGL GPU programs are called shaders, while OpenCL GPU programs are called kernels. Why would you want to run something on a GPU? Because this way you can take advantage of its parallel nature and do things much faster. For example, when you want to set the color of the pixels of your screen depending on their position, you can write an OpenGL shader that will set the color of all pixels simultaneously. And when you want to add two arrays A and B, each with 1000 elements, you can write an OpenCL kernel that will add A[0] with B[0], A[1] with B[1], ... etc. simultaneously.

Regarding your questions now, yes, the language I used above is python, and you can use both OpenGL and OpenCL with it. You can also use them with other languages [1, 2] like C++, Java and more...

[1] http://www.opengl.org/wiki/Language_bindings#OpenGL_Language_Bindings
[2] http://en.wikipedia.org/wiki/OpenCL#Language_bindings_and_wrappers

[h4ever]

When I would use Python (and learn it...), is it possible to compile the library needed or does the user need to have installed Python with its packages? I would like to create program which is simply to install (or not to install but just to extract it from a archive, this way I prefer).

[m4ster r0shi]

[...] is it possible to compile [...] ?

Short answer: Yes.

Long answer: There are several tools you can use for this purpose. My personal choice is pyinstaller [1], as I find it easier to use. This doesn't mean that it doesn't have its bad days though. In order to convert the above python script to an exe I had to...

* Edit PpmImagePlugin.py in PIL and add import encodings above all other imports.

* Edit __init__.py in pyopencl and change this:

def _find_pyopencl_include_path():
    from pkg_resources import Requirement, resource_filename
    return resource_filename(Requirement.parse("pyopencl"), "pyopencl/cl")

to this:

def _find_pyopencl_include_path():
    return "pyopencl/cl"

[1] http://www.pyinstaller.org/

[h4ever]

While my learning C++ I use great tutorial found here on this site. I like that there is lots of examples and very simple way to understand. Looks like they covered all most basic topics. What I like most, they include guide and perfect navigation menu. In contrary autohotkey documentation seems a bit chaotic, because they have no such nice organization. Does Python and OpenCL/OpenGL have similar page where I could start to learn on some very simple tutorial page? And in what program do you write you python program, does it can to test syntax before you compile? Or you have no need to compile until you have python installed?

[m4ster r0shi]

Does Python and OpenCL/OpenGL have similar page where
I could start to learn on some very simple tutorial page?

Python's official docs [1] include a tutorial which is a great starting point.
For OpenGL and OpenCL, you'll have to dig here and there...
Here are a couple of links that helped me when I started:

http://arcsynthesis.org/gltut/
http://www.opengl-tutorial.org/
http://documen.tician.de/pyopencl/
http://stackoverflow.com/a/17383647

And in what program do you write you python program,
does it can to test syntax before you compile?

I use PyScripter (it has autocompletion and basic on-the-fly syntax
checking), but there are many other IDEs you can choose from [2].

[1] http://docs.python.org/2.7/
[2] http://wiki.python.org/moin/IntegratedDevelopmentEnvironments

[h4ever]

Ok, thanks for the links. I am saving the page for my later use. What I forget to mention about program in which I could write in the python, I need a program which enables to enlarge font size and make black background to the code (and set highlighting so that eyes does not hurt / get tired). So any recommendation welcome. Last thing I wanted to ask you if the Python is simpler or harder than C++. As I see C++ is very exact when it comes to define types. Do you also use C++ or are you only Python programmer? From the piece of code you send it looks that Python is not so strict for type declarations and variable declarations. That could be advantage if it will make programming more simpler and faster than in C++. I have downloaded numpy-unoptimized-1.7.1.win32-py2.5.exe and pyopencl-2013.1.win32-py2.6.exe that I could test your code. What version of Pyhton do I need to download be it compatible with these programs? I think I will need to get the Open GL too?

[m4ster r0shi]

I need a program which enables to enlarge font size and make black background
to the code (and set highlighting so that eyes does not hurt / get tired).

PyScripter can do that [1].

I have downloaded numpy-unoptimized-1.7.1.win32-py2.5.exe and pyopencl-2013.1.win32-py2.6.exe that I could test your code. What version of Pyhton do I need to download be it compatible with these programs?

I'm using...

pyopencl-2013.1.win-amd64-py2.7.‌exe
numpy-MKL-1.7.1.win-amd64-py2.7.‌exe
Pillow-2.1.0.win-amd64-py2.7.‌exe

...from here -> [2], with python 2.7.5 64bit (Windows X86-64 MSI Installer (2.7.5) from here -> [3]).

If you want 32bit versions, I suggest using...

pyopencl-2013.1.win32-py2.7.‌exe
numpy-MKL-1.7.1.win32-py2.7.‌exe
Pillow-2.1.0.win32-py2.7.‌exe

...from here -> [2], with python 2.7.5 32bit (Windows x86 MSI Installer (2.7.5) from here -> [3]).

I think I will need to get the Open GL too?

You don't need OpenGL to test the above code.

Last thing I wanted to ask you if the Python is simpler or harder than C++. As I see C++ is very exact when it comes to define types. Do you also use C++ or are you only Python programmer? From the piece of code you send it looks that Python is not so strict for type declarations and variable declarations. That could be advantage if it will make programming more simpler and faster than in C++.

In general, python code is smaller than C++ code for the same task. Also, in general, python code is slower than C++ code for the same task. When you use python with OpenCL though, the latter doesn't matter, as the hard work is done by your GPU. This way, you get the best of both worlds: small code size and fast execution.

When you test the code above, make sure that you pick
a GPU device. It just happens for me that this code...

platform = cl.get_platforms()[0]
device = platform.get_devices()[0]

...picks my GPU, but this may not be the case for you.

[1] http://stackoverflow.com/q/2246710
[2] http://www.lfd.uci.edu/~gohlke/pythonlibs/
[3] http://www.python.org/download/releases/2.7.5/