CPU or GPU on amazon aws & how to handle their GPU?

[Gyiove]

Hello everyone!

I'm having a really big neural network written in c++ using matrices.
It is an fully connected neural network using rprop.

Code is highly parallel until to the part it does the rprop and calculates
weights.

There are just like 12-20 really really big matrices what gets multiplied/added/etc with another big matrix.

rprop however has a big loop

    for (int i=1; i< layers; ++i) {
        for (int j=0; j< rows; ++j) {
            for (int k=0; k< cols; ++k) {
                 weight[i](j,k) += doRprop(i, j, k);
            }
        }
    }

sadly I can't get my head over it to make it more parallel, rprop function
is pretty complicated and I don't wish to bother you guys with complex details.
each step depends on previous one, I can't calculate steps independently using multiple threads.

Anyhow,
Do you think I should still aim for GPU?

I must say that im new to GPU.
I know that it should do alot better in parallel computation.

I've worked with openGL before but never with CUDA. ( learning/reading about it currently to make hello world program on it. )

Looking at details on aws.amazon
https://aws.amazon.com/ec2/pricing/on-demand/

c4.8xlarge having 36 virtual cpu's, each one being Intel Xeon E5-2666 v3
source:https://aws.amazon.com/ec2/details/

Intel Xeon E5-2666 v3 being 2.60 GHz
source:http://ark.intel.com/products/81706/Intel-Xeon-Processor-E5-2660-v3-25M-Cache-2_60-GHz

c4.8xlarge is being really powerful can probably could do the job needed in only 5-12 hours.


now looking at GPU Instances
p2.16xlarge having 64 virtual CPUs

from here:https://aws.amazon.com/ec2/details/

I found that:
p2.16xlarge: 16 x NVIDIA K80 GPU (Kepler GK210), 64 vCPUs, 732 GiB memory,64-bit platform

I fail to understand the power of GPU.
What would p2.16xlarge mean to my code what is that much parallel?

And the last question,
I'm assuming that p2.16xlarge would need NVIDIA's CUDA sdk
to operate, correct?

Thanks!