you only need to check up to (or down to, if reverse) sqrt N, in your loop, also.
Brute force is not so hot here, but you can at least greatly reduce the work done with that tweak. When you find a factor, divide the number by it and pick up its counter, for example if it divides by 3, you can pick up 3*x = yournumber, solve for x (yournumber/3). Factors come in pairs.
the first factor you find, its counterpart is the largest factor, then need to check if that # is prime. I cant see that you check to see if your factors are prime here (?).