Your problem is the time of evaluation. line 1 is evalutated at runtime and so is line 4 which leads to the compiler error.
what you need is evaluation at compile time. i.e. that the compiler doesn't see line 4 or 2 depending on the type. For this check out the boost enable_if: