Are you using a stock headunit?
If so, you might be sending a distorted signal to the subs.
Another thing could be that the box has a leak which can sound like distortion.
As for the amp if your gains are set incorrectly and you are sending a clipped signal to the subs-- you might fry the coils.
I would suggest using that kicker amp and one of those ID subs-- that would be perfect with 250watts of extra headroom. That way you are sending more than sufficient amount of power to the sub and there would be less clipping involved.