What USB ports are you using? Make sure they are 2.0. Also the cable from the PC to the Audigy is 2.0 compliant. Do not share it with anything give it its own line, no hubs. Update the drivers as well.
Without the USB 2.0 connection right it sounds like crap, all distorted at low volume.
Sounds to me it could be you have a USB 1.0 cable or port.