hehe, about 5 years back, i bought an adapter for my hugh noise cancelling studio headphones and only one ear worked. I got ****ed and swapped out everything, including the headphones. To my embarrassment, alter on, i figured out the problem. The moral of the story: make sure you didn't buy mono

.
Buy the sound card. It shoudl provide better sound anyways and if you have a 4 channel amp, its the best way to utilise all that the amp can give. Ofcourse, you can bridge the amp (use only two channels and it will power as a two channel amp) and run you speakers in parallel or serial (one side of your car on one output, the other on the other). That takes alot of effort and rewiring. Just get the soundcard

.