If a wire is warm or hot it means that it is carrying too many amps, and in your case the 8ga can handle it but the 12ga can't. I don't think your carpc should be drawing that many amps. You might want to cut off that connector and solder on a new one, maybe the wires are too thin and so it's heating up.
Eh this is probably old news, but I'd be nervous about using such a heavy heatsink in a carpc, that's a lot of stress on the motherboard. I underclock my Barton 2500+ by 40% by reducing the fsb from 166 to 100. I can't boot my carpc without this underclock (M2ATX).