Since you usually don't want audio before the PC starts up create a timer that doesn't turn the amp on till windows is up.
Some of the car power supplies already have that built in (may or may not be the correct amount of time for you)
You can buy ready made delay switches or search for a power on delay circuit to build.
Use software called volumouse to make the mouse scroll wheel act as a volume control. Simply use the mouse as is or hack it apart and create your own custom volume knob. (doesn't work till Windows is up)
Buy a Joycon and make a knob or button to have it control the volume (doesn't work till Windows is up)
Add a pot to the input side of your amp. (This was the first link in a google search)
This would give you direct control regardless of Windows' state
I'd want to have a delay like first suggestion regardless of any other volume control options.
Then the easiest is a simple pot on the input side of your amp.
The other two are only for after Windows starts.