There was a thread on here, that someone gave the advice that you should angle the two vents down. Then gently push down/pull on the bottom to get the vent/hu console out.
I just received my spare 2004 vents today. I can snap a picture of the back so you know what you are dealing with. Apparently, once you get the console piece out there are screws you can use to unfasten the HU and then pull that out too.