First, make sure in the RRConfiguration program, you have defined the name of the visulization you are using. I reccomend using Milk-Drop to start with, as it is tried and true. If you get that one working, you can move on to something else. As your visulization is starting, it sounds like you have not set the name of the visulization correctly in the RRconfig.
In winamp, (While winamp is in focus press COntrol K) and set the configuration for Milk-Drop and make sure that "Integrate with Winamp" is unchecked. This will do away with the visulizations title bar and borders.
Next, you have to have a skin with a defined "Application area" that the visu will show up in.
There are quite a few other things to check as well, but this should get you started.
If you need some examples, download a couple different skins, and see how they are incorporating the visulizations.