do you tell it to clear the index prior to re-indexing? I have about 40GB of music, and use rhapsody..whenever i download a new album, i tell it to reindex, and chose "No" when it asks if it should clear the index first. The process then takes about 3-5 minutes to complete...reasonable for me. FYI - I've never had it result in duplicates in the library/index.
EDIT - and i also only use one folder for music, no separate import path.