You have the process correct. You just need to keep filling and sanding. It is not a quick process with such complex curves as you have. You will end up sanding most of the filler off. That is what you want. Thin layers are better.
Keep filling and sanding. Then use a glazing putty to finish it off.
Or, you have the rough shape you want, take it to a paint and body shop and have them finish it for you.