I honestly think that fiberglassing the box you have is really more work than it will be worth in the end.
I know you want to learn more about using fiberglass, and want to seal the box, but I really think that the result you are after is not easy, especially for a newbie.
To seal the box, you should build the box from the sub hole piece first, that way you can seal the opposite side from the sub hole. Silicone, wood glue, whatever seals the air would be sufficient. For the outside, if you want a perfectly smooth surface to apply vinyl or carpet, or even paint, you can countersink the screws, and then use bondo or some kind of filler, and sand it down.
Fiberglass is usually better used when you want more curvy shapes, and in fact it's strongest when it's curved.