It works!!! Finally
Here's where I was getting screwed up.
I was setting evtouch in the xorg.conf to use
Option "Device" "/dev/input/event1"
No errors or anything, but didn't fix anything. Apparently that's the event for the mouse
Though it didn't effect the mouse at all or report any errors. After "cat"-ing the events, what I needed was
Option "Device" "/dev/input/event5"
option "SwapY" "true" to fix the reversed Y axis.