You're absolutely correct (about the need for a great set of programmers to assist the linux space).
You are incorrect however, about the Microsoft value-add proposition. Using MSFT you're likely to spend too much on licensing fee's and you'll give too much power to users that _think_ they know what they're doing. That environment would be an absolute nightmare to support. You'd have every kid trying to get his fave game ported to CE; and he'd likely know just enough about Windows to be able to completely clobber it. Where-to-fore, he'd be keeping your support lines hot trying to get his system back to normal.
Anyway, using X adds a _ton_ (in linux terms) of overhead to the system. X.org is a bit of an upgrade, but the only realistic solution is direct framebuffer, or SDL code. I like SDL, but for portability purposes, we're developing on framebuffer using an open toolkit.
Even X on Framebuffer is slow (albeit not quite as slow as straight X on the rocks).