Expat is short for expatriate, not ex-patriot. Both expatriate and patriot do come from the same Latin root (patria, fatherland) via French.

But yes, expatriate means someone who lives outside (ex) of their fatherland (patria) - can be used for both immigrant or emigrant, depending on context.