Remember the core of free trade, according to Adam Smith, is the free circulation of labor. Meaning: you can go anywhere you want.
I, wholeheartedly, support that. (Actually, it's more important to restrict the free movement of capital. Not people.)