It's just simple protectionism. Banning outsiders from working helps with wages and unemployment. At least in the short term. The idea was to thwart the "steal our jobs" claim.

That said, the working bans - that get lifted when people are recognized as refugees - only explain a portion of the high unemployment and welfare rates. Besides the language issues, it should be self-evident that people who grew up in war torn countries are less likely to well educated and mentally sound than people who grew up with a reasonably effective welfare state watching over them.