They nearly went that way during the Industrial Revolution in my world. A lot of the fae disappeared from our plane for a century or so. Luckily, the Age of Enlightenment managed to create a niche for oddity scholars, and the Guardians came out of that. So the refugees are coming back, one hill at a time.