What about when they're temporary, and go back home with their money?

That's not an "immigrant", that's a seasonal migrant worker. Typically not legally capable of competing with legal citizens. They're picking fruit, not edging you out of a Target cashier job, much less anything else. If they're actual immigrants, and not seasonal workers, this is home.

What about when they come from a country so desperate and poor that they're terrified of being forced to go back, and so they'll do anything for work, even work half of minimum wage? You don't think that might drive down wages?

If they live here, they live here. If they're legally able to apply to jobs that pay above minimum wage, why would they be forced to go back? Are there any employment-dependent visas for minimum wage work? I'm certainly not aware of any.

What if they're so terrified that they won't form a union? Shouldn't the IWW care about that?

What?

What if this is all a trick by the capitalists themselves?

Immigration? I mean yeah, they're the driving force behind seasonal migrant workers. But actual immigrants?