It's either that, or not rinsing the dirty dishes with tap water before starting the soap-cleaning procedure. Salt, in particular, totally wrecks the cleaning power of the soap. There's an entire science around this, actually.

I find that it's more effective to put detergent on the dish, scrub with wet sponge, rinse. That keeps the detergent at high concentration at the surface, where the action happens. Detergent on the sponge, on the other hand, tends to migrate away towards the center of the sponge.

I switched from soaping the sponge to soaping the dish like 10 years ago and never looked back.