Smaller cars I would definitely look back, because you get a wider view, but it's also situational. If you need to quickly alternate between left and right side I would use mirrors, especially if squeezing in somewhere.

Especially with an SUV like we have now, looking back does nothing. Small rear window, all I see is back seats. Now I use mirrors and top-view mode + rear view on reversing cameras that surround the vehicle.