Depends where you're at. Tokyo they will mostly speak to you in Japanese, so i would attempt there. But I was in Okinawa for most of it, and most locals were good at English and used you to practice.

Helped many kids fulfill their school projects by letting them ask you questions on the street, in English lol.

It's no different to seeing someone in your country who can't speak. You both just figure it out together.