How about kicking Israel out completely?

I mean, they seemingly don't believe in the idea of Eurovision.

Wouldn't surprise me. Israel's continued, if declining, support by Western powers is dependent on Israel's own ability to project soft power, especially the cultural perception of Israel as a 'normal' Western country. Success in Eurovision assists that goal - thus, the Israeli state has a vested interest in succeeding in Eurovision.