Your points are well taken, but it also incentivizes abandoning a losing party after an election - even if that party still represents the interests of the constituency.
The broader solution is to deeply gut the party system, starting with true proportional representation. Banning whipped votes would be ideal, but not possible to implement.