It makes sense, but once it's pushed there is no way to know if it's been cloned or kept somewhere else. The only real mitigation is to rotate the keys or password that was leaked.
If it's something else you can't rotate, you're screwed.