You're correct. There are more factors involved too, including climate change, since more ice melting means more water as well, moving the mass away from poles to the center to also be affected by the mind pull too.

All the factors end up changing things by about a millisecond per century. The effect is akin to a spinning figure skater having their arms up over their head vs directly out from their body.