Yep, this only counts years since he left Valinor for Middle-earth.

Which is kinda odd, if you reset his age every time he's rebirthed then Gandalf the White is a newborn