Maybe you number skills are fading because you waste your time and energy on pointless computations. Maybe they just fade with time...
The brain is a muscle that atrophies without use.
This is anti-scientific. But again I don't think doing multiplication problems is really going to help anything.