Maybe something along the lines of, if you're eligible to collect social security, then you're too old to serve in government.

I disagree. Age limits would disallow competent people who happen to be old from serving (Bernie, for instance). Age does not correlate well to competency, but if you're young you might believe that. Competency itself can't be measured well enough by a test to be meaningful, so that is out.

There already is an effective age limit in that people typically run for office the first time when they're relatively young, and a term limit would kick them out before they got old. Old people running for the first time would likely be seen as out of touch, unless they make compelling arguments (again, see Bernie), and so would not be elected.

Having only an age limit would still allow people decades to entrench themselves and do great damage.

In short, the problem isn't competency, it's corruption. If anything, they're too competent at being corrupt. It's the same reason we have a term limit for the President, without the limit, we'd have Putin.

Finally, I think there's a chance term limits could pass, whereas an age limit wouldn't.

You mean like corporations are people and they can run for congress also? New nightmare unlocked.