Good point. Ultimately this leads me to question the existence of some fixed quality of intelligence. People are growing, adapting, and learning through their lives, so a fixed number defining general intelligence is likely a moot concept.
On top of the prior point lies another major issue with any sort of "general intelligence" test: defining "general intelligence". Intelligence comes in many forms: linguistic, logical-mathematical, musical, visual-spatial, bodily-kinesthetic, interpersonal, intrapersonal, naturalistic, existential intelligence, and more. The IQ test does not test all forms of intelligence.
This being said, It is likely impossible to test all forms of intelligence in one test; and even if we could create this test, how would this test handle differently abled people. For example, a completely blind person would fail the visual intelligence portion every time (for obvious reasons).