People assume intelligence is based on IQ which is a metric intentionally made to be normalized on the general population. So in that case average = median.

However I agree with you, because I believe in the Idiocracy notion that dumb people (not just uneducated) will have more kids and skew the median lower than the average.