There's a lot of comments in here addressing the social skill reduction, as if those kids in gifted programs (hello, fellow former gifted kids) didn't still socialize with their peers in just about every other aspect.
Even the kids in 'charter school programs' here were just separated from a group of 400 to a group of 50 or so kids for half or so of the day and then the rest of the stuff they attended classes with the other 350 kids. Even if they were completely separated off, they still have peers (admittedly, also 'gifted' peers).
Ignoring that portion, and you've still got the fact that you MUST challenge a child while developing. If I didn't get put in the 'gifted' track, I'd have goofed off even more and paid even less attention. NONE of my peers had their parents doing their homework (like some commentera have put), we just finally had homework we couldn't do on our own on the bus ride home. If you don't challenge a child's mind, they can't grow. And people who think every kid learns at the same pace, and that learning slower than the pace your brain can handle has no negative side effects, have no idea what they're talking about and should look into child development as a focus of psychology and come back to this comment thread.