You might find these tidbits interesting and relevant to your point!
Children weren't named until their 8th (female) or 9th (male) day alive, precisely because of the extremely high chance of them dying before that. Likewise, young children who are interred in family tombs before a certain age are sometimes left unnamed; while children over a certain age (nonstandard, but around ~6 years) tend to be named on the tombs and mourned deeply with specific epitaphs.
There's actually some evidence that there wasn't a large gap in the lifespans of the poor and the rich. What was most likely to kill a person - disease - was pretty evenly distributed. The low level of medical understanding at the time meant that high-quality physicians weren't much help for the biggest killers - much of their advice was useful, but applicable primarily as prevention and things that ordinary people could (and would) do; while 'cures' for diseases were generally not much more than placebo. Roman surgery and injury treatment was fantastically advanced, though, probably no better place in the world at the time to break something or get something lodged in you, lmao.
If you were lucky, you could be like the Emperor Vespasian (wealthy, but not fabulously so before his Emperorship), and have all three of your kids live to adulthood. If you were unlucky, like the Emperor Marcus Aurelius (who was ultrawealthy before he was even married), you might have only 5/14 kids live to adulthood - or fewer. The Emperor's rhetoric tutor, Fronto, had only 1/6 of his kids survive to adulthood, and none of them were even 'lucky' enough to be alive simultaneously.
Modernity can be terrible, but damn, is it a step up from the past!