PCIe 1x -> 4 serial port cards exist. Multilane PCIe slots can be bifurcated to the point that, in theory, you could have as many cards as the CPU had PCIe lanes. The biggest number I've seen for number of PCIe lanes in one system is 256 (in a dual-socket Epyc server), which means it could theoretically support 1024 serial modems and a connection speed of 57 Mbps.
The practical limit is gonna be a lot lower though, because the biggest actual motherboard I've found had 3 PCIe 16x slots, 2 PCIe 8x slots, 1 M.2 (x4) slot, 5 SlimSAS (U.2 x4) ports, 1 external serial port and 1 internal serial header, for a total of 88 accessible PCIe lanes and thus 162 modems. That would equate to 9 Mbps of bandwidth.