The largest spaceship today is the falcon heavy with a payload of just over 140k lbs. To make the math easy let's say the falcon can lift 1000 people into space at a time.

If "millions of people" means 2 million people then we would need to launch 1000 people into space twice per week for 20 years. So 8 fully loaded falcon heavys per month for 240 months straight.

So.... No. We will not have millions of people in space any time soon.