The key is, subsistence farming is not small scale. It's why subsistence farming is often described as extensive farming as opposed to modern intensive farming. Natural yields suck and very very quickly degrade land. About the only real intensive agriculture historically is rice paddies where you'd have large amounts of labor planting and tending individual rice plants for a very large yield by area and to a much lesser extent but still significant terraced potato farms. Everything else used tons of land.