Agriculture also happened as an on/off season thing, with groups being nomad for a season, then "settling" and planting for another. No point in keeping a land year-round when half of that time there's no actual land (underwater) or people around and the other half needs a lot of manpower to work it; manpower that is very ready to pack their shit and leave. There are many ways agriculture showed up and not all of them had an asshole bossing everyone while saying "MINE!"