Agriculture was something we knew about for a long, long time before we did it as our main thing. Turns out it's actually way more work and involves staying in one place a lot longer than you might be used to.

So people would plant seeds like you said and hope that they had fruit the next year when they came back to that spot.

Eventually as populations increased we started relying more and more on agriculture and had people stay back to protect the crops. That being said, this was NOT a linear process. Agriculture came in waves until it eventually just stuck around for good.

Disclaimer: I only know about pre-history in the Americas, but I think this is pretty much true everywhere.