I do not have any recommendations, but the category of device you're looking for is called a voice recorder.
Back in the 90s, tape voice recorders were all the rage. The talkboy was featured in Home Alone 2. The tapes got miniaturized with entries like the Sony's M470, which evidently stayed in production long enough to still be findable on Staple's website these days. An modern digital example is Sony's ICD-PX370.
Good luck in your search!
Sincerely, a child of the 80s who grew up in the 90s.