I have a similar feeling for my barely-an-ebike (250w geared folder, ~45lbs, bought on-sale). I sometimes wonder how a non-e bike would compare, it seems like it might be easier so long as I'm half-in-shape (though I don't have a lot of incentive to ride, so I don't exactly stay half-in-shape, so I'd probably need a nice stationary bike too).
Wish I could try out a minivelo, probably not much chance of that in USA particularly not for a cheap price. Maybe a folder that's better-designed than mine (D4S), Brompton is almost a dream (aside from even tinier wheels than 20") except way too expensive. Similarly, a lighter-weight frame like magnesium or titanium would be nice but probably also no.
EDIT: I forgot to say, if I didn't go with a derailleur a retro-direct drive would be neat. Maybe even cool to make a minivelo with that myself, not that I have really anything required to do so (converting it, maybe). Better if it were somewhat easy to swap gears out (like per-season sort of thing).