I think our assumptions are not shared, so arguing more isn't going to be productive until that's straightened out.
When you say difficulty settings, I think of lowering enemy effectiveness, raising player effectiveness, and removing consequences for bad play (eg: permadeath of characters). Is that what you mean?
You mention less annoying backtracking. Can you imagine a game where the "annoying back tracking" is fulfilling an important role (eg: resources attrition, encouraging revisiting areas)?
If so, is there a threshold beyond which is too much? If there's a slider that adjusts enemy damage, should it go to zero? If no, how do you decide the limits? What about the players who want to exceed them?
It seems like you have the assumption that everyone should be able to complete every game. Is that correct? Is that true for all media, or only video games?
I would write more but I'm on my phone and almost to my destination.