For me, I tend to focus on specific directories I know I'd need data from (or that will just be a hassle to rewrite config for). I have a scripts folder that gets backed up, Books, .mozilla, etc. A lot of things I just know I won't need like .cache. That folder is 7GB and mostly just the cache from yay needing to be cleared out.

I don't backup my entire home directory because I'm worried ACLs may change or other little issues that will take more time than its worth to correct. That said, you could. You worried about something like that, you could pull the existing ACLs: find ~/ -type f -exec getfacl --absolute-names {} + > home_acls_backup.txt and then restore them: setfacl --restore=home_acls_backup.txt

I haven't really used KDE much, but I know it has a theme data in .local/share that you'd want (and probably the .cache folder as well). GNOME keeps theme data in .themes, .icons, .fonts. They might just be defaults, but if you have anything custom, you'd want those folders too.