Is it possible to remove this button when there is only one (or no) search engine in the submenu?
I don't think that's possible. At least, not exactly how you you described it. You could get pretty close like this:
#urlbar:not([searchmode]) > .urlbar-input-container > .searchmode-switcher{
margin: 0 !important;
padding: 0 !important;
max-width: 8px;
overflow: hidden;
opacity: 0;
}
#urlbar > .urlbar-input-container:has( > .searchmode-switcher-panel > #searchmode-switcher-panel-list-urlbar > panel-item + panel-item) > .searchmode-switcher {
max-width: unset;
opacity: 1;
}
However, when you disable or enable search-providers, the state of the panel only updates when it is shown the next time. So if the panel had only single item and thus it is not shown, and you go to settings and enable another one, then the button will stay hidden until you open the panel once - which is a bit tricky since the button is hidden.
As for the separators, they seem to have their margin set with !important your userChrome probably won't override it. But luckily the value is set to var(--space-small) so you could possibly just modify that. Something like this:
panel-list > hr{
--space-small: 0;
}