Don't know for Sweden but for Germany the reason that it is not an tax is, that the gouvernements get to decide what is done with the tax money and that would heavily limit the freedom of the public broadcasting channels.

That's why it is a fee collected by the channels themself.