Ok, then here's a short answer to your points:

Aesthetics. Annotations are just uglier than modifiers. Due to their special syntax instead of just a naked keyword.

If the language has annotations already, then you have paid the tax of having "special syntax" in your language in any case.
(Except Swift maybe, which has "attributes" and more than 200 keywords.)

Annotations take up more space

I don't consider this a drawback. In fact, many languages with modifiers have the same rules about modifier placement.
I actually want annotations on their own line, such that all my actual keywords start at the same column.

Disorder

Many languages with modifiers have the exact same issue, and address the issue the same way I'd address it for annotations:
Define a desired order of annotations and let the compiler/linter/IDE/formatter deal with it.

Downgrading

Let the IDE/editor pick a different color for you "more important" annotations, if you like.

“your language should not have modifiers, do annotations instead” instead of “if your language has both, remove modifiers”

That's just nitpicking the wording. Ok.

Namespacing ... is not objectively better. I don’t want to import “public” in every single file.

Then don't? Most languages don't make you import String either.

Namespacing ... Or even worse, a non-annotation (function, class) named “public”.

Have a separate namespace for for annotations, or treat @ as part of the name.
Though it's not something I would spend effort on – sometimes the best answer to "X does not work" is "then don't".

Variable declarations do have modifiers too (for example “const” in C).

I replaced ...

const foo: Foo = ...

... with ...

@constantValue
let foo: Foo = ...

... in my language a short while ago. It's fine.