That explains my problem :/

It’s so heinous that their little oopsies or „genuine effort to combat spam“ basically force self hosters to use their services.

Thank you too. I’m glad I don’t have that incorrect factoid in my head now! :D

Yes indeed! Changing the sender name to Matrix Authentication Service @ <my_homeserver> works, with successful dkim: pass.

From original source:

Authentication-Results: mx.google.com;
       dkim=pass [...]
[...]
From: "Matrix Authentication Service @ <my_homeserver>" <matrix@matrix.<my_homeserver>>
Reply-To: "Matrix Authentication Service @ <my_homeserver>" <matrix@matrix.<my_homeserver>>

So sending mails with quoted names that do not have to be quoted leads to DKIM failures at gmail.

Nice idea!

Ah, but I was right for the wrong reasons, I also only now understand it fully. Thanks for the nice discussion.

That’s the unfortunate reality. Google can force their will on the internet because they own around 35% of all email. And Microsoft owns another ~35%, so if the two decide to change how email works, it’s everyone else who has to conform.

Google decides what is ok

Good luck getting Google to do anything about it

Yep, you are right. So Gmail is following the spec, but still altering headers. So at least it’s not breaking the email, just breaking the authentication.

Atom is defined as such:

atext           =       ALPHA / DIGIT / ; Any character except controls,
                        "!" / "#" /     ;  SP, and specials.
                        "$" / "%" /     ;  Used for atoms
                        "&" / "'" /
                        "*" / "+" /
                        "-" / "/" /
                        "=" / "?" /
                        "^" / "_" /
                        "`" / "{" /
                        "|" / "}" /
                        "~"

atom            =       [CFWS] 1*atext [CFWS]

So since atom does not contain white space, there should be no white space. Only comments can separate 1*atom.

If they wanted to allow separating atoms by a space, it would be written:

atom *(SP atom)

Edit: wait, I’m wrong. CFWS includes white space. You’re right.

phrase is 1*word, not word. The difference to obs-phrase is that it allows dots (".") and folding whitespace. Not that it allows whitespace.

In the same example, the Mary Smith is not the issue, rather the @machine.tld, as is written in the description.

John Q. Public is an issue because of the dot, not the spaces. The spaces are an issue in jdoe@test . example, as that's actually an address, not a name

Folding white space is a different issue. It’s the atom / quoted-string part. That’s the standard form of a display name. Meaning if it’s more than one word (separated by white space), it should use the quoted string form. It does list obs-phrase as an alternative, but using obsolete syntax should be avoided when possible.

See here: https://gehirneimer.de/m/sysadmin@lemmy.world/t/918726/-/comment/8449841

I had to think about this for a while, but the standard is only obsoleting folding white space, i.e. white space that wraps lines, such as:

Subject: This
 is a folded line

which is equivalent to

Subject: This is a folded line

As I understand it, white space is allowed before and after obsoletion. Or do I understand it wrong?

Edit:

I think in the obsoleted language the following would have been allowed for a From: field as well:

From: John
 Smith <j.smith@example.com>

The section right below shows that it’s obsolete:

A.6.1. Obsolete addressing

   Note in the below example the lack of quotes around Joe Q. Public,
   the route that appears in the address for Mary Smith, the two commas
   that appear in the "To:" field, and the spaces that appear around the
   "." in the jdoe address.

----
From: Joe Q. Public <john.q.public@example.com>
To: Mary Smith <@machine.tld:mary@example.net>, , jdoe@test   . example
Date: Tue, 1 Jul 2003 10:52:37 +0200
Message-ID: <5678.21-Nov-1997@example.com>

Hi everyone.
----

And in the spec itself, that syntax is named as “obs-phrase”.

But yes, though obsolete, it is still legal syntax. So I guess I shouldn’t say it “does not conform to the standard”, but rather “just barely conforms to the standard”.

The unquoted names with whitespace are legal, as stated in A.5: https://www.rfc-editor.org/rfc/rfc2822#appendix-A.5

The section of obsolete syntax in the appendix starts just below A.5., unfortunately.

I agree though, that a quoted string seems to be perfectly legal as well, so there is no good reason for google to change it.

Lettre is following the spec, where if the display name contains a space, the modern way to encode it is to put quotes around it.

Quotes around the display name in a header certainly conforms to the standard (and in fact, the way Gmail rewrites it does not conform to the standard when it contains a space, but is the obsolete form), and I would expect any decent mail program to leave it alone. Then again, Gmail is not a decent mail program, and hasn’t been for a long time.

Edit: @w2xel@gehirneimer.de points out below that Gmail’s rewrite does follow the spec, so it is conforming to the standard when white space is in the display name. It’s only when there is a dot (.) that it would be using the obsolete form.

Yes, it seems it does that. I assume it has some processing chain involving decode -> process -> encode of the whole mail, and usually that works out and the DKIM check passes. Apparantly, if you send something non-standard the decode and encode are not symmetrical and this happens. IMHO it shouldn't, so I agree. Especially, showing users mails with broken authentication seems broken as well to me.

Yeah, I understand how the mail that Gmail receives is not necessarily the mail that the user sent (in regards to headers), and I understand Gmail can add headers (like auth results, spam scores, other sorts of records), but is Gmail really changing the “From” header or other headers included in the DKIM signature?? I would think that would be absolutely unacceptable.

Usually, the important parts of the mail, such as subject, sender and contents are protected by DKIM authentication. Unfortunately this is usually not visible to the end-user, i.e. as in my case, where mails fail DKIM, but are still presented in my inbox.

Mail servers and relays add headers to the mail as it goes, for example their own IPs to trace the mail, or authentication results if authentication happens at various endpoints.

In the end, the mail as in the gmail postbox is the result of the original mail, and all these additions of the mail relays. In an ideal world only DKIM authenticated would be presented to the end-user, but the world of mail seems to be so broken, that many sending servers just do not apply DKIM/DMARC correctly, and thus many receivers accept broken mail.

Will play around with that and report back :)

Wait, so when you download what is supposed to be the original email from Gmail, it’s not actually the original? How is that ok?

I think it's technically an encoding bug in lettre, which is used in matrix authentication services: https://github.com/lettre/lettre . As far as I can tell, exim is relaying the messages "correctly" or at least without altering them.

I.e. lettre should not add quotes for whitespace. But also google shouldn't alter messages before authenticating. In an ideal world, both sides are fixed ^^