FIX: Add random suffix to outbound Message-ID for email (#15179)

Currently the Message-IDs we send out for outbound email
are not unique; for a post they look like:

topic/TOPIC_ID/POST_ID@HOST

And for a topic they look like:

topic/TOPIC_ID@HOST

This commit changes the outbound Message-IDs to also have
a random suffix before the host, so the new format is
like this:

topic/TOPIC_ID/POST_ID.RANDOM_SUFFIX@HOST

Or:

topic/TOPIC_ID.RANDOM_SUFFIX@HOST

This should help with email deliverability. This change
is backwards-compatible, the old Message-ID format will
still be recognized in the mail receiver flow, so people
will still be able to reply using Message-IDs, In-Reply-To,
and References headers that have already been sent.

This commit also refactors Message-ID related logic
to a central location, and adds judicious amounts of
tests and documentation.
This commit is contained in:
Martin Brennan
2021-12-06 10:34:39 +10:00
committed by GitHub
parent 11d1c520ff
commit 3b13f1146b
12 changed files with 304 additions and 104 deletions

View File

@ -236,7 +236,7 @@ module Imap
trashed_email_uids = find_uids_by_message_ids(message_ids)
if trashed_email_uids.any?
trashed_emails = emails(trashed_email_uids, ["UID", "ENVELOPE"]).map do |e|
BasicMail.new(message_id: Email.message_id_clean(e['ENVELOPE'].message_id), uid: e['UID'])
BasicMail.new(message_id: Email::MessageIdService.message_id_clean(e['ENVELOPE'].message_id), uid: e['UID'])
end
end
end
@ -253,7 +253,7 @@ module Imap
spam_email_uids = find_uids_by_message_ids(message_ids)
if spam_email_uids.any?
spam_emails = emails(spam_email_uids, ["UID", "ENVELOPE"]).map do |e|
BasicMail.new(message_id: Email.message_id_clean(e['ENVELOPE'].message_id), uid: e['UID'])
BasicMail.new(message_id: Email::MessageIdService.message_id_clean(e['ENVELOPE'].message_id), uid: e['UID'])
end
end
end
@ -266,7 +266,7 @@ module Imap
def find_uids_by_message_ids(message_ids)
header_message_id_terms = message_ids.map do |msgid|
"HEADER Message-ID '#{Email.message_id_rfc_format(msgid)}'"
"HEADER Message-ID '#{Email::MessageIdService.message_id_rfc_format(msgid)}'"
end
# OR clauses are written in Polish notation...so the query looks like this:

View File

@ -138,7 +138,7 @@ module Imap
else
# try finding email by message-id instead, we may be able to set the uid etc.
incoming_email = IncomingEmail.where(
message_id: Email.message_id_clean(email['ENVELOPE'].message_id),
message_id: Email::MessageIdService.message_id_clean(email['ENVELOPE'].message_id),
imap_uid: nil,
imap_uid_validity: nil
).where("to_addresses LIKE ?", "%#{@group.email_username}%").first