Commit Graph

311 Commits

Author SHA1 Message Date
95b71b35d6 FEATURE: IMAP delete email sync for group inboxes (#10392)
Adds functionality to reflect topic delete in Discourse to IMAP inbox (Gmail only for now) and reflecting Gmail deletes in Discourse.

Adding lots of tests, various refactors and code improvements.

When Discourse topic is destroyed in PostDestroyer mark the topic incoming email as imap_sync: true, and do the opposite when post is recovered.
2020-08-12 10:16:26 +10:00
ed68ac054d DEV: Cover other case for generated message ID in Email::Receiver 2020-08-03 13:37:06 +10:00
7f55ed1a4a DEV: Clarify Email::Receiver existing incoming email behaviour 2020-08-03 13:31:34 +10:00
2920988b3a FIX: IMAP sync email update uniqueness across groups and minor improvements (#10332)
Adds a imap_group_id column to IncomingEmail to deal with an issue where we were trying to update emails in the mailbox, calling IncomingEmail.where(imap_sync: true). However UID and UIDVALIDITY could be the same across accounts. So if group A used IMAP details for Gmail account A, and group B used IMAP details for Gmail account B, and both tried to sync changes to an email with UID of 3 (e.g. changing Labels), one account could affect the other. This even applied to Archiving!

Also in this PR:

* Fix error occurring if we do a uid_fetch and no emails are returned
* Allow for creating labels within the target mailbox (previously we would not do this, only use existing labels)
* Improve consistency for log messages
* Add specs for generic IMAP provider (Gmail specs still to come)
* Add custom archiving support for Gmail
* Only use Message-ID for uniqueness of IncomingEmail if it was generated by us
* Various refactors and improvements
2020-08-03 13:10:17 +10:00
e0d9232259 FIX: use allowlist and blocklist terminology (#10209)
This is a PR of the renaming whitelist to allowlist and blacklist to the blocklist.
2020-07-27 10:23:54 +10:00
c72bc27888 FEATURE: Implement support for IMAP and SMTP email protocols. (#8301)
Co-authored-by: Joffrey JAFFEUX <j.jaffeux@gmail.com>
2020-07-10 12:05:55 +03:00
8d5750d90a FIX: Catch all kinds of exceptions when processing email 2020-07-09 13:41:51 +01:00
bd842cd2b0 FEATURE: Parse images in email signatures (#10137)
* FEATURE: Parse images in email signatures

* DEV: Fix tests

* Code review
2020-07-08 15:50:30 +10:00
a047004c9a FIX: Specific email error for replies to digest emails (#9770) 2020-05-14 09:04:58 -05:00
9bff0882c3 FEATURE: Nokogumbo (#9577)
* FEATURE: Nokogumbo

Use Nokogumbo HTML parser.
2020-05-05 13:46:57 +10:00
453bec9394 FEATURE: Add small action post to indicate forwarded email
This happens only when the sender of the email didn't write anything in their email.
2020-02-12 16:23:17 +01:00
e84d88ddea FIX: Email attachments with a size of 0 bytes caused error 2020-02-04 12:30:43 +01:00
62c21ba649 Remove bounce_score_threshold_deactivate setting.
Removed `bounce_score_threshold_deactivate` setting as the deactivate
threshold is not getting reached.
2020-01-30 16:17:31 +05:30
8883cca373 enqueue spam/dmarc failing emails instead of hiding (#8674)
* enqueue spam/dmarc failing emails instead of hiding

* add translations for dmarc/spam enqueued reasons

* unescape quote

* if email_in_authserv_id is blank return gray for all emails
2020-01-21 11:12:00 -05:00
0c52537f10 DEV: update rubocop to version 0.77
We like to stay as close as possible to latest with rubocop cause the cops
get better.

This update required some code changes, specifically the default is to avoid
explicit returns where implicit is done

Also this renames a few rules
2019-12-10 11:48:39 +11:00
c88797bf0e FIX: Ignore DMARC for emails sent to mailing list mirror 2019-12-06 13:29:39 +01:00
2714149fd2 FEATURE: hide posts from incoming email based on dmarc verdict (#8333) 2019-11-26 15:55:22 +01:00
c32bd8ae48 FEATURE: Remove attachments and truncate raw field for incoming emails (#8253)
Adds the settings: 

raw_email_max_length, raw_rejected_email_max_length, delete_rejected_email_after_days. 

These settings control retention of the "raw" emails logs.

raw_email_max_length ensures that if we get incoming email that is huge we will truncate it removing uploads from the raw log.

raw_rejected_email_max_length introduces an even more aggressive truncation for rejected incoming mail. 

delete_rejected_email_after_days controls how many days we will keep rejected emails for (default 90)
2019-10-30 16:54:35 +11:00
e7ff6809a3 FEATURE: add SES spam header to recognised spam headers (#8254) 2019-10-28 12:46:53 -04:00
427d54b2b0 DEV: Upgrading Discourse to Zeitwerk (#8098)
Zeitwerk simplifies working with dependencies in dev and makes it easier reloading class chains. 

We no longer need to use Rails "require_dependency" anywhere and instead can just use standard 
Ruby patterns to require files.

This is a far reaching change and we expect some followups here.
2019-10-02 14:01:53 +10:00
6296ae3d31 FEATURE: add setting to show content of forwarded emails in topics (#7935) 2019-08-07 12:32:19 +02:00
fd12c414e7 DEV: Refactor helper methods for upload markdown
Follow-up to a61ff167
2019-07-25 16:36:35 +02:00
a61ff16740 DEV: Make attachment markdown reusable 2019-07-25 14:04:18 +02:00
b3e5f7a8c6 SECURITY: Sanitize email id for use as mutex key 2019-07-24 13:45:02 +01:00
42ab016856 FIX: Use markdown for images and attachments in Email::Receiver. 2019-06-11 14:49:46 +08:00
87d3b86484 FIX: better error message when user without permissions replies via email 2019-06-04 16:39:41 +08:00
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
4a1755b780 FIX: Don't create staged users when processing forwarded email fails 2019-05-09 23:47:47 +02:00
152238b4cf DEV: Prefer public_send over send. 2019-05-07 09:33:21 +08:00
0644c10bfe FIX: Prioritize VERP key over final_recipient header in bounced email
Per RFC3464 2.3.2, the final_recipient header may not match the address we originally sent the email to.
2019-05-03 12:12:44 +01:00
1be01f8dd4 DEV: Add support for Rails 6
Minor fixes to add Rails 6 support to Discourse, we now will boot
with RAILS_MASTER=1, all specs pass

Only one tiny deprecation left

Largest change was the way ActiveModel:Errors changed interface a
bit but there is a simple backwards compat way of working it
2019-05-02 16:23:25 +10:00
35a866fe22 FIX: Do not strip email lines having lists. 2019-04-16 11:39:35 +03:00
e92cd5318b FEATURE: Add setting to strip whitespaces from incoming emails. (#7375)
Some email clients add leading whitespaces which get are transformed in
code blocks when processed.
2019-04-15 16:26:00 +10:00
6a05f190c6 PERF: Do not create staged users for most rejected incoming emails (#7301)
Previously we would create users, then destroy them at the end of the job if the post was rejected. Now we do not create users unless required.
2019-04-08 10:36:39 +01:00
b58867b6e9 FEATURE: New 'Reviewable' model to make reviewable items generic
Includes support for flags, reviewable users and queued posts, with REST API
backwards compatibility.

Co-Authored-By: romanrizzi <romanalejandro@gmail.com>
Co-Authored-By: jjaffeux <j.jaffeux@gmail.com>
2019-03-28 12:45:10 -04:00
9334d2f4f7 FEATURE: add more granular user option levels for email notifications (#7143)
Migrates email user options to a new data structure, where `email_always`, `email_direct` and `email_private_messages` are replace by

* `email_messages_level`, with options: `always`, `only_when_away` and `never` (defaults to `always`)
* `email_level`, with options: `always`, `only_when_away` and `never` (defaults to `only_when_away`)
2019-03-15 10:55:11 -04:00
Sam
b2187301fd FEATURE: allow users to easily track/watch/mute topics via email
If you reply to an email with the word "mute" a topic will be muted
If you reply to an email with the word "track" a topic will be tracked
If you reply to an email with the word "watch" a topic will be watched

These ninja command can help advanced mailing list ex-users, saves a trip
to the website
2019-03-06 18:38:49 +11:00
51fdf7a11d FIX: don't duplicate attachments 2019-01-28 18:40:52 +01:00
db76e5a368 FIX: properly handle attachments in received emails
This regressed in 1ac3e547 when we added support for eml attachments.
2019-01-25 19:13:34 +01:00
fc0b7c9e26 FIX: incoming email matches the wrong user if null bounce key available in db 2018-11-30 12:29:51 +05:30
bfb3c4d9f9 DEV: create bounce alert earlier if email_log detected from bounce_key 2018-11-28 21:13:06 +05:30
25253dec56 FIX: Get email address from email_log if bounced with verp
We can not access mail.final_recipient attr if it bounced with verp
2018-11-28 19:04:09 +05:30
7dbf709467 FIX: create whisper post in PMs when bounces with verp and user is staged 2018-11-28 08:24:23 +05:30
cedd2118c4 FEATURE: If PM email bounced for staged user then alert in whisper reply (#6648) 2018-11-27 00:29:37 +05:30
6b51d84dc5 FIX: Don't enqueue topics if the user can't create them
Co-authored-by: Vinoth Kannan <vinothkannan@vinkas.com>
2018-11-09 18:24:28 +01:00
1ac3e5473a FIX: don't strip eml attachments from received emails 2018-11-05 09:35:22 +01:00
341836eb42 Fix the rake task and importer instead 2018-10-17 16:48:09 +02:00
ee18d9ace0 FIX: mbox importer and rake task were broken 2018-10-17 16:34:18 +02:00
b06dccac49 FIX: force enable a user's email_private_messages option when user replies via email (#6478)
* Enable user email PM when posting to group or replying to topic via email

* remove extra line

* Add test and fix snake_case

* Only reenable email_private_messages for PM replies
2018-10-16 10:51:57 +11:00
7a41a783a4 FIX: Don't reply to Unsubscribe email sent to mailing list mirror 2018-10-11 16:09:22 +02:00