Hi Deoren,

Deoren Moor wrote:

My apologies if this is answered somewhere,

Don't be afraid to ask, we'll redirect you to the correct places if needed :-)

but I've been unable to find out how (through the interface, not via code browsing/mods) Redmine filters incoming emails.

Redmine and ChiliProject don't filter emails but for a very few things: making sure the sending address belongs to someone that has the permission to create/update the corresponding object (unless the adequate options are set on the CLI…), make sure a project is set, and so on. A failure at any stage will cause the email to be rejected. There's only one place that does "content" filtering: the Administration > Settings > Incoming emails > Truncate emails after those lines (or something like), and I'm not sure what Redmine/ChiliProject does when the resulting string is empty.

Does ChiliProject offer a way to tune email scraping?

Other than the truncate email thing: no. Email filtering is something that's very delicate to get right, and that's something MTAs/mail server do much better than we ever could.

Thanks for the reply.

The following shell script is used to fetch emails:

# Purpose: Fetch emails and import into Redmine

# References:
#   * http://www.redmine.org/projects/redmine/wiki/RedmineReceivingEmails

cd /opt/redmine

sudo -u www-data rake -f /opt/redmine/Rakefile redmine:email:receive_imap \
    RAILS_ENV="production" \
    host=host.example.com username=user@example.com password=example \
    move_on_success=accepted_emails move_on_failure=rejected_emails \
    project=unassigned category=unassigned tracker=support \
    no_permission_check=1 unknown_user=accept port=993 ssl=1 \
    allow_override=tracker,priority,project,status,category > /dev/null 2>&1

The project, category and tracker are all forced, and we're accepting all email per the unknown_user=accept option. Anything wrong with that setup?

Thanks for your time.

This is still a problem, and I'm not sure what else to do other than closely monitor the "rejected_emails" folder for emails that are filtered out. Any suggestions?

It appears others are having the same problem:

• http://superuser.com/questions/265288/fetching-e-mails-into-redmine-via-imap
• http://www.redmine.org/boards/2/topics/23012

Thanks in advance for any tips/tricks/etc.

Deoren Moor wrote:

Thanks in advance for any tips/tricks/etc.

I guess the only way to really debug this is to go in and put some additional log statements into the whole thing to see where it breaks, at least I don't see any other practical way to do it.

My guess why some mails get filtered out but processed correctly once you've moved them is that by moving them, you have touched them at some part Redmine/ChiliProject barfs on. What program do you use to move the rejected mails back into the incoming folder? Could you try to get the raw email before and after the move and see if there's any difference?

Thanks for the tips, I'll look into doing that.

I usually use Outlook to move them manually from the rejected folder to the Inbox. In most cases the emails will be picked up on a second attempt, but occasionally that trick fails to work and one of us has to enter them manually. I've also used Mozilla Thunderbird in the past and got the same results. We provide tech support, so I manually check the folder several times an hour to make sure one didn't get placed there.

We have a project setup that all incoming emails to go to, and then we move from there into other projects where they get assigned to the applicable person. Hence the forcing of project and category values.

Good tip on the raw email too. I didn't even think about an email being modified when it was moved. I just assumed it was the same email, with maybe some sort of metadata update.

Thanks again for the reply.

I had been meaning to reply back several times, but always just manually moved the emails, marked them unread and just kept on going. Unfortunately our volume of email is ever increasing, and we're still having wanted emails rejected.

I checked the source of the email before the move and after the move, and there was no change:

# diff -u before.txt after.txt
#

Any other ideas? I'd like to force the script to pickup all emails in the Inbox, with no filtering whatsoever via Redmine.

Thanks for your time.