Deluged by spam

[SteveRiley]

When email programs search for settings, what are they doing? Are they trying different configurations in a pre-determined order, or is there a file that email servers present to clients as a guide?

I presume you mean various email client programs and their built-in account discovery mechanisms.

There is, alas, no standard mechanism for querying IMAP or SMTP servers for configuration details. See my post #18 for a correction.

The developers of KMail, Thunderbird, Android email, etc. all have included automated setup of well-known email providers because the details rarely change. For configuring your clients to communicate with your personal email server, your only option is to go through the manual steps.

[Feathers McGraw]

I presume you mean various email client programs and their built-in account discovery mechanisms.

There is, alas, no standard mechanism for querying IMAP or SMTP servers for configuration details. The developers of KMail, Thunderbird, Android email, etc. all have included automated setup of well-known email providers because the details rarely change. For configuring your clients to communicate with your personal email server, your only option is to go through the manual steps.

Yes that's what I meant, sorry for being so ambiguous.

That's a pain, any idea why? Are people concerned about making it easier for spammers? Or is it just poor communication? It would be really useful if "we" could agree a standard.

Feathers

[SteveRiley]

Actually, I was wrong before. Turns out there is a standard: RFC 6186. You can place the relevant SRV records in your DNS server; there's no security risk or spammer utility. The bigger question is how many email clients support it? I'd guess not too many.

[Feathers McGraw]

The standard doesn't even have a Wikipedia page! That alone says quite a lot.

This is useful though:
https://www.rfc-editor.org/rfc/rfc6186.txt

It sounds like it could be a really useful standard - if my uni had been supporting it they would have saved themselves a lot of time and effort fielding questions about different connecting with different MUAs. As it was, their "outlook or nothing" approach to support probably caused them unnecessary hassle!

P.S. thanks Steve and Snowhog for pushing me to rebuild, it has been fun and I've learned loads already (and a little revision for the things I thought I knew already hasn't hurt either ).

[SteveRiley]

During the course of running my own server, I've rebuilt at least a dozen times. You're obviously on the same path

[Feathers McGraw]

I use a plugin on Wordpress called "Better WP Security". One of the options it has is temporarily locking out hosts that request lots of pages that don't exist and get 404 errors (this is useful because it can block scripts that scan for the phpmyadmin page etc).

I was looking through the 404 error logs and saw that the page below was requested by 192.168.1.1 (my router):

/.well-known/autoconfig/mail/config-v1.1.xml?emailaddress=<USER>%40samhobbs.co.uk

I think this was when I was setting up Thunderbird to connect to my email server. IIRC, you see two messages during the setup, one about trying to fetch connection settings from the mail server, and if that fails another about looking them up in a Mozilla database.

Could this be Thunderbird implementing RFC6186 and checking my site for connection settings?

Feathers

[SteveRiley]

Not sure. You could test this by running tail -f $LOGFILE and watching the output while creating your account in Thunderbird.

Also: http://tools.ietf.org/html/rfc5785

[Feathers McGraw]

Thunderbird's log file on my laptop, or one of the server's log files?

Feathers

[MoonRise]

Do any of those use "default" passwords? We had a system installed a long while back. We had a security assessment done and found those installers left a default PW the same. Two things learned from that. Always verify what the installer does (though we were new and wasn't aware enough to ask or look .... bad combo there) and DON'T assume the installer has security for you in mind.

Kind of side tracked there but the point is, could there be a default UN/PW you may not be aware of that these accounts got created from?

[MoonRise]

@ Steve...

Yes, I can say these BotNets are vicious. We had a user stupidly open a Trojan e-mail that slipped through (these things (people) are learning how to get around filters at an alarming rate). Seeing how that Trojan operated was fascinating to say the least but really caused havoc to our e-mail. That is what this Trojan was trying to set-up, a BOT. Luckily not all of it installed due to other measures in place. We also were pounded for WEEKS by that new one that encrypts your files. That one was really bad and it tried its hardest to get through.

[Feathers McGraw]

Do any of those use "default" passwords? We had a system installed a long while back. We had a security assessment done and found those installers left a default PW the same. Two things learned from that. Always verify what the installer does (though we were new and wasn't aware enough to ask or look .... bad combo there) and DON'T assume the installer has security for you in mind.

Kind of side tracked there but the point is, could there be a default UN/PW you may not be aware of that these accounts got created from?

The accounts that were created have no passwords. Not sure if they're created by default or if they were created by the spammer. Citadel is easy to set up but has pretty poor documentation.

After asking questions in the support forum, a citadel dev said that the system is initially set up to allow self service user account creation (maybe a script found it just at the right moment?). That would explain the account creation.

Also, although I'd set the default new user setting to restrict internet mail, apparently this wasn't observed because I hadn't set a global setting to restrict internet mail :/ so essentially, as far as I can tell, the settings have a non-functional and misleading tick box.

[SteveRiley]

Thunderbird's log file on my laptop, or one of the server's log files?

The log file(s) that you referred to in your post #21.

[Feathers McGraw]

So, I just remembered this thread. I tried connecting an account with Thunderbird again and I got this pop up in the logs:

/.well-known/autoconfig/mail/config-v1.1.xml?emailaddress=USER%40samhobbs.co.uk

So yes, it is Thunderbird, and it must be looking for a configuration file that tells it which settings to use. Cool

############################################

And now for something completely different...

This is really going to confuse any newcomers to the thread, but I'd like to jump right back to the beginning.

I'm curious about your spam blocking arrangement. From the example you posted, I'd guess that the majority of that spam was blocked by the "reject_unknown_helo_hostname" parameter?

What other measures do you have? I'm currently using this:

smtpd_helo_restrictions =
        permit_mynetworks,
        reject_invalid_helo_hostname,
        reject_non_fqdn_helo_hostname,
        reject_unknown_helo_hostname,
        check_helo_access hash:/etc/postfix/helo_access,

With this:

#/etc/postfix/helo_access
samhobbs.co.uk          REJECT          Get lost - you're lying about who you are
www.samhobbs.co.uk      REJECT          Get lost - you're lying about who you are

Which blocks loads of spam. So many spammers try to helo/ehlo with my IP address, or my domain name, or localhost.

I've only had 2 unwanted emails so far, one of which seems to have been sent by an actual person...which makes it half spam, I guess...

I'm not sure whether it's worth running spamassassin or not, if the volume of emails picks up then I might do. How many emails actually get through to the stage where they're evaluated by spamassassin (I'm trying to get a feel for whether it would bog my Pi down much)?

Feathers

[SteveRiley]

My STMP restrictions...

smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_helo_hostname, reject_invalid_helo_hostname, check_helo_access pcre:/etc/postfix/helo_access.pcre

smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_sender_domain, reject_non_fqdn_sender, reject_sender_login_mismatch

smtpd_recipient_restrictions = reject_unknown_recipient_domain, reject_non_fqdn_recipient, check_policy_service, inet:127.0.0.1:7777, check_policy_service inet:127.0.0.1:10031, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

smtpd_data_restrictions = reject_unauth_pipelining

smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:10031

My helo_access.pcre is the one included with iRedMail (pastebinned for your perusal).

The vast majority of my spam is getting tossed by the reject_unknown_sender_domain restriction. HELO restrictions aren't useful in my case. Remember that DYN is my MX -- inbound mail for me always comes from their servers, so the only EHLO I see is from them.

The amount of rejected mail has decreased considerably since I first wrote this post. Some stats:

• Nov 17 - Nov 23: 4326 rejected, 618/day
• Nov 24 - Nov 30: 5760 rejected, 823/day
• Dec 1 - Dec 7: 1324 rejected, 189/day
• Dec 8 - Dec 14: 1 rejected
• Dec 15 - Dec 21: 5 rejected

[Feathers McGraw]

My STMP restrictions...

smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_helo_hostname, reject_invalid_helo_hostname, check_helo_access pcre:/etc/postfix/helo_access.pcre

smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_sender_domain, reject_non_fqdn_sender, reject_sender_login_mismatch

smtpd_recipient_restrictions = reject_unknown_recipient_domain, reject_non_fqdn_recipient, check_policy_service, inet:127.0.0.1:7777, check_policy_service inet:127.0.0.1:10031, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

smtpd_data_restrictions = reject_unauth_pipelining

smtpd_end_of_data_restrictions = check_policy_service inet:127.0.0.1:10031

My helo_access.pcre is the one included with iRedMail (pastebinned for your perusal).

The vast majority of my spam is getting tossed by the reject_unknown_sender_domain restriction. HELO restrictions aren't useful in my case. Remember that DYN is my MX -- inbound mail for me always comes from their servers, so the only EHLO I see is from them.

The amount of rejected mail has decreased considerably since I first wrote this post. Some stats:

• Nov 17 - Nov 23: 4326 rejected, 618/day
• Nov 24 - Nov 30: 5760 rejected, 823/day
• Dec 1 - Dec 7: 1324 rejected, 189/day
• Dec 8 - Dec 14: 1 rejected
• Dec 15 - Dec 21: 5 rejected

Really interesting, thanks mate. There are some things I haven't seen before in amongst all of that, will investigate further.

Do you get to configure DYN at all or do they just forward everything to you?