this presentation for the SNE Master Students at Innopolis University
low-level infrastructure services are critical
live acceptance testing of the some Postfix anti-spam features
Everything clear about SMTP chains? What about smarthosts? (+ auth vs network?)
There are several ways to handle SPAM.
You can let the users deal with spam on their side e.g. using their own MDA/MUA e.g. Thunderbird Bayesian anti-spam feature.
You can filter it out from users' inboxes and let them review those in a dedicated spambox folder (for that they should use webmail or IMAP).
But the most effective way is to block the unsollicited messages directly during the SMTP session. This is what this PoC is about. We are going to proceed with an full acceptance testing of a DIY anti-spam gateway. With this method, the SPAM is refused by the Mail eXchanger and you are making users happy as long as there are not too much false-positives or inter-MX issues. Be ready to send Zircon missiles to other postmasters that sometimes do not understand anything about computing have no clue what an RFC is.
Those methods can be cumulated but one gets great results with filtering during the SMTP session already.
Looking at a message's source:
Connect with telnet,
telnet stdpierre.os3.site 2525
Who you are (helo YOURSELF),
Who's email account are you impersonating (this is for the
Who are you wiling to send a message to?
rcpt to:root rcpt to:<email@example.com>
What are you willing to send (including the message headers)?
data From: Me Me Me <firstname.lastname@example.org> To: Target <email@example.com> Subject: you have been spammed spam content
==> Do not forget to make the customer sign it (even if some parts failed)!
(quick reminder of the major SMTP daemons available out there)
One could consider distributing the postfix instances on different bare-metal or VMs or Docker containers against a shared storage. One could consider firewalling, load-balancing, CARP and DNS round-robin but this is out of topic. We are not reviewing infrastructure architecture for heavy production but only anti-spam specific tuning.
One could also consider the linkage between the SMTP services and the Calendar Webmail Chat etc. To undoom your customer or the company you will be working from from MS hosted mail facilities, the Open Source community does not have ideal answers yet. Microsoft is strong on the server market with their Exchange and Outlook products. Actually it is the only thing they got left. But I believe that we are going to take over this remaining peace too in the future (what else do they got anyway?).
By the way on the Desktop OS market, we (the Open Source community) are also becoming stronger with Unity, Cinnamon, MATE and Ubuntu Budgie.
Yes, given the labs you are working on, you are part of the community now.
myorigin = $mydomain #(default to FQDN minus the first component) mydomain = stdpierre.os3.site myhostname = mx.stdpierre.os3.site mydestination = $mydomain mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 172.17.0.0/16 smtpd_banner = $myhostname ESMTP biff = no append_dot_mydomain = no delay_warning_time = 4h readme_directory = no compatibility_level = 2 alias_maps = hash:/etc/aliases
alias_maps because the default is
alias_maps = hash:/etc/aliases, nis:mail.aliases.
delay_warning_time for user's convenience.
2525 to workaround the blocked outgoing SMTP sessions from Innopolis networks as well as from Google Cloud (so we can also do some tests from those locations),
vi /etc/postfix/master.cf #smtp inet n - y - - smtpd 2525 inet n - y - - smtpd
Create a dummy mail user called
groupadd mailuser useradd -g mailuser -m -s /sbin/nologin user
Check that you can reach out the target SMTPD from different locations/clients,
host stdpierre.os3.site ping stdpierre.os3.site
and that ports smtp and submission are not filtered,
nmap -p 25,587 stdpierre.os3.site
Check that your own presence on the public network has a PTR and resolves,
curl -s ip.nethence.com | egrep '^your ip|resolves'
Setup Expect and Autoexpect to automate telnet test sessions for this PoC acceptance testing,
apt install expect expect-dev
Re-do the sample SMTP session with Expect,
./script0.1.no.mail.from.exp ./script0.2.otherdomain.exp ./script0.3.basic-pass.exp
Receivedfields in the message source tell us everything about message's path?
Most important importantly (here used config = default),
#smtpd_relay_restrictions (default: permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination) smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
Beware of the Postfix version. The syntax is continuously changing esp. between versions
3.x. Do NOT copy/paste configuration snippets from the web. Read The Freacking Manual
man 5 postconf and Postfix Documentation. How to find relevant manual pages by the way?
The tested configurations are enabled then disabled so the next one can actually be tested (it also helps to get back to the scientific negative control status anyway).
DO NOT FORGET TO READ THE LOGS IN REAL TIME,
tail -F /var/log/mail*
and for production progressively seperate the output, at least look at both logs separately,
tail -F /var/log/mail.log tail -F /var/log/mail.err
Note: On Redhat systems there is
from an NXDOMAIN client
450 4.7.1 Client host rejected: cannot find your reverse hostname, [184.108.40.206] 450 Requested mail action not taken: mailbox unavailable
unknown_client_reject_code = 554
from an NXDOMAIN client
554 5.7.25 Client host rejected: cannot find your hostname, [220.127.116.11] 554 Transaction failed
Note: the client IP has changed in the meamwhile. This has nothing to do with the Schmilblick.
example on production system
reject_rbl_client bl.spamcop.net reject_rbl_client cbl.abuseat.org reject_rbl_client b.barracudacentral.org reject_rbl_client zen.spamhaus.org # this is too restrictive # reject_rhsbl_sender dsn.rfc-clueless.org, # apews is blocking bsd.nethence.com (online.net) # reject_rbl_client l2.apews.org, # sarbl.org not found # reject_rbl_client public.sarbl.org,
example on prod server
Sep 14 10:05:37 postfixprod postfix/smtpd: NOQUEUE: reject: RCPT from dsl-197-245-184-22.voxdsl.co.za[18.104.22.168]:50029: 554 5.7.1 Service unavailable; Client host [22.214.171.124] blocked using cbl.abuseat.org; Blocked - see http://www.abuseat.org/lookup.cgi?ip=126.96.36.199; from=<firstname.lastname@example.org> to=<email@example.com> proto=ESMTP helo=<dsl-197-245-184-22.voxdsl.co.za> Sep 10 01:47:37 postfixprod postfix/smtpd: NOQUEUE: reject: RCPT from 69-171-232-140.outmail.facebook.com[188.8.131.52]:31497: 554 5.7.1 Service unavailable; Client host [184.108.40.206] blocked using bl.spamcop.net; Blocked - see http://www.spamcop.net/bl.shtml?220.127.116.11; from=<firstname.lastname@example.org> to=<email@example.com> proto=ESMTP helo=<mx-out.facebook.com> Sep 13 14:21:18 postfixprod postfix/smtpd: NOQUEUE: reject: RCPT from mail4.hop-digital.net[18.104.22.168]:37099: 554 5.7.1 Service unavailable; Client host [22.214.171.124] blocked using b.barracudacentral.org; Client host blocked using Barracuda Reputation, see http://www.barracudanetworks.com/reputation/?r=1&ip=126.96.36.199; from=<firstname.lastname@example.org> to=<email@example.com> proto=ESMTP helo=<mail4.hop-digital.net> Aug 11 05:49:19 postfixprod postfix/smtpd: NOQUEUE: reject: RCPT from ext0.benefit-sharing.info[188.8.131.52]:58944: 554 5.7.1 Service unavailable; Client host [184.108.40.206] blocked using zen.spamhaus.org; https://www.spamhaus.org/sbl/query/SBLCSS; from=<firstname.lastname@example.org> to=<email@example.com> proto=ESMTP helo=<server.benefit-sharing.info>
Note: IAP's IP address ranges are blocked! What does this mean for SMTP servers at home?
from any client
(echo helo crap; echo mail from:whatever; echo rcpt to:firstname.lastname@example.org) | telnet stdpierre.os3.site 2525
returns (instead of Relay Access Denied),
Client host rejected: Improper use of SMTP command pipelining
enabling SPF policy,
on gandi.net hosted NS with WEB GUI
from a system that can do outgoing tcp/25,
telnet mx.nethence.com 25 (could not reproduce -- TXT record broken?)
on production system
Sep 14 15:28:38 postfixprod postfix/smtpd: NOQUEUE: reject: RCPT from unknown[220.127.116.11]:44712: 550 5.7.1 <unknown[18.104.22.168]:44712>: Client host rejected: Message rejected due to: SPF fail - not authorized. Please see http://www.openspf.net/Why?s=mfrom;email@example.com;ip=22.214.171.124;firstname.lastname@example.org; from=<email@example.com> to=<firstname.lastname@example.org> proto=ESMTP helo=<adeptmachinetools-com.mail.protection.outlook.com>
on production system
vi /etc/postfix/client_access postmap /etc/postfix/client_access ls -lhF /etc/postfix/client_access
from a system that can do outgoing tcp/25,
telnet mx.nethence.com 25
554 5.7.1 <ec2-18-221-91-30.us-east-2.compute.amazonaws.com[126.96.36.199]:47566>: Client host rejected: compute.amazonaws.com is identified as a spam domain
smtpd_helo_restrictions = permit_mynetworks, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname, reject_unknown_helo_hostname, regexp:/etc/postfix/helo.regexp #reject_unknown_helo_hostname --> unknown_hostname_reject_code unknown_hostname_reject_code = 554
gives among others,
554 5.7.1 <this.is.me.domain>: Helo command rejected: Host not found
#postmap /etc/postfix/sender_access smtpd_sender_restrictions = permit_mynetworks, check_sender_access hash:/etc/postfix/sender_access, reject_non_fqdn_sender, reject_unknown_sender_domain # warn_if_reject, #too restrictive, this prevents unreal addresses to send #you messages. try to book a hotel or a flight with that #and you will feel the pain, # reject_unverified_sender #unverified_sender_reject_code = 550 #unverified_sender_reject_reason = Address verification failed #address_verify_map = proxy:btree:$data_directory/verify_cache #address_verify_cache_cleanup_interval = 72h # # Postfix 2.6 and later. # unverified_sender_defer_code = 250 # #proxy_write_maps = $smtp_sasl_auth_cache_name $lmtp_sasl_auth_cache_name $address_verify_map $postscreen_cache_map
smtpd_recipient_restrictions = permit_mynetworks, reject_non_fqdn_recipient, reject_unknown_recipient_domain #reject_unknown_sender_domain --> unknown_address_reject_code #reject_unknown_recipient_domain --> unknown_address_reject_code unknown_address_reject_code = 554 # DATA restrictions # Block clients that speak too early. smtpd_data_restrictions = reject_unauth_pipelining
man 5 postconf