Nethence Newdoc Olddoc Lab Your IP BBDock  

Anti-spam Freakness v0.1 (Sep 2017)

this presentation for the SNE Master Students at Innopolis University

Before we start

Everything clear about SMTP chains? What about smarthosts? (+ auth vs network?)

DNS and SMTP are so critical they are often outsourced

MDA/MUA vs spambox vs smtp session

There are several ways to handle SPAM.

Those methods can be cumulated but one gets great results with filtering during the SMTP session already.

Message’s source

Looking at a message’s source:

Layout of a standart SMTP session

Connect with telnet,

telnet stdpierre.os3.site 2525

Who you are (helo YOURSELF),

helo this.is.me.domain

Who’s email account are you impersonating (this is for the Return-Path)?

mail from:<this.is.my.email@example.com>

Who are you wiling to send a message to?

rcpt to:root
rcpt to:<happy@target.com>

What are you willing to send (including the message headers)?

data
From: Me Me Me <this.is.my.email@example.com>
To: Target <happy@target.com>
Subject: you have been spammed

spam content

Introduction about Acceptance Testing

==> Do not forget to make the customer sign it (even if some parts failed)!

(quick reminder of the major SMTP daemons available out there)

IMAGE HERE IMAGE HERE IMAGE HERE

One could consider distributing the postfix instances on different bare-metal or VMs or Docker containers against a shared storage. One could consider firewalling, load-balancing, CARP and DNS round-robin but this is out of topic. We are not reviewing infrastructure architecture for heavy production but only anti-spam specific tuning.

One could also consider the linkage between the SMTP services and the Calendar Webmail Chat etc. To undoom your customer or the company you will be working from from MS hosted mail facilities, the Open Source community does not have ideal answers yet. Microsoft is strong on the server market with their Exchange and Outlook products. Actually it is the only thing they got left. But I believe that we are going to take over this remaining peace too in the future (what else do they got anyway?).

By the way on the Desktop OS market, we (the Open Source community) are also becoming stronger with Unity, Cinnamon, MATE and Ubuntu Budgie.

Yes, given the labs you are working on, you are part of the community now.

Preparation – Initial server setup

myorigin = $mydomain
#(default to FQDN minus the first component)
mydomain = stdpierre.os3.site
myhostname = mx.stdpierre.os3.site
mydestination = $mydomain
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 172.17.0.0/16

smtpd_banner = $myhostname ESMTP
biff = no
append_dot_mydomain = no
delay_warning_time = 4h
readme_directory = no
compatibility_level = 2
alias_maps = hash:/etc/aliases

Forcing alias_maps because the default is alias_maps = hash:/etc/aliases, nis:mail.aliases.

Enabling delay_warning_time for user’s convenience.

Changing port 25 to 2525 to workaround the blocked outgoing SMTP sessions from Innopolis networks as well as from Google Cloud (so we can also do some tests from those locations),

vi /etc/postfix/master.cf

#smtp      inet  n       -       y       -       -       smtpd
2525      inet  n       -       y       -       -       smtpd

Create a dummy mail user called user,

groupadd mailuser
useradd -g mailuser -m -s /sbin/nologin user

Preparation – Initial client(s) setup

Check that you can reach out the target SMTPD from different locations/clients,

host stdpierre.os3.site
ping stdpierre.os3.site

and that ports smtp and submission are not filtered,

nmap -p 25,587 stdpierre.os3.site

Check that your own presence on the public network has a PTR and resolves,

curl -s ip.nethence.com | egrep '^your ip|resolves'

Setup Expect and Autoexpect to automate telnet test sessions for this PoC acceptance testing,

apt install expect expect-dev

Re-do the sample SMTP session with Expect,

./script0.0.sample.exp

Acceptance Testing with default relay restrictions (negative control)

Proceeding!

./script0.1.no.mail.from.exp
./script0.2.otherdomain.exp
./script0.3.basic-pass.exp  

– SPOILER –

Hint on # RELAY restrictions

Most important importantly (here used config = default),

#smtpd_relay_restrictions (default: permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination)
smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

Acceptance Testing # NETWORK restrictions (smtpd_client_restrictions)

Beware of the Postfix version. The syntax is continuously changing esp. between versions 2.x and 3.x. Do NOT copy/paste configuration snippets from the web. Read The Freacking Manual man 5 postconf and Postfix Documentation. How to find relevant manual pages by the way?

The tested configurations are enabled then disabled so the next one can actually be tested (it also helps to get back to the scientific negative control status anyway).

DO NOT FORGET TO READ THE LOGS IN REAL TIME,

tail -F /var/log/mail*

and for production progressively seperate the output, at least look at both logs separately,

tail -F /var/log/mail.log
tail -F /var/log/mail.err

Note: On Redhat systems there is /var/log/maillog.

enabling in main.cf,

reject_unknown_reverse_client_hostname

returns,

from an NXDOMAIN client

450 4.7.1 Client host rejected: cannot find your reverse hostname, [188.130.155.155]
450     Requested mail action not taken: mailbox unavailable

replacing with,

reject_unknown_client_hostname

enabling,

unknown_client_reject_code   = 554

returns,

from an NXDOMAIN client

554 5.7.25 Client host rejected: cannot find your hostname, [188.130.155.154]
554     Transaction failed

Note: the client IP has changed in the meamwhile. This has nothing to do with the Schmilblick.

enabling,

example on production system

    reject_rbl_client bl.spamcop.net
    reject_rbl_client cbl.abuseat.org
    reject_rbl_client b.barracudacentral.org
    reject_rbl_client zen.spamhaus.org

# this is too restrictive
#   reject_rhsbl_sender dsn.rfc-clueless.org,

# apews is blocking bsd.nethence.com (online.net)
#   reject_rbl_client l2.apews.org,

# sarbl.org not found
#   reject_rbl_client public.sarbl.org,

logs,

example on prod server

Sep 14 10:05:37 postfixprod postfix/smtpd[18751]: NOQUEUE: reject: RCPT from dsl-197-245-184-22.voxdsl.co.za[197.245.184.22]:50029: 554 5.7.1 Service unavailable; Client host [197.245.184.22] blocked using cbl.abuseat.org; Blocked - see http://www.abuseat.org/lookup.cgi?ip=197.245.184.22; from=<service@vanwertvet.com> to=<emailpbraun@nethence.com> proto=ESMTP helo=<dsl-197-245-184-22.voxdsl.co.za>

Sep 10 01:47:37 postfixprod postfix/smtpd[9923]: NOQUEUE: reject: RCPT from 69-171-232-140.outmail.facebook.com[69.171.232.140]:31497: 554 5.7.1 Service unavailable; Client host [69.171.232.140] blocked using bl.spamcop.net; Blocked - see http://www.spamcop.net/bl.shtml?69.171.232.140; from=<update+ta9c==6@facebookmail.com> to=<pbraun@nethence.com> proto=ESMTP helo=<mx-out.facebook.com>

Sep 13 14:21:18 postfixprod postfix/smtpd[17080]: NOQUEUE: reject: RCPT from mail4.hop-digital.net[82.96.169.4]:37099: 554 5.7.1 Service unavailable; Client host [82.96.169.4] blocked using b.barracudacentral.org; Client host blocked using Barracuda Reputation, see http://www.barracudanetworks.com/reputation/?r=1&ip=82.96.169.4; from=<emma@smtp.on-compare.fr> to=<pbraun@nethence.com> proto=ESMTP helo=<mail4.hop-digital.net>

Aug 11 05:49:19 postfixprod postfix/smtpd[19570]: NOQUEUE: reject: RCPT from ext0.benefit-sharing.info[37.187.180.196]:58944: 554 5.7.1 Service unavailable; Client host [37.187.180.196] blocked using zen.spamhaus.org; https://www.spamhaus.org/sbl/query/SBLCSS; from=<return@benefit-sharing.info> to=<pbraun@nethence.com> proto=ESMTP helo=<server.benefit-sharing.info>

Note: IAP’s IP address ranges are blocked! What does this mean for SMTP servers at home?

enabling,

reject_unauth_pipelining

telnet bulk,

from any client

(echo helo crap; echo mail from:whatever; echo rcpt to:abuse@nethence.com) | telnet stdpierre.os3.site 2525

returns (instead of Relay Access Denied),

Client host rejected: Improper use of SMTP command pipelining

enabling SPF policy,

on aws

less /etc/nsd/stdpierre.os3.site

on gandi.net hosted NS with WEB GUI

https://gandi.net/

and testing,

from a system that can do outgoing tcp/25,

telnet mx.nethence.com 25

(could not reproduce -- TXT record broken?)

usually returns,

on production system

Sep 14 15:28:38 postfixprod postfix/smtpd[19305]: NOQUEUE: reject: RCPT from unknown[217.10.204.238]:44712: 550 5.7.1 <unknown[217.10.204.238]:44712>: Client host rejected: Message rejected due to: SPF fail - not authorized. Please see http://www.openspf.net/Why?s=mfrom;id=onxoae@adeptmachinetools.com;ip=217.10.204.238;r=pbraun@nethence.com; from=<onxoae@adeptmachinetools.com> to=<pbraun@nethence.com> proto=ESMTP helo=<adeptmachinetools-com.mail.protection.outlook.com>

enabling client_access,

on production system

vi /etc/postfix/client_access
postmap /etc/postfix/client_access
ls -lhF /etc/postfix/client_access

testing,

from a system that can do outgoing tcp/25,

telnet mx.nethence.com 25

gives,

554 5.7.1 <ec2-18-221-91-30.us-east-2.compute.amazonaws.com[18.221.91.30]:47566>: Client host rejected: compute.amazonaws.com is identified as a spam domain

Acceptance Testing # HELO/EHLO restrictions

enabling,

smtpd_helo_restrictions = permit_mynetworks,
    reject_invalid_helo_hostname,
    reject_non_fqdn_helo_hostname,
    reject_unknown_helo_hostname,
    regexp:/etc/postfix/helo.regexp

#reject_unknown_helo_hostname --> unknown_hostname_reject_code
unknown_hostname_reject_code = 554

gives among others,

554 5.7.1 <this.is.me.domain>: Helo command rejected: Host not found

Acceptance Testing # MAIL FROM restrictions

#postmap /etc/postfix/sender_access
smtpd_sender_restrictions = permit_mynetworks,
    check_sender_access hash:/etc/postfix/sender_access,
        reject_non_fqdn_sender,
        reject_unknown_sender_domain


#        warn_if_reject,
#too restrictive, this prevents unreal addresses to send
#you messages.  try to book a hotel or a flight with that
#and you will feel the pain,
#       reject_unverified_sender
#unverified_sender_reject_code = 550
#unverified_sender_reject_reason = Address verification failed
#address_verify_map = proxy:btree:$data_directory/verify_cache
#address_verify_cache_cleanup_interval = 72h
#
# Postfix 2.6 and later.
# unverified_sender_defer_code = 250
#
#proxy_write_maps = $smtp_sasl_auth_cache_name $lmtp_sasl_auth_cache_name $address_verify_map $postscreen_cache_map

Acceptance Testing # RCPT TO restrictions

smtpd_recipient_restrictions = permit_mynetworks,
        reject_non_fqdn_recipient,
        reject_unknown_recipient_domain

#reject_unknown_sender_domain --> unknown_address_reject_code
#reject_unknown_recipient_domain --> unknown_address_reject_code
unknown_address_reject_code  = 554

# DATA restrictions
# Block clients that speak too early.
smtpd_data_restrictions = reject_unauth_pipelining

References

Further readings


Home | GitHub | Donate | Contact