Consulting Ian Tighe Putting Technology To Work

Adding Value

Technology and Project Advice




Making Things Work Better

Know-How
Linux :: Fedora 7 :: postfix :: spam :: counter measures
When you run your own smtp server then a strategy for deaing with spam is an essential thing to have. Some 71% of mail recieved by this site's server was spam. A real waste of bandwidth but there is something we can do about that - refuse to receive it.

Many people both have worked hard and continue to do so to bring us relief from spam. The Postfix MTA has seen considerable effort along these lines and is the choice for this site. It has some excellent features for combating those seeking out open relays and it uses DNSBL ( black listing ) techniques which is a very valuable type of service from people like spamhaus.org/dsbl.org. Greylisting also plays a part in effectively checking if sending servers are properly configured and will retry a message send following an initial "please try again later response".  Combine these features with:
  • our own DNSBL dns zone
  • body checks
  • header checks
  • a bash script to review message logs each 5 minutes picking out those IPs/domains that get around all the above measures (but are nevertheless relayers or they got sucked into the honey pot) for insertion into the firewall dynamically
  • along with any DNSBL finds being entered into the firewall rules
  • finally any hack attempts added dynamically to the firewall

This is a formidable level of protection. Using a firewall in this way means you even keep known spammers away from the online DNSBL checks and they do not even reach postfix. This is judged to be a good strategy not just because of its breadth but also because the likes of spamhaus.org cannot keep expanding forever. It makes sense that once spamhaus.org has identified a spammer through its DNSBL services then firewalling it locally takes the load off spamhaus.org/dsbl.org for that IP and it keeps connections off your SMTP server. This is a fair array of counter measures and seems to take out about 95% of spam or relay attempts so far. This is quite pleasing. So the checks are at least as follows:

  • In the postfix config file check for bad HELOs, bad source domains, bad destination domains and unknown sender addresses, bad hostnames as a minimum.
  • Header and body check content is then analysed for any matching rules - exclude messages on the presence of certain words or phrases so, for instance, avoid viagra offers and those great gold Rolex watches for 50 bucks.
  • Build and utilise your own black-list dns zone for addresses gathered and researched and known to be baddies.
  • Next utilise the dsbl.org and both spamhaus.org sbl and xbl zones to see if the incoming mail emanates from a known spammer or hacker. This includes virus and some hack attempts by the way.
  • Utilise greylisting too as this brings spam quantities down quite effectively.
  • Finally review the logs every 5 minutes and build a list of IP addresses that have emerged as hack or spam attempts (including those blocked by your DNSBL service provider) and insert rules to block those addresses in the firewall. Keep all IPs in a database and age them for deletion later.
  • Not really email connected but then scan the logs for any attempts to connect through ports for which visitors are not authorised and which represent unwelcome traffic - these also get added to the firewall. You will need to have set your firewall to trap these attempts so you can get the addresses involved.
Note that the log analysis is undertaken every five minutes. Only the latest log file is used. IPs are stored in a database and aged to make sure they come back into play later. That way it means they are barred for a period and then released so they have to be proven again to be "baddies" to get blocked in the future. Clearly if they are still baddies they get listed again based on evidence and the cycle starts again until they are released again. Anything other than this would block addresses for ever which is not what is required. Below are typical Postfix main.cf directives for protection:
#
# spam defence
#
smtpd_require_helo = yes
disable_verify_command = yes

smtpd_helo_restrictions =
reject_invalid_helo_hostname,
reject_unknown_hostname,
reject_rbl_client zulu.renegade-hosting.co.uk,
reject_rbl_client list.dsbl.org,
reject_rbl_client sbl.spamhaus.org,
reject_rbl_client xbl.spamhaus.org

smtpd_sender_restrictions =
reject_unknown_sender_domain,
reject_unknown_address

strict_rfc821_envelopes = yes

smtpd_recipient_restrictions = permit_mynetworks,
reject_invalid_hostname ,
reject_non_fqdn_sender ,
reject_unknown_recipient_domain ,
reject_non_fqdn_hostname,
reject_non_fqdn_recipient,
reject_unauth_destination,
check_policy_service inet:127.0.0.1:60000,
permit

smtpd_data_restrictions =
reject_unauth_pipelining,
permit

Statistics

Below are the stats for 3 days of activity by vector and category of attack. This sample represents the spam or attacks detected and trapped.

Error in SQL : select count(*) from ip where archived = 'n' : Unknown column 'archived' in 'where clause'Error in SQL : select name , count(*), c.category from ip i left join categories c on c.type=i.reason where i.archived = 'n' group by i.reason : Unknown column 'i.archived' in 'where clause'Sample Size is
VectorQuantityPercentage of all IncidentsCategory

There has been some aggregation of the data above. Also if a SMTP connection/session fails on more than one detection measure (e.g. bad HELO and known spammer in DNSBL check) then it is the latest reason for failure that is measured here. For example - if a spammer fails HELO the IP address is categorised and counted as such. Later if the spammer fixes the HELO issue but fails a DNSBL check then that IP is measured as a failed DNSBL and the previous HELO is deleted.

Unauthorised access attempts blocked at the firewall :

The above shows firewall blocks: includes persistant spammer IPs, excessive/intrusive ping activity and security classes of incident.

A list of currently blocked IPs is shown here

Note also that greylisting is in use here. We changed from the standard Fedora 11 Unix socket config and implemented this with a server on port 60000.  To succeed in getting a mail message through the sender's smtp server must be properly configured and respond to a requested delay; most spammers do not set up their servers properly so they get dumped and the mail never gets onto our server.

Before finishing with this whirlwind topic we should say that we tried a number of providers of DNSBL type services and whilst we have settled on two of them there are others that you should try. Reference should be made, though, to SORBS before we finish. This too is a site for, amongst other services, DNSBL type checking but it is woefully inadequate compared to the others. It really should be avoided as it adds no value to the data provided and treats whole swathes of IP addresses in a way that immediately leads people to believe they are guilty of spamming - when in fact they are not. Their defence is that it is for others to interpret what being in their database means and they make no judgements. The very fact they offer such a service and return data on innocent IP addresses makes it such that context alone brings about the assertion of guilt and the tangible feeling that something is wrong with an IP address. The issue here is that ranges of addresses marked as dynamic but given out by ISPs as static (they commonly class long lease dynamic as static) means some businesses get their smtp servers et al rejected. This is one such business with a number of static addresses allocated from Virgin Media's dynamic block used for small business. Efforts have been made to persuade but after being told we were only semi-literate on these matters we gave up. Their comment below:

I think you are not understanding the problem (don't worry it's quite
common).  I choose to block how I want, as others choose how they want.

 

tells quite a story. Our advice - avoid the Neanderthal-like SORBS until they see the light. Indeed avoid any service that blocks on the basis of range checking of IP addresses - they are blunt instruments!


If you would like to have an implementation of this approach then please contact us and we will be happy to help you at competitive consultancy rates. 22 Aug 2007

Return to Know-How Index of articles
Return to Home Page
Copyright © Consulting Ian Tighe 2005-2008.