Exim SpamAssassin at SMTP time
What's that?
mail from: merlin@gandalf
250 OK
rcpt to: merlin@gandalf
250 Accepted
data
354 Enter message, ending with "." on a line by itself
From: merlin@gandalf
To: merlin@gandalf
Subject: $$$ Make Money Fast $$$ !!!
viagra 100% GARANTEE AMAZING FULL REFUND
This is not spam
.
550 Rejected
(logs would show something like this:
2004-03-10 08:27:18 1B16Y8-0001UP-4R SA: Action: permanently rejected message: hits=14.8 required=7.0 trigger=11.0 ( scanned in 2/2 secs | Message-Id: CCQPVENACPQBFLTRLICXWQVEK@gandalf). From (host=gandalf [127.0.0.1]) for merlin@gandalf)
An example of teergrube would return this instead
data
354 Enter message, ending with "." on a line by itself
(...)
body SEE_FOR_YOURSELF /See (?:for|it) yourself\b/i
describe SEE_FOR_YOURSELF See for yourself
body ORDER_NOW /\border (?:now|soon|fast|quickly|while)\b/i
describe ORDER_NOW Encourages you to waste no time in ordering
.
451- wait for more output
451- wait for more output
451- wait for more output
(... one line every 10 secs, 15 minutes elapse ...)
450 Please try again later
The idea here is to stall and waste the resources of the remote sender (BTW
teergrube comes from german, and means tar-pitting, or stopping someone in his
tracks)
Unmaintained
As indicated in sa-exim list message, I haven't maintained sa-exim since 2006. It does work as shipped in Debian and is still used by people, including myself, but I just don't have time to spend on it anymore. Don't let that stop you from using it, just please don't ask me to add features :)
Why?
SpamAssassin can be run inside exim after the mail has been accepted, as shown
here,
but if you're not going to use my patch and you just want to run SA as an exim
transport,
this
version is recommended
Now, while this will work, we can do better, hence the reason for my code
(just to make things clear, you do not want to run both my code, and dman's
transports. It'd work, but you'd be scanning the message twice)
The reason why I wanted SpamAssassin in local scan is that I don't want to
accept the damn spam in the first place.
- While my code lets you do that, I don't like to send mails to the bit
bucket, so you need to bounce them.
- Once you accept the spam, you can't bounce it half the time, or you
bounce it to an innocent whose Email was forged as an envelope sender
(some spam even forges the bounce address to you)
- If I refuse spam at SMTP time, it will remove the spam addresses from at
least a few lists (they gotta clean their lists eventually otherwise they'd
spend more time Emailing dead addresses than good ones)
- I have the option of toying with spammers and stall their connections and
waste their resources (see the following page for details on
teergrubing
Note that you can also use this code to simply run SA on all your mails (or
portion thereof as configured with SAEximRunCond) without having to configure SA
in your exim.conf. In other words, this code can be configured to not reject
any mails.
SpamAssassin? What's that?
Ah, you need to visit this page first
then
How does it work, what knobs are there?
You need to configure spamassassin to flags mails as spam after a certain
threshold (7 for instance). After that, this code can be configured to
- Pretend to be processing the Email and send continuation lines to the
remote server until it gives up (aka
teergrubing)
- Accept but not deliver mail with a high threshold (i.e. devnull the mail)
- Reject mail with a lower threshold
- Temporarily reject mail with a still lower threshold (you can then inspect
your logs to decide if you want to tweak SA so that next time the mail
is sent, you can receive it)
- In all 5 cases, mail can be optionally saved to disk so that you can
inspect all the mails you've rejected or /dev/nulled
You can also (and probably should ) use the new greylisting support for even
better spam control
For more details, you should look at the self-documented
config file and you can see
some sample rejects and what you get in the logs
While when sa-exim first came out, its strongest point was being one of the
first programs (if not the first) that let you reject Spam at SMTP time, its
coolest feature now is adaptive greylisting support
In a nutshell, you get the advantages of greylisting without the disadvantages:
- mails with a low spam score are accepted without delay
- mails with an average spam score are greylisted,
and only those are delayed
- mails with high spam scores are rejected regardless (no greylisting)
This method is the best combination I've seen out there so far, and
while I've been talking about it for a while, I don't yet know of other
programs that implement this method (if you do, please let me know so that
I can acknowledge them)
For more details on how this works, check out the greylisting README
Ok, where's the code? / Downloads
As explained in the archive, you can either copy sa-exim.c over exim's
src/local_scan.c You need to copy local_scan in src in the exim source
tree and rebuild it, or you can build sa-exim as a loadable module (you need
to patch exim to support loadable modules though)
You can also browse all my exim files here
Mailing list
You should probably subscribe to this low traffic
mailing list if
you download the code to keep apprised of bug fixes and enhancements
Integration with Exim 4
This code works without anything in the exim conf, but you probably want to use
some knobs to disable scanning for some users (like setting
X-SA-Do-Not-Rej or X-SA-Do-Not-Run in the rcpt ACL and
removing those headers in the right places).
See my exim4 conf tree
and more specifically the
exim4.conf
file
You can look at the README for more
integration details.
Changelog/Download
2006/01/09 - v4.2.1 (sa-exim.tar.gz
or local_scan only)
Security update (reported by Chris Morris)
- Modified Greylisting.pm not to generate tuplets with spaces, although
the cleaning cron job is now safe with regard to whitespace
- Included Mark Lawrence's perl script to better clean old tuplets
- The default config now ships with the greylistclean cronjob running
as nobody (which spamd should be running as, too)
2005/01/17 - v4.2 (sa-exim.tar.gz
or local_scan only)
Do not use, greylistclean is insecure, use 4.2.1 instead
- Fixed code so that it compiles inside the exim tree too (thanks Jason)
- Support SA 3.0 new score reporting format (score= instead of hits=)
- Proper SA 3.0 plugin support
- Documentation updates
- greylisting files now also save the last SA score, just in case
- Saved files are now group read/write too, if you need to shut off group
access, do it at the directory level
- Only rewrite Content-Type/Content-Transfer-Encoding if SARewriteBody is
on now (this used to be useful with old SA versions and defang_mime
but those options are gone). Suggested by Adam Tilghman.
- Finally made all score threshold variables exim conditions that are
evaluated at runtime
- SAteergrubecond is deprecated as a result
2004/08/16 - v4.1 (sa-exim.tar.gz
or local_scan only)
Please see the mailing list, or use the CVS version if you are compiling
sa-exim inside your exim tree (there is a small mistake in the source
which will prevent proper building) / You also need CVS if you are
using SA 3.0
- Deal with being called with the number of recipients is 0 (reported by
John Horne)
- Made hopefully fully 64 bit clean (well, it was written that way,
except for that pesky printf in C, which got in the way). Kudos go to
Adam D. Barratt for finding and explaining the bug
- Switched to using strchr instead of index to remove compilation warnings
on some systems (reported by John Horne/Jeff Carter)
- Added SAspamcHost/SAspamcPort at the request of several people
- Fixed mbox From file to be standards complaint (reported by Cliff Hones)
- Added SAFEMESGIDCHARS option for safer message-id based filenames
- New version of the SA patch, with safe file creation (clean env from
and rcpt to before using as directory names)
- IMPORTANT: if you are using greylisting, and applied the
previous greylisting patch, it didn't create shell safe filenames,
it is therefore highly recommended that you apply the new version
- Added a quick patch to sa-exim to deal with headers that are longer
than 8K (and drop the remaining lines)
2004/03/16 - v4.0 (sa-exim.tar.gz
or local_scan only)
- Added SAspamcpath (from Richard Lithvall)
- Makefile fixes for version.h and dependencies
- Added exim acl_m trick from Chirik in README
- New code to read and pass on any X-Spam- header from SpamAssassin
- Message-Id is now logged in SA log entries to allow you track down
a message in other logfiles, like SA's log
- Add X-SA-Exim-Connect-IP header for greylisting
- Allow for tempreject on messages not flagged as spam by SA (for
greylisting)
- Added greylisting support (see README.greylisting)
- Added the following options to sa-exim.conf:
- SAspamcSockPath
- SAgreylistiswhitestr
- SAgreylistraisetempreject
- Lots of documentation fixes and updates
- Basic debian tree shipped by default (but real deb package from Sander
Smeenk)
2003/08/18 - v3.1 (sa-exim tar.gz
or local_scan only)
- Fixed SA: Action: teergrubed sender until full configured duration output
- spamassassin.conf was renamed sa-exim.conf (bad initial name choice)
- Changed teergrubing so that it works with exim 4.20 and above
- Added the much requested SARewriteBody option for SA 2.50's report_safe
(courtesy of Richard Lithvall)
- Added SAaddSAEheaderBeforeSA to deal with SARewriteBody and privacy
- Added SAPrependArchiveWithFrom option (Richard Lithvall)
- As announced in the previous version, SAStallSender was removed.
Use Teergrubing instead
- Message-Id is read directly from Exim now that the API allows for that
(we used to parse the Message-Id header ourselves) -> except that
we don't use this for logging or saving messages: as soon as we can
read the real Message-Id from the headers, we use that Message-Id for
logging (this is to facilitate tracking of messages from the logs and
correlating with something like SA timelog files)
- Logging improved some more: Message-Id is added to log entries, as
well as the full mailinfo whether the message is spam or not
2003/04/30 - v3.0 (sa-exim tar.gz
or local_scan only)
- Makefile can generate a short sa-exim.conf (for Brian Kendig)
- Added a contrib directory with 3rd party scripts
- Since we already had X-SA-Exim-Rcpt-To: to show the envelope to
(disabled by default), I added X-SA-Exim-Mail-From: to show the
envelope from (always enabled, but you can delete it in system_filter
if you wish). If you can't easily see or parse the envelope sender
in your mails, this should definitely help
- Much improved directory creation and error handling for the reject
save directories
- Added SIGCHLD patch from David Woodhouse
- Added version header that looks like this:
X-SA-Exim-Version: 2.2.x-cvs (built Tue Apr 22 10:28:25 PDT 2003)
- Rewrote pretty much every log message to be more consistent and grepable
(the previous ones were quite bad). Suggested by multiple people
including Brian Kendig
- Made the SMTP error messages finally configurable (suggested by
several people). This is now possible since exim lets me output a
different message in the log than in the SMTP session. Consequently, new
default messages do not show the spam score, this only goes to the exim
now.
- Made SAmaxrcptlistlength and option to control how long of a
X-SA-Exim-Rcpt-To header you can output if you want to output it at
all
- Do not output "savemail condition expanded to false" if not in debug
mode (as reported by Brian Kendig)
- Now ships with a sample local_scan.h if we can't find the exim source
- Change of logic to delete SA headers that were in the original mail
but weren't outputted by the SA run (like X-Spam-Flag, as reported
by Chad Leigh)
- Modified local_scan dlopen patch to deal with updates to the exim
local_scan API
- The default location for spamasassin.conf was changed to /etc/exim4
(debian default)
- Updated README and INSTALL
- Updated localscan_dlopen.patch to deal with minor/major version numbers
- Added X-SA-Exim-Version
2002/10/28 - v2.2 (sa-exim tar.gz
or local_scan only)
- Fixed a bug that affected all mails bigger than what spamc would
accept. Doh! (it not clear why, but spamc would hang and stop reading
after it had been fed more than it was willing to accept)
- Added more debugging code to help track the above problem
- Depending on SATruncBodyCond will now either not pass a message that's
too big to spamc, or will optionally truncate it first
- Now strips any X-SA-Exim-* headers already present in the message before
scanning it
- Added new X-SA-Exim-Rcpt-To: header (see privacy section in README)
after a suggestion from Brian Kendig
- Teergrube is now spelled correctly (note that the option names in
sa-exim.conf changed as a result)
- Teergrubing has been re-implemented to detect that the other side went
away so that exim doesn't stay around for nothing.
- Added SAteergrubecond so that you don't teergrube your neighbours
- Small cleanups
2002/10/13 - v2.1 (buggy) (sa-exim tar.gz
or local_scan only)
- Default mail config is now in /etc/exim instead of /etc/mail
- Fixed SAstallsender so that exim doesn't leave spool files behind
- Fixed comparison for spam values as suggested by Patrice Fournier
- Returned a header saying whether SA-Exim was able to scan the mail
or not (based on a suggestion from Patrice Fournier)
- Added support for teergrubing as suggested by Robert Strickler
(see
http://www.iks-jena.de/mitarb/lutz/usenet/teergrube.en.html)
- We now make sure that we feed spamc a little more data than it's
willing to accept. That way, we don't feed a truncated message
- sa-exim.conf now appends /new on save directories so that they
are valid maildir spool and you can run mutt -f dirname to read
- Small cleanups
2002/07/07 - v2.0.1 (sa-exim tar.gz
or local_scan only)
- Removed unneeded ## concatenation in macro
- Fixed lseek call, as reported by Peter N Lewis
- Improved Makefile for documentation
- Suggestion from dman to allow for config options overrides in Makefile
- Fixed all the compiler warnings that were fixable
- Handle more kinds of Message-Id as suggested by Patrice Fournier
2002/06/14 - v2.0 (sa-exim tar.gz
or local_scan only)
- Allow for stalling SMTP sessions from spammers (caution)
- Support X-Spam-Status from SA pre 2.30
- Build patches from Norm
- Documented that spamd shouldn't be run with -c
- dman's patch to allow for this to be a shared library
- dman's indirect request for saved files to be 660
2002/06/01 - v2.0b1
This is how 1.0 would have been if I had done it right :-)
- We now attempt to create directories we write to
(suggestion from Patrice Fournier)
- Log envelope sender, recipients and connecting host
on rejected messages (suggestion from Patrice Fournier)
- We can now save non rejected messages whether they were
flagged as spam or not (based on request from Patrice)
- Each save rule now has its own condition that lets you
decide which messages you want to save or not
- Fixed parsing of Message-Id. May not parse all of them
but won't do anything stupid with bad input
- Fixed a buglet where we removed old headers before knowing
whether they were going to be replaced.
- Added much needed internal alarm so that we don't run
too long
- I added lots of options, but weren't able to test all
of them and their combinations, please reports problems
you may notice and documentation issues (in config file)
- Thanks also go to dman for his plentiful feedback
2002/05/21 - v1.3
- We now close the files we create on disk. Duh!
2002/05/17 - v1.2.2
- SA options are now kept in memory. Small cleanups
2002/05/13 - v1.2.1
- v1.2 (unreleased) didn't work right: it tagged messages
properly, but failed to see what was marked as spam and
couldn't reject messages. Fixed.
- Stripped newlines in header lines (better for logging)
- fixed header_add bug if headers contained '%'
2002/05/12 - v1.2 (unreleased)
- According to Craig R Hughes, any X-Spam header can be multiline.
Let's parse them accordingly
2002/05/08 - v1.1.1
- Added fake envelope from to mails that we save on disk
so that they can be opened with MUAs (idea from dman)
2002/05/07 - v1.1
- Cleaned up and fixed multiline header parsing
- Added parsing of Subject back from Spamc (suggestion from dman)
- We now log how long spamc took, and all messages processed
by spamc get logged in mainlog so that we know the outcome
- Added custom header compare function to handle mixed case headers
2002/05/06 - v1.0.1
- Fixed problems with reading Content-Type back from spamc
(we used to read whole body too, ooops...)
- Handle multi-line Content-Type headers.
- All log entries now start with "SA: "
- If SA gets run twice, we drop the previous X-Spam headers
- Support for X-Spam-Prev-Content-Transfer-Encoding
2002/05/05 - v1.0
- *Many* changes. This is now a real program with hopefully
most of the configuration options people could need
- Fully configurable through external config file
- Condition to run or not run SA against a message
- 3 Levels of spam handling
- Option to save messages that are rejected or cause errors
- Very complete error checking
2002/04/17 - v0.9.1
- Added a few checks for possible failures
2002/04/16 - v0.9
More generally, all the files can also be found here
Feedback is appreciated (but please
prefer the use of the
sa-exim list)
Acknowledgements
While I wrote SA-Exim after realizing that I didn't want to accept Spam in the
first place, this package would not have been put together without the help
and contributions of the following people:
- Philip Hazel
We wouldn't have exim without him :-)
- Justin Mason, Craig R Hughes, Dan Quinlan, and the rest of the SpamAssassin
crew
- Derrick 'dman' Hudson
Early adopter, feedback, ideas, first integration with localscan_dlopen
- David Woodhouse
Excellent original localscan_dlopen patch for exim 4
SIGCHLD patch (was set to ignore by exim)
- Norm
Build patches and trailing slash problem for mkdir on netbsd
- Patrice Fournier
Several suggestions, including adding the X-SA-Exim-Scanned header
- Peter N Lewis
Spotted a buglet in lseek call
- Robert Strickler
Suggested that I add teergrube support
- Tim Jackson
Narrowed the problem with SA Exim 2.1 to a hang when we feed a message
that's too big to spamc and did a lot of research on the bug
- Brian Kendig
Suggested the functionality behind X-SA-Exim-Rcpt-To:
Noted and requested logging changes
- Paul Matthews
Reported string length problem with X-SA-Exim-Rcpt-To:
- Chad Leigh
Reported that SA-Exim doesn't delete a X-Spam-Flag header in the original
mail if the local config decides that the mail isn't spam
- Martin Balvers
Reported that
SA: Action: teergrubed sender until full configured duration
logging wasn't consistent.
Author of http://nossie.addicts.nl/projects/sa-exim-stats/
- Andreas Metzler
Has provided invaluable help by being the main maintainer of the exim4
debian package and doing most of the work for the sa-exim package, while
patiently helping me with my debian build questions
- Richard Lithvall
Added the much requested SARewriteBody option for SA 2.50's report_safe
Also added SAPrependArchiveWithFrom option
Also added SAspamcSockPath for SA 2.60 and better
- Justin F. Knotzke
Reported that Dec 31st 1969 isn't a good idea and upsets some MUAs like mutt
- Norihisa Washitake
Pointed out that I should be using date -R to override foreign locales
- Chirik
Made suggestions on increasing logging consistency
Gave example on how you can trigger mail scans without using headers
(using internal exim acl variables)
- Ross Boylan
Suggestions for documentation tweaks
- Sander Smeenk
Made sa-exim show up in Debian. Yeah! :)
Major doc diffs
- Stéphane MANHES
Pointed out that greylisting requires X-SA-Exim-Rcpt-To:
- John Horne
Reported crash on discard, due to the number of recipients being 0
- Adam D. Barratt
Found the problem with sa-exim's indirect use of printf and causing a crash
in exim on 64 bit architectures, and gave a detailed explanation on what
happened and how to fix it
- Jeffrey D. Carter
Suggested strchr instead of index for easier compilation.
- Danilo Lotina F. & Others
Suggested adding a destination host for spamc (I added a port too)
- Cliff Hones
Reported that From mbox line was in an incorrect format
- Jason John Schwarz
Reported compile problem inside the exim tree (i.e. without local_scan)
- Kristopher Austin
Initial port of the SA patch to an SA 3.0 plugin
- Adam Tilghman
Pointed out that I shouldn't be rewriting Content-Type headers if we don't
rewrite the body (and defang_mime is dead now)
- Mark Lawrence
He is the author of the greylistclean cron job
- Chris Morris
Reported the ealier tuplet deletion cronjob in shell as insecure
- Your name here
If you contribute or if you did and I forgot your name (sorry, let me know)
Email
Link to Home Page