Index of /computing/parp/files

	Name	Last modified	Size	Description
	Parent Directory		-
	Parp-0.60.tar	2002-04-29 16:17	240K
	parp.Xrefs	2001-11-01 15:53	201K
	ChangeLog	2002-04-29 16:17	62K
	parp.tar.gz	2002-04-29 18:37	58K
	Parp-0.62.tar.gz	2002-04-29 18:37	58K
	Parp-0.60.tar.gz	2002-02-24 12:25	55K
	Parp-0.56.tar.gz	2002-01-02 00:54	43K
	parp.orig	2001-05-14 16:07	42K
	MyFilter.pm.adam	2001-11-23 19:54	42K
	parp-0.55.tar.gz	2001-10-21 21:59	38K
	MyFilter.pm.sample	2001-11-14 14:54	30K
	Makefile.old	2002-02-20 13:53	25K
	Makefile	2004-08-15 20:04	22K
	parp-inject	2002-01-02 00:32	15K
	primitives.diff	2002-02-24 23:52	15K
	README.html	2003-11-01 22:59	12K
	stats	2002-04-23 13:34	10K
	Mail-Field-Received-0.23.tar.gz	2000-10-13 16:45	9.9K
	parp	2002-04-29 16:16	6.6K
	sample-stats.txt	2000-05-02 12:17	6.0K
	COPYING	2002-02-24 00:24	6.0K
	TODO	2002-04-02 14:12	4.8K
	parp-inject.c	2002-01-02 00:29	1.4K
	Makefile.PL	2002-02-23 19:24	819
	MANIFEST	2002-04-29 16:14	568
	Mail-Box.patch	2001-11-22 16:00	460
	FAQ	2002-02-23 19:41	230
	pm_to_blib	2002-04-29 16:17	0
	t/	2002-02-23 19:24	-
	sample-confs/	2002-04-29 18:12	-
	blib/	2002-02-24 12:23	-
	Parp/	2002-04-29 18:12	-
	Mail/	2002-03-12 11:26	-

parp - Perl-based Anti-spam Replacement for Procmail

<#exe>parp is a powerful, extensible, hackerware e-mail filter with sophisticated anti-spam capabilities. It's written in <#perl>, so theoretically it can be run on about 70 different platforms. However, it was written with UNIX systems in mind, and so far has only been tested on RedHat Linux.

N.B. The user configuration file is written in pure Perl. Please understand this means that unless you have a certain degree of experience in Perl or similar programming languages, you currently stand very little chance of being able to use the filter to its full potential. That's what I meant when I said it was hackerware. If you're in any doubt as to whether you'll be able to "hack it" (literally), take a look at the sample configuration files provided. If they look fairly understandable then chances are you can use parp. If not, you're welcome to try anyway, but I can't be held responsible for the consequences! (Not that I can anyway.)

Also please note: the filter is currently working beautifully for me, but has had next to no external testing. If you let it loose on your own e-mail, make sure you take the necessary precautions!

Motivation and goals

This was yet another personal itch which needed scratching. I receive between 5 and 20 spam e-mails most days. It wasn't only mildly annoying to have to hit delete more than normal, but I also forward all e-mail which ends up in my main inbox to my mobile (cell) phone via email2sms and an Internet/SMS gateway, and I was sick to death of my phone bleeping a lot through the day purely due to junk mail.

I started looking at all the available anti-spam filters. Over a period of two years, I looked at many, including the NAGS filter, despam, various complex anti-spam procmailrcs, the spamometer, blackmail, filter.plx, zfilter, spamstop, junkfilter ... but various things put me off all of them:

• Some weren't written in <#perl>. Call me a Perl bigot, but if there was ever a case of Perl being the right tool for the job, it's an e-mail filter. Extensibility and maintainability were very high on my list.
• Some were terribly coded. I refuse to put my e-mail at the mercies of bad code (and that includes sendmail ;-).
• Some insisted that you use a particular MDA or MUA. I have no intentions of changing from mutt and qmail.
• Many filtered on only the headers, or only the body. I want to filter on both, not all the time, but in some circumstances.
• None were as accurate as I wanted. My goal was at least 99% accuracy. (At the time of writing, parp's accuracy is hovering around the 99.8% mark.)

So, I resolved to write my own anti-spam filter, taking the best features of all the others. As I started, I realised that it wouldn't be that much more effort to scrap my .procmailrc and rewrite it as part of this filter, and it would be well worth the power gained. Besides, I was fed up of procmail's rather clumsy configuration language, and I missed not being able to use <#perl>'s regular expression syntax.

Features

• Can act as a filter in a similar manner to procmail, or directly on files in Mbox format (and possibly other formats via <#pm>Mail::Box - untested), or as a daemon processing mails from a spool. In the latter case, mails are injected into the queue via a tiny (15k on my system) executable which handles locking correctly.
• Standard filtering actions are available (deliver to mailbox, pipe to command, reject as junk etc.)
• Highly sophisticated spam detection heuristics: currently around 40 different tests performed in a worst case scenario, although all tests optimised for speed (e.g. fast tests performed on headers, then slower tests only performed on body if necessary). N.B. I'm considering incorporating the SpamAssassin ruleset at some point too.
• Optional cross-checking with the Open Relay Database.
• Filter adds X-Parp-Accepted: and X-Parp-Rejected: headers so that you can easily monitor its filtering strategy without leaving your mail reader.
• MIME multi-part aware, e.g. will not be confused by binary attachments.
• Berkeley DB format friends database, for keeping false positives to an absolute minimum.
• Automatic extraction of addresses into the friends database from emails which pass the spam tests. Semi-automatic removal of addresses from the friends database on the rare occasions parp gets it wrong. The friends database is also easily editable with my dbm utility.
• Other `grace' tests allowing bona fide persons' communications through (e.g. passworded e-mails) just in case all the other tests go badly wrong.
• The configuration files are written in raw <#perl>, so you can extend the filter arbitrarily using the main program's API.
• Comprehensive logging and error-trapping systems.
• Auxiliary program to print out comprehensive statistics on all aspects of filtering (see the sample output).
• Ability to log false positives/negatives when spam detection has gone wrong in a way which can be interpreted by the statistics program to determine the filter's current accuracy of spam detection.
• Mostly RFC822-compliant state machine parser of Received headers, enabling extensive spam trace analysis and retaliative action. Read its man page or source if you're curious.
• Duplicate removals (by message id).
• Emails which have already been filtered can be used as regression tests, to easily spot problems when you make changes to your filtering logic.

Limitations

• Limited documentation so far. This is gradually improving.
• Requires some knowledge of Perl / programming. (Ironically, if it didn't, there would be far greater limitations to the filter's flexibility.)

Wishlist

Prerequisites

You'll need <#perl> 5.6.0 or later, and the following Perl modules, available from <#cpan>:

• <#pm>Digest::MD5 (for calculating a unique message ID)
• <#pm>Mail::Box
• <#pm>Mail::Filter
• <#pm>Mail::Address
• <#pm>Mail::Internet
• <#pm>Mail::Field
• <#pm>Mail::Field::Received (this one also available locally)
• <#pm>Net::DNS (only if you want to enable RSS cross-checking)

Download

• a tarball including the latest version of everything
• browse all related files
• sample configuration files
• a program which calculates interesting statistics from the log file (see the sample output)
• the change log

Installation

Here are brief instructions (hey, I did say it was hackerware):

1. Ensure you have the prerequisites installed.
2. Install the <#pm>Parp::* modules via the standard Perl <#keyinput>perl Makefile.PL; make install procedure.
3. Edit the configuration files <#pm><#samples> to suite your own filtering needs, and save them in <#dir>~/.parp (or elsewhere if you know how to instruct Perl to find it).
4. Install <#exe>parp somewhere in your $PATH and check that it happily compiles by running it with the <#keyinput>-h option.
5. Invoke <#exe>parp in test mode (the <#keyinput>-t option) as a filter (taking one e-mail from STDIN) or on one or more folders (with the <#keyinput>-f option).
6. If you want to use parp in daemon mode (highly recommended), type <#keyinput>make parp-inject to compile the executable which inserts mails into the daemon queue. Then put something like <#keyinput>/path/to/parp-inject /home/me/mail/spool in your <#file>.forward or <#file>.qmail, or do whatever else is necessary to get your MDA using parp as a filter. Finally, start up the daemon with the option <#keyinput>-D /home/me/mail/spool.
7. When you invoke parp, don't forget to enable the nice extras if appropriate, like the <#keyinput>-d option for discarding duplicates, or the <#keyinput>-r option for RSS cross-checking.
8. Send yourself some test e-mails to make sure they're getting dealt with as you'd wish.
9. Enjoy an almost spam-free life immediately.
10. If the filter mistakenly classifies a spam as bona-fide, or vice-versa, invoke the filter on that e-mail with the <#keyinput>-w option. (Binding a key press in your favourite e-mail reader to pipe a selected e-mail to <#keyinput>parp -wv is recommended.) This writes a comment to the logfile that the filter got it wrong, which can be used by the accompanying statistics program. It also removes the appropriate addresses from the friends database if they happened to be in there already.
11. Tweak the filter logic every time you notice the filter behaving an a way which doesn't satisfy you. Bind a key press in your mail reader to pipe a selected e-mail to <#keyinput>parp -vt. Test your new configuration easily using the above key press.

Documentation

There isn't currently much documentation except fairly completel pod for each of the modules. This will hopefully change soon. However, as noted above, you probably shouldn't be using this filter unless you have some experience as a (Perl) programmer, in which case you should hopefully find the code very readable. In particular, read the documentation for <#pm>Parp::Filter and <#pm>Parp::Config via <#keyinput>perldoc, as these should give you a fair idea of how to get started customising the filter to your own tastes.

Feedback