Jari's Procmail Tips Page

Author: Jari Aalto.

Table of contents

1.0 Document id
  • 1.1 General
  • 1.2 What is Procmail?
  • 1.3 Abbreviations and thanks
  • 1.4 Version information
  • 1.5 Document layout and maintenance
  • 1.6 About presented recipes
  • 1.7 Variables used in recipes
  • 1.8 About "useless use of cat award"
2.0 Procmail pointers
  • 2.1 Where is procmail developed
  • 2.2 Procmail resources
  • 2.3 Procmail mode for Emacs
  • 2.4 Procmail module library project
  • 2.5 Procmail code to filter UBE
3.0 Dry run testing
  • 3.1 What is dry run testing?
  • 3.2 Why the From field is not okay after dry run?
  • 3.3 Getting default value of a procmail variable
4.0 Things to remember
  • 4.1 Get the newest procmail
  • 4.2 Csh's tilde is not supported
  • 4.3 Be sure to write the recipe starting right
  • 4.4 Always set SHELL
  • 4.5 Check and set PATH
  • 4.6 Keep the log on all the time
  • 4.7 Never add a trailing slash for directories
  • 4.8 Remember what term DELIVERED means
  • 4.9 Beware putting comment in wrong places
  • 4.10 Brace placement
  • 4.11 Local lockfile usage
  • 4.12 Global lockfile
  • 4.13 Gee, where do I put all those ! * $ ??
  • 4.14 If you Send an automatic reply, use X-loop header
  • 4.15 Avoid extra shell layer and check command for SHELLMETAS
  • 4.16 Think what shell commands you use
  • 4.17 Using absolute paths when calling a shell program
  • 4.18 Disabling a recipe temporarily
  • 4.19 Keep message backup, no matter what
  • 4.20 Order of the procmail recipes
5.0 Procmail flags
  • 5.1 The order of the flags
  • 5.2 Flags HB at top of recipe (warning)
  • 5.3 Flag w and recipe with |
  • 5.4 Flag w, lock file and recipe with |
  • 5.5 Flag f and w together
  • 5.6 Flags h and b
  • 5.7 Flag h and sinking to /dev/null
  • 5.8 Flag i and pipe flag f
  • 5.9 Flag r
  • 5.10 Flag c's background
  • 5.11 Flag c before nested block forks a child
  • 5.12 Flag c and understanding possible forking penalty
  • 5.13 Flags before nested block
  • 5.14 Flags aAeE tutorial
6.0 Matching and regexps (regular expressions)
  • 6.1 Philosophy of abstraction in regexps
  • 6.2 Matches are not case-sensitive
  • 6.3 Procmail uses multi line matches
  • 6.4 Headers are unfolded before matching
  • 6.5 Improving Space-Tab syndrome
  • 6.6 Handling exclamation character
  • 6.7 Rules for generating a character class
  • 6.8 Matching space at the end of condition
  • 6.9 Beware leading backslash
  • 6.10 Correct use of TO Macro
  • 6.11 Procmail's regexp engine
  • 6.12 Procmail and egrep differences
  • 6.13 Understanding procmail's minimal matching (stingy vs. greedy)
  • 6.14 Explaining \/ and ()\/
  • 6.15 Explaining ^^ and ^
  • 6.16 ANDing traditionally
  • 6.17 ORing traditionally
  • 6.18 ORing and score recipe
  • 6.19 ORing by using De Morgan rules
7.0 Variables
  • 7.1 Setting and unsetting variables
  • 7.2 Variable initialization and sh syntax
  • 7.3 Testing variables
  • 7.4 What does $\VAR mean?
  • 7.5 Common pitfalls when using variables
  • 7.6 Quoting: Using single or double quotes
  • 7.7 Quoting: Passing values to an external program
  • 7.8 Passing values from an external program
  • 7.9 Incrementing a variable by a value N
  • 7.10 Comparing values
  • 7.11 Strings: How many characters are there in a given string?
  • 7.12 Strings: How to strip trailing newline.
  • 7.13 Strings: deriving the last N characters of a string.
  • 7.14 Strings: Getting partial matches from a string.
  • 7.15 Strings: Procmail string manipulation example
  • 7.16 How to raise a flag if the message was filed
  • 7.17 Dollar sign in condition lines.
  • 7.18 Finding mysterious foo variable
  • 7.19 Storing code to variable
  • 7.20 Getting headers into a variable.
  • 7.21 Converting value to lowercase
8.0 Suggestions and miscellaneous
  • 8.1 Speeding up procmail
  • 8.2 See the procmail installation's examples
  • 8.3 Printing statistics of your incoming mail
  • 8.4 Storing UBE mailboxes outside of quota
  • 8.5 Using first 5-30 lines from the message
  • 8.6 Using cat or echo in scripts?
  • 8.7 How to run an extra shell command as a side effect?
  • 8.8 Forcing "ok" return status from shell script
  • 8.9 Make your own .procmailrc available to others
  • 8.10 Using dates efficiently
  • 8.11 Keep simple header log
  • 8.12 Gzipping messages
  • 8.13 Emergency stop for your .procmailrc
9.0 Scoring
  • 9.1 Using scores by an example
  • 9.2 Brief Score tutorial
  • 9.3 Score's scope
  • 9.4 Counting length of a string
  • 9.5 Counting lines in a message (Adding Lines: header)
  • 9.6 Determining if body is longer than header
  • 9.7 Matching last Received header
  • 9.8 Testing value range with scoring (bogofilter)
  • 9.9 How to add Content-Length header
  • 9.10 Testing message size or number of lines
  • 9.11 Counting commas with recursive includerc
10.0 Formail usage
  • 10.1 Fetching fields with formail -x
  • 10.2 Always use formail's -rt switch
  • 10.3 Using -rt and rewriting the From address
  • 10.4 Formail -rt and Resent-From header
  • 10.5 Quoting the message
  • 10.6 Without quoting the message
  • 10.7 How to include headers and body to the reply message
  • 10.8 Adding text to the beginning of message
  • 10.9 Adding text to the end of message
  • 10.10 Adding text before quoted message
  • 10.11 Adding extra headers from file
  • 10.12 Splitting digest
  • 10.13 Mailbox: Splitting to individual files
  • 10.14 Mailbox: Extracting all From addresses from mailbox
  • 10.15 Mailbox: Applying procmail recipe on whole mailbox
  • 10.16 Mailbox: run series of commands for each mail (split mailbox)
  • 10.17 Option -D and cache
  • 10.18 Option -D and message-id in the body
  • 10.19 Reducing formail calls (conditionally adding fields)
  • 10.20 Formail -A -a options
  • 10.21 Formail -e -s options
11.0 Saving mailing list messages
  • 11.1 Using subroutine pm-jalist.rc to detect mailing lists
  • 11.2 Using plus addressing [email protected]
  • 11.3 Using RFC comment trick for additional information
  • 11.4 Simple mailing list handling
  • 11.5 Archiving according to TO
  • 11.6 Using Return-Path to detect mailing lists
12.0 Procmail, MIME and HTML
  • 12.1 Mime Bibliography
  • 12.2 Mime notes
  • 12.3 Software to deal with mime or HTML
  • 12.4 Mime content type application/ms-tnef
  • 12.5 Trapping HTML mime messages
  • 12.6 Complaining about HTML messages
  • 12.7 Converting HTML body to plain text
  • 12.8 Getting rid of unwanted mime attachments (HTML, vcard)
  • 12.9 Sending contents of a HTML page in plain text to someone
13.0 Simple recipe examples
  • 13.1 Saving: MH folders -- numbered messages
  • 13.2 Saving: to monthly folders
  • 13.3 Modifying: Filtering basics
  • 13.4 Modifying: Squeezing empty lines around message body
  • 13.5 Modifying: shuffling headers always to same order
  • 13.6 Service: Auto answerer to empty messages
  • 13.7 Service: Ping responder
  • 13.8 Service: simple vacation with procmail
  • 13.9 Service: vacation code example
  • 13.10 Service: Auto-forwarding
  • 13.11 Service: forward only specific messages
  • 13.12 Service: Making digests
  • 13.13 Kill: killing advertisement headers and footers
  • 13.14 Kill: simple kill file recipe with procmail
  • 13.15 Kill: duplicate messages
  • 13.16 Kill: spam filter with simple recipes
  • 13.17 Kill: (un)subscribe messages
  • 13.18 Time: Once a day cron-like job
  • 13.19 Time: Running a recipe at a given time
  • 13.20 Time: Triggering mail and using cron
  • 13.21 Decoding: Uudecode
  • 13.22 Decoding: MIME
  • 13.23 How to send commands in the message's body
  • 13.24 Matching two words on a line, but not one
  • 13.25 How to define personal XX macros?
  • 13.26 How to change subject by body match
  • 13.27 How to change Subject according to some other header
  • 13.28 How to call program with parameters
14.0 Miscellaneous recipes
  • 14.1 Matching valid Message-Id header
  • 14.2 Sending two files in a message
  • 14.3 Excessive quoting of message
  • 14.4 Sending message to pager in chunks
  • 14.5 Playing particular sound when message arrives
  • 14.6 Combining multiple Original-Cc and Original-To headers
  • 14.7 Forwarding sensitive messages in encrypted format
15.0 Procmail and PGP
  • 15.1 Decrypt pgp messages automatically
  • 15.2 Getkeys from key server
  • 15.3 Auto grab incoming pgp keys
16.0 Includerc usage
  • 16.1 Using: multiple rc files
  • 16.2 Using: call rc file conditionally
  • 16.3 Using: autoloading an rc file
  • 16.4 Making: naming of the rc file
  • 16.5 Making: Using name space when saving procmail variables
  • 16.6 Making: Public and private variables in rc file
  • 16.7 The rules of thumb for constructing general purpose rc file
  • 16.8 An includerc skeleton
17.0 Mailing list server
18.0 Common troubles
  • 18.1 Procmail modes: normal, delivery, and mail filter.
  • 18.2 Procmail as sendmail Mlocal mail filtering device
  • 18.3 Procmail doesn't pass 8bit characters
  • 18.4 My ISP isn't very interested in installing procmail
  • 18.5 My ISP has systemwide procmailrc; is this a good idea?
  • 18.6 Procmail changes mailbox and directory permissions
  • 18.7 Changing mbox permission during compilation to 660
  • 18.8 The .forward file must be real file
  • 18.9 Using .forward if procmail already is LDA
  • 18.10 Mail should be put in the mailqueue if write fails
  • 18.11 Qmail: how to make it work with procmail
  • 18.12 Qmail: Procmail looks file from /var/spool/mail only
  • 18.13 Qmail: patch to procmail 3.11pre7 to work with Maildirs
  • 18.14 AFS: How to use Procmail when HOME is in AFS cell
  • 18.15 Help, some idiot sent my address to 30 mailing lists
  • 18.16 Help, Procmail beeps and prints to my console
  • 18.17 Help, procmail dumps mail to console
  • 18.18 Help, corrupted From_ line in mailbox
  • 18.19 Directing user's mail to HOME instead of /var/spool/
  • 18.20 NFS mounting /var/mail is a good way to get bad performance
  • 18.21 I can't see the sendmail's response in LOGFILE
  • 18.22 Compiling procmail and choosing locking scheme
  • 18.23 Forwarding lot of mail causes heavy load
  • 18.24 What happens to mail if MDA Procmail fails
  • 18.25 Procmail reads entire 90Mb message into memory
  • 18.26 Help, procmail uses occasionally huge chunk of memory
  • 18.27 Procmail signaled out of memory in my verbose log
  • 18.28 Variables DEFAULT and ORGMAIL
  • 18.29 When DEFAULT cannot be mailed to
  • 18.30 Variable DROPPRIVS
  • 18.31 Variable HOME
  • 18.32 Variable HOST
  • 18.33 Variable LINEBUF
  • 18.34 Variable LOG and LOGFILE
  • 18.35 Variable TRAP
  • 18.36 Variable UMASK
  • 18.37 UMASK and permissions
  • 18.38 Performance difference between back tick and "|" recipe
  • 18.39 Procmail's temporary file names while writing file out
  • 18.40 Parameter $@
  • 18.41 Procmail variables are null terminated (detecting null string)
  • 18.42 FROM_DAEMON TO and TO_ and case-sensitiveness
  • 18.43 TO_ macro deciphered
  • 18.44 TO_ macro and RFC 822
  • 18.45 FROM_DAEMON deciphered
19.0 Technical matters
  • 19.1 List of exit codes
  • 19.2 List of precedence codes
  • 19.3 Sendmail and -t
  • 19.4 RFC822 Reply-To and formail problem with multiple recipients
  • 19.5 Procmail and IMAP server
  • 19.6 Machine which processes mail
  • 19.7 Compiling procmail and MAILSPOOLHOME
20.0 Procmail software for Emacs
  • 20.1 What is Emacs
  • 20.2 Emacs procmail-mode and Procmail code checking (Lint)
  • 20.3 Why use procmail with Gnus
  • 20.4 Setting up Gnus for procmail - Basics
  • 20.5 Gnus for procmail - More about it
  • 20.6 Emacs and Gnus -- Fiddling with spool files
  • 20.7 Gnus article snippets
21.0 RFC, Request for comments
  • 21.1 RFCs and their jurisdiction (munged Addresses)
  • 21.2 Comments about addresses munging
  • 21.3 RFC and valid mail address characters
  • 21.4 RFC and login-name@fdqn
  • 21.5 RFCs and messages signature
  • 21.6 RFC and using MIME in Usenet newsgroups
  • 21.7 Some RFC Pointers
22.0 Introduction to E-mail Headers
  • 22.1 To find out more about mail (Resources)
  • 22.2 Lecture by Alan Stebbens
  • 22.3 Applied to received messages
  • 22.4 Bcc lecture by Alan Stebbens
  • 22.5 Bcc lecture by Philip Guenther
23.0 Message headers
  • 23.1 What is correct From address syntax
  • 23.2 What's that X-UIDL header?
  • 23.3 What is that first From_ header?
  • 23.4 Message-Id header
  • 23.5 Received header
  • 23.6 Return-Path
  • 23.7 Errors-To
  • 23.8 X-Subscription-Info
  • 23.9 Reply-To header
  • 23.10 Mail-Copies-To header
  • 23.11 Mail-Followup-To and Reply-To-Personal headers
  • 23.12 Content-Length header and From_ specification
  • 23.13 Moral about CC copies in Usenet

1.0 Document id

1.1 General

$Id: pm-tips.txt,v 2.28 2004/10/06 13:55:39 jaalto Exp $
$URL: http://pm-doc.sourceforge.net/ $
$UrlLinksLastChecked: 2002-07-11 $

This is a Procmail Tips page: a collection of procmail recipes, instructions, howtos. The document also contains URL pointers to the procmail mailing list and sites that fight against Internet UBE. Procmail is powerful mail handling tool and a lot of space here has been devoted to discuss about UBE (aka Spam) and its essence. You will also find many other interesting subjects that discuss about internet mail in general: mail headers, MIME and RFCs. Another part of this document is dedicated to Emacs and Emacs plug-in package Gnus.el, simply because Emacs is the best tool you can use to deal with your mail and news reading. Nowadays Emacs is also available in Windows platform as well. This is not to say that existing Unix elm(1), mutt(1) or pine(1), slrn(1) mail/news programs are bad, they are just limited in power compared to Emacs and usually tied to Unix platform. Finally, to your blessing or curse (smile) the author happens to know Emacs quite well. The tips are compiled from the procmail discussion list, from comp.mail.misc and from the author's own experiences with procmail.

This document does not intend to teach you the basics of procmail, instead you have to be familiar with the procmail man pages already. Procmail manual pages exists primarily on Unix/Linux platform, If You're using Windows operating system, see Cygwin at http://www.cygwin.com/

You may want to read Nancy's and Era's procmail FAQ pages before this page. Ther are wealth of useful procmail links and pointers to Unix programs that deal with mail. If you find errors or things to improve in this document, please send mail to this document's Maintainer.

If any mentioned URL is not alive, you may still be able to successfully find it using the WWW search such as http://www.google.com/

1.2 What is Procmail?

[FAQ] Procmail is a mail processing utility, which can help you filter your mail, sort incoming mail according to sender, Subject line, length of message, keywords in the message, etc, implement an ftp-by-mail server, and much more. Procmail is also a complete drop-in replacement for your MDA. (If this doesn't mean anything to you, you may not want to know.) Procmail runs under Unix. See Infinite Ink's Mail Filtering and Robots page for information about related utilities for various other platforms, and competing Unix programs, too (there aren't that many of either).

1.3 Abbreviations and thanks

People and documents, abbreviations referred to, tokens used, are in no particular order.

[stephen] Stephen R. van den Berg, Author of Procmail Last heard from stephen 1997-08 in procmail mailing list by using address [email protected]. Later 1998 due to his regular work activities and lack of time he nominated Philip Guenther to the head of Procmail development.

[aaron] Aaron Schrab aaron+procmail A T schrab com
[alan] Alan K. Stebbens alan.stebbens A T openwave com
[dan] Daniel Smith J.Daniel.Smith A T WriteMe dt com
[david] David W. Tamkin dattier A T panix com
[ed] Edward J. Sabol sabol A T alderaan gsfc nasa gov
[elijah] Eli the Bearded process A T qz little-neck ny us
[hal] Hal Wine hal A T dtor com
[jari] Jari Aalto jari aalto A T poboxes dt com
[philip] Philip Guenther guenther A T gac edu
[richard] Rich