


          ()                                                             ()














                             TThhee EEllmm FFiilltteerr SSyysstteemm GGuuiiddee




                      _W_h_a_t _t_h_e _f_i_l_t_e_r _p_r_o_g_r_a_m _i_s_, _w_h_a_t _i_t _d_o_e_s_,
                                  _a_n_d _h_o_w _t_o _u_s_e _i_t



                                 The Elm Mail System
                                    (Version 2.4)


                           Syd Weinstein, Elm Coordinator
                               Datacomp Systems, Inc.
                                   3837 Byron Road
                          Huntingdon Valley, PA 19006-2320

                          email: elm@DSI.COM  or  dsinc!elm
















                       ((CC)) Copyright 1986,1987 by Dave Taylor
                ((CC)) Copyright 1988-1992 by The USENET Community Trust







                                                                          1





          ()                                                             ()







                             TThhee EEllmm FFiilltteerr SSyysstteemm GGuuiiddee

                         (The Elm Mail System, Version 2.4)

                                   October 1, 1992


                                    Syd Weinstein
                               Datacomp Systems, Inc.
                                   3837 Byron Road
                          Huntingdon Valley, PA 19006-2320

                          email: elm@DSI.COM  or  dsinc!elm


                                    Derived from
                         "The Elm Mail System, Version 2.0"
                                         by
                                     Dave Taylor
                                  Intuitive Systems
                              Mountain View, California
                    email: taylor@intuitive.com  or  limbo!taylor



          One  of the greatest problems with the burgeoning electronic mail
          explosion is that I tend to get lots of mail that  I  don't  care
          about.   Amusingly,  perhaps, I have the equivalent of electronic
          junk mail.  Not amusing, however,  is  the  fact  that  this  can
          rapidly  accumulate  and  end  up taking over my mailbox.  At the
          same time I often get mail that,  while  it  is  interesting  and
          important,  can  easily  be  filed to be read later, without ever
          actually having to cluttering up my incoming mailbox.


          This, then, is what _f_i_l_t_e_r does!  The _f_i_l_t_e_r program  allows  you
          to  define  a  set  of rules by which all incoming mail should be
          screened, and a subsequent set of actions  to  perform  based  on
          whether  the  conditions  were  met  or not.  _F_i_l_t_e_r also has the
          ability to mail a summary of what actions  it  performed  on  the
          incoming mail as often as you'd like.

          The language for writing _f_i_l_t_e_r rules is pretty simple, actually.
          The fundamental structure is;
               if  (_c_o_n_d_i_t_i_o_n)  then  _a_c_t_i_o_n
          Where _c_o_n_d_i_t_i_o_n is constructed by an arbitrary number of individ-
          ual  conditions  of  the  form  ``_f_i_e_l_d   _r_e_l_a_t_i_o_n  _v_a_l_u_e''.  (an
          optional further type of rule is of the  form  ``always  _a_c_t_i_o_n''
          but  should  only  be  used  as the last rule in the ruleset, for



                                                                          1





          ()                                                             ()


          obvious reasons).  The _f_i_e_l_d value can be;
               subject
               sender
               from
               to
               lines
               contains
          where, if ``lines'' is chosen, the _r_e_l_a_t_i_o_n can  be  any  of  the
          standard  relationships (`>', `<', `>=', `<=', `!=' and `=').  If
          another action is chosen, ``contains'' can be used as  the  rela-
          tion,  ``='',  or,  if  you'd like, you can skip the relationship
          entirely (e.g. `subject "joe"').  The _v_a_l_u_e is any quoted  string
          that is to be matched against or number if ``lines'' is the field
          being considered.

          The relation ``matches'' (or ``~'') takes an egrep  like  regular
          expression  as the _v_a_l_u_e. The regular expression must be enclosed
          in pairs of ``/''.  If you need a ``/'' in your expression  write
          it as ``\/''.

          Individual  conditions  are  joined  together  by  using the word
          ``and'', and the logic of a condition can  be  flipped  by  using
          ``not''  as the first word (e.g. `not subject "joe"').  We'll see
          more examples of this later.

          Note that the ``or'' logical conjunction isn't a  valid  part  of
          the _f_i_l_t_e_r conditional statement.

          Finally, <_a_c_t_i_o_n> can be any of;
               delete
               save   _f_o_l_d_e_r_n_a_m_e
               savecopy  _f_o_l_d_e_r_n_a_m_e
               forward  _a_d_d_r_e_s_s
               forwardc  _a_d_d_r_e_s_s
               execute  _c_o_m_m_a_n_d
               executec  _c_o_m_m_a_n_d
               leave
          where  they  result  in the actions;  ddeelleettee deletes the message;
          ssaavvee saves a copy of the message  in  the  specified  foldername;
          ssaavveeccooppyy  does  the  same  as  save, but also puts a copy in your
          mailbox; ffoorrwwaarrdd sends the message to the specified address; ffoorr--
          wwaarrddcc  sends  the  message  to the specified address and leaves a
          copy in your mailbox; eexxeeccuuttee feeds the message to the  specified
          command (or complex sequence of commands) as standard input; eexxee--
          ccuutteecc is like eexxeeccuuttee but also leaves a copy in your mailbox; and
          lleeaavvee leaves the message in your mailbox.

          NOTE:  The  execute  command assumes that you are doing something
          useful with incoming mail.  The command  does  _n_o_t  automatically
          append the message to your mail spool.  Use executec for that.  A
          command such as:

           if (from = "dave") then execute "echo mail from dave > /dev/tty"




                                                                          2





          ()                                                             ()


          will result in _l_o_s_i_n_g your mail.  A more suitable use for execute
          would be:

           if (from = "boss") then execute "vacation pat"

          Foldernames can contain any of a number of macros, too, as  we'll
          see  in  the example ruleset below.  The macros available for the
          string fields are;
          MMaaccrroo   MMeeaanniinngg
          %%dd      ddaayy ooff tthhee mmoonntthh
          %%DD      ddaayy ooff tthhee wweeeekk ((00--66))
          %%hh      hhoouurr ooff tthhee ddaayy ((00--2233))
          %%mm      mmoonntthh ooff tthhee yyeeaarr ((11--1122))
          %%rr      rreettuurrnn aaddddrreessss ooff mmeessssaaggee
          %%ss      ssuubbjjeecctt ooff oorriiggiinnaall mmeessssaaggee
          %%SS      ````RRee:: _s_u_b_j_e_c_t _o_f _o_r_i_g_i_n_a_l _m_e_s_s_a_g_e''''
          %%tt      ccuurrrreenntt hhoouurr aanndd mmiinnuuttee iinn HHHH::MMMM ffoorrmmaatt
          %%yy      yyeeaarr ((llaasstt ttwwoo ddiiggiittss))
          %%&&      tthhee ssttrriinngg tthhaatt mmaattcchheedd tthhee llaasstt rreegguullaarr eexxpprreessssiioonn
          %%11--%%99   tthhee ii::tthh ssuubbeexxpprreessssiioonn iinn tthhee llaasstt rreegguullaarr eexxpprreessssiioonn tthhaatt mmaattcchheedd

          If a message has a subject of _T_h_i_s _i_s _a _t_e_s_t, and the rule is

          if (subject matches /([a-z]+) a ([a-z]+)/) then "echo %& %1 %2"

          then %& expands to _i_s _a _t_e_s_t, %1 expands to _i_s and %2 expands  to
          _t_e_s_t.

          Foldernames  may contain a leading '~/' which will be expanded to
          be your home directory.

          The rules file can also contain comments (any line starting  with
          a  `#')  and blank lines.  If you need a `%' character as part of
          an argument (such as an address), use `\%'.

          Unless you specify an alternate file via  the  `-f'  option,  the
          rules  file  needs to reside in your .elm directory off your home
          directory and be called _._e_l_m_/_f_i_l_t_e_r_-_r_u_l_e_s.  Here's an example:

           ## $$HHOOMMEE//..eellmm//ffiilltteerr--rruulleess
           ##
           ## FFiilltteerr rruulleess ffoorr tthhee EEllmm FFiilltteerr pprrooggrraamm.. DDoonn''tt cchhaannggee wwiitthhoouutt ssoommee
           ## sseerriioouuss tthhoouugghhtt.. ((rreemmeemmbbeerr -- oorrddeerr ccoouunnttss))
           ##
           ## ((ffoorr DDaavvee TTaayylloorr))
           ## rruullee 11
           iiff ((ffrroomm ccoonnttaaiinnss ""!!uuuuccpp"")) tthheenn ddeelleettee
           ## rruullee 22
           ttoo ""ppoossttmmaasstteerr"" ?? ssaavvee ""//ttmmpp//ppoossttmmaasstteerr--mmaaiill..%%dd""
           ## rruullee 33
           iiff ((ttoo ""ccuullttuurree"" aanndd lliinneess >> 2200)) ?? ssaavvee ""//uusseerrss//ttaayylloorr//MMaaiill//ccuullttuurree""
           ## rruullee 44
           ssuubbjjeecctt == ""ffiilltteerr tteesstt"" ?? ffoorrwwaarrdd ""hhppllddaatt!!tteesstt""
           ## rruullee 55



                                                                          3





          ()                                                             ()


           iiff [[ ssuubbjjeecctt == ""eellmm"" ]] ssaavveeccooppyy ""//uusseerrss//ttaayylloorr//MMaaiill//eellmm--iinnccoommiinngg""
           ## rruullee 66
           ssuubbjjeecctt == ""ddiissppllaayy--ttoo--ccoonnssoollee"" ?? eexxeeccuuttee ""ccaatt -- >> //ddeevv//ccoonnssoollee""
          (notice the loose syntax -- there are lots of valid ways to spec-
          ify a rule in the _f_i_l_t_e_r program!!)

          To translate these into English;

            1. All messages from uucp should be summarily deleted.

            2.  All  mail  to postmaster should be saved in a folder (file)
               called /tmp/postmaster-mail._n_u_m_e_r_i_c_-_d_a_y_-_o_f_-_t_h_e_-_w_e_e_k

            3. All mail addressed to  `culture'  with  at  least  20  lines
               should    be    automatically   appended   to   the   folder
               /users/taylor/Mail/culture.

            4. All messages that contain the subject `filter  test'  should
               be  forwarded  to  me, but via the address `hpldat!test' (to
               force a non-user forward)

            5. All messages with a subject that  contains  the  word  `elm'
               should  be  saved  in  the  folder ``/users/taylor/Mail/elm-
               incoming'' and also dropped into my mailbox.

            6. Any message with the subject ``display-to-console'' will  be
               immediately written to the console.

          Notice that the _o_r_d_e_r of the rules is very important.  If we, for
          example, were to get a message from `uucp' that had  the  subject
          `filter  test',  the _f_i_l_t_e_r program would match rule 1 and delete
          the message.  It would never be forwarded to  `hpldat!test'.   It
          is  for  this  reason  that  great  care should be taken with the
          ordering of the rules.

          The _f_i_l_t_e_r program has a convenient way of check  out  the  rules
          you have written.  Simply invoke it with the --rr (rrules) flag;
               % ffiilltteerr --rr
               RRuullee 11:: iiff ((ffrroomm == ""!!uuuuccpp"")) tthheenn
                       DDeelleettee
               RRuullee 22:: iiff ((ttoo == ""ppoossttmmaasstteerr"")) tthheenn
                       SSaavvee  //ttmmpp//ppoossttmmaasstteerr--mmaaiill..<<ddaayy--ooff--wweeeekk>>
               RRuullee 33:: iiff ((ttoo == ""ccuullttuurree"" aanndd lliinneess >> 2200)) tthheenn
                       SSaavvee  //uusseerrss//ttaayylloorr//MMaaiill//ccuullttuurree
               RRuullee 44:: iiff ((ssuubbjjeecctt == ""ffiilltteerr tteesstt"")) tthheenn
                       FFoorrwwaarrdd  hhppllddaatt!!tteesstt
               RRuullee 55:: iiff ((ssuubbjjeecctt==""eellmm"")) tthheenn
                       CCooppyy  aanndd  SSaavvee  //uusseerrss//ttaayylloorr//MMaaiill//eellmm--iinnccoommiinngg
               RRuullee 66:: iiff ((ssuubbjjeecctt==""ddiissppllaayy--ttoo--ccoonnssoollee"")) tthheenn
                       EExxeeccuuttee ""ccaatt -- >> //ddeevv//ccoonnssoollee""

          There  are  a  few  things  to notice -- first off, these are the
          parsed and rebuilt rules, so we can see that they are  all  in  a
          consistent  format.  Also, notice on the filename for rule 2 that



                                                                          4





          ()                                                             ()


          the program has correctly expanded the ``%d'' macro to be the day
          of the week.

          It  is  hhiigghhllyy  recommended  that  you  always check your ruleset
          before actually letting the program use it!

          Now the bad news.  If you aren't running _s_e_n_d_m_a_i_l or  some  other
          Mail Transport Agent that supports delivery to a program you can-
          not use this program as currently  written.   Why?   Because  the
          _f_i_l_t_e_r program expects to be put in your _._f_o_r_w_a_r_d file (or equiv-
          alent concept for MTAs other than sendmail) that causes the  mes-
          sages  to  be delivered to the filter program.  That is something
          that only _s_e_n_d_m_a_i_l and other smart MTAs look at!

          The format for the entry in the _._f_o_r_w_a_r_d file  (located  in  your
          home directory) is simply;
               "| /usr/local/bin/filter"
          Allright,  it  isn't  quite  _t_h_a_t  simple!   Since _f_i_l_t_e_r will be
          invoked by processes that don't know where you are logged in, you
          need  to  have  some way to trap the error messages.  For ease of
          use, it was decided to have all the messages written to the  file
          specified  by `-o' (or _s_t_d_e_r_r) which means that you have two main
          choices for the actual entry.  Either;
               "| /usr/local/bin/filter -o /dev/console"
          which will log all errors on the system console  (each  error  is
          prefixed with ``filter (_u_s_e_r_n_a_m_e)'' to distinguish it), or;
               "| /usr/local/bin/filter -o /tmp/joe.filter_errors"
          If you want to have a copy saved to a file.  Note that the quotes
          are a required part of the line.  A possible strategy would be to
          have the errors written to a file and to then have a few lines in
          your _._l_o_g_i_n script like:
               iiff (( --ff //ttmmpp//jjooee..ffiilltteerr__eerrrroorrss)) tthheenn
                  eecchhoo  ""  ""
                  eecchhoo ""FFiilltteerr pprrooggrraamm eerrrroorrss;;""
                  ccaatt //ttmmpp//jjooee..ffiilltteerr__eerrrroorrss
                  eecchhoo ""  ""
               eennddiiff
          You can also use the --vv flag in combination  with  the  above  to
          have a more verbose log file saved by having your _._f_o_r_w_a_r_d file;
          "| /usr/local/bin/filter -vo /tmp/joe.filter_errors"
          Suffice to say, you can get pretty tricky with all this!!

          The  _F_i_l_t_e_r  program  can  keep  a  log of all actions performed,
          including what rules it matched against, in your  .elm  directory
          in a file called _._e_l_m_/_f_i_l_t_e_r_l_o_g.  You can either directly operate
          on this file, or, much more recommended, you can one of  the  two
          summarize flags to the program and let _i_t do the work for you!

          The difference between the two is best demonstrated by example:
          % ffiilltteerr --ss
                               SSuummmmaarryy ooff FFiilltteerr AAccttiivviittyy
                               ----------------------------------------------------
               AA ttoottaall ooff 441188 mmeessssaaggeess wweerree ffiilltteerreedd::
               TThhee ddeeffaauulltt rruullee ooff ppuuttttiinngg mmaaiill iinnttoo yyoouurr mmaaiillbbooxx



                                                                          5





          ()                                                             ()


                        aapppplliieedd 336644 ttiimmeess ((8877%%))
               RRuullee ##11:: ((ddeelleettee mmeessssaaggee))
                        aapppplliieedd 11 ttiimmee ((00%%))
               RRuullee ##22:: ((ssaavvee iinn ""//uusseerrss//ttaayylloorr//FFiilltteerreedd--MMaaiill//nneettnneewwss..1122""))
                        aapppplliieedd 88 ttiimmeess ((22%%))
               RRuullee ##33:: ((ssaavvee iinn ""//uusseerrss//ttaayylloorr//FFiilltteerreedd--MMaaiill//ppoossttmmaasstteerr..1122""))
                        aapppplliieedd 1144 ttiimmeess ((33%%))
               RRuullee ##55:: ((ssaavvee iinn ""//uusseerrss//ttaayylloorr//FFiilltteerreedd--MMaaiill//rriisskkss..1122""))
                        aapppplliieedd 33 ttiimmeess ((11%%))
               RRuullee ##66:: ((ssaavvee iinn ""//uusseerrss//ttaayylloorr//FFiilltteerreedd--MMaaiill//rraayyss..1122""))
                        aapppplliieedd 2288 ttiimmeess ((77%%))
          versus:
               % ffiilltteerr --SS
               _t_h_e _o_u_t_p_u_t _a_s _l_i_s_t_e_d _a_b_o_v_e_, _f_o_l_l_o_w_e_d _b_y_:
               EExxpplliicciitt lloogg ooff eeaacchh aaccttiioonn;;
               MMaaiill ffrroomm ttaayylloorr aabboouutt FFiilltteerr SSuummmmaarryy
                        PPUUTT iinn mmaaiillbbooxx:: tthhee ddeeffaauulltt aaccttiioonn
               MMaaiill ffrroomm nneewwss@@hhppllaabbsszz..hhppll..hhpp..ccoomm aabboouutt NNeewwssggrroouupp ccoommpp..eeddiittoorrss ccrreeaatteedd
                        PPUUTT iinn mmaaiillbbooxx:: tthhee ddeeffaauulltt aaccttiioonn
               MMaaiill ffrroomm rroooott aabboouutt LLoogg ffiillee:: cclleeaannuupplloogg
                        PPUUTT iinn mmaaiillbbooxx:: tthhee ddeeffaauulltt aaccttiioonn
               [etc etc]
          To  actually use either of the summarizing options, there are two
          ways that are recommended;

          The preferred way is to have a line in either your  _c_r_o_n_t_a_b  (ask
          your  administrator  for  help with this) that invokes the _f_i_l_t_e_r
          program as often as you desire with the --ss flag.  For example,  I
          have a summary mailed to me every morning at 8:00 am:
             00 88 ** ** ** ""//uussrr//llooccaall//bbiinn//ffiilltteerr --ss || eellmm --ss ''FFiilltteerr SSuummmmaarryy'' ttaayylloorr""

          An  alternative  is  to have your _._l_o_g_i_n execute the command each
          time.


          Note that if you want to have your log  files  cleared  out  each
          time  the  summary  is generated you'll need to use the '-c' flag
          too.  Also, if you want to keep a long list of actions  performed
          you can do this by saving it as you display it.  A way to do this
          would be, if you were to  have  the  invocation  in  your  _._l_o_g_i_n
          script, to use:
               eecchhoo ""FFiilltteerr LLoogg;;""
               ffiilltteerr --cc --ss || tteeee --aa PPEERRMM..ffiilltteerr..lloogg
          which  would  append  a  copy  of  all  the  output  to  the file
          `PERM.filter.log' and would avoid you having to read  larger  and
          larger summaries of what the program had done.


          If  you  prefer not to have _f_i_l_t_e_r generate log files, use the --qq
          flag.  Messages that are written to _s_t_d_e_r_r or  the  --oo  file  are







                                                                          6





          ()                                                             ()


          still generated; only message to the log files are quashed.

          With  the _r_e_a_d_m_s_g command available, it is quite easy to test the
          rules you've written to see if they'll do what you desire.

          For example, we can use the --nn flag to _f_i_l_t_e_r, which means `don't
          actually do this, just tell me what rule you matched, if any, and
          what action you would have performed' (you can see why  a  single
          letter  flag is easier to type in!!), and feed it each message in
          our mailbox by using a command like;
               % sseett mmeessssaaggee==11
               % sseett ttoottaall__mmeessssaaggeess==``mmeessssaaggeess``
               % wwhhiillee  ((11))
               > iiff (($$mmeessssaaggee >> $$ttoottaall__mmeessssaaggeess)) eexxiitt
               > eecchhoo pprroocceessssiinngg mmeessssaaggee $$mmeessssaaggee
               > rreeaaddmmssgg --hh $$mmeessssaaggee || ffiilltteerr --nn
               > eecchhoo "" ""
               > @@ mmeessssaaggeess++++
               > eenndd
          which will then hand each of the messages in your mailbox to  the
          _f_i_l_t_e_r program and display what action would have been taken with
          that message and why.

          For example, if we do this for a few interesting messages  in  my
          mailbox, we'd end up with output like:
               MMaaiill ffrroomm ttaayylloorr aabboouutt ffiilltteerr tteesstt
                    FFOORRWWAARRDDEEDD ttoo hhppllddaatt!!ttaayylloorr bbyy rruullee;;
                      ssuubbjjeecctt==""ffiilltteerr tteesstt""  ?? ffoorrwwaarrdd ""hhppllddaatt!!tteesstt""
               MMaaiill ffrroomm bbrraaddlleeyy%%hhppllkkaabb@@hhppllaabbsscc aabboouutt RRee:: AAII--EEDD mmaaiilliinngg aaddddrreessss ffoorr HHPP
                    PPUUTT iinn mmaaiillbbooxx:: tthhee ddeeffaauulltt aaccttiioonn
               MMaaiill ffrroomm ttaayylloorr aabboouutt ddiissppllaayy--ttoo--ccoonnssoollee
                    EEXXEECCUUTTEEDD ""ccaatt -- >> //ddeevv//ccoonnssoollee""
          (sharp  users will notice that this is exactly the same format as
          the longer summary listing)

          When a message is forwarded to another user by the  _a_c_t_i_o_n  being
          specified  as  ``forward _a_d_d_r_e_s_s'', then the program can generate
          one of two styles of message.  If the message  is  to  you,  then
          it'll  simply  add  it to your mailbox in such a way as to ensure
          that the return address is that of the person who sent  the  mes-
          sage and so on.

          If not, then the message is enclosed in a message of the form:
             FFrroomm ttaayylloorr TThhuu OOcctt  22 1155::0077::0044 11998866
             DDaattee:: TThhuu,, 22 OOcctt 8866 1155::0066::5588 ppddtt
             SSuubbjjeecctt:: ""ffiilltteerr tteesstt""
             FFrroomm:: TThhee ffiilltteerr ooff ttaayylloorr@@hhppllddaatt <<ttaayylloorr>>
             TToo:: hhppllddaatt!!ttaayylloorr
             XX--FFiilltteerreedd--BByy:: ffiilltteerr,, vveerrssiioonn 11..44
             ---- BBeeggiinn ffiilltteerreedd mmeessssaaggee ----

                FFrroomm ttaayylloorr TThhuu OOcctt  22 1155::0066::4411 11998866
                DDaattee:: TThhuu,, 22 OOcctt 8866 1155::0066::3333 ppddtt
                FFrroomm:: DDaavvee TTaayylloorr <<ttaayylloorr>>



                                                                          7





          ()                                                             ()


                SSuubbjjeecctt:: ffiilltteerr tteesstt
                JJuusstt aa ssiimmppllee tteesstt..
             ---- EEnndd ooff ffiilltteerreedd mmeessssaaggee ----
          The  subject  of the actual message is the same as the subject of
          the message being forwarded, but in quotes.  The  `From:'   field
          indicates  how  the  message  was  sent, and the `X-Filtered-By:'
          identifies what version of filter is being used.

          While the _f_i_l_t_e_r program as presented herein is obviously a  nice
          addition  to  the  set  of tools available for dealing with elec-
          tronic mail, there are some key features  that  are  missing  and
          will be added in the future based on demand.

          As I see it, the main things missing are;

            1. Perhaps more _a_c_t_i_o_n_s available (but what?)

            2. Certainly the ability to filter based on any field or combi-
               nation of fields.

          Since this is a pretty simple program, there are a few  pitfalls,
          some of which have already been mentioned;

          OOrrddeerr counts in the rules.  Beware!

          MMaattcchhiinngg  is  pretty simple -- make sure your patterns are suffi-
          ciently exclusive before having any destructive rules.

          The log files can grow indefinitely unless you use the --cc  or  --qq
          options or you trim the files by other means.


          Finally, as with the rest of the EEllmm mail system, I welcome feed-
          back and suggestion on how to improve this program!!























                                                                          8


