| # The SRE-Filter For GoServe Web Server
## Users guide

### 1) Introduction.

Welcome to SRE-Filter, version 1.2k (`and counting`). For those of you who've stumbled upon this document:

> SRE-Filter is a HTTP (Web) Server for OS/2. Written in REXX, SRE-FILTER is designed to run under the IBM EWS GoServe program.

Narrowing our focus just a bit ...

> SRE-Filter is designed to be a relatively full-featured package for non-cutting edge, small-to-medium load sites being maintained by non-professionals

In fact, SRE-Filter has many obscure, and useful, features. These include:

- Support for multiple hosts
- Remotely configurable
- Common-log auditing
- Caching support for NCSA HTTPD-style server side includes
- Support for server-side-processing using SRE-Filter add-ons and CGI-BIN scripts
- Bundled accessories include a text-file search engine, and a caching directory display processor.

At this point your are either long gone, or still interested. Assuming the latter, the goal of this document is to outline some of the possibilities of SRE-Filter, and point you to the relevant documentation. Before we do that, it is de-rigeur to ask you to read the disclaimer. You might also want to peruse the basic terminology section of the introduction (we use some not-necessarily-intuitively-obvious terms in this documentation).

> Bored already? You may find the FAQ more informative, the manual more immediately useful, and the list of documentation files a useful index.
> Not bored? You can proceed linearly through this documentation, or you can use the table of contents to jump to the desired section.

> > Please be aware that this guide strikes a balance between brevity and thoroughness. There will be features not mentioned. So repeat after me: the manual, the discussion of initialization paramters, and the list of documentation files are the places to go for more detailed discussion.

### Are you looking for a quick start?

Although this guide attempts to be succinct, there are a fair amount of details to bog down the unwary.
So... if you really don't need all those `paranoid` access controls, and your needs don't require refining, re-refining, and otherwise mangling requests, you probably can skim right through & and over this user guide! That is, if you are reasonably careful during installation, and answer all the questions that the install program asks, you'll be ready to go. I suppose you might want to read the installation section (section 3), but even that's not absolutely necessary (the INSTALL program will cover most of the details).

Let me add one proviso: if you want to use server side includes, you should carefully read the relevant section of the SRE-Filter manual. They can do a lot for you...

### 2) Table of Contents

Introduction | you just read it.
Table of contents | you're reading it now.
Installation | some installation instructions, and some basic terminology.
Multiple Hosts | using SRE-Filter to support multiple hosts and IP aliases
Controlling Access | controlling access to your server's resources
Resolving requests | redirection and interpretation of the client's request
Server side includes | creating dynamic HTML documents
CGI-BIN, Imagemaps, SRE-Filter addons | the wonderful world of server side processing (FORMS and beyond)
Uploads, HEAD method requests, and other methods | some powerful, but more obscure, WWW tricks
Auditing, Pre-filter processing, and Post-filter processing | initializations, anticipations, are record keeping.
Optimization hints | a few suggestions on how to improve SRE-Filter's performance.
The synopsis of features | an outline of what SRE-Filter can do
The disclaimer and acknowlegments | the documentation wouldn't be complete without one

### 3) Installation

All good things must start somewhere, and that means you have to install SRE-Filter before you can use it (or have you already done that).

First, have you installed GoServe? I'll assume that you have.
Second do you have the most recent copy of SRE-Filter? Assuming that you have, you should create (or clear out) a temporary directory, and UNZIP the (possibly several) SRE-Filter distribution files.

1. Look at the READ.ME file that you just unzipped -- it outlines the installation steps (more impatient souls can skip this file with little ill effect).
2. Ready to go? Run the INSTALL program from an OS/2 prompt (be sure you are in the temporary directory, or you might end up running somebody else's installation program!).
3. INSTALL is self explanatory. It does require that you enter the GoServe working directory and the GoServe data directory (see the GoServe documentation, or the SRE-Filter manual, for details).
4. INSTALL also let's you set a few parameters, and will ask if you want to add a SUPERUSER entry to SRE-Filter's username database. We especially recommend adding the SUPERUSER entry, without it you may have a hard time accessing SRE-Filter's configurator!

When you are done installing, it's a good idea to review the notes available at the end of the INSTALL program.

#### Some Basic Terminology

The SRE-Filter documentation uses some not-necessarily-intuitively-obvious terms. The following lists a few of the more important ones. For a more complete list, see the SRE-Filter manual.

URL
  Universal Resource Locator (URL) is a scheme for specifying Internet resources using a single line of printable ASCII characters. A URL should contain a protocol, domain name, port number (optional), the location of the resource, and an (optional) option list (following a ?).
  Examples:
  - `http://your.server.net/dir1/sample.htm`
  - `http://pet.store.com/prices?type=mammals&class=retail`
Selector (also referred to as the request selector )
  The selector is the location of the resource on the server -- it's the part of the URL after the domain name. Examples (assuming the above URLs):
  - `dir1/sample.htm`
  - `prices?type=mammal&class=retail`

  In many cases, the connection between the selector and a server resource it refers to is simple (such as when the selector is a filename relative to the GoServe data directory). In other cases, it's completely virtual (as when the resource consists of `the output of a program that's run using information provided by the client`).

Selector-specific (usually written as SEL-specific )
  SEL-specific refers to SRE-Filter configuration information that is specific to a selector -- such as resource privileges, or a virtual directory.

  > In much the same way that OS/2 `extended attributes` contain additional information about a file, SRE-Filter's SEL-specific information contain additional information about a selector ; or, to be bit more precise, about the server resource identified by the selector (needless to say, this information is only used by SRE-Filter).

Privileges
  SRE-Filter uses the concepts of resource and client privileges to control access to server resources. The basic notion is that access can be regulated by requiring that a request for a server resource come from a client with a client privilege that matches a resource privilege. Note that resources privileges are assigned by the webmaster on a selector specific basis, a given selector can be assigned multiple resource privileges. Simiarly, client privileges are assigned on a user specific basis, a given user may have multiple client privileges.
Realms
  Realms are a commonly used means of subsetting a server's resources. Most browsers, when asked to provide a username and password, will use (if avaialable) the username and password specific to both server and realm; where realm will be provided, on a selector specific basis, to the client's browser by the server software.
  In a sense, realms and resource privileges are parallel concepts; they both are used to identify a subset of a server's resource.

Return to table of contents.

### 4) Multiple Hosts

SRE-Filter is a multi-host (multi-homing) web server -- it can handle requests for several IP hosts simultaneously. This does require some configuration; in particular, you must define a host nickname for each host. Note that each host can be a unique IP address, or can be an IP alias.

The easiest way to add (and remove) hosts is through the Multiple Hosts section of the simple-mode configurator. You may also want to examine the description of the HOSTS parameter in INITFILT.DOC, and the discussion of hosts in the SRE-Filter manual.

Return to table of contents.

### 5) Controlling Access to your Site

SRE-Filter has several levels of access control, including Logon controls, SEL-specific access controls, and file specific access controls. In addition, you can identify resources that are open to the public.

Type of control | Description | Configuration Information?
LOGON controls | At the coarsest level, you can require all clients to LOGON. In most cases, logging on requires that the client provide a username and password, which are then compared against the SRE-Filter user database. If there is no match, SRE-Filter will return a simple you are not authorized response. Alternatively, you can construct a customized response. Since this logon requirement is a bit of a hassle, you can specify OWNERS and In-house clients; for whom logon requirements are waived. This specification is by IP address, with the possibility of using wildcards to grant access rights access to entire domains. | The Logon Controls section of the simple-mode configurator can be used to enable this Logon requirement, to add and remove users, and to add and remove In-house clients. Advanced User Notes: The following SRE-Filter parameters may be of interest (see INITFILT.DOC for details). LOGON_LIMITS: limits the number of logon attempts per minute DNS_CHECK: denies access to clients with no IP Name. UNALLOWED_IPS: identify IP addresses that will not be allowed onto your site. THE_REALM: the default `REALM` name (can be changed with the simple mode configurator)
Public areas | The basic idea is that before logon or access_controls are checked, SRE-Filter checks if the request selector points to a PUBLIC area. If so, no logon or access controls are attempted. In a sense, the PUBLIC areas are purposely placed outside of the protection of SRE-Filter's various access controls. | You can use the Public Areas section of the simple mode configurator to add and remove public areas. For a more nuanced control, you probably will want to use the Modify Initializaton Parameters section of the the intermediate configurator -- look for the sub-section on the PUBLIC_URLS parameters. Advanced users note: Significant performance advantages can be realized by using the SREFQUIK variant of SRE-Filter. This requires that you establish as many literal PUBLIC_URLS as possible. The intermediate mode configurator can be used to accomplish this.

Since many sites will contain a continuum of private and public resources, this coarse level of control will often be inappropriate. SRE-Filter offers two finer methods of access controls: SEL-specific access controls, and the HTACCESS method.

SEL-specific access controls | Selector-specific access (aka SEL-specific access controls ) are based on a comparison between the request selector (the request selector is part of the http request string that the client sends to the server... ... it's the URL minus http://x.y.z/) and a a list of SEL-specific access controls. This list (which may contain wildcarded entries) dictates who can access the various resources (files, etc.) on your site. Given a match, SRE-Filter will then compare the list of resource privileges assigned to the selector to the client privileges granted to the client. In general, the client must have at least one client privilege that's also in the resource privilege list (though you can specify more complicated conditions). If no such matching privilege exists, SRE-Filter returns an unauthorized access response. This response can be a generic response, a customized-for-your-site response, or a customized-for-this- selector response. For a more detailed description of SRE-Filter's various access control mechanisms, see the the SRE-Filter FAQ | The Access Controls section of the simple-mode configurator lets you enable access controls, as well as choose between the generic and customized unauthorized access response. You can also use the simple-mode configurator to add and remove entries in the list of SEL-specific access controls. Note that when you use the simple-mode configurator to create a selector specific access control , you will be asked to provide the selector (with optional wildcards) a list of resource privileges, and an optional permission list. You should use the Modify SEL-specific Access Controls section of the intermediate mode configurator to: specify a selector-specific access-failure response (for details, see the description of ACCESS_FAIL_FILE in INITFILT.DOC), specify more complicated access-control conditions, grant SEL-specific permissions. specify an advanced options file Client privileges are granted when you set up user entries. You may also want to use the intermediate mode configurator to modify the INHOUSE_PRIVS and PUBLIC_PRIVS parameters. If you want to get fancy, see ADDPRIVS.DOC for details on how you can specify dynamic client privileges.
The HTACCESS method | The HTACCESS method (which is something of a standard) uses special HTACCESS files located in the directory (and in the parent directories) of the requested file. This (or these) HTACCESS files contain information on who has access rights to files in the directory (and in child directories). | HTACCESS files can be created using your favorite text-editor. You might want to read some documentation on the HTACCESS method of access control. Alternatively, you can use the intermediate configurator's Modify HTACCESS Contols options to change HTACCESS files.

The choice between the SEL-specific and HTACCESS methods is a matter of taste and convenience -- controlling directories is a sure way to prevent access to files, but controlling selectors is more flexible (if your physical directory structure changes more frequently then your pages). In fact, these two methods (SEL-specific and directory-specific) can be used jointly, which implies multiple checks. But be careful when using both methods, it is possible to get stuck in authorization loops!

> Since SEL-specific controls are native to SRE-Filter, they will tend to be faster (and better tested) then the HTACCESS method. Therefore, our recommendation is to use the SEL-specific method.

Return to table of contents.

### 6) Resolving requests

Since all kinds of requests might show up on your server's doorstep, SRE-Filter offers a slew of techniques and tricks for resolving requests. These include specifying your home-page, indicating directory specific documents, establishing virtual directories, redirection to other servers, and revising the request selector .

The problem | Solutions
A document was not specified | You can use the Default Documents section of the simple-mode configurator to: identify your home page -- it is sent in response to empty selectors. identify possible files (including directory specific file names) to use when a request for a directory appears .. or, generate a cachable directory listing instead (see DIR.DOC for details). Advanced Users Note: see the description of the NOEXT_TYPE parameter (in INITFILT.DOC) for an alternative approach.
Document could not be found | By default, SRE-Filter will: send a generic response that mentions the colloquial name of your site Alternatively, you can modify the generic response, create a customized document not found response file, or return a "directory specific" document not found response file. Use the Site Identification variables section of the simple-mode configurator to modify the colloquial name, the generic response, and the document not found response file.
A document has been moved to another location | SRE-Filter uses aliases to redirect requests to another URL. This URL need not be on the same server, and it need not have the same name. You can use the Home Directory, virtual directories, and redirection section of the simple-mode configurator to create these redirection aliases.
You want to catch common mistaakes | SRE-Filter uses aliases to specify local redirections: which involve textual substitutions in the recieved request selector. Among other advantages, this gives you quite a bit of control over how "SRE-Filter facilities, and external procedures" percieve The Request. The intermediate mode's Modify Redirection Aliases section can be used to create these local redirection aliases.
You want to specify a ~ directory | The home directory is used as a text replacement whenever a ~ is encountered in a request selector. A typical value of this would be "Users/". Further refining the use of home directory, you may wish clients to have access to particular subdirectories of your "Users" directories. For example, all "students" may have space on the server machine, some of which is used for web, and some for "personal" purposes. The goal is to give clients direct access to the "web" related sub-directories but not to the "personal" sub-directories. To modify these home directory options, see the Home Directory, virtual directories, and redirection section of the simple-mode configurator
You want to transfer files from outside of GoServe's data directory | By default, SRE-Filter will match a request selector to a file in the `GoServe data directory`. While a good security feature (files not in or under the GoServe data directory are inaccessible), this can be an inconvenience. To remedy this inconvenience, one can define virtual directories Basically, SRE-FILTER will compare the starting portion of the request selector to see if it matches one of your virtual directory entries. If it does, the target directory listed in the matching entry is used (instead of the GoServe data directory). Thus, you can make available a wide, but controllable, set of directories (on or LAN accessible from) your server. To create virtual directories, see the Home directory, virtual directories, and redirection section of the simple-mode configurator. Advanced users notes: Virtual directories can point to "remote" directories on other http servers -- SRE-Filter will attempt to retrieve the file from the remote server, without using a redirection (see the SRE-Filter manual for details). You can also use virtual directories with CGI-BIN scripts, file uploads, and SRE-Filter add-ons (and you can specify which virtual directories are to be used for each of these several types of requests). Although somewhat obsolete, you can use special transfer alias to transfer selected files from outside the GoServe data directory (see the SRE-Filter manual for details).

Return to table of contents.

### 7)Server Side Includes

SRE-Filter supports a rich variety of server side include (SSI) features.

The feature | Discussion | More details
Caching of documents that contain SSIs. | To improve performance, SRE-Filter will "cache" HTML documents that have server side includes (SSI). This SSI-caching is of the document after the SSIs have been performed. In simple cases SRE-Filter can reuse this cached file, without having to repeat the actual process of SSI lookups, etc. Needless to say, this can greatly improve server performance. | You can use the Server Side Includes section of the simple-mode configurator to enable SSI-caching. For a detailed discussion of SSI-Caching, see SSICACHE.HTM.
Suppressing Server Side Includes | You can suppress server side includes for everyone, or for selected HTML documents. The NO_INCLUDE parameter can be used to suppress all SSI's. A less drastic approach is to use a NO_SSI access-control permission. | See INITFILT.DOC for a discussion of NO_INCLUDE. To set access control permissions, see the Access Controls section of the intermediate configurator.
Limiting SSIs to a subset of your HTML documents | SRE-Filter can be instructed to attempt server side includes (SSIs) only on a subset of HTML documents; such as those with .SHT or a .SHTML extension. In other words, HTML documents with .HTM or .HTML extension will not be checked for server side includes. This can speed up file transfer (especially when used in conjunction with SSI-caching), but does require more care when naming html files. | See the Server Side Includes section of the simple-mode configurator to enable this SSI on SHTML files only feature. Advanced users notes: If you wish to use extensions other then .SHT or .SHTML, you can change the SSI_EXTENSIONS parameter (described in INITFILT.DOC). You may also need to add `text/html` mime type entries to the MEDIATYP.RXX file (see the SRE-Filter manual for details).
Headers and Footers | You can include headers and footers in all your HTML documents. They can contain Server Side Include keyphrases (see below). | You can set both headers and footers in the Server Side Include section of the simple-mode configurator (or see the SRE-Filter manual for further details).
The NSCA HTTPD-style server side include syntax | The NSCA HTTPD-style server side include syntax is fully supported by SRE-Filter. These include: INCLUDEing files Reporting size and creation date of files Displaying selected system variables (such as current date and time, and the servername and client's IP addresses). Including the output of CGI-Bin scripts | The SRE-Filter manual also describes how to use NCSA HTTPD-style server side. You may also want to see TIMEFMT.DOC for a description of time and date display formats.
The SRE-Filter syntax | The SRE-Filter syntax overlaps the NCSA HTTPD-style syntax, but there are some important additions (note: you can use both types of SSIs in your documents). Additional features include: INTERPRETing REXX code, either in-line or in external files SELECTive excludes are supported You can specify a large set of static variables to include in your documents Several built-in text counters are available. Of particular interest, SRE-Filter can suppress augmenting the hit counter when: Repetitive hits arrive from the same client Requests arrive from OWNERS OPTIONs passed to the html document can be incorporated into your document | The SRE-Filter manual contains a detailed discussion of the SRE-Filter SSI syntax, including a discussion of the suppression of hits feature. The SRE-Filter FAQ contains a discussion of the various "hit" counting mechanisms available to SRE-Filter, as does COUNTER.DOC. You can use the Server Side Include section of the simple-mode configurator to modify most of the static variables.

Return to table of contents.

### 8)Server Side Processing: CGI-BIN, imagemaps, SRE-Filter add-ons

SRE-Filter supports a variety of server side processing features. These include support for server-side imagemaps and for searchable indices, CGI-Bin support, and support for SRE-Filter add-ons.

The feature | Discussion
ImageMaps | SRE-Filter supports NSCA and CERN server-side imagemaps. The SRE-Filter manual contains a detailed discussion of how to specify and configure imagemaps. You can also play with SAMPMAP.HTM (a sample image map document, it uses the SAMPMAP.MAP imagemap file).
Searchable Indices | SRE-Filter is shipped with DOSEARCH, a moderately powerful text-file search engine. When used in conjunction with aliases, it is easy to implement searchable indices. The SRE-Filter manual contains a short discussion of how to setup searchable indices, as does the ALIASES.IN file.
Special Requests | SRE-Filter supports several special requests, all of which start with an exclamation point (!) The current set of special requests are: !authorize !ping !statistics !ssi?option !host+ip.add.res.s !reset !save !variable?varname See the SRE-Filter manual for details.
CGI-Bin support | SRE-Filter offers full support for the Common Gateway Interface. SRE-Filter offers a few enhancments, including: an ability to specify alternative directories (within which to find CGI-Bin scripts) an ability to use PERL scripts
SRE-Filter add-ons | As an alternative to the somewhat clunky CGI-Bin interface, SRE-Filter add-ons can be used. The STATUS program is a sample of one such add-on. But that's a trivial example. Far more useful addons are available -- the following lists some of the larger ones, all of which can be obtained from the SRE-Filter home page. Customize your web site on a per client basis (CUSTOMIZ.ZIP) Generate text-graphics GIF files on-the-fly, using all kinda fonts (GIF_TEXT.ZIP) Create "news-group and list-server" discussion groups (FORUM.ZIP). A full featured "web based bulletin board system" (BBS.ZIP) A directory displayer, with user-settable display features (GETAFILE.ZIP) A powerful, character-mode calculator with a web interface (CALCWWW.ZIP) Lots of server status info, and server configuration tools (NUSTATUS.ZIP) A front-end to the SWISH directory indexer/search-engine (SRCHINDX.ZIP) A simple database engine, designed for fast web access of flat files (SREDATA.ZIP) Notes: Ambitious programmers can look in the SRE-Filter manual for more information on writing to the SRE-Filter API. If you are really serious, you'll want to get the source code and documentation for SRE-Filter's macrospace library & independent threads. If desired, you can suppress execution of CGI-BIN scripts and SRE-Filter add-ons, both across-the-board (the NO_PROCESSING parameter), and on a SEL-specific basis (the NO_SSP permission). SRCHINDX.DOC describes how to use the SWISH directory indexing tool. SHOWDIR.DOC describes how to use the SHOWDIR dynamic directory processor (as an alternative to SRE-Filter's !DIR facility).

Return to table of contents.

### 9)Uploads, HEAD requests, and other methods

SRE-Filter supports some of the less frequently used HTTP 1.0 protocols, as well as a few proposed HTTP 1.1 protocols.

The protocol | Discussion
HEAD requests | Although not frequently used, HEAD method requests offer some throughput advantages when searching over a broad range of sites. To fully support such requests, SRE-Filter can parse the <HEAD> section of HTML documents, and return (in the response header) the contents of LINK, NAME and META-EQUIV elements. In fact, SRE-Filter can parse the <HEAD> for GET requests too (though this may not be a very useful activity). For further discussion, see the description of the AUTO_HEADER parameter in INITFILT.DOC, or see the SRE-Filter manual.
File Uploads using HTML FORMS | SRE-Filter's PUT_FILE facility supports the multipart/form-data method of file upload (as supported by NetScape 2.01). This requires the use of an HTML document that contains a special type of <FORM>. Notes: For an example of such a document, you can examine UPLOAD.HTM . The SRE-Filter FAQ contains a discussion of file uploads using HTML forms. For further discussion, see the SRE-Filter manual.
Uploads from other servers | Though somewhat obsolete with the advent of multipart/form-data method of file upload, SRE-Filter's GET_URL facility permits clients to transfer files from different http servers to your server. Notes: For an example of the use of GET_URL, you can examine UPLOAD.HTM . The UPLOAD_MINFREE and UPLOAD_MAXSIZE parameters (as described in INITFILT.DOC) can be used to control the size and extent of file upload capabilities. For further discussion, see the SRE-Filter manual.
The PUT and DELETE methods | The PUT and DELETE HTTP 1.1 methods allow clients to upload, and delete, files from your server. Most browsers do not support these methods. Notes: The DOPUT.CMD and DODELETE.CMD files can be used to PUT and DELETE files from your server. As a security measure, special SEL-specific access-control permissions must be granted before the PUT and DELETE methods are recognized. For further discussion, see the SRE-Filter manual.
Byte-range retrieval | SRE-Filter supports byte-range retrieval. Byte-range retrieval, which consists of sending selected portions of a document, is used by Adobe Acrobat to retrieve selected pages from large .PDF files. For further discussion, see the description of the ACCEPT_RANGE parameter in INITFILT.DOC.
Setting the expiration date of documents | To work around NetScape's tendency to over-refresh documents that have an immediate expiration date, you can instruct SRE-Filter to modify GoServe's default `temporary file` response header. Specifically, you can change the immediate expiration date to an expire in 2 hours expiration date. The simple mode configurator's Miscellaneous section can be used to select this option. For further discussion, see the description of FIX_EXPIRE in INITFILT.DOC.

Return to table of contents.

### 10)Auditing, Pre-filter, and Post-Filter Processing

SRE-Filter supports several forms of auditing, including the common-log format. In addition, SRE-Filter can do a few things before and after processing the request.

Action | Description
SRE-Filter Audit files | SRE-Filter allows you to use several different audit mechanisms. The common-log, browser, and referer audit files . SRE-Filter can maintain a log of all requests in a common-log file (the common-log format is something of a WWW standard). In addition, SRE-Filter can maintain seperate logs of browsers, and referers. You can even archive these log files on a scheduled basis. For details, see the SRE-Filter manual, the description of the WRITE_LOGS parameter in INITFILT.DOC, or see SREFLOGS.DOC for a detailed description. The RECORD_ALL_FILE SRE-Filter has a special audit mechnanism to record all requests. This recording consist of augmenting a unique counter for every file (or selector ) requested. That is, it does not record who made the request. For details on SRE-Filter's record all requests option, see the SRE-Filter manual, or see the description of the RECORD_OPTION and RECORD_CACHE_LINES parameters in INITFILT.DOC. GoServe's GOAUDIT file. GoServe maintains an audit file (typically called GOAUDIT.80). It contains a lot of information, but is a bit difficult to read (see GOSERVE.DOC for further details on how to configure the GOAUDIT file).
Load Balancing | SRE-Filter supports a very simple form of load balancing. Specifically, when the load on your server (measured in number of current transactions) gets too high, SRE-Filter can redirect the request to (one of several) backup servers. For details, see the description of the BACKUPSERVERLIST and LOADTHRESHOLD parameters (in INITFILT.DOC), or see the SRE-Filter manual.
Special Directives | SRE-Filter understands several special directives. These are prefixes added to request selector that can modify SRE-Filter's logic (they all start with a ! ) !force/sel : Forces a logon (if needed), and suppresses the SSI-Cache !norecord/sel : Do not record this request (in the RECORD_ALL_FILE, the COUNTER.RXX files, or the COUNTER.CNT file) !sendas_mimetype_mimesubtype/URL : Send the request with a `content-type: mimetype/mimesubtype` response header !delsend/filename.ext : Return a transient file from (from the TEMPFILE_DIR directory), and then delete it. See the SRE-Filter manual for more details on these special directives.
Pre-filter processing | There may be occasions when you want some other filter to process a request before SRE-Filter get's a shot at it. For example, ambitous programmers may wish to implement a more sophisticated load balancing algorithim; or you might wish to implement an alternative access control mechanism. As a more practical example, SRE-Filter comes with PREFILTR.80; a pre-filter that passes a request to the GOREMOTE.80 server remote control package. The SRE-Filter manual describes how to enable pre-filters; or you can read the description of the PRE_FILTER and PREFILTER_NAME parameters in INITFILT.DOC.
Post-filter processing | After SRE-Filter has responsed to a request (and GoServe has closed the connection), you may wish to perform further actions. For example, you may wish to notify the webmaster of special events (see POSTMAIL.80 for an example), or you may wish to record select requests (see POSTRCRD.80 for an example). The SRE-Filter manual describes how to enable post-filters; or you can read the description of the POST_FILTER and POSTFILTER_NAME parameters in INITFILT.DOC.
| Some advanced-users options
Mid-Filter requests | You can specify, on a SEL-specific basis the execution of "mid-filter" REXX procedures. These "mid filter" procedures are run after resolution of a request (and after access control checks), but before the actual transmission of any information. For details, see the advanced options file
Custom initialization | Using the CUSTOM_INITS parameter (located in the SRE-Filter file) you can specify a set of REXX procedures to run when SRE-Filter is first initialized. For example, a PRELOAD procedure (available as an SRE-Filter addon) can be specified to "preload" commonly called files. For the details, see the description of the CUSTOM_INITS parameter in INITFILT.DOC.
Scheduled Programs | You instruct SRE-Filter to execute REXX procedures on a scheduled basis (such as hourly, daily, or monthly). For details, see the description of the SCHED. parameter(s) in SREFLOGS.DOC

Return to table of contents.

### Optimization Hints

Let's admit it: `SRE-Filter is not a super-fleet web server.` The design goal has always been comprehensiveness (without an enormous amount of complication). Within the context of an interpreted language (REXX), that has a price.

That said, a lot has been done to improve throughput. In particular; the extensive use of macrospace, the use of daemons (independent threads) to handle much of the mundane processing, and the incorporation of an SSI-cache help keep SRE-Filter's performance respectable. Still, it would be nice if it were faster.

Well, if you are willing to work on it, there's actually quite a bit you can do to improve it's performance. The following outlines some of these `optimization strategies`. If you want more specific information, the best place to go is the first section of INITFILT.DOC -- it contains a number of `optimzation hints`.

Action | Description
The GoServe Cache | By far the best optimization strategy is to take advantage of GoServe's cache (look at the Options-General tab under GoServe). Unfortunately, there is a potential drawback -- anything that's cached will be free of access controls (since GoServe sends back cached items without fussin' around). Therefore, if you have any security on your site, SRE-Filter will disable the cache. To work around this problem, you can use the CACHE access permission, or the PUBLIC_URLS public area identifiers.
The SREFQUIK and SREFAST variants of SRE-Filter | For many requests you won't need all the bells and whistles built into SRE-Filter. Recognizing this, SRE-Filter includes two "small and simple" filter. These filters (SREFQUIK and SREFAST) will process a select set of requests, and pass the one's it can't handle to the standard version of SRE-Filter. SREFQUIK can handle "cached" requests, and it can handle literal PUBLIC_URLS. The SREFQUIK documentation thoroughly describes the use SREFQUIK. You should also read the description of PUBLIC_URLS in INITFILT.DOC. SREFAST is more flexible then SREFQUIK (for example, it can process most of the server side includes), but does require a fair amount of configuration. If you are willing to do this configuration (it's a bit tedious, but not complex), use of SREFAST is recommended.
Suppressing unneeded features | If you know you aren't going to use one of SRE-Filter's features; you can and should instruct SRE-Filter not to use it. In general, there are two ways of doing this: across the board, or on a selector specific basis. For example, you can turn off Virtual Directory lookup for everyone (by setting VIRTUAL_FILE=' '), you can use the NO_VIRTUAL permission to suppress virtual directory lookup on a case by case ( selector by selector ) basis, or you can set up a literal PUBLIC_URL. Another example is the use of SHTML files to indicate "SSI-capable" documents; normal HTML files will not be subject to SSI compilation (a time consuming action).

Return to table of contents.

### Synopsis of Features

The following summarizes most of SRE-FILTER's features:

- Multiple hosts (either unique IP addresses, or multiple aliases per IP address) are supported.

  > Seperate data directories, aliases, etc. can be specified for each supported host

- Logon requirements may be required of no one, everyone, or the general public.
  - A set of public resources can be specifed, which will be available to all clients.
  - OWNERS (with SUPERUSER privileges) and IN-HOUSE users can be automatically detected (using their numeric IP address), with logon requirements waived.
  - A set of "unallowed" numeric IP addresses can be specified.
  - Wildcards can be used in the list of IN-HOUSE and "unallowed" numeric IP addresses
- Several tools for resolving and redirecting requests are provided
  - A "directory specific" document can be sent when a request selector specifying a directory, but no file is name, is recieved
  - Instead of a default document; a "directory listing" can be produced, with file descriptions generated for all plain, html, and .ZIP files. These directory listings can be cached, and used for subsequent requests.
  - ALIASes can be used to redirect requests for moved documents, to resolve commonly occuring "misspelled requests", and to transfer specific files from anywhere on your server.
  - A customized "not found" document (complete with suggested link) can be sent when the requested document is missing.
  - The ~ shorthand for the "home directory" is supported. In addition, ~ can point to www-specific subdirectories of the "home directory"
  - Transfer of files from outside the "default data directory" is easily supported through the use of "virtual directories".
  - Transfer, with optional processing, of files from remote servers (without using redirection) is supported through the use of "remote" virtual directories.
- A variety of server-side includes are easily achieved
  - A special caching algorithim selectively caches documents that contain server side includes
  - Headers and Footers can be included in all HTML documents
  - Recursively processed keyphrases, that dictate server side includes, can be placed in your HTML documents
    - Dynamic and Static String REPLACEments
    - Inclusion of files (from anywhere on your system)
    - Inclusion of OPTIONs that appear in the request selector (information after a ?)
    - Execution of REXX-code blocks, and insertion of the results
    - SELECTive excludes of HTML code blocks
  - The NSCA HTTPD server side include syntax is fully implemented
- Standard server features include:
- NCSA style IMAGEMAPS are supported, and easily requested.
- Searchable indices on text files are easy to setup
- A flexible mechanism for controlling access to explicit selectors (and the files they correspond to); including assignation of realms on a SEL-specific basis.
- Support for the HTACCESS method of access control
- Several mechanisms for recording hits
- Several forms of redirection can be specified
- A simple form of load balancing is supported
- Support for "common-log" format audit files (as well as browser and referer log files)
- Special SRE-Filter built-in features include:
- A file display and transfer facility with easy directory traversal
- A simple but powerful "text file" search facility
- Dynamic assignation of "short-term" privileges to clients
- Tracking of recent hits to prevent recording of repetitive requests
- Optional limits on frequency of unsuccesful "logon" attempts, with options for custom response files in the event of "logon" or "access" denial.
- Implementation of "byte range retrieval" (which is used by Adobe Acrobat to access subsections of documents)
- Reponse headers can be automatially generated, using LINK and META HTTP-EQUIV elements in the <HEAD> portion of the document.
- Several file upload methods are supported
- Client directed retrieval of files from other HTTP servers
- Browsers that support the `type=file` FORM attribute (such as NetScape 2.01), can upload files from their own machines.
- The PUT verb, with byte ranges, can be permitted on a per selector (or per "set of selectors") basis
- The DELETE method can be permitted on a per selector (or per "set of selectors") basis
- User written server side processing (gateway programs) options include:
- Support for SRE-Filter add-ons (REXX routines that return results directly to the client).
- CGI-BIN scripts (both REXX and binary executables, and PERL scripts if you have a PERL interpreter installed) are supported (using procedures adapted from the GoHTTP package).
- A "pre-filter" can be called prior to invoking SRE-Filter
- SRE-Filter comes with a special pre-filter that implements the the GoRemote package.
- A "post-filter" can be called after SRE-Filter has responded to the request.
- SRE-Filter comes with a special post-filter that uses socket calls, to an SMTP server, to e-mail "event specific" alerts.

Return to the table of contents.

### The Disclaimer

In a nutshell:

> SRE-Filter, and related products, are to be used at your own risk; the authors assume no liability for any damage that may be caused by the use or misuse of these software products.

The nice news is:

> SRE-Filter is copyright by Daniel Hellerstein, and hereby placed in the the public domain (yes, I realize that's somewhat of a legalese contradiction). You may use is it any manner you deem fit, up to and including use of the code in your own software.

Out of the nutshell:

      Copyright 1996,1997 by Daniel Hellerstein.
      Permission to use this program  for any purpose is hereby granted without fee,
      provided that the author's name not be used in advertising or publicity
      pertaining to distribution of the software without specific written
      prior permision.
      With some exception, this includes the right to subset and reuse the code,
      with proper attribution. The exceptions are two fold:
         1)  Portions of the code adapted from other author's work
            (these are noted where appropriate); you'll need to contact these other
            authors for appropriate permissions.
        2)  SRE-Filter uses Quercus System's REXXLIB procedure library.  The
            license for REXXLIB gives the author the right to distribute REXXLIB
            without charge.  This right may NOT extend to redistributors.
            Please contact Quercus Systems for details.
            SRE-Filter also uses Info-Zip's UNZIPAPI library, and Jeff Glatt's
            FILEREXX library.  Although these are freely available software,
            you may wish to contact their author's for details.
      Furthermore you may also charge a reasonable re-distribution fee for
      SRE-Filter; with the understanding that this does not remove the
      work from the public domain.

        THIS SOFTWARE PACKAGE IS PROVIDED "AS IS" WITHOUT EXPRESS
        OR IMPLIED WARRANTY.
        THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE PACKAGE,
        INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS.
        IN NO  EVENT SHALL THE AUTHOR (Daniel Hellerstein) OR ANY PERSON OR
        INSTITUTION ASSOCIATED WITH THIS PRODUCT BE LIABLE FOR ANY
        SPECIAL,INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER
        RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION
        OF CONTRACT,NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
        IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE PACKAGE.

       SRE-FILTER was developed on the personal time of Daniel Hellerstein,
       and is not supported, approved, or in any way an official product
       of my employer (USDA/ERS).

### Acknowlegments

Though my wife probably put up with the most grief, the following people all contributed:

- Steve Ryckman provided great help: good suggestions, bug finds, and hosting the partial mirror site (http://srefilter.simsware.com).
- Don Meyer, author of the GoHTTP filter, was very gracious in allowing me to "build on his work". In particular, much of the source code for HTACCESS file access-controls, CGI-BIN script processing, and image map processing was adapted from his code.
- Derek Sims (author of the IntrFilt filter) for helpful advice regarding multi-host servers.
- Charles Daney for providing the REXXLIB library at a very reasonable price.
- Dave Briccetti for providing the "SMTP using socket calls" code.
- Ariel L. Szczupak for advice on working with queues and semaphores.
- Richard Hughes for useful thoughts on discussion group software
- The people at the INFO-ZIP project, for providing the UNZIPAPI dynamic link library.
- the several early adopters (may they rest in peace) for their patience in dealing with stupid bugs
- several individuals (mm, dr, sr, et al) for good suggestions that are now realities
- .... and (of course) Mike Cowlishaw for creating GoServe, and always finding time to try and answer.

Return to table of contents.

Do you have questions, comments, or suggestions relating to SRE-Filter? Send 'em in ( Daniel Hellerstein.) -- that's DANIELH@ECON.AG.GOV if your browser can't deal with MAILTO:.