			THE SYNTAX OF CACHE FILES:
	DECOUPLING THE GN HIERARCHY FROM THE FILE SYSTEM HIERARCHY

Normally the gn menu hierarchy is closely linked to the hierarchy of
the file system containing the files being served, and this is as it
should be.  However, in certain circumstances it may be desirable to
decouple these two hierarchies.  One might, for some reason, want
all the files to be in a single directory but still have an extensive
hierarchy of menus.  One reason for doing this would be to facilitate
using gn as a front end for some other programs which produce data.

This document describes a mechanism for completely separating the
the gn menu hierarchy from the filesystem hierarchy.  To do this 
we need to describe the format of entries in a cachefile.  Such
an entry looks like


	TMenu Title <tab> selector <tab> host <tab> port <CRLF>

The first character (T in this prototype) is one of the single
character standard gopher types (see the gopher protocol from Univ of
Minn for for more information), i.e. it is "0" for a file, "1" for a
directory, "7" for a search, "I" for an image, etc.  The phrase "Menu
Title" or anything replacing it (which must not contain a <tab>) is
what will appear on the clients menu. The host is the fully qualified
hostname and the port is the port at which the server is running.  The
selector is allowed by the gopher protocol to be more or less any
string used by the server to identify the file or menu which the
client wants.  The normal transaction is the server sends the contents
of a cachefile to the client which displays a menu based on it.  The
user picks an item and the client sends the selector to the server.
This tells the server to send a document to the client if that was
what the user chose or a new menu (i.e. another cachefile) if the
user chose that.

For gn the selector in a cachefile entry produced by mkcache is 
precisely the contents of the "Path" entry in the menu file.  Thus
a menu file which contains

	Name=My favorite file
	Type=0
	Path=0/dir1/dir2/filename

will result in a cachefile entry like

	0My favorite file <tab> 0/dir1/dir2/filename <tab> host.edu <tab> 70

or for a directory a menu which contains 

	Name=My favorite directory
	Type=1
	Path=1/dir1/dir2

will result in a cachefile entry like

	1My favorite directory <tab> 0/dir1/dir2 <tab> host.edu <tab> 70


When the gn server receives a selector like 0/dir1/dir2/filename it
knows it should send the file rootdir/dir1/dir2/filename to the client
and it knows this is a text file (because of the leading type 0).
However, before sending it the gn server checks the cachefile which
contained the entry with this selector to make sure the selector is a
legitimate one and not one that an unscrupulous client has produced to
get access to some private file on the server's host.  Gn will send
the file rootdir/dir1/dir2/filename if and ONLY if it finds the
selector 0/dir1/dir2/filename in an entry in the appropriate
cachefile, which by default is the file rootdir/dir1/dir2/.cache.  The
point is that gn constructs the name of the cachefile by taking the
file path, deleting the file name and tacking on .cache.  (There is
one exception to this for structured files [type 1m].  See the
installation guide.)  This requires that the cachefile containing an
entry for a file be in the same directory as the file and a cachefile
containing an entry for a directory be in the parent directory of that
directory.  This is what ties the filesystem and menu hierarchies
together.

To decouple the menu and filesystem hierarchies we use an alternate
syntax for the selector which gn versions 0.5 and later recognize.  In
this form the selector for the file above would be

	0/dir1/dir2/filename(/dir1/dir2/.cache)

We have placed the name of the cache file containing this item in
parentheses and appended it to the selector.  When gn receives this
selector it knows what file is requested and that it is a text file and
it knows the name of the cachefile in which to check for security
purposes for an entry containing the selector 0/dir1/dir2/filename.
(This security check only checks the selector up to the parentheses,
since that is all that is necessary to know the file is legitimately
being offered by the server).

In this form there is no need for the cachefile to be located in any
particular place relative to the files its entries reference, nor any
need for the cachefile to have the name ".cache".  Thus the selector

	0/dir1/dir2/filename(/dir3/cfile)

is fine provided the file /dir3/cfile is a legitimate cachefile containing
an entry whose selector is 0/dir1/dir2/filename, so that gn can check
this is a valid file to be sent to a client.

There is one other very important difference in the syntax of selectors
of this type.  Items of gopher type 1 are menus to the client and 
normally directories to the server, but in this scheme for gn they
no longer correspond to directories but to cachefiles which will present
a menu to the client.  Thus the selector 1/dir1/dir2 in the example above
will translate into the selector

	1/dir1/dir2/.cache(/dir1/.cache)

in the alternate scheme.  In other words to specify a menu (even if it
corresponds to a directory) you must specify the cachefile with the
menu entries.  This is really different from the usual form where 
to refer to a directory the selector is 1/dir1/dir2 -- the path of the
*directory*.  Now to refer to the same directory the selector contains
the path of the *cachefile* containing the items in that directory.
It is the presences of the parentheses which allows gn to distinguish these
two different syntaxes.  Notice that in parentheses we put the cachefile 
*containing* this entry which is different from the cachefile we want
to send, i.e. gn checks /dir1/.cache to make sure that it is ok to 
send /dir1/dir2/.cache.

Of course, as with files, the cachefiles can have any name and be located
anywhere.  The selector 

	1/dir1/dir2/cfile(/dir3/another_cache) 

is fine if cfile is a cachefile containing the items in dir2 (or items
in any directory for that matter) and another_cache is a cachefile
containing an entry with the selector above.  

TWO FINAL NOTES: 

1. The cachefiles listed in parentheses are used only for security
checking -- to make sure permission is given to send the file.  Thus
they don't have to be "real" cachefiles.  You could, for example, have
a single file "masterlist" which contained cachefile entries for all
the selectors for your whole server and put this in parentheses at the
end of each selector, like

	1/dir1/dir2/cfile(/masterlist)

The masterlist file would be used only for security checks and never 
actually be a menu.  Moreover the security check looks only at the
selector in a cachefile entry, which it takes to be everything between
the first and second <tab> character on the line.  Thus masterlist
could be a list of all valid selectors for the server, each on a
line preceded and followed by a <tab> with nothing else on that line.
I should point out that I haven't actually tried this and it might
be inefficient if masterlist is large.

2. To do grep type searches of the files in a menu the
search menu item's selector should look like 

	7g/dir1/dir2/cfile(/dir3/anothercache)

where cfile contains the entries you want to be grepped and anothercache
contains either the entry with this selector or an entry with selector
1s/dir1/dir2/cfile(something_else) either of which indicate that the 
files listed in cfile are permitted to be searched.  The point is that
the "g" in the "7g" at the beginning is optional (for backwards 
compatibility) with the old syntax, but required for the new "parenthesis"
syntax.


              USING DECOUPLING WITH ACCESS CONTROL

One interesting thing which can be done with the decoupling feature of
gn is to make your root menu (or any other menu) appear different to
different hosts. Here's an example that sets up a root menu which
shows one item to all hosts, an additional directory to any hosts
in one selected group and an additional file to any host in a second
selected groups.

In your root directory make the subdirectories "hostgroup1" and
"hostgroup2" which contain the files and directories that you want to
be visible only to a certain group of hosts.  In this example we
assume that there is one file in /hostgroup2/privatefile and one
directory /hostgroup1/dir.  In each of the directories /hostgroup1 and
/hostgroup2 put a .access file containing the hosts permitted to see
the information in that directory.

The menu file for the root directory would look like

##########################################
Name=Public Stuff                      # This is visible to any host
Path=1/public    

Name=Directory for Group 1 only        # This is invisible except for group1
Path=1/root1/dir/.cache(/root1/.cache)

Name=File for Group 2 only             # This is invisible except for group2
Path=0/root2/privatefile(/root2/.cache)
##########################################

While the menu file in root1 would contain

##########################################
Name=Directory for Group 1 only
Path=1/root1/dir/.cache(/root1/.cache)
##########################################

and the one for root2 would contain

##########################################
Name=File for Group 2 only
Path=0/root2/privatefile(/root2/.cache)
##########################################

Directories below /hostgroup1/dir would be ordinary, i.e. they need
not use the decoupling mechanism.