%
%  The Regina Rexx Interpreter
%  Copyright (C) 1992-1994  Anders Christensen <anders@pvv.unit.no>
%

\chapter{Extensions}
\label{ch:options}


\index{extensions|(bold}
\begin{it}
This chapter describes how extensions to {\regina} are implemented.
The whole contents of this chapter is specific for
{\regina}\index{Regina@{\regina}}.
\end{it}

\section{Why Have Extensions}

Why do we need extensions? Well, there are a number of reasons,
although not all of these are very good reasons:

\begin{itemize}
\item
   Adaptations to new environments may require new functionality in
   order to easily interface to the operating system. 

\item
   Extending the language with more power, to faciliate programming. 

\item
   Sometimes, a lot of time can be saved if certain assumptions are
   met, so an extention might be implemented to allow programmers to
   take shortcuts. 

\item
   When a program is ported from one platform to another, parts of the
   code may depend of non-standard features not available on the
   platform being ported to. In this situation, the availability of
   extensions that implement the feature may be of great help to the
   programmer. 

\item
   The implementor had some good idea during development.
\end{itemize}

Extensions arise from holes in the functionality. Whether they will
survive or not depends on how they are perceived by programmers; if
percieved as useful, they will probably be used and thus supported in
more interpreters. 



\section{Extentions and Standard {\rexx}}

In standard {\rexx}, the {\tt OPTIONS}\ttindex{OPTIONS} instruction
provides a ``hook'' for extensions. It takes any type of
parameters, and interprets them in a system-dependant manner. 

The format and legal values of the parameters for the {\tt OPTIONS}
instruction is clearly implementation dependent \cite[p.\ 61]{trl2}.


\section{Specifying Extensions in {\regina}}

In {\regina} there are three level of extensions. Each independent
extension has its own name. Exactly what an independent extension is,
will depend on the viewer, but a classification has been done, and is
listed at the end of this chapter. 

At the lowest level are these ``atomic'' extensions.  Then there are
some ``meta-extensions''. These are collections of other extensions
which belongs together in some manner. If you need the extension for
creating ``buffers'' on the stack, it would be logical to use the
extension to remove buffers from the stack too. Therefore, all the
individual extensions for operations that handle buffers in the stack
can be named by such a ``meta-extensions''. At the end of this
chapter, there is a list of all the meta-extensions, and which
extensions they include.

At the top is ``standards''. These are sets of extensions that makes
the interpreter behave in a fashion compatible with some standard.
Note that ``standard'' is used very liberally, since it may refer to
other implementations of {\rexx}. However, this description of how the
extensions are structured is only followed to some extent. Where
practical, the structure has been deviated. 




\section{The Trouble Begins}

There is one very big problem with extensions. If you want to be able
to turn them on and off during execution, then your program has to be
a bit careful. 

More and more {\rexx} interpreters (including {\regina} seem to do a
parsing when the interpreter is started. The ``old'' way was to
postpone the parsing of each clause until it was actually executed.
This leads to the problem mentioned.

Suppose you want to use an extension that allows a slightly different
syntax, for the sake of the argument, let us assume that you allow an
expression after the {\tt SELECT}\ttindex{SELECT} keyword. Also assume
that this extension is only allowed in extended more, not in
``standard mode''.  However, since {\regina} parses the source code
only once (typically at the starts of the program), the problem is a
catch-22: the extension can only be turned on after parsing the
program, but it is needed before parsing. This also applies to a lot
of other {\rexx} interpreters, and all {\rexx} compilers and
preprocessors.

If the extension is not turned on during parsing, it will generate a
syntax error, but the parsing is all done before the first clause is
executed. Consequently, this extension can not be turned on during
execution, it has to be set before the parsing starts. 

Therefore, there are two alternative ways to invoke a set of
extensions. 
\begin{itemize}
\item
   It can be invoked by using the {\tt -e}\ttindex{-e}
   option\index{options} to the interpreter.  The word following the
   option is the extension or standard to invoke. Multiple {\tt -e}
   options can be specified.

\item
   It can be invoked by setting the environment variable {\tt
   REXXEXTS}\ttindex{REXXEXTS}, which must be a string of the same
   format as the parameters to the {\tt OPTIONS}\ttindex{OPTIONS}
   clause.
\end{itemize}



\section{The Format of the {\tt OPTIONS} clause}

The format of the {\tt OPTIONS} cluase is very simple, it is followed
by any {\rexx} string expression, which is interpreted as a set of
spaceseparated words. The words are treated strictly in order from
left to right, and each word can change zero or more extension
settings. 

Each extension has a name. If the word being treated matches that
name, that extension will be turned on. However, if the word being
treated matches the name of an extension but has the prefix {\tt NO},
then that extension is turned off. If the word does not match any
extensions, then it is simply ignored, without creating any errors or
raising any conditions. 

\begin{example}[Extensions changing parsing]\label{exam:chext}
An example of the same is the {\tt UPPER} instruction. In the
following piece of code the same clause is interpreted in two
completely different ways:
\begin{verbatim}
   options 'NOUPPER'
   do i=1 to 2
      if i=2 then options 'UPPER'
      upper foo bar
      end
\end{verbatim}
In the first iteration of the loop, the clause starting with the token
{\tt UPPER} will be a command, issuing the string resulting from
evaluating the expression {\tt upper foo bar}. However, in the second
iteration of the loop, the same clause is interpreted as an {\tt
UPPER} instruction. Since these two statements has very different
syntax, it seems impossible to handle both in the same program.
{\regina} tries to handle this by ``allowing'' both syntaxes when
parsing the source code, and selecting the right one when interpreting
the statement in question. 
\end{example}

{\regina}s frequent usage of extensions may slow down execution. To
illustrate how this can happen, consider the {\tt OPEN()} extra
built-in function. As this is an extension, it might be dynamically
included and excluded from the scope of currently defined function.
Thus, if the function is used in a loop, it might be in the scope
during the first iteration, but not the second (similar to example
\ref{exam:chext}). Thus, {\regina} can not cache anything relating to
this function, since the cached information may be outdated later. As
a consequence, {\regina} must look up the function in the table of
functions for each invocation. To avoid this, you can set the
extension {\tt CACHEEXT}, which tells {\regina} to chache info
whenever possible, without regards to wether this may render useless
later executions of {\tt OPTIONS}. 



\section{Why You Should Seriously Consider Not Using Extensions}




\section{The Fundamental Extensions}

Here is a description of all ``atomic'' extensions in {\regina}:
%
\begin{description}
\item[\tt BUFTYPE\_BIF~]
   Allows calling the builtin function {\tt BUFTYPE()}, which 
   will write out all the contents of the stack, indicating the
   buffers, if there are any. The idea is taken from VM/CMS, and its
   command named {\tt BUFTYPE}.

\item[\tt CACHEEXT~]\ttindex{CACHEEXT}
   tells {\regina} that information should be cached whenever
   possible, even when this will render future execution of the {\tt
   OPTIONS} instruction useless. Thus, if you use e.g.\ the {\tt
   OPEN()} extra builtin function, and you set {\tt CACHEEXT}, then
   you may experience that the {\tt OPEN()} function does not
   disappear from the current scope when you set the {\tt NOOPEN\_BIF}
   extension. 

   Whether or not a removal of an extension really do happen is
   unspecified when {\tt CACHEEXT} has been called at least once.
   Effectively, info cached during the period when {\tt CACHEEXT} was
   in effect might not be ``uncached''. The advantage of {\tt
   CACHEEXT} is efficiency when you do not need to do a lot of
   toggling of some extension. 

\item[\tt CLOSE\protect\_BIF~]\ttindex{CLOSE\protect\_BIF}
   allows the {\tt CLOSE()}\ttindex{CLOSE()} extra built-in function,
   which allows the program to explicitly close a stream
   \refer{ebif:close}. 

\item[\tt DESBUF\protect\_BIF~]
   Allows calling the builtin function {\tt DESBUF()}, to remove all
   contents and all buffers from the stack. This function is an idea
   taken from the program by the same name under VM/CMS.

\item[\tt DROPBUF\protect\_BIF~]
   Allows calling the builtin function {\tt DROPBUF()}, to removed one
   of more buffers from the stack. This function is an idea take from
   the program by the same name under VM/CMS.

\item[\tt FIND\_BIF~]\ttindex{FIND\protect\_BIF}
   allows calling the {\tt FIND()}\ttindex{FIND()} extra built-in
   function, which is a compatibility function with VM/CMS. This
   function is really equivalent to {\tt POS()}\ttindex{POS()}, but
   the parameters are somewhat reversed, and some find {\tt FIND()}
   more intuitive. Besides, this extension helps porting. 

\item[\tt FLUSHSTACK~]\ttindex{FLUSHSTACK}
   \label{ext:flushstack} Tells the interpreter that whenever a
   command clause instructs the interpreter to flush the commands
   output on the stack, and simultaneously take the input from the
   stack, then the interpreter will not buffer the output but flush it
   to the real stack before the command has terminated. That way, the
   command may read its own output.  The default setting for {\regina}
   is not to flush, i.e.\ {\tt NOFLUSHSTACK}, which tells
   interpreter to temporary buffer all output lines, and flush them to
   the stack when the command has finished.

\item[\tt LINEOUTTRUNC~]\ttindex{LINEOUTTRUNC}
   \label{ext:truncate} This options tells the interpreter that
   whenever the {\tt LINEOUT()}\index{LINEOUT()@{\ptt LINEOUT()}}
   built-in function is executed for a persistent file, the file will
   be truncated after the newly written line, if necessary. This is
   the default setting of {\regina}, unless your system does not have
   the {\tt ftruncate()}\index{ftruncate()@{\ptt ftruncate()}} system
   call. The complement option is {\tt
   NOLINEOUTTRUNC}\index{NOLINEOUTTRUNC@{\ptt NOLINEOUTTRUNC}}. This
   is further documented in section \ref{sect:reginaIO}.

\item[\tt MAKEBUF\_BIF~] 
   Allows calling the builtin function {\tt MAKEBUF()}, to create a
   buffer on the stack. This function is an idea taken from a program
   by the same name under VM/CMS. 

\item[\tt OPEN\_BIF~]\ttindex{OPEN\protect\_BIF}
   adds the extra built-in function {\tt OPEN()}\ttindex{OPEN()}
   \refer{ebif:open}, which is used for explicitly opening streams. 

\item[\tt PRUNE\_TRACE~]\ttindex{PRUNE\protect\_TRACE}
   makes deeply nested routines be displayed at one line. Instead of
   indenting the trace output at a very long line (possibly wrapping
   over several lines on the screen). It displays {\tt [...]} at the
   start of the line, indicating that parts of the white space of the
   line has been removed. 

%\item[\tt UPPER] 
%   Extends the language with the clause {\tt UPPER}
\end{description}




\section{Meta-extensions}


\begin{description}
%\item[\tt UPPER~]
%   Combines the effect of {\tt UPPER\_CLAUSE} and {\tt UPPER\_BIF}.
%
\item[\tt BUFFERS~]\label{ext:buffer}
   Combination of {\tt BUFTYPE\_BIF}, {\tt DESBUF\_BIF}, {\tt
   DROPBUF\_BIF} and {\tt MAKEBUF\_BIF}.

\item[\tt FILEIO~]
   Introduces some commonly used extra features for handling files.
   This is a combination of {\tt OPEN\_BIF()} and {\tt CLOSE\_BIF()},
   which allow the programmer to explicitly open and close files. 
\end{description}



\section{Semi-standards}

\begin{description}
\item[\tt CMS~]
   A set of extensions that stems from the VM/CMS operating system.
   Basically, this includes the most common extensions in the VM/CMS
   version of {\rexx}, in addition of some functions that perform task
   normally done with commands under VM/CMS

\item[\tt VMS~]
   A set of interface functions to the VMS operating system.
   Basically, this makes the {\rexx} programming under VMS as powerful
   as programming directly in DCL.

\item[\tt UNIX~]
   A set of interface functionality to the Unix operating system.
   Basically, this includes some functions that are normally called as
   commands when programming Unix shell scripts. Although it is
   possible to call these as commands in {\regina}, there are
   considerable speed improvements in implementing them as builtin
   functions. 
\end{description}





\section{Standards}

\begin{description}
\item[\tt ALL~]
\item[\tt ANSI~]
\item[\tt DEFAULT~]
\item[\tt NONE~]
\item[\tt SAA~]
\item[\tt TRL1~]
   {\rexx} Language level 3.50, as described in \cite{trl1}.

\item[\tt TRL2~] 
   {\rexx} Language level 4.00, as described in \cite{trl2}.

\end{description}

Also, for those of these standards that have a accepted {\rexx}
language level number, that number can be used, provided that it
matches character by character (i.e.\ not by numeric value). Thus, you
can use {\tt 3.50} as a synonym for {\tt TRL1}, and {\tt 4.00} as a
synonym for {\tt TRL2}.

\begin{center}
\begin{tabular}{|l|c|c|c|c|c|c|c|} \hline
\bf Option\tt       & ALL &ANSI & DEF &NONE & SAA &TRL1 &TRL2 \\\hline\hline
{\tt BUFTYPE\_BIF}  & yes & ??  & yes & no  & ??  & no  & no  \\
{\tt CLOSE\_BIF}    & yes & ??  & yes & no  & ??  & no  & no  \\
{\tt CACHEEXT}	    & no  & no  & no  & no  & no  & no  & no  \\
{\tt DESBUF\_BIF}   & yes & ??  & yes & no  & ??  & no  & no  \\
{\tt DROPBUF\_BIF}  & yes & ??  & yes & no  & ??  & no  & no  \\
{\tt FIND\_BIF}	    & yes & ??  & yes & no  & ??  & no  & no  \\
{\tt FLUSHSTACK}    & yes & ??  & no  & no  & ??  & no  & no  \\
{\tt LINEOUTTRUNC}  & yes & ??  & yes & no  & ??  & no  & no  \\
{\tt MAKEBUF\_BIF}  & yes & ??  & yes & no  & ??  & no  & no  \\
{\tt OPEN\_BIF}     & yes & ??  & yes & no  & ??  & no  & no  \\
{\tt PRUNE\_TRACE}  & yes & no  & yes & no  & no  & no  & no  \\
{\tt UPPER\_CLAUSE} & yes & ??  & yes & no  & ??  & no  & no  \\\hline
\end{tabular}
\end{center}

Note that the standard and default interpreter is a {\rexx} language
level 4.00 interpreter. All other functionality is extensions. In
fact, the features in 4.00 that does not exist in 3.50 are ``inverse''
extensions, i.e.~the extension is to remove the functionality only in
4.00. 
\index{extensions|)}
