

                             Do-It-Yourself Hypertext:
                       Techniques and a Pascal Implementation

                                  Steve T. Jones,
                       Assistant Professor of Data Processing
                            Indiana University at Kokomo
                            2300 South Washington Street
                                  Kokomo, IN 46902
                               KCDZ100@INDYCMS.BITNET

       ABSTRACT

       Hypertext is an attractive vehicle for computer assisted instruction,
       intelligent help systems, and other applications which invite the user
       to navigate through a sea of text.  There are a number of products
       available which make hypertext accessible to end-users; this paper
       discusses implementation of hypertext from the academic programmer's
       point of view.

       Two hypertext browsers are described.  The first is a public-domain
       program which uses an ASCII text file for a data base.  The second is
       an integrated help system developed by the author using some of the
       techniques from the first program.  This system is composed of two
       parts.  Part one is a hypertext compiler which converts an ASCII text
       file into a simple network database.  Part two is a separate browser
       (implemented as a Pascal function) which retrieves text frames from
       the data base and displays them in overlapping windows.

       AN EMBEDDED HELP SYSTEM

       This integrated hypertext help system arose out of the need to provide
       on-line documentation for a data dictionary program planned for
       student use.  The system had to provide information about the
       operation of the program, about the operating environment (MS-DOS),
       and about various kinds of data dictionary notation.  It was
       inevitable that many windows would include references to topics which
       were themselves the subjects of other windows.

       Many programs provide context-sensitive help; indeed, it has become a
       minimum standard for commercial microcomputer applications.
       "Context-sensitive" generally means that when the user requests help
       the application displays a relevant help window based on what the user
       was doing at the time of the request.  If further information is
       needed, the user usually must resort to an on-line index of help
       topics or to the manual.  Hypertext appeared to offer a significant
       improvement over this approach.

       HYPERTEXT ORIGINS

       Hypertext, in one form or another, has been around since the 1960's.
       The central idea behind hypertext is that "everything is deeply
       intertwingled" (Nelson, 87),  Every product of human intellect is
       related to others which influenced its creation.  Additional
       connections may be discovered by others.  Together, these
       relationships form a complex multi-layer network of ideas (nodes)
       connected by associations (links).

       A parallel network of documentation, the "docuverse", embodies and
       describes these products.  The docuverse consists mostly of text --
       books, articles, manuals, ledgers and such.  Unfortunately, the rich

                       Academic Microcomputing Conference, Indianapolis, 1989
                                                                       Page 1



       complexity of the links in the network of ideas is not reflected in
       the docuverse.  Within a single document, the ideas are arranged
       sequentially, with perhaps an index and notes such as "see Chapter 3"
       providing links.  Links to other documents are provided by citations
       and references within the document and by bibliographic data bases and
       indexes without.  Still, text in the docuverse is basically
       sequential.  "Often, there are simultaneous and interpenetrating
       structures  which need to be understood separately and together."
       (Nelson, 87) Computers and video displays offer the promise of
       realizing these complex structures by enriching the links in the
       docuverse.

       Ted Nelson coined the term hypertext to refer to non-sequential text.
       He defined three distinct types of hypertext.  Basic hypertext is a
       network composed of fairly small chunks of text, each concerning a
       specific fact or idea.  Chunks (nodes) are linked to other relevant
       chunks by keywords or notes which enable the reader to navigate from
       chunk to chunk by association instead of following a predetermined
       sequence.  This internal navigation -- staying within the text and
       following links displayed there -- is sometimes supplemented with
       external navigation aids such as a graphic map of the hypertext.

       Collateral hypertext is a way of working with pairs of related
       documents; it shows the similarities, differences, and links between
       them.  A variation of this approach was implemented by Doug Englebart
       and others and demonstrated in 1968 (Englebart).  Figure 1 shows a
       collateral link between this paper and Dream Machines (Nelson 87).

           Jones   Nelson Ŀ
          and by bibliographic data      and to present, this true     
          bases and indexes without.     structure in a way that       
          Still, text in the docuverse   someone can explore it -- for 
          is basically sequential.       learning, for ready reference.
          Often, there are simultaneous = Often, there are simultaneous 
          and interpenetrating          = and interpenetrating          
          structures which need to be   = structures which need to be   
          understood separately and     = understood separately and     
          together.  Ted Nelson coined  = together.  TO FOLLOW          
          the term hypertext to refer    REFERENCES.  Some people want 
          to non-sequential text.        to study one connection       
          
                          Figure 1.  Collateral hypertext


                ͻ
                 This article is a survey ͻ
                 of hypertext systems,     This article is a survey  
                 applications, and design. of existing hypertext     
                 It is an introduction to  systems, their            
                 hypertext and a survey of applications, and their   
                 design issues.            design.  It is both an    
                ͹ introduction to the world 
                                            of hypertext and, at a    
                                            deeper cut, a survey of   
                                            some of the most          
                 Figure 2.  Stretchtext.    important design issues   
                 (The text is from          that go into fashioning a 
                 Conklin.)                  hypertext environment.    
                                           ͼ


                       Academic Microcomputing Conference, Indianapolis, 1989
                                                                       Page 2



       Stretchtext is truly three-dimensional text.  The viewer reads not
       only from left to right and top to bottom, but, using a joystick or
       similar  device, descends into the text as well.  As illustrated in
       Figure 2, the text stretches to accommodate the increased detail
       rising to view.  This approach seems not to have been implemented yet.

       Halasz has suggested a three-dimensional taxonomy for describing
       hypertext systems.  The scope of a system denotes the size of the
       information base it accesses.  The range is from small to vast (all
       the world's literary output, see Nelson, 83).  Browsing versus
       authoring denotes the extent of user involvement.  A browser simply
       presents the information; an authoring system allows the user to
       create or modify a hypertext.  An authoring system must also
       accommodate versioning -- maintaining different copies from different
       authors or different times.  Target task domain denotes the task the
       system supports, from highly specific to generic.

       THE HELP SYSTEM

       A basic hypertext system with only internal navigation was a logical
       choice for the help application.  The system described here is a
       small-to-medium size general-purpose browser.  Although the help task
       is specific, the basic platform could be used for a wide variety of
       stand-alone and embedded browsing systems.

       When the user requests help (as determined by the host application), a
       text window containing information relevant to the current state of
       the program opens.  It contains a brief explanation of what the
       program is doing or what it expects the user to do.  If the
       explanation contains words or short phrases which an inexperienced
       user might not be familiar with, they are displayed in enhanced text
       (either underscored or in a different color).  These terms are
       "buttons"; "pressing" a button causes a window devoted to that topic
       to open.  The text in this window may in turn contain more buttons.

       To press a button, the user first selects it by moving a reverse video
       highlighter bar to it and then presses the "Enter" key.  The
       highlighter is moved with the cursor keys and can only hop from button
       to button.  When a window is first displayed, the highlighter is
       hidden; pressing a cursor key makes it visible.  Moving it past the
       last button hides it again.  During display, the only active keys are
       the cursor movement keys, the "Enter" key and the "Esc" (escape) key.
       The "Esc" key terminates the help display and returns to the
       application.  If more than one window has been opened, the "Backspace"
       key is also active; it removes the current window and backs up to the
       previous one.  The "Esc" key still terminates the entire help session.

       The Programs [1]

       The public-domain program HYPE.PAS (Thompson) provided a
       starting-point for this project.  HYPE is a stand-alone browser which
       uses an ASCII text file as its database.  Lines of text are of two

       
       1.  All of the programs were written in Turbo Pascal for use on
           IBM-compatible PCs.  A reasonable attempt was made to avoid or
           isolate language extensions specific to Turbo Pascal and machine
           characteristics peculiar to IBM-compatible PCs running MS-DOS.
           The use of secondary storage, windows with enhanced text, and
           strings, however, virtually guarantees the use of such features.

                       Academic Microcomputing Conference, Indianapolis, 1989
                                                                       Page 3



       different types.  Lines which contain control information begin with
       ".." while any other character or combination of characters indicates
       a line of displayable text.

       Control lines delimit and identify chunks of displayable text.  A
       chunk begins with "..label" where "label" is a word or phrase which is
       the subject of the chunk.  "..end(label)" marks the end of the chunk's
       text.  Within the text,  Words or phrases which are the subject of
       other chunks are enclosed in backslashes ("\").  Thus, the appearance
       of "\label\" in one chunk of text threads to another chunk.  See the
       example below.

                      ..main
                      This is a sample of HYPE's \source text\.
                      The lines above and below which begin
                      with ".." are \control lines\
                      ..end(main)
                      ..source text
                      The source text in HYPE forms the actual
                      database.
                      ..end(source text)

       It is the responsibility of the text's author to ensure that each word
       or phrase which is enclosed in backslashes is indeed the subject of a
       chunk and that the chunk is identified with an appropriate "..label"
       line.  Control lines also can also be used to specify window
       information and to incorporate text from other files into the
       database.

       When the program starts, it reads the initial chunk of text into
       memory.  Lines are stored as a linked list with each line allocated as
       much space as it requires; little space is wasted.  The first chunk of
       text is then displayed.  The buttons appear in reverse video.  The
       left and right cursor keys highlight each button in turn.  When the
       user presses the "Enter" key, the chunk threaded to the highlighted
       word or phrase is located and displayed in a new window.  Text already
       in memory is searched first.  If the desired chunk is not found, the
       program reads the text file sequentially in order to find it.  The
       "Esc" key backs up to the previous window while The "F10" key
       terminates the program.
       
       Three major changes were necessary to adapt the techniques in HYPE to
       suit the help system.  First, the display mechanism had to be modified
       so that it could be included as part of a host application --
       preferably with as little modification to either the host or the
       display mechanism as
       possible.  Second, it was necessary to modify the approach to storing,
       threading and searching the text in order to control the help system's
       consumption of memory and speed up searches in a large database.
       Third, the host application had to be able to determine which chunk to
       display first; that is, it had to be context-sensitive.

       Together, these changes necessitated the use of a more sophisticated
       database.  The requirements of the database were simple:  It had to
       follow the network model, permitting any node to be linked to any
       other node.  It had to permit reasonably efficient retrieval of chunks
       of text of arbitrary size.  Furthermore, it had to be easy for the
       author of the underlying text base to create and modify both the nodes
       and the links.


                       Academic Microcomputing Conference, Indianapolis, 1989
                                                                       Page 4




       The Hypertext Compiler

       The solution presented here uses a separate compiler to convert an
       ASCII text file similar to that used by HYPE into a hypertext
       database.  The basic unit of information in the system's database is
       the frame (a term borrowed from artificial intelligence).  A frame
       consists of the text to be displayed, information about the window it
       is to be displayed in, the text and screen coordinates of the buttons,
       and database pointers which implement the links.

       A sample text file is shown below.  The lines which begin with a colon
       name the frame and can specify optional window information (not shown
       here).  All succeeding lines (until the next ":" line or the end of
       the file) are assumed to be the text associated with the frame.  Like
       HYPE, backslashes are used to mark the buttons in the text.
       
                       :Intro
                       This is a sample \hypertext\.  It contains
                       only a few \frames\ but will illustrate the
                       concepts.
                       :Hypertext
                       Hypertext is non-sequential text.  The
                       reader navigates among \frames\ by
                       selecting \buttons\.
                       :Frames
                       A frame is a window containing text.  The
                       text may or may not contain \buttons\.
                       :Buttons
                       A button is a word or phrase which appears
                       in enhanced text.  Buttons link the text in
                       one frame with other \frames\.

       Figure 3 is a schematic of the resultant hypertext database with four
       nodes (frames) and six links.  Each frame is stored as one or more
       80-byte records.  The first record for a frame is a window header
       which defines the size of the frame's display window and specifies the
       colors to be used for the various components of the window.  It also
       holds the number of text and button records associated with the frame.
       Text records follow, one record per line of text, followed by button
       records.  A button information block (two per 80-byte button record)
       includes the button's tag (the word or phrase between the
       backslashes), its window coordinates, and the relative record number
       of the window header record of the frame it references.

                        Intro                       Hypertext
                       Ŀ             Ŀ
                       TextPointers>TextPointersĿ
                                      
                                        
                       Frames                     Buttons         
                      Ŀ            Ŀ  
                    >TextPointers<<Ŀ   >TextPointers<
                                  
                                               
                                           
                         Figure 3.  The hypertext database.




                       Academic Microcomputing Conference, Indianapolis, 1989
                                                                       Page 5



       The structure of the database records is shown below:

                 database  =
                     * all records are 80 bytes long *
                     1 {frame} n
                 frame =
                     window-record + 0 {text-record} num-of-text-lines
                     + 0 {button-record} number-of-button-records
                 window-record =
                     * all values determined by the hypertext compiler *
                     border-foreground-color + border-background-color
                     + text-foreground-color + text-background-color
                     + button-foreground-color + button-background-color
                     + hilite-foreground-color + hilite-background-color
                     + window-width + border-type + number-of-text-lines
                     + number-of-button-records
                 text-record =
                     0 {char } 78
                 button-record =
                     1 {button-tag + x-coord + y-coord + frame-file-loc} 2

       Because the source is a text file, creation and modification of the
       database is done with a word processor.  Chunks and links can be added
       and updated without disturbing the host application.  Different
       versions of both the source and the compiled database can be
       maintained.

       The source text is passed through the compiler twice.  On the first
       pass, the compiler scans for frames and builds a symbol table in
       memory.  The symbol table is used primarily to determine where the
       window header record for each frame will appear in the compiled
       database.  The data appearing in the window header record is also
       gathered at this time.

       After the entire source file has been processed, the symbol table is
       scanned for duplicate, undefined, and unreferenced symbols.  During
       this scan, a frame index file is generated.  This is a sequential file
       containing only tags and relative record numbers; it is used by the
       display engine as described below.

       The second pass produces the actual database.  For each frame in the
       text file, the window header data is retrieved from the symbol table
       and written to the database.  Each line of text is written.  As each
       text line is processed, the buttons are located and the backslashes
       are removed from the text.  Each button's tag is looked up in the
       symbol table, the frame's file location is retrieved, the button's
       window coordinates are determined, and the information is stored in
       memory.  After all text lines have been written, the button
       information is retrieved and the button records are written.

       The compiler checks for a number of error conditions and prints an
       error message for each one found.  Table 1 (next page) shows these
       conditions.  All errors result in the display of an error message.
       Fatal errors will abort the second pass.


       The Display Engine

       The display engine is implemented as a single function which accepts a
       frame tag as its input and returns an error code.  This allows the

                       Academic Microcomputing Conference, Indianapolis, 1989
                                                                       Page 6



                 Ŀ
                           Error Condition            Fatal? Action 
                 ͵
                 Frame referred to in text not defined   Y          
                 Frame defined more than once            Y          
                 Unpaired backslash in text              Y          
                 Frame has too many text lines           Y          
                 Line begins with ":", no tag            Y          
                 Source file contains no frames          Y          
                 Frame defined, never referenced         N          
                 Frame contains no text lines            N          
                 Button/frame tag too long               N  truncate
                 Text line loo long                      N  truncate
                 Non-numeric window parameters           N  ignore  
                 
                        Table 1.  Hypertext Compiler Errors

       host application to optionally specify which frame to display first
       and to handle errors by checking the error code.  The procedure is
       self-contained except for the following constants and variables which
       must be declared in the host application:

            hyperText file name constant -- determined by host application
            hyperText file variable -- opened and closed by host application
            index file name constant -- determined by host application
            pointer to frame index -- must remain between calls
            return code -- returns result so host application can handle
             errors

       When the display function is called, it stores the application's
       screen and displays the first frame.  It first checks to see if the
       frame index has been read into memory (index pointer <> nil) and, if
       not, reads it in.  If called with a blank frame tag, it displays the
       frame starting at relative record number 0.  Otherwise, it looks up
       the tag up in the frame index and retrieves the frame beginning at the
       indicated location.  From that point, the display engine behaves as
       described above.  Errors such as missing files or frames and memory
       allocation errors are trapped.  When the help session is terminated,
       the application's screen is restored and the return code is set.
       Figure 4 shows the screen after selecting "frames" from the "Intro"
       window and "buttons" from the "Frames" window.

             Ŀ
              Intro ͻ           
              This is a sample hypertext.  It contains             
              only a few frames but will illustrate the            
              concepts. Frames ͻ
             ͹ A frame is a window containing text.  The 
                         text may or may not contain buttons.      
                   Buttons ͼ
                   A button is a word or phrase which appears      
                   in enhanced text.  Buttons link the text in     
                   one frame with other frames.                    
                  ͼ    
             
                         Figure 4.  The hypertext display.





                       Academic Microcomputing Conference, Indianapolis, 1989
                                                                       Page 7



       NOTES ON WRITING SOURCE TEXT

       The use of a word processor for producing and maintaining the
       hypertext database carries with it both opportunities and problems.
       The opportunities include the use of existing text in a variety of
       applications.  For example, help frames describing the operating
       system, diskette handling, and backup procedures could be easily
       transported from one host application to another.  Also, text written
       for other purposes such as printed documentation can be incorporated
       into the online help system.  (For some cautions about recycling text,
       see James and Brockmann.)

       The greatest problem is chunking the text.  An authoring system
       provides tools for structuring and linking chunks of text while the
       author who uses a word processor is has few tools available.  The
       article by Rubens and Krull which discusses the designing of hypertext
       may prove helpful here.

       The word processor can help in two ways, however.  By setting a line
       length of 78 characters and a page size of 23 lines (the frame header
       line and 22 lines of text), the word processor can help keep text down
       to the desired size.  Also, the program's search function can be used
       to ensure that each button tag has a corresponding frame definition.

       Finally, the use of a help index should be considered.  A frame
       containing nothing but buttons used elsewhere in the text would act as
       an index.  The index would be made available by adding an index button
       ("\Help Index\") to every frame.

       THE FUTURE

       Two enhancements are planned for the future -- addition of mouse
       support and a way of backing up more than one frame.  If the 22-line
       restriction on text proves unworkable, it will be necessary to add
       scrolling to the display engine.

       On a grander scale, a structure editor and modified version of the
       compiler could be added to the browser.  These would enable the user
       to customize the help system by adding tips, reminders, work-arounds
       and such.

       A routine could be added to keep track of users' navigation through
       the system.  Coupled with interviews or questionnaires, this would be
       helpful in evaluating and revising the help text.

       AVAILABILITY OF CODE

       Source code and a compiled version of the compiler are available from
       the author.  Readers who wish to implement the display function in
       another language should refer to the pseudocode in the Appendix.
       Implementation in C or Modula-2 should be straightforward.

       REFERENCES

       Brockmann, R. J.  Exploring the connections between improved
            technology -- workstation and desktop publishing and improved
            methodology -- document databases.  In Text, ConText, and
            HyperText.  E. Barret, Ed.  MIT Press, Cambridge, Ma., 1988.
       Conklin, J. Hypertext: An introduction and survey. IEEE Computer  2, 9
            (Sept. 1987), 17-41.

                       Academic Microcomputing Conference, Indianapolis, 1989
                                                                       Page 8




       Englebart, D. C. and English, W. K. A research center for augmenting
            human intellect.  In Proceedings of the 1968 Fall Joint Computer
            Conference (Montvale, N.J., Fall 1968), AFIPS Press, 395-410.
       Halasz, F. Reflections on NoteCards: Seven issues for the next
            generation of Hypermedia systems.  Communication of the ACM 31, 7
            (July 1988), 836-852.
       James, G.  The ethics of automated publishing systems (a response to
            Dr. Brockmann).  In Text, ConText, and HyperText.  E. Barret, Ed.
            MIT Press, Cambridge, Ma., 1988.
       Nelson, T. Computer Lib/Dream Machines. Tempus Books, Redmond,  Wa.,
            1987.
       Nelson, T. Literary Machines 5/e. Ted Nelson, Swarthmore Pa.,  1983.
       Price, J. Creating a style for online help.  In Text, ConText, and
            HyperText.  E. Barret, Ed.  MIT Press, Cambridge, Ma., 1988.
       Rubens, P. and Krull, R.  Designing online information.  In Text,
            ConText, and HyperText.  E. Barret, Ed.  MIT Press, Cambridge,
            Ma., 1988.
       Thompson, B. and Thompson, B.  Hyping text: Hypertext and knowledge
            representation.  AI Expert 2, 8, (Aug. 1987), 25-26+.

                   APPENDIX -- Pseudocode for the Display Engine

       Display Engine Function (name of frame to be displayed)

       if frame index not in memory then read it in
       if name of frame to be displayed is null then
           file location = 0
       else
           search frame index for file location
       end-if
       display the frame (file location)
       repeat
           get a key
           if no buttons in current frame then
               select
                   case key = escape: clear window stack
                   case key = backspace: remove current, restore previous
                                          screen
            else
               select
                   case key = escape: clear window stack
                   case key = backspace: remove current, restore previous
                                         screen
                   case key = right arrow: move button to next button
                   case key = left arrow: move button to previous button
                   case key = enter: retrieve file location from button info;
                                     display the frame (file location)
           end-if
       until key = escape or window stack is empty
       restore application screen

       display the frame (file location)
       save screen on heap
       read window header record at file location
       open window for this frame
       read and display text lines for frame
       read and store button info for frame
       position highlighter on first button


                       Academic Microcomputing Conference, Indianapolis, 1989
                                                                       Page 9

