*Clipper in Network, tips, suggestions...

                  Clipper in Network, tips, suggestions...
                                    alias
                    Network Guide for Clipper programmer
                      
                                     by
                               Daniel Docekal


Clipper application in network is something that should work on first moment
of start if some guidelines are followed and some conditions fulfilled. If
program you are writing is written with record and file locking and is taking
care of everything that can happened in file sharing, there shouldn't be
problem to run it and use it. Reality is often different, many problems can
rise out from nowhere which will prevent your application from running or
will give difficulties to application's run. What this article will try is to
give helpful hand to those who are starting in networking with Clipper, but
also for those who are swimming in problems with written Clipper application.
I would like to ask anybody who wishes to contribute in this Network Clipper
guideline to do so, then i can collect opinions and release them in new
edition.

NETWORK APPLICATIONĿ


Network application in general must follow idea that it's not only one
program running at the same time. Network application which is using ANY from
resources of network must take good care of fact that in most cases it WILL
SHARE them with other applications running at the same time and requiring
access to the same resources. In most cases it's matter of programming
languages to give tools for networking rather than programmers knowledge.
Clipper is high level programming language in difference from assembler or
basic of C language, therefore is expected that networking behavior will be
covered with filling needs.

NETWORKĿ


Network is something most horrible and bad for every programmer, nightmare
like environment with every problem hidden after not sufficient knowledge and
not experienced people around. Believe or not, it's right definition of
network for beginning. Users of network which are in most cases not even good
equipped with knowledge of MS DOS are of course able to broke all rules which
logically thinking programmer will have in mind when programming, therefore
programmer must start thinking like average (or even, like under average)
users. Difficulty of this mind change is paid in most cases by application
which is created with best wishes and is ending SOMETIME in troubles.

CLIPPERĿ


Clipper is programming language offering enough to start writing network
application. General guidelines are very simple:

ڷ  if database file has to be used at one time by MORE users (workstations
Լ  will be used in all next text) then it must be opened as SHAREABLE

    What this is requiring is very simple, either USE..SHARED or
    SET EXCLUSIVE OFF before all USE which are not equipped with SHARED nor
    EXCLUSIVE keyword.

     Experienced users are probably falling away, but please don't rather
     skip those basic informations, later are coming more handy ones.

    What can be often source of problems:

    WS-A has file FILE opened as EXCLUSIVE and WS-B is trying top open it
    by USE (of any form). As far as WS-A has it exclusive there is no way
    how to access this file.

    WS-A has file opened as SHARED and WS-B is trying open it as EXCLUSIVE
    which cannot because may not gain EXCLUSIVE access to already SHARED
    file.

    WS-A has file opened with fopen() (or just by program like Norton
    Commander), no any other workstation is able to access file.

    USE command is NOT well made for network operations. it's requiring
    little effort from programmer because needs test of NETERR() function
    after USE if required operation was finished:

       USE ADRES SHARED NEW
       IF !NETERR()
           SET INDEX TO ADRESI1,ADRESI2,ADRESI3
       ELSE
           //  error situation
       ENDIF

    This is base fragment of opening a files in network. ANY application
    can be built with this method because it WILL work perfect even without
    network and what it also will - work inside of MS WINDOWs or any other
    multitasking software which also requires network methods of file access.

    Here we do have of course little problem, USE attempt is just made one
    time and not repeated again and again for case that condition will change
    and file will be ready for opening. That's of course trouble of
    programmer which must know in what conditions will program run. 24hours
    running mail server should again and again try to open files, but user
    program just shall only tell to user about problem and then exit. It's
    user's decision what to do, call supervisor or try it later.

ڷ  if any file has to be used at one time by MORE workstations then it must
Լ  be opened also in SHARED mode.

    This must be based on FOPEN() function which IS allowing some MORE
    attributes of open

        0               FO_READ              Open for reading (default)
        1               FO_WRITE             Open for writing
        2               FO_READWRITE         Open for reading or writing
    addition which should be done:
        16              FO_DENYALL           Not allow ANYBODY else access
        32              FO_DENYWRITE         Write access of others denied
        32+16           FO_DENYREAD          Read access of others denied
        64              FO_DENYNONE          Full access of others permitted

    This extension of fopen() function will allow to control if two
    workstations can simultaneously access the same text file (for example).

ڷ  Any updates to database files are requiring record or file locking,
Լ  unless database is opened in EXCLUSIVE mode.

    SHARED open database is suitable for READING records but not for
    writing. Basic conflict which can happened when two workstations will
    try to change database is that both of them will come into the very
    same record. It's something that may not be. Therefore RECORD LOCK is
    required before ANY operation which CHANGES value of FIELD in RECORD
    of database. Record lock is in general result of RLOCK() function which
    will TRY to lock record, but it must not be successful. In general
    record locking mechanism should follow:

    -   try to lock record and if it goes fine just continue with
        operations
    -   if request for locking failed there are several possibilities:

        -   try it for INFINITE time
        -   try it until user will stop it with STOP key
        -   try it for DEFINED time (let say one minute)
        -   try it for DEFINED number of attempts
        -   try FIX reason of failed record lock request
        -   stop immediate and tell user and not continue

        In general there are combinations usual given by:

        -   it's trying some time and user can stop it by pressing
            STOP key

        Many from third part add-ons are offering replacement for
        rlock() function, therefore let them to do job. BBS systems
        and Compuserve will have for sure enough of those examples...

    The same as above written is valid about locking of WHOLE file. Locking
    of record is needed in case of operation which will change ONE record,
    locking of FILE is needed on operation which will be changing WHOLE FILE
    or in cases when is NEEDED to be sure that no other workstation can
    jump in middle of counting values and change some of them just second
    after they were counted. Function for file lock is FLOCK() and works
    exactly the same way as RLOCK(), only effect is on COMPLETE file. After
    FLOCK() issued is file still accessible for READING, but no-one can write
    to any from file's records. All rlock() or flock() operation tried by
    other workstation will be unsuccessful. Requirement for successful
    flock() is therefore:

    -   no one has locked ANY from records in database
    -   no one else has flocked() database

    General required are flock() or rlock() in functions/commands:

        rlock()                             flock()
        
    :=                                  APPEND FROM
    DELETE (for one record)             DELETE (for more records)
    RECALL (for one record)             RECALL (for more records)
    REPLACE (for one record)            REPLACE (for more records)
    @..GET (operating on field)         UPDATE ON
    fieldput()                          dbeval() (changing records)
    fieldwblock()

    Considerably required is flock() in:

        SUM                             COPY TO         LIST
        AVERAGE                         JOIN            DISPLAY
        TOTAL                           LABEL
        dbeval() (counting values)      REPORT
        LOCATE and CONTINUE             SORT

        Those commands DO NOT change values in database at all, but are
        dependent FROM values in database. In some cases it's therefore
        good to think if it's normal and good that someone else will be
        able change values in database during some counting processes.

    Clipper is placing OWN method of record locking on file which is NOT\
    compatible with methods used by dBASE, FOXBASE, "C" language or others.
    in general is this method based on locking one byte areas beyond any
    possible end of file. This method is picked up because of needed side
    effect, records are still READABLE even when lock is placed on them.
    In normal case, when record is locked by locking AREA of file in which
    record is, any locked area UNREADABLE on the same way as no-one can
    WRITE to them. Clipper record which is LOCKED _must_ be readable from
    other workstations, therefore locking is done by simulation of fake
    records beyond possible end of file. All of this has next bad side
    effect - any other software which wants to access Clipper files at the
    same time as Clipper will have to follow exactly the same method of
    record locking, otherwise it can only make big mess from Clipper database
    files.

ڷ  Several database operations are requiring EXCLUSIVE use of file and
Լ  without this they will not operate at all.

    EXCLUSIVE use of database is giving application right to do ANYTHING
    application wants with database. It's therefore required for some
    Clipper operation. EXCLUSIVEly used database is not allowed even to
    open from other workstations. From this point of view is power of
    "locking" in this order:

        EXCLUSIVE used      no one else can ACCESS database file, database
                            MAY NOT be open by other workstations, otherwise
                            is impossible to make USE...EXCLUSIVE

        flock()             no one can CHANGE thing in database, rlock() or
                            flock() from other workstations will fail, no one
                            can EXCLUSIVE USE database

        rlock()             no one can CHANGE record in database, rlock() or
                            flock() from other workstations will fail, no one
                            can EXCLUSIVE USE database

        SHARED used         everyone can open database, use flock() or
                            rlock(), but no can EXCLUSIVE open database

    Functions and commands for which EXCLUSIVE is required are:

        INDEX (*)                   PACK
        REINDEX                     ZAP
        dbcreateind() (*)           dbreindex()

    INDEX/dbcreateind() are partially excluded from this list in case that
    on one is interested in multi-use of created indexes. INDEX/dbcreateind()
    are generating index files which are not available for other workstations
    by default, therefore EXCLUSIVE use of database is NOT required. But
    there is one important detail:

    Index file generated on non-exclusive (or non-flocked()) database is
    quite unstable in case someone just changed some values of keys in
    middle of creating index key, therefore as general requirements can
    be given:

      Database _must_ be either file-locked or EXCLUSIVE used for
      case of index generation. Only with this one can overcome possible
      index destruction with later IE1210 (well know to most of us).

ڷ  Appending of new record is not requiring rlock(), but it's requiring
Լ  test of success.

    APPEND/dbappend() are used for ADDING ONE new record at END of database.
    INSERT command which is present for example in FoxBase doesn't exist
    in Clipper and therefore problem with new records is eliminated only to
    appending at END of database.

    APPEND/dbappend() is by default trying to make one NEW record and then
    place rlock() at this record. If something fails (two workstations
    are doing dbappend() at the same time) dbappend() will not succeed and
    NETERR() will be again returning information about error. Good sequence
    therefore is:

        dbappend()
        while neterr()
            dbappend()
        enddo

    This sample of course is limited in never ending waiting for append
    of record, but what was told about flock() can be applied also in
    custom appending function. There are therefore few points about
    appending:

    -   record is ALWAYS locked if dbappend()/APPEND _was_ successful,
        no need for additional RLOCK(), it's wasting of time
    -   record is EMPTY after APPEND and MUST NOT be currently updated
        in database YET nor in index files

    There is no need for checking success of APPEND/dbappend() in cases
    of USE...EXCLUSIVE or flocked() databases, but considerable is fact
    that flock() active _before_ dbappend() is replaced with rlock()
    active _after_ dbappend() (see following comments).

ڷ  Clipper is limited in number of rlock()/flock() on ONE per DATABASE
Լ  and per WORKSTATION (program).

    Limitation of Clipper is that there can be ONLY ONE rlock() or flock()
    per ONE database and workstation. One program can therefore place just
    one lock per one open file. It's somehow limitation for many
    applications, therefore many add-ons for Clipper are offering multiple
    locks per file (DBFSIX, NETLIB for example).

    Also important point is that ANY RLOCK() or FLOCK() is removing
    previously placed RLOCK() or FLOCK() in the same file, therefore
    programmer MUST be careful when placing locks in the same file.

ڷ  EVERY rlock() or flock() should have own dbunlock() or dbunlockall()
Լ  function to assure proper unlocking and proper updating of data

    From moment of placing rlock() at record is very important to remember
    that ANY changes made to this record MUST NOT be visible for other
    workstations, even they have READ access to this record. They can see
    still OLD values and old KEY values in index! dbunlock() is only one
    which is REQUIRED to make changes visible. If this is not true in your
    network, then believe or not - your network or your DOS version has
    VERY important BUG, don't blame Clipper.

    On the same way as WHILE has proper ENDDO, IF has proper ENDIF, CASE
    proper ENDCASE it's very important to have for every RLOCK() proper
    DBUNLOCK(). There is of course possible to use DBUNLOCKALL(), but that's
    solution not very clean - in some cases it can unlock something that
    should stay YET locked.

    Thinking that NEXT rlock() (or flock()) will make dbunlock()
    automatically is of course right, but it has one side effect, one can
    miss easy proper lock-unlock combination in case of longer source files
    with more actions between flock() and later flock() again. It's of course
    easy to use it in way:

        while !oef()
            if rlock()
                ADRES->CITY := upper(ADRES->CITY)
            endif
            dbskip(1)
        enddo
        dbunlock()

    This sample is CLEAR on FIRST SIGHT, therefore nothing against this way
    of coding program, but in case that this source will become FEW pages
    of lines, it's much better use dbunlock() on right place. Time saving
    given by again and again using dbunlock() separately from rlock() are
    not very important because rlock() in above example will have to make
    unlock() of previous record anyway (with subsequent update actions).

ڷ  Using of dbcommit()/COMMIT can be sometime too much
Լ
    dbcommit()/COMMIT/dbcommitall() are in most cases very powerful.

    In processing od DATABASE files of Clipper are several levels of
    buffering (CACHING). First there is Clipper internal buffering of data
    in EMS or conventional memory (if available) which buffers complete
    records of database when possible, then there is DOS buffering (and
    Caching) which works between disk hardware and Clipper requests. It
    uses either buffers created by CONFIG.SYS command BUFFERS= or cache
    buffers created by CACHE software (SMARTDRV.SYS in standard). In
    network environment there is in some cases network buffering at
    workstation which is NOT active for Clipper files opened as SHAREABLE
    and which IS active for EXCLUSIVE open files. COMMIT/dbcommit()
    possibility of CLipper is related to those buffers:

    -   DOS buffers are flushed TO DISK in versions 3.3+
    -   CLIPPER buffers are flushed TO DOS in versions under 3.3
    -   SERVER buffers are flushed to SERVER disk in Netware 3.11 with
        NetWare Shell of versions 3.22+

    It's generally known and approved that dbunlock() is far enough to
    make visible record changes in all normal cases. Committing data is
    in most cases even not very good idea because it can make additional
    unwanted traffic inside of server between server CACHE memory and
    physical disk.

    Warning anyway, there ARE revisions of DOS which are affecting file
    handling functions and which HAVE significant problems with file and
    record related function (not even locking) when running under Networks.
    One sample can be TANDON DOS version 3.3 which is exception from case
    that dbunlock() is enough for making updates visible, dbcommit() IS
    required in this case to SEE dbunlock() activated.

ڷ  DOS based networks are REQUIRING use of SHARE.EXE
Լ
    Rule about DOS based network is that ANYTHING that is coming to be
    SERVER and will have to share files for other workstation MUST have
    loaded SHARE.EXE program BEFORE any other network programs OR in
    some SPECIAL order when it's specified by documentation for network.

    What is coming to be problem is that SHARE.EXE by default is ready
    only for SINGLE user machine with limited number of files opened.
    MultiUser SHARE.EXE requires some changes in parameters. SHARE.EXE
    has TWO parameters and common knowledge of them is not very good.

    /F:n is used to specifying SIZE OF FILE NAME POOL which SHARE.EXE will 
    allocate. Standard size of this pool is only 2048 bytes and therefore 
    very small. ONE opened file from server will in general require about 
    20bytes for name + 20 bytes (exactly 11bytes of information and full size 
    of path+name) for informations, and that means that 2048 is space for 50 
    files. Common Clipper programs are easily opening this number of files 
    PER PROGRAM.  Therefore take number of workstation accessing your server, 
    count let say 60 files to open from every workstation and result will be 
    <n>*60*40. Be careful in thinking, because MEMORY WILL BE ALLOCATED from 
    CONVENTIONAL memory of server (in most cases) and therefore CAN limit 
    significantly any other operations.

    /L:n is used for specifying SIZE OF LOCK POOL which SHARE.EXE will
    allocate. Standard size of this POOL is allowing 20 locks. That's also 
    not good enough because Clipper program CAN make the same number of
    locks as is number of files it's opening. Therefore get number of 
    workstations multiplied by number of files (<n>*60 in sample case) and
    that's /L:n which should be used. This parameter is NOT increasing
    memory usage so much, therefore don't worry about it.

ڷ  DOS based network are requiring .EXE and .OVL files marked as READ ONLY
Լ
    DOS based network (and may be that some others also) are requiring that
    .EXE and .OVL files from Clipper program will be marked as READ ONLY to 
    allow more than one workstation run the same file at the same time. It's 
    problem of DOS default method of opening non-Read-only file which is 
    making impossible to open it twice by default. Therefore just don't 
    forget to mark them with ATTRIB as +R.

ڷ  WORKSTATION is NOT requiring SHARE.EXE
Լ
    ANY workstation running in ANY network doesn't need SHARE.EXE, don't
    worry about clever recommendations to load SHARE.EXE on all workstations 
    of your network. It's wrong. It will only take up your memory (which is
    critical in many cases) and will do nothing, or worse will conflict with
    your programs and make problems. 

    From this rule is only ONE exception. Microsoft DOS 4.01 running 
    workstations with HARD DISK bigger than 32MB _are_ _requiring_ SHARE.EXE 
    to be loaded. IF it's not loaded computer will run smoothly until disk is 
    started to be allocated in area over 32MB and some stupid program using 
    old way of file handling (like SideKick for example) will be started. 
    After it harddisk of workstation is overwritten...

ڿ  Computers or workstations running MULTIUSER software and planning 
  running multiple Clipper programs over the same data NEED SHARE.EXE

    Anything running DesqView, MultiDos, MS Windows or anything else which is 
    providing multi tasking capabilities should load SHARE.EXE before start
    of multiuser software otherwise lower copy of DOS will not be able to 
    handle file/record locking and shared accessing. In general multi task 
    environment SHOULD BE TAKEN AS EQUAL TO SIMPLE NETWORK, therefore 
    anything written before or after this statement can be active...

ڷ  Novell Netware SHAREABLE flag is DANGEROUS, DO NOT try mark your files
Լ  as SHAREABLE (FLAG +S)

    SHAREABLE flag of Novell Netware files is something that MUST be avoided 
    to use for Clipper data files. SHAREABLE flag has for Novell Server only 
    one meaning that it should ignore any file/record locking on given file 
    because applications using file will maintain OWN method of locking and 
    checking. As far as CLipper is still using standard method of locking, 
    only far beyond end of file it means that any locks can went well wrong 
    when file is marked Shareable. Please try understand this statement as 
    copied out from official Novell statement which can be made available and 
    therefore please understand this as end of myth of Shareable attribute.

    There is no reason to mark Clipper database file on any special way in
    any known network, especially not in Novell Netware.

ڷ  Be sure that Clipper application users have enough rights in directories
Լ  where CLipper program will be working.

    Often problem is that there is not enough rights for Clipper .EXE file to
    run and to access his data. In Clipper it's not very difficult, just give 
    ALL rights in directory where .EXE is running, in TEMPorary directory and
    in all directories where database related files are located. I didn't yet
    made experiments with giving less rights, but generally it shouldn't be
    a problem because CLipper application will need anyway separate direc-
    tories for program and data and giving rights in secured network will not
    be very big problem.

    There is also little bug in OLDER versions of Netware. FILES in Netware
    are ALWAYS created WITH OWNER. It's good idea to create ALL files AS
    SUPERVISOR because this one will NEVER be deleted. IF is installed 
    limiting of space by users it can be that file create by let say user
    named FRED will give strange errors for one simple reason. User FRED
    is already deleted from server and therefore space available checking
    mechanism is unable to determine if it can append new records to file
    and will give "Disk Full" messages even when there are MEGABYTES of
    free space. Just change owner of files to SUPERVISOR or somebody who
    is existing in network and problem will be solved.

ڷ  Don't forget for TEMP drives!
Լ
    TEMP drive specified for application in SET CLIPPER can be source of
    problems when someone will forget to create this directory or don't give 
    rights for users in this directory. Clipper application WILL create 
    sooner or later some swap and temporary files in there and if it will be
    unable to locate directory? It will just jump out of program without 
    possibility to catch this error!

    It's GOOD idea to collect ALL temporary files in separate directory 
    called for example TEMP (or $TEMP$) because they are garbage in meaning
    of network. If workstation HAS harddisk or RAMDISK is good to redirect
    those temporary files on them. It can save LOT of megabytes transferred 
    through network and also therefore can speed up your application.
    
ڷ  What about marking some directories as PURGE in 3.11 Netware 
Լ
    Since Novell Netware 3.11 is Novell server SAVING _all_ deleted files
    for undeletion. What it means is usage of disk space on server (he will 
    purge them automatically when needed) and also time required for those 
    and later for PURGE operation. It's nice to be able undelete by accident 
    deleted file two weeks ago, but in some cases it's good to use PURGE flag 
    for directories (command is FLAGDIR <dir_name> P). Directories marked 
    with this Purge flag will not try to keep files and all deleted will be 
    automatically immediately purge. Good candidate for FLAGDIR P command is 
    TEMP directory in ALL cases, otherwise Novell Netware server will have 
    after week of working with your Clipper program thousands of files stored 
    in salvageable state and for nothing.

ڷ  Quest for file handles
Լ
    Coming into troubles with file handles is very easy. General rules will 
    be described just step by step to help cover this kind of problems.

    First of all, Clipper is VERY hungry for file handles (all other xBase
    derivates are anyway also because of using the same technique of file
    management), one therefore must be able calculate needs of his own 
    program for file handles:

    -   one file handle for .EXE file                   *ALWAYS*
    -   one file handle for every .OBL file             *WHEN USED*
    -   two file handles for swapping files             *ALWAYS*
        this can be even more when one allows more,
        SET CLIPPER=DYNF:<n> is used for this
    -   one file handle per .DBF file                   *ALWAYS*
    -   one file handle per .DBT .FPT file              *WHEN USED*
    -   one file handle per every .NTX .IDX .NDX        *WHEN USED*
    -   one file handle per every .CDX                  *WHEN USED*
    -   one file handle per every fopen(), fcreate(),
        report, label, set alternate, set print to,
        set device to, append from, copy to, copy file,
        copy structure, create, create from, save, 
        restore, sort, type, update                     *WHEN USED*
    -   one or more handles when CLD.EXE or CLD.LIB     *WHEN USED*
        is used (.PRG is open at least).

    Most file hungry moment from all those is fact of .DBF needs another file 
    handle for any memo fields (.DBT file) and then for EVERY index related 
    to one .DBF is required additional file handle for .NTX (or .IDX or .NDX) 
    file. It's most limiting issue in using file handles and it can be solved 
    only by two ways - one is requiring structural changes of applications, 
    second one using of SIX RDD which allows using of .CDX file which can 
    include ALL (and even more) .NTX in one file handle... (our main 
    application which is user part of FAX/TELEX/MAIL program was using 45 
    file handles with DBFNTX driver, now it's using 26 file handles with 
    DBFSIX driver).

    After counting this "BIG" number of files which application is using is 
    good to write this number down and start setting up system to allow use 
    of them.

    First place where one meets FILE HANDLEs specification is CONFIG.SYS,
    command FILES=<n> is giving maximum number of file handles which may be 
    used in computer (workstation) and is by default ONLY 20, therefore for 
    Clipper needs to be a bit more. Every file handle from FILES= will of 
    course grab additional memory, therefore is not good to try allocate TOO 
    much of them. But for sure it MUST be _more_ than "BIG" number from 
    Clipper application. In general "BIG"+5 is good for <n>, but it's true 
    for standard MS DOS without TSR programs (Norton Guide engine NG.EXE is 
    for example using one file handle for NG.INI file!). 4DOS replacement of 
    COMMAND.COM can keep open additional file handles for swapping files, 
    therefore "BIG" and <n> should be more far from each other or "BIG" 
    should be bigger than real application need.

    One thing is for sure anyway, LOCAL running application WILL NEVER be 
    able open more than <n> files (allocated in CONFIG.SYS).

    Little warning, QEMM has nice utility called FILES.COM which CAN create 
    additional file handles in High Memory and save conventional memory for 
    application. In this case is FILES=8 most sufficient in CONFIG.SYS and 
    later FILES <n>-8 in AUTOEXEC.BAT will do additional allocation of 
    required. But warning, FILES utility from QEMM is _not_ compatible with 
    file handles used in DR DOS 6.0 and even if it will not complain it will 
    NOT give required file handles and Clipper application will be magically 
    refusing to work even if all looks fine.

    SECOND place where is requirement to specify file handles in command 
    line for your .EXE (or in SET CLIPPER or in linker burn-in script). 
    Clipper written application is using Microsoft like run engine which by 
    default is getting only 20 file handles from DOS under. Call for DOS 
    service to getting additional file handles is required for opening more 
    than 20 of them. Clipper is not so clever to just call at beginning for 
    maximum (255) and is requiring programmer (user) to give request for this 
    number. SET CLIPPER=F<x> or //F<x> for .EXE run is therefore REQUIRED to 
    get more than 20 file handles available. 

    GOOD TIP. If your linker is allowing burn-in of Clipper environment, just
    burn in F255 and forget about this nasty and stupid parameter. It will 
    not give any harm or extra memory used (yes, ONE byte per file handle 
    rounded to 16bytes boundary which is harmless in Clipper memory usage) 
    and no longer thinking about which F<x> parameter is active now.

    Until now we were talking about SINGLE user/task environment on single
    PC. Different situation is coming in network and that's what we should
    discuss especially.

    Novell Netware is most often used network, therefore we can start there.

    First, Novell Netware 2.x servers are GENERATED with FIXED number of
    available file handles and MAXIMUM possible is only 1000 (that's 
    sufficient for 16 workstations opening 60 files for example!). Because
    using of more available file handles on servers requires more memory in
    server some technicians generating 2.x servers were afraid of putting 
    more MB of memory and generated servers with 200 handles available (for 
    example). That's something that can break your plans to run application 
    from CLipper in give server immediately. Therefore make always check
    for how much of files is server generated. Ask supervisor of network for 
    look into SYSCON utility, Statistics submenu and General statistics. All 
    number are there, even number talking about current and peak number of 
    opened files. From those three numbers is possible start thinking about 
    ability to run Clipper program. Example:

            Maximum     Peak        Current
                600         560         320

        This will be not very good place for running CLipper program, unless 
        this program will be the one running at most workstations at the same 
        time which will slow down normal workload of files. 560 in peak is 
        VERY near of configured MAXIMUM and that's a sign of need to change 
        maximum value.

    Problem. Novell 2.x server is GENERATED from about 30 floppies and then 
    LOADED into harddisk by special loader, ANY changes of parameters of 
    server's operating system are requiring NEW generation based on OLD 
    definitions. In most cases 2.x server is impossible to generate again 
    because original supporter has not floppies anymore or doesn't exist.

    Novell Netware 3.11 server is NOT limited in number of files for opening 
    in generation or maximum given by exact number. Only available memory of 
    server is limitation of number of files to open (and speed of course), 
    therefore if your application is going to be running on 3.11 Network and 
    server, no problem at all.

    Every workstations in Novell Netware network will have available 40 file 
    handles by default WITHOUT giving any specification. If there is 
    requirement for more it's needed to place request for it in NET.CFG (or 
    SHELL.CFG) file. SHELL.CFG is _old_ used name which is replaced by 
    NET.CFG in last days. Command to place in there is FILE HANDLES=<m> and 
    it will assure ability to open maximum of <m> files FROM NETWORK.

    IMPORTANT NOTE. Files allocated in CONFIG.SYS by FILES=<n> and files 
    allocated from network by FILE HANDLES=<m> are INDEPENDENT from each 
    other. <n>+<m> MAY NOT exceed 254 and in old versions of Netware Shell 
    (NETX) it will NOT give any warning when this is happening and will 
    allocate just default (40 or even less). New netware Shell is giving 
    warning and refusing to load when <n>+<m> is over 254.

    <n>+<m> is also NUMBER OF FILES AVAILABLE for OPEN _AT_ workstation,
    but <n> is talking about LOCAL FILES and <m> about NETWORK FILES (files 
    from network). This fact can also affect CLipper application with F<x> 
    setting by the way (that's why F255 is most nice solution also). From 
    this comment is also coming out one other warning, NUL device which is 
    using file handle when opened more times is using ALWAYS local file 
    handle and not Netware file handle when used for testing number of file 
    handles available for opening.

    Non-Netware networks can have similar limitations of files available to 
    open in server. Maybe someone else is able to elaborate on this for Lan 
    Manager networks, but we do not support them nor use them (for enormous 
    slowness) and it's hard to say.

    DOS based network servers are using different ways of overcoming problem 
    that MS DOS allows maximum of 255 file handles (file handle is 8bit 
    number only) and therefore is ALWAYS matter of configuration of server 
    HOW much of files it will allow to open simultaneously. Some from DOS 
    based networks are even limited to 255 files maximum (old versions of 
    Lantastic were for example) or have significant difficulties when trying 
    to successfully and reliable work with more than 255 files. Always check 
    with network supervisor for setting and count them in Clipper application 
    installation.

ڷ  Versions of network support software or network operating systems
Լ
    There was already article devoted to Novell Netware 3.11 patches and 
    their importance in correct working network and also article about 
    Netware Shell releases and their troubles. Both are available in 
    Clipper BBS Magazine number 14 and are also uploaded to Compuserve's
    forum CLIPPER and DBADVISOR as text files free to download.

    In general, IF application is not running properly be sure to take a look 
    and check if networking software is latest version or it's patched with 
    latest available patches. In world doesn't exist software WITHOUT bugs 
    and shortcuts and therefore patches are something very usual in serious 
    companies (NANTUCKET was exception because waiting for 5.01 patch of 
    CLipper 5.0x was very, very long). Example can be taken on Novell Netware 
    3.11 operating system which has few very important bugs related 
    immediately with Clipper (and any other database applications) which can 
    lead in major problems. On the same way it's VERY important to use latest 
    Shell programs (Shell programs are those who allow workstation to access 
    network files and services - those like IPX/NETX or ODI/NETX in Novell 
    Netware, NET in Lantastic and so on) are of latest versions also. MANY 
    from problems are related to old version of Shell program which has well 
    known and already fixed bugs. In Novell Netware it's matter of having 
    just latest 3.26 version of NETX because before this one is one important 
    problem related to VISIBILITY of updates in network...

ڷ  Memory problems
Լ
    We all know that CLipper is very tight with memory usage. RTLINKed 
    application is maybe better to not describe so much because standard 
    RTLINK distributed with Clipper is not very good in handling big programs 
    with C, ASM and CLIPPER combination. Fortunately we do have our BLINKERs, 
    WARPLINKs which are better in memory handling, but still it's not 
    optimal and it's caused by Clipper philosophy.

    Clipper us using VERY large runtime engine which is present in every .EXE 
    created even if it's one line program. It makes some base of memory usage 
    which CANNOT be reduced very much. All additional CLipper code and other 
    libraries are sooner or later coming to create monstrous applications with 
    .EXE file over 1MB in size and load sizes over 350KB (load size is size 
    reported by linker when finished work on bunch of .OBJ and .LIB files). 
    Those monstrous programs are requiring about 350KB of just memory loaded 
    with programs and at least another 100KB of memory used for program to 
    have it running. It comes in 450KB of memory needed for POSSIBILITY to 
    run program. That's not so big problem in SINGLE user computer because 
    typical free memory in there can be some 550KB in older DOS versions and 
    some 600KB in DOS 5.0 with DOS loaded HIGH. Where problem is coming is 
    network because Shell drivers are requiring some memory for work also. 
    Free memory for Clipper program then can end in between 50KB till 90KB 
    less than in non-network case. And that can become critical in monstrous 
    megabyte applications. 

    This part of our long article will therefore try to give little light and 
    magic into memory issue. Don't expect too much, it's just list of 
    practical experiences from own life.

    -   reduce size and load size of Clipper application

       Do not use PLL for living versions of your applications. It's saving 
        of course size on your disk but .PLL is requiring some additional
        impact on load and run size of application

       Do not use incremental linked application in living environment. 
        Sounds silly, but that's often happening that application linked with 
        INCREMENTAL ON (we are NOT talking about RTLINK where incremental 
        link is now working at all) is by mistake used. Incremental linking 
        is making .EXE file bigger by padding all functions by additional 
        empty space for later replacing relink. It requires therefore more 
        memory for running of program (much more).

       Do not use application linked with debugger or debugging code for 
        living environment. /B switch of CLIPPER.EXE will generate object 
        file which will force linkers to make .EXE file which is bigger and 
         also requires more memory. Be sure that NO ONE from .OBJ files in 
        your link script (or in your libraries) is compiled with /B because 
        it's enough to have one to have included all debug informations. It's 
        good (and needed for testing), but unwanted for living environment.

       Reduce size of your .OBJ code by excluding line number informations 
        for living environment. /L switch of CLIPPER.EXE is required for 
        doing so. Side effect of this is of course that all ERROR reports 
        will have line number ZERO, but living application shouldn't produce 
        so much errors anyway. Line number is TWO BYTES in .OBJ for EVERY 
        line of your SOURCE included in source and that can mean significant 
        saving of space!

       In debugging cycle link CLD.LIB into your application instead of 
        using separated CLD.EXE. Save of memory is NOT SO BIG, but can make 
        significant difference in ability of debugging own code. Don't forget 
        in this case that CLD.LIB _must_ be included in linking script as 
        FILE CLD.LIB and not as SEARCH CLD (or ALLOCATE CLD, LIB CLD) because 
        it's nice present from Nantucket. But then, DO NOT forget to REMOVE 
        CLD.LIB from link script for living environment application, 
        otherwise anyway inactive CLD.LIB will take additional memory.

       Don't think that just specifying list of libraries and object modules 
        for your linker will take care of proper and optimal overlaying. Most 
        from libraries are combination of CLipper and C/ASM code and should 
        be overlaid explicitly by specification. Most from CLIPPER.LIB can be 
        also forced to overlay and all this can save lot of memory. Don't 
        forget in this case of course that invoking static overlay manager 
        will require some additional memory (RTLINK about 18KB) and therefore 
        effect from overlaid code must be bigger than impact of overlay 
        manager present in .EXE code. Some examples of scripts for BLINKER 
        1.51 are here:

        TELEPATHY - communication library
        Ŀ
           search TPROOT                               
           BEGINAREA                                   
               allocate TPOVL                          
           ENDAREA                                     
        

        NOVLIB - Novell Netware library
        Ŀ
           BEGINAREA                                   
               allocate NLEXTEND, NLMAIN               
           ENDAREA                                     
        

        EXTEND - Part of Clipper libraries
        Ŀ
           BEGINAREA                                   
               ALLOC extend                            
           ENDAREA                                     
        

        CLIPPER - Main Clipper library
        Ŀ
           SEARCH CLIPPER                                                
           # tbrowse are forced in ROOT otherwise it's too slow          
           MOD tbrowse0, tbrowse1, linelen                               
                                                                         
           # needed for NETLIB library                                   
           MOD txopen, eve, filesys                                      
           # those marked with ## is possible to overlay too but         
           # main issue can be slowness of system in some cases          
           BEGINAREA                                                     
               MOD accept, acopy, adel                                   
           ##  MOD aeval                                                 
               MOD ains, appinit, atail                                  
           ##  MOD box , color                                           
               MOD cmem                                                  
           ##  MOD date                                                  
               MOD dbcmd0, dbcmd1, dbcmd2, dbcmd3, dbcmd4, dbcmd5        
               MOD dbcreate                                              
           ##  MOD dbf0, dbf1, dbfdyn                                    
               MOD dbjunct, dbnubs, dbstruct, delimdyn, dlm0, dlm1       
           ##  MOD dtx0, dtx1, dtxdyn                                    
               MOD diskio , dynina                                       
           ##  MOD errorsys                                              
               MOD errsys0, errsys1                                      
           ##  MOD event, extend, fget, field                            
               MOD getenv, gets0, gets1, gets2, gx, joinlist, lupdate    
           ##  MOD maxrow                                                
               MOD memory, mrelease, msave, net, nmsghdr                 
               MOD oldbox, oldclear, philes, run, saverest, scroll       
               MOD sdf0, sdf1, sdfdyn, send, seq                         
           ##  MOD set, setcurs, sortbloc                                
               MOD sortof, squawk, startsym                              
           ##  MOD symsys                                                
               MOD tb                                                    
           ##  MOD txopen      ##  NETLIB needs it in root               
               MOD vall                                                  
           ##  MOD vblock, vdb                                           
               MOD vdbg, version                                         
           ##  MOD vmacro, vnone, vops, vpict, vterm, workarea, wrt2err  
               MOD xmacro                                                
                                                                         
               MOD _afields, _appini, _atpromp, _century                 
               MOD _dbcopy, _dbdelim, _dbflist, _dbghelp                 
               MOD _dbginsp, _dbgmenu, _dbjoin, _dblist                  
               MOD _dblocat, _dbsdf, _dbsort, _dbstrux                   
               MOD _dbtotal, _dbupdat, _fledit                           
           ##  MOD _getmsg, _getsys                                      
               MOD _helpkey, _input, _readvar,                           
           ##  MOD _savescr, _setfunc                                    
               MOD _setta, _text, _wait                                  
                                                                         
               MOD memoedit, memotrain, memoread, memowrit               
               MOD memoline, mlcount, mlpos                              
                                                                         
               MOD is, examplec, hardcr, directry, copyfile              
               MOD typefile, diskspac, achoice, osdate                   
               MOD examplea, initexit, strtran                           
                                                                         
           ENDAREA                                                       
                                                                         
           SEARCH terminal                                               
           # i do not use DBFNTX anymore, therefore i removed it out     
           # SEARCH dbfntx                                               
        

        DBFSIX - RDD replacement of DBFNTX
        Ŀ
           FILE DBFSIX.OBJ                                               
           #FILE DBT.OBJ                                                 
           FILE DBCREATE.OBJ                                             
                                                                         
           # Speeds up index creations when marked out                   
           #MODULE SXCREATE                                              
           # using standard overlay link from Successware                
           @\dbfsix\dbfsix.lnk                                           
                                                                         
           BEGINAREA                                                     
               ALLOCATE DBFSIX                                           
           ENDAREA                                                       
        

        NETLIB - Networkin library
        Ŀ
           MOD A0FIX, C1LOCK, A0BREAK                                   
                                                                        
           BEGINAREA                                                    
               allocate NL150, HORIZ50                                  
           ENDAREA                                                      
        

        Dr.Switch ASE - swapping and making program TST is ALL in ROOT
        Ŀ
           MODULE ASE                                                  
           SEARCH ASE                                                  
        

        NANFORUM TOOLKIT - best public domain library
        Ŀ
           BEGINAREA                                                   
               ALLOC nanfor                                            
           ENDAREA                                                     
        

        BLINKER general script for LIVING environment application
        Ŀ
           # use a fixed overlay area instead of the free pool         
           BLINKER OVERLAY FIXED                                       
           #                                                           
           # allocate 60K to the overlay pool                          
           BLINKER OVERLAY OPSIZE 60                                   
           #                                                           
           # full link only                                            
           BLINKER INCREMENTAL OFF                                     
           #                                                           
           # call stack size and fixes stack size                      
           BLINKER PROCEDURE DEPTH 55                                  
           STACK 5120                                                  
           #                                                           
           # overlay in EMS aare disable because of using EMS in others
           BLINKER OVERLAY PAGEFRAME OFF                               
           #                                                           
           # and use upper memory blocks for overlay pool to save memor
           BLINKER OVERLAY UMB ON                                      
           #                                                           
           # Burn in environment                                       
           BLINKER EXECUTABLE CLIPPER F255;CGACURS;SWAPPATH:'TEMP\'    
           #                                                           
           # allow users override environment with SET CLIPPER         
           BLINKER ENVIRONMENT OVERRIDE                                
           #                                                           
           # set different than SET BLINKER                            
           BLINKER ENVIRONMENT NAME APPSET                             
           #                                                           
           # don't delete .EXE file when error                         
           BLINKER EXECUTABLE NODELETE                                 
           #                                                           
           # burn-in serial numbers                                    
           BLINKER EXECUTABLE SERIAL (c)NETCONSULT 1992, (DD)          
           #                                                           
           # display list of all duplicates                            
           BLINKER MESSAGE DUPLICATES                                  
           #                                                           
           # don't blink, it's bothering me                            
           BLINKER MESSAGE NOBLINK                                     
           #                                                           
           # don't search for default libraries!                       
           NODEFLIB                                                     
        

        BLINKER general script for TESTING of application
        Ŀ
           # uncomment this when needed listing on the screen          
           #VERBOSE                                                    
           #                                                           
           # uncomment this when needed debugging/profiling            
           #DEBUG                                                      
           #                                                           
           # uncomment this for creating a MAP file                    
           #MAP                                                        
           #                                                           
           # use a fixed overlay area instead of the free pool         
           BLINKER OVERLAY FIXED                                       
           #                                                           
           # allocate 60K to the overlay pool                          
           BLINKER OVERLAY OPSIZE 60                                   
           #                                                           
           # incremental link as standard                              
           BLINKER INCREMENTAL ON                                      
           #                                                           
           # call stack size and fixes stack size                      
           BLINKER PROCEDURE DEPTH 55                                  
           STACK 5120                                                  
           #                                                           
           # overlay in EMS aare disable because of using EMS in others
           BLINKER OVERLAY PAGEFRAME OFF                               
           #                                                           
           # and use upper memory blocks for overlay pool to save memor
           BLINKER OVERLAY UMB ON                                      
           #                                                           
           # Burn in environment                                       
           BLINKER EXECUTABLE CLIPPER F60;CGACURS;INFO                 
           #                                                           
           # allow users override environment with SET CLIPPER         
           BLINKER ENVIRONMENT OVERRIDE                                
           #                                                           
           # set different than SET BLINKER                            
           BLINKER ENVIRONMENT NAME APPSET                             
           #                                                           
           # don't delete .EXE file when error                         
           BLINKER EXECUTABLE NODELETE                                 
           #                                                           
           # burn-in serial numbers                                    
           BLINKER EXECUTABLE SERIAL (C) NETCONSULT 1992, (DD)        
           #                                                           
           # NO display list of all duplicates                         
           #BLINKER MESSAGE DUPLICATES                                 
           #                                                           
           # don't blink, it's bothering me                            
           BLINKER MESSAGE NOBLINK                                     
           #                                                           
           # don't search for default libraries!                       
           NODEFLIB                                                    
        

       Using of Clipper code is better for memory than C/ASM written code in 
        cases when SPEED is not very important. Dynamic overlaying of Clipper 
        code in ALL cases is giving better memory usage with side effect on 
        speed of course. Most from C/ASM code is very fast, small but 
        overlaid in not so good way as Clipper. But all this is question of 
        used linker of course.

       do not allocate unneeded variables, PRIVATE or PUBLIC variables. Make 
        variables which are really needed and do them as LOCAL or STATIC. All 
        others can have impact on memory when used. It's saving of bytes, but 
        who knows maybe some of them will be needed...

       When using C/ASM written code try to use Clipper VMM system as less 
        as possible because it requires additional allocations of memory 
        which are impossible to release until is specifically told by your 
        C/ASM program.

    - make as much as possible of free memory for Clipper program.

       Use MS DOS 5.0 (or DR DOS 6.0 but there are known compatibility 
        problems of DR DOS with MS DOS which can affect any application) with 
        HIMEM.SYS and DOS=HIGH or with something providing UMB and load 
        everything high that can be loaded high.

       Don't make unneeded FILES=, BUFFERS=, STACKS=, FCBS= allocations
        in CONFIG.SYS, for network environment can be told

        FILES= can be 8 in Novell Netware because FILE HANDLES= in NET.CFG
            are more important
        BUFFERS= can be 8 in any Network because buffers will not be used
            in networking access and therefore are relevant. Every buffer
            needs own memory for allocation. BUFFERS are anyway good in
            multiplications of at least four. If BUFFERS= is possible to
            move in EMS, HIGH memory do so (HIBUFFERS, BUFFERS=/X ...)
        STACKS=0,0 always. Nobody needs additional STACKs from MS DOS and
            some resident programs are requiring this anyway. Can save
            some space
        FCBS=at absolute possible minimum (version of DOS dependent)

       Use replacements of COMMAND.COM like 4DOS (or NDOS) which are using
        only about 3KB of memory for COMMAND.COM and loading rest in EMS,XMS
        or just leaving as overlay on disk. I'm using 4DOS already several
        years and it's fully compatible COMMAND.COM replacement which cannot
        cause any troubles to any programs.

       Do not use relevant device drivers like ANSI.SYS in system where
        nothing with ANSI will be used, COUNTRY.SYS when nobody needs COUNTRY 
        support, EGA.SYS, DRIVER.SYS or so. Every device driver needs memory 
        which can be handy for Clipper program. If those drivers are needed,
        use HIGHDEVICE, DEVICEHIGH or so for loading them to available high
        memory to save conventional memory for Clipper program.

       Do not load relevant TSR programs like FASTOPEN in network, caching 
        software in network, DOSKEY when user will not be using DOS command 
        line at all because will walk whole day in menu. All neccesary TSR 
        programs is again good to load high (when possible)

       Don't load relevant parts of network drivers. When nobody needs 
        NETBIOS emulation don't load it in Novell network. When nobody needs 
        SPX and Diagnostic communication don't load those parts of IPXODI 
        (when using DOS ODI drivers)

       Use DOS ODI drivers instead of IPX/NETX in Novell Netware because 
        it's offering less memory usage and can even be reduced by excluding 
        mentioned SPX/Diagnostic part.

       Use memory managers software like QEMM, QEMM386, QRAM, 386MAX, 
        BLUEMAX, NETROOM, HIDOS with possibility of creating up to 300KB of 
        HIGH memory for loading of ALL used TSR programs, device drivers, 
        files, buffers, fcbss. With programs like this can complete 
        workstation in DOS 5.0 really end in nearly 640KB free after loading 
        all programs including network drivers.

       Use memory managers providing EMS memory (QEMM, EMM386, 386MAX, 
        NETROOM and others) which can then be used by Clipper paging and 
        overlaying system. It can help in speed increase of CLipper 
        application and also in some memory savings because of more accessible 
        memory.

ڷ  That stupid hanging SHIFT key no my keyboard!
Լ
    In most from the networks is happening that SHIFT key is sticking ON
    even it's not really pressed. It's not problem of hardware, it's problem
    of software. Redirection of keyboard interrupt routines can become too
    long that it can start loosing status informations. In Networks it's
    common problem. There are PATCH programs which will solve this and is
    therefore highly recommended to get them before complaining about this
    Clipper programmers which are messing my keyboard. Look for KBFIX, 
    KBDFIX, KEYBFIX, SHIFTFIX, INTFIX INTxxFIX and similar words when 
    scanning for this patch. It's for sure available in NOVLIB...


ڷ  Data integrity problems are in most cases problem of hardware
Լ  Hardware CAN cause terrible problems which looks like software bug

    Believe or not, there were MANY cases of software-like problems which 
    were caused by hardware problems, therefore following is a list to check 
    when problems exist but all other possibilities are already checked out.

    -   faulty network card in server or in workstation

        ANY faulty network card which is successfully working for common
        DOS tasks can fail under heavy Clipper load. It can fail most easy 
        because of amount of transferred data which Clipper applications 
        normally does. Any network MUST have possibility to CHECK network 
        cards for transferring errors. In normal cases must LAN have 100% 
        error free transfer between workstation and server. If it's not true 
        for some from network cards then replace them for sure.

    -   memory problems of workstation or server

        Because of heavy user of memory by Clipper application can easily 
        happen that faulty memory chip is not failing under normal 
        conditions, but only with Clipper programs (using for example EMS 
        memory somewhere at end of memory). If Parity error is not generated 
        wrong information can be obtained from memory and then used.

        Weak memory chips (SIMMs or SIPPs) are often successfully working with
        programs like MS DOS and are even able to pass initial memory test of 
        your computer (anyway this is not real test as it should be), but are
        visible under programs really checking memory (like CHECKIT). 
        Sometime computer MUST run longer time to get overheated a bit and 
        then test can be made to see problem.

    -   conflict in memory or in hardware

        Believe me that technicians installing network cards on IRQ2 are 
        still between us (IRQ2 is used in EVERY AT computer for all IRQs 
        8 till 15 and therefore is NOT free for network card). Computer
        or server with like this installed card WILL work. Will work
        slower and will have reliability problem in some cases because at
        least HARDDISK controller will be located between IRQ8 till IRQ15 and
        therefore IRQ2 will be shared by TWO devices.

        ANY conflict of hardware or software (like non deallocated ROM for
        QEMM which must be) can be source of problems. If program is running
        OK on _one_ computer and WRONG on _other_ computer, then be sure
        it's not problem of program (in 99% cases) but problem of computer.

    -   conflict with resident software

        There are MANY resident program which can cause problems for not only 
        Clipper application. One example should be USER.COM from LANSIGHT 
        software. When this resident program is loaded, Clipper applications
        acts like mad program, crashing in unpredictable moment or smashing 
        your database files. Therefore be always careful what TSR is loaded. 
        Test problematic program also on PURE MS DOS loaded computer. 

    -   bad sector on your server

        There MAY be problem for example in Novell Netware server that sector
        which is BAD is _not_ marked as bad and is used again and again. It's
        hard to understand how this can be happening, but i'm suspecting that 
        HotFix on some servers can be full (or off) and server can still run
        without someone notice it. In this case your database files will get
        corrupted on network, but will run fine in local disk or on other 
        volume. What can also make problems is:

         too old version of disk driver
         IDE (ISA) disk with bad driver or wrong setup in CMOS
         disk which has ON-BOARD cache memory (like some Seagates) and
          this memory is not switched OFF. In server there MAY NOT be
          onboard cache memory on harddisk!
         NLM which is responsible for disk/io operations is not well
          in good health
        

ڷ  PURE MS DOS loaded computer is best place for testing your software if
Լ  any problems are out.

    In most cases computers on which your programs will run are filled with
    different configuration. That means that sometime configuration of 
    computer (hardware, software, TSR, DOS versions) can be causing problems. 
    To see if that's the case i have used BOOTSYS program which allows me 
    boot different configuration of my 386/40 for testing. One from them can 
    be called PURE MS DOS test configuration. It looks like this:

        CONFIG.SYS
            FILES=20
            BUFFERS=16
            DOS=HIGH
            DEVICE=C:\HIMEM.SYS
            STACKS=0,0
            SHELL=C:\COMMAND.COM /E:768 /P

        AUTOEXEC.BAT
            @ECHO OFF
            PROMPT $P$G
            \DOSODI\LSL
            \DOSODI\EXP16
            \DOSODI\IPXODI
            \DOSODI\NETX
            F:
            LOGIN
        
        NET.CFG
            FILE HANDLES=100

    See, NOTHING TSR loaded, NO device drivers at all. NO SETVER.EXE (anyway
    why to use it when all must be already DOS 5.0 aware). Just load MS DOS
    and then DOS ODI network drivers and log into network. NO EMS memory. 
    For more PURE configuration can be HIMEM.SYS and DOS=HIGH removed and
    network drivers not loaded and application tested from local harddisk.

    With this configuration one can be sure to eliminate ALL drivers, TSR,
    additions to your BIOS or VIDEO BIOS and test Clipper application.

    IF THIS is working and others are not? Look for configuration problems..

ڷ  What about this SET magic of my Novell Netware 3.11 server, isn't good
Լ  to try to tune it?

    Novell Netware 3.11 operating system is selfconfiguring itself against 
    needs from workstations and server. In many cases is good anyway to tune 
    server manually for better performance, or at least take a look what's 
    going on. Following is list of all SET parameters which CAN be related
    to SPEED of Clipper applications.

     Maximum Concurrent Disk Cache Writes

        Increase for WRITING having better performance
        Decrease for READING having better performance

     Dirty Disk Cache Delay Time

        Increase for Small writes having better performance
        Decrease for better security of data (are sooner written to disk)
            but with performance going down

     Dirty Directory Cache Delay Time

        Increase for better performance and less data security
        Decrease for better security of data, but slows down

     Maximum Concurrent Directory Cache Writes

        Increase for better writing
        Decrease for better reading

     Directory Cache Allocation Wait Time

        Increase will slow down directory accessing time
        Decrease will FAST directory accessing time

     Directory Cache Buffer Non Referenced Delay
        
        Increase will give faster access to directories
        Decrease will slow access to directories, but will give more 
            memory for other server tasks

     Maximum Directory Cache Buffers
        
        Increase will give better directory access
        Decrease when memory in server is required, slows directory access

     Minimum Directory Cache Buffers

        Increase will give better initial directory access to server,
            it's effective only for time of start of server

     Immediate Purge Of Deleted Files
        
        if ON, all deleted files are immediately removed from server, 
            SALVAGE utility has not effect after. Can make server MUCH
            faster, but make impossible to undelete files

     Turbo FAR Re-Use Wait Time

        Increase is good for database files because Turbo-Fat is kept
            longer in memory. But needs more server memory
        Decrease saves memory of server

     Minimum File Delete Wait Time

        How long file MUST stay (may not be marked ready for Purge).
            Can help with keeping deleted files.

     File Delete Wait Time

        After how long file is marked ready for purge and may be 
            therefore automatically purges from disk. Can help to 
            manage deleted files.

     Maximum Subdirectory Tree Depth

        Automatically it's 25 directories level maximum which Netware
            will allow. If your Clipper application runs with more
            directories level, increase this number.

     NCP File Commit

        When it's ON (default) any application calling File Commit NCP  
            command WILL make HARD flush of data from memory to harddisk
            of server. Set it to OFF for better performance of server when
            CLipper application is using too much of DBCOMMIT() calls.

     Console Display Watchdog Logout

        Can help you to determine WHY your programs are disconnected from
            network without visible reason.

     Maximum Packet Receive Buffer

        It's limiting number of receive buffers for data from workstations.
            In case of network error without hardware cause increase this
            number because server cannot server all your workstation 
            requests.

     Maximum Record Locks Per Connection

        Defaults to 500 and means that your program can make maximum of
            500 records locked in your databases. If Clipper application is
            receiving errors of Record locking kind, increase this number.
            It's PER WORKSTATION, not FOR WHOLE NETWORK.

     Maximum File Locks Per Connection

        Default to 250 and means that your program can lock maximum of
            250 files (by flock()). It's quite enough because it's also
            real maximum of files available from network at all.

     Maximum Record Lock

        Defaults to 20000 and means maximum of record locks per whole server
            from all workstation. By default it's space for 40 workstations
            using full available 500 records lock each. It should be enough,
            otherwise can be increased. Decreasing can save some server
            memory and keep better performance.

     Maximum File Lock

        Default to 10000 and means maximum of file locks per whole server 
            from all workstations. By default it's space for 40 workstations
            using full available 250 file locks each. It should be enough,
            otherwise can be increased. Decreasing can save some server
            memory and keep performance better.

     Auto TTS Backout Flag

        When ON (default) server will automatically after crash backout all
            active transaction during start. When OFF supervisor must react
            for server's questions.

     TTS Abort Dump Flag

        When ON will create every time TTS$LOG.ERR file in root of volume.
            This file can log all transactions for analyzing.

     Maximum Transactions

        Maximum number of transactions per server from all workstations. 
            Should be enough, but can be changed.

     Enable Disk Read After Write Verify

        Can make server as twice fast when it's OFF, but then there will be
            NO verification of data after writing which can lead in data 
            integrity problems. In disk with own READ/WRITE verify on board
            it's possible switch this OFF.

     Maximum outstanding NCP searches

        When corrupted or invalid directories are encountered in some 
            application is good time to increase this.

     Allow Unencrypted Passwords

        Novell 3.11 introduced all passwords as encrypted when travelling
            through network. If 2.x server with non-update utilities is
            in the same network as 3.11 server is good to set this to ON.

     Maximum Service processes

        Increasing makes better performance
        Decreasing slows down, but gives more memory for other server tasks.

    Most from those SET command should be checked once a while in 3.11 server
    via MONITOR program where are visible most important setting. This check
    should include check for:

                                Minimum     Maximum     Sample Reality
    Service Processes                         20               4
    Packet Receive Buffers      10           100              10
    Directory Cache Buffers     20           500             110

        In this sample is good time to start about thinking parameter called
        Minimum Directory Cache Buffers because it can fast up starting of
        server and users coming to access their directories. It can be inc-
        reased to 100 or 110. Other parameters are in order and nothing is
        near of maximum, therefore it has enough room to grow.


ڷ  When the SPEED is main problem
Լ
    SPEED of Clipper application is main problem always and in network it's 
    one from most important moments. What to do for maximum speed of your
    program:

    -   make sure that network is enough fast. It can be done by checking
        transfer speed of your network by simply running SystemInfo from
        Norton Utilities set. Example numbers:

        Novell Netware 3.11, Ethernet 8bit card     280~300 KB/sec
        Novell NEtware 3.11, Ethernet 16bit card    320~370 KB/sec
        Novell Netware 2.2, Ethernet 16bit card     230~270 KB/sec
        Lantastic 4.01, Ethernet IDEAL condition    200~250 KB/sec
        Lantastic 4.01, Ethernet REAL condition     150~200 KB/sec
        Lantastic older versions                    50~100 KB/sec
        Novell Netware 3.11, Arcnet (2.5MB)         100~170 KB/sec
        Lan Manager, Ethernet                       100~150 KB/sec
        Lantastic 4.01, 2MB twisted pair             30~ 50 KB/sec
        Serial network (RS232C)                      10~ 30 KB/sec

        If your number are much lower than those it's time to thinking
        about:

         network card is on wrong IRQ and is slowed down
         network card is faulty and should be replaced
         network drivers (Shell) is loaded high and therefore slow
          use standard one (like NETX) and don't load it high
         network card should be upgraded from 8bit card to full
          16bit card when possible
         computer itself is very slow
         network card is using DMA or MEMORY technique of communication
          upgrade to better card using only IO and IRQ
         server has problems with network card. In server it should
          be ALWAYS _very_ fast 16bit (minimally) card, otherwise
          whole network is slow
         on Novell try to use BNETX for increasing of speed
         Server is very short on memory and has not enough resources
          to react to network requests
         Server is very slow (typical server should be at least 386/25)
         Server has VERY slow disk or faulty controller or driver
         There are wiring problem, check number of errors in card
          statistics or check connection between stations and recheck
          cabling. Weak connector, missing terminator, interference,
          not correct card can be reason for slow communication
         upgrade your network version to something significantly faster
          and reliable (like Lantastic to Novell Netware, Novell Lite to
          Lantastic or Novell Lite to real Netware).
         Upgrade your network topology and type. ARCNET is slower than
          Ethernet (2.5MB against 10MB) and so on.

    -   Make as much as memory available for your application

        This is covered somewhere else in this article, therefore go there.
        If Clipper application has not enough memory it must make more
        disk swapping and overlaying and therefore is slowing down

    -   Make EMS memory available for your Clipper application

        Clipper 5.01 can use EMS memory for overlays and data, therefore
        enough of EMS (2MB at least) can make significant difference in
        speed - disk overlay is not used and EMS is MUCH faster.

    -   Place temporary files on RAMDISK or HARDDISK instead of network.

        If your network is not faster than your harddisk, try to point TEMP
        files to harddisk (or ramdisk). It's faster and also network traffic
        will be much smaller

    -   When linking application try to move often used modules in ROOT 
        instead of letting overlay them. It can have impact on using of
        memory, but can make application MUCH faster. When for example is
        your application using function CHECKIT() every start of MENU on
        screen, then maybe place it to ROOT and it will not be loaded 
        again and again over. But think about memory usage of course...

    -   place your .EXE file on your local harddisk or RAMDISK and access
        only data files if it is possible. .EXE will be loaded faster from
        your disk and also all reading of overlaid part will be faster

    -   make sure that your machine is running at highest possible speed.
        Most of AT machines have speed switch on chassis or possibility to use
        CTRL ALT + (big gray plus, not the small one) for switching to high
        speed. Use some SystemInfo or SPEED utility to see if it's really
        running fastest as can. 

    -   Do not expect that 8088 or 8086 machines will be speed deamons for
        Clipper application. It will be VERY slow. Average SLOW AT286/12Mhz
        is still 4 times faster than average 808x machine. Upgrade those 
        machines as fast as possible or use them for other jobs (like
        print servers they are of great use for matrix printers!). 

    -   When upgrading 8088, 8086 or 80286 machines it's good to upgrade them
        at least to 80386SX/16Mhz with some 2MB or 4MB of memory. 

    -   Make sure that your computer has not:

         slow harddisk which can make lot of troubles
         slow harddisk controller
         slow video card. Upgrade all those CGA, MCGA and HERCULES cards to
          VGA and all will be better in displaying speed
         slow memory chips requiring additional WAIT STATES
         keyboard in bad mode because it will work, but slowing down
         faulty card in system bus because it will slow down all I/O
          operations

    -   make sure that your program is written as much effective as possible
        because that's main point of many programs running very slow. Some
        tips for this:

        - do not use PRIVATE or PUBLIC
        - use your indexes as good as possible. When one can make SEEK
          instead of skipping through database, it's much faster
        - use DBFSIX replacement of DBFNTX because it's much faster
          in network and offers much more for optimization
        - use MACH6 query optimizer for your database operations
        - EXTENDBASE software for your server should help also
        - don't generate unneeded files
        - optimize your databases to not have so much of them without
          reason
        - pack sometime .DBT files and .DBF files because if file is
          too big, it's slower
        - reports generated from your files which are not ORDER 
          dependent is good to generate with SET ORDER TO 0 because
          it will save time to access your index and also will reduce
          network traffic
        - SUBNTX or DBFSIX is also handy for creating subsets of
          index files

ڷ  OOPS and what about those terrible problems with indexes
Լ
    Most from us experienced already some 1210 Internal errors or index 
    corruptions. There is NOT most probably bug in Clipper 5.01 way of
    handling .NTX files, but there WAS bug in 5.0 version causing corruptions
    in case of BIG applications. Most from index corruption are related to:

    -   index key expression is NOT STABLE. It means it HAS NOT EXACTLY the
        same size EVERY time it's invoked. TRIM() functions are suicide when
        used in index key. Remember that index key MUST have always the same
        length which MUST be the same also when EMPTY parameters are passed to
        index key (that's a way how Clipper is making initial size for 
        generating a index files).

    -   index key expression is NOT UNIQUE. It means it's RELATED to some
        other WORK AREAS or FUNCTIONS which can give different results
        when making and when updating index. Be absolutely sure that there
        is NO WAY how to make INDEX ON ADRES->NAME+DATA->CODE even when
        SET RELATION is invoked between ADRES and DATA. No discussion about
        this, it's even documented in Clipper documentation

    -   Index files are not opened in THE SAME order. It's VERY important
        that two programs using the same .DBF with the same .NTX (or any
        other) file is opening them in the same order. What is therefore
        wrong is for example:

        App1    SET INDEX TO NAME,LEVEL,CODE
        App2    SET INDEX TO NAME,CODE,LEVEL

        In this example most sure LEVEL and CODE are two indexes ready for
        become corrupted. Why? Because Clipper will update index keys in
        order they are defined in application and when two Applications 
        will come to LEVEL and CODE it will get updated in different order
        and can destroy index.

    -   Index file are not opened always when database is updates. It's 
        again VERY important to open ALL index files and have them OPEN
        at moment database is updates. Don't try to stick on rule that
        'when index key related fields are updated', stick on rule that
        ALWAYS when database is updated. Therefore this is wrong:

        App1    SET INDEX TO NAME,LEVEL,CODE
        App2    SET INDEX TO NAME,LEVEL

        CODE index in this case is subject of destruction which will be
        manifested by missing keys from index because APP2 will not 
        update correctly this index (because it's not open).

    -   if DBFSIX is your horse for race, then don't forget to update to
        latest revision (which is 2:30am) and which is fixing few little
        index related problems finally.


A word from authorĿ


    Final word. I tried in two days compile as much as knowledge which i was
    keeping in my head and occasionally just copying out to Compuserve Mail as
    answers to most common question. I'm sure that i forget something that
    might be interesting or important, therefore if it's your experience or
    knowledge that can be added into this Network Guide for Clipper, please
    feel free to write me or tell me (CIS ID is 100064,2343 and i'm checking
    mail every working day). I can later release newer versions with updates.

    Finally for those speaking English as mother tongue. I'm CZECH and i'm
    working now in The Netherlands (which anyway speaks Dutch) and therefore
    my English for sure look funny (formulation) and must have many mistakes
    or misused word. Take a fun i have it also. Anyone wishing make language
    correction of this text is anyway welcome.

    And please don't write angry letter or angry comments about something
    you don't agree with. It can be in many cases.

    Daniel

    100064,2343 Compuserve
    2:285/608@FidoNet
    (31)-10-4157141 FAX to NETCONSULT/DANIEL
    (31)-10-4157140 VOICE to NETCONSULT/DANIEL
    
