
DGROUP.TXT version 2.0

Copyright(C) 1993, Robert G. Montgomery.  All Rights Reserved.

The following is a discussion of DGROUP, and its relationship to the
following error messages:

  Unrecoverable Error 650 Processor Stack Fault (UE 650)

  Unrecoverable Error 667 Eval Stack Fault (UE 667)
  Unrecoverable Error 668 Eval Stack Fault (UE 668)
  Unrecoverable Error 669 Eval Stack Fault (UE 669)


The following discussion is based upon my understanding what is going on
with DGROUP, as it relates to programming with CA-Clipper (hereafter
referred to as Clipper), version 5.0 through 5.2c, and probably later.  This
is based mostly on information obtained through messages on the Clipper
forum on CIS.  I am not an authority on this, just a common user of Clipper
who has figured a way around a problem I was having with Clipper.  I furnish
the following as a favor to all Clipper users, and I furnish the following
information without any guarantee that this is the way it is, and therefore
cannot accept any responsibility for anything which may come of it.  All I
know is that it works for me.  Please use at your own risk, and please don't
sue me <bg>.

If anyone has anything to add, or wishes to point out where I might be
downright wrong on something discussed here, feel free to contact me through
CIS at 75660,31.

*****************************************************************************
* DGROUP                                                                    * 
*****************************************************************************

DGROUP is an area of memory which exists in large model programs (like
our Clipper apps).  This is where many Clipper internals are stored during
the execution of your programs.

DGROUP structure:

  direction of growth           <---|--->                             <---|
  +---------------------+-----------+------------+---------+--------------+
  | C & ASM static data | CPU stack | Eval Stack | "space" | Memvar Table |
  +---------------------+-----------+------------+---------+--------------+
  |   allocated at link-time        |       allocated at run-time         |
  
                                    |<------ DS Avail from //INFO ------->|

  |<---------------------------- total of 64K --------------------------->|
  

DGROUP is the area of memory where the DS (Data Segment) and SS (Stack
Segment) registers of the CPU point to during the execution of large model
programs.  This way, all pointers to static data during execution of a
program can be done with near pointers, which are faster since they are only
16 bit pointers.  By assuming SS=DS, access to C and ASM local variables may
also happen with near pointers.  And by putting the Eval Stack and Memvar
Table in DGROUP, they too can be accessed with near pointers.  This is all
in the interest of speed.

Some background information.


What is a VALUE?

A VALUE is a data type defined internally to Clipper which is a 14 byte
block of memory stored in DGROUP which contains the information necessary to
define a Clipper value.  Stored within the VALUE is the type of the Clipper
value (Character, Numeric, etc.), and the value itself, if possible.  If the
value is too large to be stored within the 14 byte VALUE record (i.e.
character strings, arrays, etc.), a VM segment handle is stored within the
VALUE, and the actual value is stored in a VM segment somewhere outside of
DGROUP.  Both the Eval Stack and the Memvar Table are arrays of 14 byte
VALUE records.


What is an ITEM?

An ITEM is a near pointer to a VALUE.  ITEMs are manipulated using ITEM.API,
from C & ASM code.  When an ITEM is created, a VALUE is allocated in the
Memvar Table, and a near pointer to the 14 byte VALUE record is returned by
the _item*() function.


What is the purpose of the different areas within DGROUP?

C & ASM static

This is where static variables, and literal strings, defined within C or
Assembler code, are stored.  Many third party libraries, as well as many
parts of the Clipper run-time, are written in C and/or assembler and use up
DGROUP in this area.  The amount of its allocation is depends on the
variables and literal strings defined.  You cannot do much about this except
for not linking in code which eats it up, or modifying the C and/or ASM code
which eats it up. The amount of C & ASM may be determined with following
formula, run //INFO to get the DS Avail value.

  <C & ASM allocation> = 64K - <stack space> - <DS Avail>

Or look in map file generated by the linker, and total up everything in
DGROUP except for the stack.  If you are using Exospace, the //info switch
does not work so you must look at the map file to determine this.

CPU Stack

This is where procedure/function return addresses are stored, as well as
local variables defined within C or Assembler code.  The amount of its
allocation is dependent on the /STACK or SET PROCEDURE DEPTH parameters
given to the linker.  Its default is 4096 bytes, but could be changed by the
inclusion of .OBJ or .LIB files which request that it increase.  As long as
you don't get UE 650 processor stack fault, or system hangups, you can
reduce the size of the CPU stack to save DGROUP memory for other things.
Usually takes 4096 bytes, but may be adjusted by linker.

Eval Stack

This is where VALUEs for local and private variables and parameters are
stored.  Note that only the VALUEs associated with the variables are stored
here, private variables also eat up space in the symbol table, locals do
not.

Memvar Table

This is where clipper static variables are stored, as well as the VALUES
which are associated with the ITEMs which are allocated through ITEM.API.
It is a heap, and if all slots within the heap are filled when a call to
_itemNew() is made, it expands the static area to allow for allocation of
more ITEMs.  ITEMs are allocated internally by Clipper, as well as by C &
ASM routines, including many third party libraries.

"space"

This is the area in which the Eval Stack and Memvar Table expand into.
Hopefully, conventional memory is not in short supply, but if it is, then
some or all of the "space" may be reallocated to the "Conventional Memory
Pool".  If this happens, and then there is an attempt to expand either the
Eval Stack or Memvar Table into the reallocated "space", you will get UE 668
or UE 669, Eval Stack Fault.  The functions fix_UE668() and/or fix_UE669()
will allow you to push the boundaries of the "space" inward toward the
center of the "space", reducing your chances of getting the errors.


What determines the boundaries between the 5 sections of DGROUP?

The boundary between the C & ASM static data area and the CPU Stack, as well
as the boundary between the CPU Stack and the Eval Stack, are set by the
linker and remain fixed throughout the execution of your application.  The
Eval Stack expands upward in memory, and the boundary between the Eval Stack
and the "space" is set by a "watermark" which marks the highest extent in
which the Eval Stack has been expanded throughout the execution of the
application.  The Memvar Table expands downward in memory, and the boundary
between the Memvar Table and the "space" is set by a "watermark" which marks
the lowest extent in which the Memvar Table has been expanded.  The memory
between these "watermarks" is the "space".  Under low memory situations, the
VM will reallocate the "space" to the conventional memory pool.

Determination of the position of the "watermarks" may only be done through
access to UNDOCUMENTED public variables defined in CLIPPER.LIB, and they may
only be accessed from C and/or ASM code.  Specifically, the variable
_evalhigh (near pointer to void) is the high "watermark" for the Eval Stack,
and the variable _estatlow (near pointer to void) is the low "watermark" for
the Memvar Table.  In C, it is useful to type-cast their pointer values to
unsigned integers (USHORT), and pass their integer values back to Clipper
for the debugging programmer to see.  It is useful to consider the values of
these pointer variables when debugging programs, especially those which
access ITEM.API, or use third party libraries which access ITEM.API.
Clipper callable functions for determination of these pointer values, as
well as other useful functions for determination of the state of DGROUP, are
included in the file DGROUP.ZIP in lib 2 in the CA-Clipper forum on
Compuserve.  Writing of routines which access these pointer values is
recommended only for debugging purposes, since they may be changed in
upcoming releases of Clipper.



*****************************************************************************
* Unrecoverable Error 650                                                   * 
*****************************************************************************

Unrecoverable Error 650 - Processor Stack Overflow / Out of Stack Space.

Your program has exhausted the CPU Stack memory.

The most straightforward cause is having function and procedure calls nested
too deep for the CPU stack to hold all of the required activation records
for the called procedures and functions.  This may be fixed by using the
linker STACK command (or SET PROCEDURE DEPTH command) to expand the CPU
stack for an application.  Note that any memory added to the CPU stack gets
taken from the Eval Stack and Memvar Table, so expand the stack carefully in
small increments, until you do not get UE 650 anymore.

Another cause is infinite recursion.  This is the case if expanding the CPU
Stack with a linker command does not work, since infinite recursion eats up
all of the CPU Stack regardless of the size of it.  Note that when running
under Clipper, when an error is generated causing a call to the error
system, and there is an error in the error system, another call to the error
system will happen, causing another error to be generated, causing another
call to the error system, and so on ... infinite recursion.  One way to
check for this is by viewing the call stack from within the Clipper
debugger, and looking for repeated procedure/function names.


*****************************************************************************
* Unrecoverable Error 667                                                   * 
*****************************************************************************

Unrecoverable Error 667 - Eval Stack Fault

This is caused by Eval Stack and/or Memvar Table Overflow, which means that
the Eval Stack and the Memvar Table have overrun each other.

One cause is not having enough memory in DGROUP (see DGROUP explanation for
more info on this) available after C & ASM static data and the CPU Stack
have been allocated.  One way to check for this is by running your
application with the //info command line switch and looking at the DS Avail
value.  Another way is to look in the map file generated by your linker and
adding up the lengths of all segments which are linked into DGROUP.  Both
methods yield the same result, but the //info switch does not work with
Exospace'd applications.  If the value is under 20K, this may be your
problem, and if it is below 10K, this is almost definitely your problem.  If
this is your problem, the only way to fix the problem is to not link in as
much C & ASM static data, and/or reducing the size of the CPU Stack.

Another cause is too many local, static, and private variables defined at
one time.  Note that ITEMs allocated through ITEM.API are basically the same
as static variables, except that they can be created and destroyed.  Static
variables cannot be destroyed.  Local variables are created when a function
is called and destroyed when a function returns.  Note that some Clipper
internals create and destroy them as well.  The total number of variables
which may be instantiated at one time is roughly equal to the DS Avail value
(see above) divided by 14.

Note that it is possible (but improbable) that this error may be generated
by infinite recursion, where the program runs out of Eval Stack before it
runs out of CPU Stack.  Also note that it is possible to get infinite
recursion by having an error in your error handling system, where an error
will generate a call to the error system, where another error generates
another call, and so on until the program runs out of Eval Stack.

See the section on DGROUP for more information.


*****************************************************************************
* Unrecoverable Error 668                                                   * 
*****************************************************************************

Unrecoverable Error 668 - Eval Stack Fault

This is caused by Eval Stack Overflow into Locked VM Segment.

Your program attempted to expand the Eval Stack into an area of DGROUP which
contains a locked VM segment.  Under low memory situations, the Clipper VM
will reallocate memory in DGROUP between the Eval Stack and the Memvar Table
to the conventional memory pool.  After this happens, if your program needs
to expand the Eval Stack into the reallocated memory, and that memory
contains a locked VM segment, you get this error.

Note that it is possible (but improbable) that this error may be generated
by infinite recursion, where the program runs out of Eval Stack before it
runs out of CPU Stack.  Also note that it is possible to get infinite
recursion by having an error in your error handling system, where an error
will generate a call to the error system, where another error generates
another call, and so on until the program runs out of Eval Stack.

When running under Protected Mode using a DOS Extender (i.e. Exospace), it is
very unlikely that this error will happen since there is plenty of memory
available, so that the VM will not need to reallocate memory from DGROUP.

See the section on DGROUP for more information.  Also, see DGROUP.ZIP in lib
2 of the CA-Clipper forum on Compuserve, which contains the Clipper source
code for fix_UE668(), a function which allows you to coerce the VMM into
leaving some more memory for the Eval Stack when it reallocates DGROUP.
Fix_UE668() is a Clipper function which creates a whole mess of local
variables and returns, which will move the "watermark" (see DGROUP) that
marks the highest expansion of the Eval Stack.  It should be called in one
of the first statements in your program.  If the number of local variables
declared and then released by Fix_UE668() is more than the total which will
be needed during the execution of your program, you will not get UE 668
anymore.


*****************************************************************************
* Unrecoverable Error 669                                                   * 
*****************************************************************************

Unrecoverable Error 669 - Eval Stack Fault

This is caused by Memvar Table Overflow into Locked VM Segment.

Your program attempted to expand the Memvar Table into an area of DGROUP
which contains a locked VM segment.  Under low memory situations, the
Clipper VM will reallocate memory in DGROUP between the Eval Stack and the
Memvar Table to the conventional memory pool.  After this happens, if your
program needs to expand the Memvar Table into the reallocated memory, and
that memory contains a locked VM segment, you get this error.

Note that it is possible (but improbable) that this error may be generated
by infinite recursion, where the program runs out of Memvar Table space
before it runs out of CPU Stack.  Also note that it is possible to get
infinite recursion by having an error in your error handling system, where
an error will generate a call to the error system, where another error
generates another call, and so on until the program runs out of Memvar Table
space.

When running under Protected Mode using a DOS Extender (i.e. Exospace), it is
very unlikely that this error will happen since there is plenty of memory
available, so that the VM will not need to reallocate memory from DGROUP.

See the section on DGROUP for more information.  Also, see DGROUP.ZIP in lib
2 of the CA-Clipper forum on Compuserve, which contains the C source code
for fix_UE669(), a function which allows you to coerce the VMM into leaving
some more memory for the Memvar Table.  Fix_UE669() is a C function which
creates a whole mess of ITEMs through ITEM.API, releases them, and returns.
This will move the "watermark" (see DGROUP) which marks the lowest expansion
of the Memvar Table.  It should be called in one of the first statements in
your program.  If the number of ITEMs allocated and then released through
fix_UE669() is more than the total which will be needed during the execution
of your program, you will not get UE 669 anymore.


