END-OF-FILE RECKONING: CLIPPER VS. dBASE

David Morgan

Copyright (c) 1988 Nantucket Corporation.


File sizes are reckoned by Clipper by truncating the file where the logically
meaningful data ends.  Clipper writes files it processes back to disk so that by
DOS's terms of measurement, they end where the data ends, after the following
number of bytes:

     header size  +  (# of records) * (1 + record length)  +  1

This accounts for the extra byte in each .dbf record for DELETE/RECALL, and the
single final byte containing the 1Ah (i.e., control-Z or CHR(26)) used by some
software, dBASE included, to signify "end-of-file" (i.e., the point where the
logically meaningful data ends).

dBASE III PLUS, on the other hand, places a 1Ah into the file when writing and
vigilantly looks out for it whenever reading, as the defining, conventional end
of the file.  If you delete the last few records in the file then PACK in dBASE,
no adjustment is made in the DOS size of the file, and the deleted material is
still in the file as viewed by DOS, but not as viewed by dBASE since it ignores
whatever follows the 1Ah.

When Clipper comes along in the wake of dBASE activity in a .dbf, a lot depends
on whether you open the file in exclusive or shared mode. In exclusive mode,
Clipper behaves like dBASE, both writing and reading 1As.  Not so when in shared
mode however, for a good reason (namely, simultaneous APPEND BLANK capability).

An example is helpful.  I create a file called Merrill..dbf in dBASE with the
following structure:

Structure for database: C:MERRILL..dbf
Number of data records:       6
Date of last update   : 05/11/88
Field  Field Name  Type       Width    Dec
    1  TOP_BROKER  Character      7
** Total **                       8

and the following contents, APPENDed in dBASE:

Record#  TOP_BROKER
      1  MERRILL
      2  MERRILL
      3  MERRILL
      4  LYNCH
      5  LYNCH
      6  LYNCH

The underlying file contents, revealed by DOS's DEBUG, look like this:

1370:0100  03 58 05 0B 06 00 00 00-41 00 08 00 00 00 00 00   .X......A.......
1370:0110  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00   ................
1370:0120  54 4F 50 5F 42 52 4F 4B-45 52 00 43 0D 00 EB 3E   TOP_BROKER.C...>
1370:0130  07 00 00 00 01 00 00 00-00 00 00 00 00 00 00 00   ................
1370:0140  0D 20 4D 45 52 52 49 4C-4C 20 4D 45 52 52 49 4C   . MERRILL MERRIL
1370:0150  4C 20 4D 45 52 52 49 4C-4C 20 4C 59 4E 43 48 20   L MERRILL LYNCH
1370:0160  20 20 4C 59 4E 43 48 20-20 20 4C 59 4E 43 48 20     LYNCH   LYNCH
1370:0170  20 1A 00 5A C3 52 E8 A9-FE 2E C4 3E 55 7A BB 19    ..Z.R.....>Uz..

The header is 65 bytes.  There are six records, each with defined size seven
bytes, therefore eight bytes long.  Thereafter is a 1Ah; altogether 114 bytes.
DOS DIR command yields the following:

 Volume in drive C has no label
 Directory of  C:\ADMIN

MERRILL  DBF      114   7-11-88   6:00a

Same number: 114.  If I now DELETE the last three records then PACK, the result
differs depending whether I do so in Clipper or dBASE.  If I do so in dBASE,
then start working with the file in Clipper in shared mode, disagreement over
where the file ends can arise.

IN dBASE:

1370:0100  03 58 05 0B 03 00 00 00-41 00 08 00 00 00 00 00   .X......A.......
1370:0110  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00   ................
1370:0120  54 4F 50 5F 42 52 4F 4B-45 52 00 43 0D 00 EB 3E   TOP_BROKER.C...>
1370:0130  07 00 00 00 01 00 00 00-00 00 00 00 00 00 00 00   ................
1370:0140  0D 20 4D 45 52 52 49 4C-4C 20 4D 45 52 52 49 4C   . MERRILL MERRIL
1370:0150  4C 20 4D 45 52 52 49 4C-4C 1A 4C 59 4E 43 48 20   L MERRILL.LYNCH
1370:0160  20 2A 4C 59 4E 43 48 20-20 2A 4C 59 4E 43 48 20    *LYNCH  *LYNCH
1370:0170  20 1A 83 26 8B 04 C4 76-18 26 89 04 83 46 18 02    ..&...v.&...F..

 Volume in drive C has no label
 Directory of  C:\ADMIN

MERRILL  DBF      114   5-11-88   6:06a

IN CLIPPER:

1370:0100  03 58 05 0B 03 00 00 00-41 00 08 00 00 00 00 00   .X......A.......
1370:0110  00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00   ................
1370:0120  54 4F 50 5F 42 52 4F 4B-45 52 00 43 0D 00 EB 3E   TOP_BROKER.C...>
1370:0130  07 00 00 00 01 00 00 00-00 00 00 00 00 00 00 00   ................
1370:0140  0D 20 4D 45 52 52 49 4C-4C 20 4D 45 52 52 49 4C   . MERRILL MERRIL
1370:0150  4C 20 4D 45 52 52 49 4C-4C 1A 10 00 83 7E 0E 01   L MERRILL....~..
1370:0160  75 35 B8 1E 0A 1E 50 FF-76 16 FF 76 14 FF 76 12   u5....P.v..v..v.
1370:0170  FF 76 10 9A F8 05 B5 16-83 C4 0C B8 72 09 1E 50   .v..........r..P

 Volume in drive C has no label
 Directory of  C:\ADMIN

MERRILL  DBF       90   5-11-88   6:11a

Note Clipper maintains the file DOS file size so as to coincide with the extent
of the actual data.  Now apply Clipper against the dBASE-PACKed version of the
file.  Specifically, for example, run the following compiled programs and see
the results:

SET EXCLUSIVE ON
USE Merrill
LIST top_broker

      1  MERRILL
      2  MERRILL
      3  MERRILL

SET EXCLUSIVE OFF
USE Merrill
LIST top_broker

      1  MERRILL
      2  MERRILL
      3  MERRILL
      4  LYNCH
      5 *LYNCH
      6 *LYNCH

A number of other commands that process the whole file (e.g., APPEND FROM)
behave just like LIST, above, in respect of the dependency of end-of-file upon
the EXCLUSIVE setting.


