INTRO             COPYRIGHT horio shoichi 1995              INTRO



NAME
     intro.txt - Introduction To Concache


A Brief Overview
     Concache.exe is for  "concurrent  disk  cache"  that  is  so
     called "advance cache", which drives disks concurrently with
     DOS/user programs. The maximum concurrency it can perform is
     up  to  25  disks, including one of two floppies, one of IDE
     disk, and up to maximum  number  of  configurabe  SCSI  disk
     units  under  DOS.  (The  term  IDE is for Integrated Device
     Interface. In this and the other texts in this package, how-
     ever,  includes  ST506  interface  and  ESDI  for  brevity.)
     Current version limits  the  maximum  number  to  16,  since
     exceeding  this limit sounds ridiculous.  However, the ulti-
     mate limit of its concurrency is imposed only by  DOS  drive
     letter availability.

     Concache.exe allows true postwrite operations  transparently
     for programs under DOS.

     The basic idea to attain concurrency is, by  catching  PC/AT
     type machine's BIOS int1590 and int1591 interrupts, to force
     switch from concache.exe to DOS/user program by  the  former
     and force back again to concache.exe by the latter.

     The SCSI hard disks are operated concurrently by the aid  of
     ASPI  (Advanced  SCSI Programming Interface) command posting
     capability, which offers the chance to synchronize with disk
     operations  exactly  the  same  way  as BIOS int1590/int1591
     allow.

     Additional programs floppies.exe and ccdisk.exe are provided
     with  concache.exe, the main cache program.  The former is a
     mini-driver which does most of  floppy  i/o  independent  of
     DOS. The latter is the reentrant disk driver for SCSI disks,
     using ASPI.

     If a disk is neither of above,  for  example  pre-ASPI  SCSI
     disk,  concurrency is lost there. On such disk, the interval
     the device made CPU idle is indiscernible from CPU overhead.
     Yet  postwrite,  although  not concurently, is supported for
     even such disk.

     Concache.exe uses memory for caching disk data. Memory types
     to be used are

     1)   Raw protected (int15) memory,

     2)   Expanded memory conformable to EMS specification 4.0,

     3)   Extended  memory  block  (EMB)  defined  by   XMS   2.0



Concache 1.00      Last Update:  18 March 1995                  1
INTRO             COPYRIGHT horio shoichi 1995              INTRO



          specification.

     If only int15 memory is to be used no additional third party
     software is needed.

     If memory manager allows to configure, these three types  of
     memory  can  be  used  freely mixed, totaling up to 64 mega-
     bytes.  The amount of EMS or EMB used  by  concache.exe  are
     changeable  without  disloading/reloading  concache.exe from
     memory.  (Int15 memory is hard to share with the other  pro-
     grams. Changing its size on the fly is not considered.)

     Concache.exe resident memory requirements are usually around
     20  kilobytes,  increased  depending on concurrency require-
     ments (stacks to keep things concurrent and sheer io buffers
     are unavoidably allocated at each startup.)

     Although concache.exe can  be  run  on  upper  memory  block
     (UMB),  its  large startup memory size (around 50kb) may not
     allow loading into UMB, even if more than 20 kb is available
     after many device drivers are loaded.

     To help this situation, there are two ways:

     1)   concache.exe can be loaded from config.sys command line
          as  a  device  driver,  giving  an earlier chance to be
          loaded into UMB.

     2)   The option load_umb lets concache.exe try to  find  UMB
          of enough size for resident part.


Prerequisites
     Concache.exe runs PC/AT class machine  and  after,  equipped
     with INTEL 80386 CPU and the later, running MSDOS3.1 and the
     later or the equivalents.


Installation
     If ROM BASIC is used then interrupt identification  of  pro-
     grams  have to be adjusted using cctame.exe. Please refer to
     the document.

     Otherwise there is no installation procedure.  Disable  disk
     cache  program  currently  used.   Simply copy to convenient
     place  and  make  calls,   either   from   DOS   config.sys,
     autoexec.bat, or command line, as appropriate.


  Quick Test
     Real programmers may skip to the next section. (Virtuals may
     better not.)



Concache 1.00      Last Update:  18 March 1995                  2
INTRO             COPYRIGHT horio shoichi 1995              INTRO



     If your system is fairly recent one and you  believe  it  is
     configured in just standard way, and you have read the Known
     Incompatibilities section below,  and  is  not  running  the
     other disk cache program, and currently on command line from
     command.com,  then  you  can  test  if  your   system   runs
     concache.exe.  (Sorry  this sentence is so obscure. Although
     there is no 100  per  cent  guarantee,  following  procedure
     should work if DOS 5.0 or the later is used.)

     You should read the related documents to find out  what  you
     have done in this test.

     -    First, go to the directory where concache.exe is copied
          and  type  simply "concache".  You will see the message
          it is not loaded. This message should appear.

     -    Next, type "concache x".  Now  concache.exe  has  allo-
          cated memory for its cache area if it doesn't complain.

          If it complains, try "concache e" and "concache  p"  in
          this order until you succeed.

          If all fails, there is no memory to be used  for  cache
          area,  and  your quick test has failed. You have to add
          memory or you must decrease some other program's memory
          usage.

     -    You should have allocated all  available  memory  of  a
          type.  See  this  by  typing "concache hm", which means
          "help about memory usage".

     -    If you succeeded in "concache "p", you can release  the
          memory  by  typing  "concache  d"  which  means "delete
          resident concache.exe from memory". If  you  have  done
          so,  you might try different amount by typing "concache
          p1m", "concache p256k", "concache p100000", and so  on,
          anything between zero and maximum value reported.

     -    If you have succeeded in "concache x" or "concache  e",
          you  can try above experiment similarly.  Try it later,
          substituting "x" or "e" with "p" as appropriate.

          For now type "concache  xm"  or  "concache  em",  which
          reallocates  zero megabyte (that is, zero byte) memory.
          In other words, it has released  all  allocated  memory
          but concache.exe is still in memory.

          Again type "concache hm" and see it is using none.

          You are now be able to allocate as much memory  as  you
          like  exactly  like  the case "concache 'p" but without
          doing "concache d". This is an advantage over "concache



Concache 1.00      Last Update:  18 March 1995                  3
INTRO             COPYRIGHT horio shoichi 1995              INTRO



          p".

     -    If you have allocated more than 256kb  of  memory,  you
          will make an interesting test at this point.

          Prepare a spare floppy (it must be  formatted,  though)
          into  floppy  drive a:, and type "copy concache.exe a:"
          (or whatever file), and type "concache hd" immediately.
          You  have  told "help about drives" by typing "concache
          hd".

          You see floppy is being written, and you get  the  list
          of how devices are treated by concache.exe.

          You will observe a line "still XX  dirty  spots"  which
          tells  how  many  sectors remain to be written into the
          floppy a:.  If you type "concache hd" again, then  this
          line is probably unseen.

          By allocating larger memory and copying more  files  at
          once  into  floppy, and typing "concache hd" in succes-
          sion, you  will  observe  the  number  of  dirty  spots
          decreasing  toward zero. Chance to observe this line on
          hard disks are less unless a large copy is  done  using
          large memory.

     If you have succeeded up to this point, your remaining thing
     to  do is to edit your autoexec.bat or config.sys file. How-
     ever, please read on.

     You have seen an  advantage  and  a  disadvantage  of  using
     concache.exe.  You  will  feel  comfortable to see that your
     computer responds  quickly  even  if  it  is  busy  writing.
     But...   While  you DON'T HAVE TO wait for the complesion of
     writes to continue work, you MUST wait for it  to  pick  the
     floppy out.

     There are many ways to await the complesion. The simplest is
     watching  the access lamp. and the second is repeatedly typ-
     ing "concache hd" to make sure there is no dirty  spots,  as
     you  have  just learned. Another method is to type "concache
     r" to tell refresh drives.  Then  concache.exe  watches  the
     progress,  and  returns  to command line when no dirty spots
     remains.

     Likewise when you shutdown your machine, there is  a  danger
     of  remaining  dirty  spots.   A recommended procedure is to
     have a shutdown procedure, like shut.bat, which contains  at
     least  "concache  r"  line (after the line which might cause
     the final writes to disks, of course).

     Remark: The author hates  follow  the  piper  approach.   It



Concache 1.00      Last Update:  18 March 1995                  4
INTRO             COPYRIGHT horio shoichi 1995              INTRO



     looks to him like a conspiracy to hide important things.  In
     fact, if you read other text files you will find many impor-
     tant informations are missing in this subsection.  This sub-
     section is written on  the  case  best  fortunate,  to  show
     although  command  line  operands of concache.exe is compli-
     cated, its genuine nature is rather its  simplicity.   Actu-
     ally,  all  but  two  of his DOS machines are set up in this
     simplest way.


  Acceptance Test
     Since there can be  many  unnoticed  incompatibilities,  and
     they  potentially  cause the worst disasters, things may not
     be as simple as preceding subsection.

     In particular, following areas need thoroughly checked.

     - Memory managers.
          Options for "unique" features for a memory manager must
          be checked with special care.  In general, simply using
          XMS or EMS would be well compatible.

     - Communication programs
          There are two considerations to be paid.  One is if the
          CPU overhead concache.exe incurs, or the pattern of its
          CPU usage, can be tolerated by communications programs.
          The  need  to  check  this applies to serial communica-
          tions.

          The other is the possibility of loss of synchronization
          caused   by   mismatched   handling   of  int2a  (which
          concache.exe may be in  part  responsible  for).   This
          caution  applies  to the server side of so called "peer
          to peer" lans.

     - Multitaskers
          Most task switchers seem  to  work.  Multitaskers  need
          special care.

     - Disk compression and encryption programs
          These programs use proprietary  driver  schemes,  which
          concache.exe doesn't know.

     - Third party block device drivers
          By definition, device drivers can do anything, and  can
          be incompatible with concache.exe. However, well behav-
          ing device drivers can coexist. If int13 and int15  are
          used  in  predefined manner, these drivers will happily
          take advantage of concurrency.

     - File maintenance programs
          These programs can use int13 directly, which can  cause



Concache 1.00      Last Update:  18 March 1995                  5
INTRO             COPYRIGHT horio shoichi 1995              INTRO



          the  mismatching  of  cached  data  and  actual data in
          disks. If only int21, int25, int26 are used  then  they
          are generally safe.

     - OEM versions of DOS
          There may be differences from what concache.exe expects
          on MS/PC-DOS, mostly on io.sys/ibmbio.com.

     - Itchy spots
          Following is the list of  current  limitations  of  the
          author's test configurations.

                 Main memories above 10mb for cache,
                 More than one SCSI host adapters,
                 More than two floppies,
                 Other than DOS partitions on SCSI disks.

          Although the author believes there is no critical boun-
          dary  conditions  to  cause  troubles,  any  difficulty
          caused by exceeding above limits is definitely  a  bug.
          You should alert yourself you are crossing the border.


     An alert tester should proceed, if in any doubt, as follows.
     If  needed,  read  concurrency  mode to write after or write
     through mode.

     1) Backup
          Make backup before conducting the tests.

     2) If Ever Loadable
          Load concache.exe setting all drives in stop mode.

     3) The Least Damage
          Put a floppy into concurrency mode. and  try  read  and
          write  on it.  At the end, check the integrity of media
          with, for example, chkdsk program.

     4) Add hard disks
          Put drives, one by one, into concurrency mode, and con-
          duct  test  on them.  Integrity check is equally impor-
          tant.

     5) Finally
          For at least a month after  acceptance,  more  frequent
          backups are scheduled.


Known Incompatibilities
     The programs are tested  against  various  environments  and
     have  identified several incompatibilities, some solved, and
     some given  up.   A  few  command  arguments  try  to  solve



Concache 1.00      Last Update:  18 March 1995                  6
INTRO             COPYRIGHT horio shoichi 1995              INTRO



     problems  rather  non-obvious ways. If you get into trouble,
     please give a look at eqanda.txt  file.   It  describes  the
     backgrounds  of  these  troubles  so you may find the way to
     proceed.

     Following are known incompatibilities:

     1)   Under NWDOS 7.0, writing to a floppy trashes  its  fat.
          To   solve   this,   either   use   write   through  or
          floppies.exe.

     2)   Desqview 2.31 and Desqview/X 1.1 cause crash.  Probably
          the  family  is  incompatible  as  a  whole.  Use write
          through method.

     3)   QEMM 7.3 stealth feature causes  hang  or  crash.  Only
          write through works.

     4)   Windows 3.1 does not implement full EMS 4.0.   Probably
          the  3.0  and  3.11 behave the same way.  EMS memory is
          freed while under it.

          Also, SCSI devices should not be  used  in  concurrency
          mode, takeover or not, where a vm (virtual machine) can
          possibly run background.   The  switching  causes  SCSI
          manager  hang.  For  SCSI disks use write after mode or
          DOS device driver.

     5)   Server.exe of Novell  Personal  Netware  maintains  its
          internal  int2a  counts incorrectly.  It causes hang or
          crash. Write after method solves the problem.

          The other TSRs that intercept int2a for synchronization
          with DOS have potentially the same problem.

     6)   Most block device redirectors such as disk  compression
          and    encryption  programs  are  not  cacheable.  Load
          concache.exe before them in order to make cacheable.

          The only compatibles are Microsoft  Dblspace  and  Stac
          Electronics  Stacker 3.12 to date.  These programs must
          be loaded before concache.exe.

     7)   Applications which use DOS "unused" stack space may  be
          corrupted  by concache.exe assumption of its being free
          at device driver calls.  It uses  rather  larger  stack
          space.


Development Environment
     The primary development environment contains  the  following
     indispensable  softwares to the author.  He devotes a lot of



Concache 1.00      Last Update:  18 March 1995                  7
INTRO             COPYRIGHT horio shoichi 1995              INTRO



     thanks to the authors of these programs.

            Borland C/C++ 4.0   Borland International
            MKS Toolkit 4.1b    Mortice Kern Systems Inc.
            Periscope 5.40


References
     Of numerous books I referenced, following  are  particularly
     important in writing this software.

         IBM Corp. "Technical Reference Personal Computer AT"

         Adaptec "Advanced SCSI Programming Interface (ASPI)"

         Andrew Shulman, et al. "Undocumented DOS" eds 1, 2

         Frank van Gilluwe "The Undocumented PC"

         Ralf Brown, Jim Kyle "PC Interrupts" eds 1, 2

         Joeff Chappel "DOS Internals"

         Microsoft "MS-DOS Programmer's Reference" V5, 6


Trademarks
     Product names in this and other files in  this  package  are
     (registered)  trademarks  of  respective  publisher's  or of
     manufacturer's.


Lastly But Most Important
     Enjoy the parallel world of  DOS  single  tasking  operating
     system !




















Concache 1.00      Last Update:  18 March 1995                  8
