
  tbackup -- general backup system for Linux.

  V0.8

  tbackup  -- do a backup
  trestore -- do a restore
  tlist    -- list contents of a backup
  tguess   -- guess the size of a backup
  tdir     -- create directory sets

   (c) 1993, 1994  Koen Holtman
   This is free software.  Use it at your own risk.
   Tbackup may be freely redistributed provided that
   1) this notice is left intact and
   2) any changes or extensions to tbackup are documented
      as such in this file.
 
  Suggestions, bug reports, etc. can be emailed to the author:  
 
                koen@stack.urc.tue.nl (Koen Holtman).


This is the tbackup manual file /usr/lib/tbackup/README.  You get it
when you do `man tbackup'.

       Using the index: to get to a chapter, e.g. number 42, search
       for _42_ with your reader.

        0 INTRODUCTION
        1 WHY NOT USE TAR?
        2 FAULT TOLERANT ARCHIVE FORMAT
        3 OTHER ADVANTAGES
        4 INSTALLATION
        5 SPACE FOR TEMPORARY FILES
        6 TBACKUP RESCUE DISK

        7 BASIC OPERATION: RUNNING TBACKUP
        8 ERROR RECOVERY AND THE OPTIONS MENU
        9 THE `SET' CHOOSE METHOD
       10 CONFIGURING DIRECTORY SETS
       11 INCREMENTAL BACKUPS
       12 COMMAND LINE ARGUMENTS
       13 KNOWN BUGS

       14 RUNNING TRESTORE
       15 SELECTIVE RESTORE
       16 RUNNING TLIST
       17 RUNNING TGUESS
       18 FREEING HARDDISK SPACE: SELECT, BACKUP, DELETE

       19 EXTENDED FLOPPY DENSITIES
       20 TAPE DRIVES
       21 TBACKUP ON A MULTIUSER SYSTEM
       22 CUSTOMIZATION
       23 FILES AND DIRECTORIES
       24 PERFORMANCE TUNING
       25 KERNEL MESSAGES
       26 ERROR CORRECTING CODES AND BAD FLOPPIES
       27 RESTORING ARCHIVES WITH MISSING DATA
       28 TBACKUP FLOPPY FORMAT
       29 RESTORING FILES WITHOUT TBACKUP

If you already know tbackup version 0.7, an overview of the new
features and changes in version 0.8 can be found in the file
/usr/lib/tbackup/NEW.0.8

There is more documentation (covering the tbackup rescue disk) in
the /usr/lib/tbackup/rescue directory.

The file README.savetree is a shorter version of this README file for
printing.  To avoid wasting paper, it omits the introduction and
installation material and only covers the basic functions of tbackup
and trestore.  Note that V1.0 of tbackup will come with hypertext and
postscript versions of the manual.  README.savetree does not contain
`emergency restore' instructions.  For those, print the
/usr/lib/tbackup/rescue/USE file.


*** _0_ INTRODUCTION

Tbackup (three backup) is a user friendly, fault tolerant package for
making backups of a Linux filesystem.  It is primarily designed to
make multi-megabyte backups to a set of floppy disks.  It has
incremental backups, backup indexes, selective restore, error
correcting codes, and more.


*** _1_ WHY NOT USE TAR?

Archiver programs such as tar(1), cpio(1) and afio(1) also allow making
backups to a set of floppy disks.   But tar stands for tape archiver,
it was not designed with floppy disks in mind.  You only need one tape
to back up a 100 meg hard disk.  But if you don't have a tape drive,
you need 50 or more floppy disks, and several things can go wrong:

1) disk 45 has a bad sector that can't be written to
2) disk 46 was not formatted 
3) you forgot to insert disk 50 and `pressed enter' with disk 49 in the drive 

Tar was not designed to handle such conditions gracefully.  It will
abort on errors 1) and 2), and will not warn you about 3).  The only
way to correct these errors is to start over again at disk 1, and that
is hardly acceptable.

Tbackup was designed to recover from all the errors above (and a lot more).

For small backups of say less than 5 floppy disks you can just as well
use tar (it is even slightly faster than tbackup), but for big backups
tbackup is better.


*** _2_ FAULT TOLERANT ARCHIVE FORMAT

Being able to quickly recover from errors when an archive is created
is nice, but not the most important aspect of a backup package.

It is the restore that really matters.  Like hard disks, the floppies
or tapes used for a backup can develop unreadable spots and give read
errors.  If this happens, the restore program should still be able to
recover as much information as possible.

If you use tar (with some auxiliary program like disksplit(1)) to make
compressed archives on a set of floppy disks, even a single read error
is disastrous.  If one sector on disk 5 is damaged, you not only
loose a part of disk 5, ALL information on disks 6-50 will also be
lost.  Compressed tar archives on tapes have the same problem.

Tbackup makes a fault tolerant kind of compressed archive: only the
the files in the archive are compressed, the file headers are left
uncompressed.  This way, a read error generally only causes the loss
of one or two files.

If floppy disks are used for the backup, tbackup can add error
correcting codes to the data on the floppy.  If a floppy then develops
unreadable spots, the restore program can often reconstruct the
missing data using the error correcting codes.  This way the the
entire archive can be restored even if there are some read errors.


*** _3_ OTHER ADVANTAGES

Apart from the error recovery mechanisms, tbackup has some other
advantages.

- No command line arguments needed:  I don't make backups all day, so
  I can't remember N options.  Tbackup asks for all relevant
  parameters and gives verbose explanations.  (You can still use
  command line arguments if you want to.)

- Hides details: You don't need to have any deep UNIX knowledge to
  use tbackup and understand its error messages.  In particular
  tbackup will hide the weirdness of the Linux floppy drivers. 
  (The drivers don't report write errors to the writing program!!)

- Customizability: You can set default values for parameters and
  preset parameters to a fixed value.

- Backup Index: A tbackup index file not only lists the files in an
  archive but also shows the numbers of the floppy disk the files were
  written to.  This is useful for making quick selective restores.

- Select, backup, delete: tbackup supports a select, backup and delete
  strategy for freeing up space on a nearly full hard drive.  You can
  move infrequently used stuff to floppies and do a selective restore
  if you need it again.


*** _4_ INSTALLATION

You have to be root to install tbackup, as the installation process
creates entries in directories that are owned by root.

1) For running tbackup, you need to have the following software:

     mtools
     fdformat
     gawk
     gzip
     afio (version 2.3.6 or higher)
     Optional: setfdprm for extended floppy disk formats

   The afio archiver program can be found at various ftp sites:

     nic.funet.fi    : /pub/OS/Linux/util/backup/afio-2.3.7.tgz
     sunsite.unc.edu : /pub/Linux/system/Backup/afio-2.3.7.tgz
     ftp.win.tue.nl  : /pub/linux/tbackup/afio-2.3.7.tgz

   Setfdprm seems to be included with all newer linux distributions, I
   don't know when it first appeared.

2) If you have no older tbackup version installed, go to 3).

   If you do have an older tbackup version installed, remove it first
   by typing
 
     rm -r /usr/lib/tbackup

   Existing tbackup directory set configuration and log files in
   /etc/tbackup can still be used with the new tbackup version.

   Existing .dea and .arg configuration files in /etc/tbackup can
   still be used.  .set files that include `not regex' instructions
   will need to be modified: the `not regex' instruction is replaced
   by a shell pattern facility in V0.8.

3) cd to the / directory and untar the tbackup archive.
   The files unpack to the directory /usr/lib/tbackup.
   The subdirectory /usr/lib/tbackup/examples contains some sample
   configuration files, the directory /usr/lib/tbackup/bin contains
   various shell scripts and programs that are used internally by tbackup.

   If you don't like to have the tbackup files under /usr/lib, you can
   move them somewhere else and put a symlink in /usr/lib, e.g.
    cd /usr/lib 
    mv tbackup /usr/local/lib
    ln -s /usr/local/lib/tbackup tbackup

4) cd to /usr/lib/tbackup

5) Type `make', this will compile some C programs that tbackup uses
   internally.

6) Type `make install'.  This will:

   a) Make symbolic links in /usr/bin to the tbackup, trestore,
      tguess, tlist, and tdir commands in /usr/lib/tbackup.  It will
      also make symbolic links in /usr/man/cat1, providing a manual
      page for tbackup (this README file) and a manual page for tdir
      (the tdir.man file in this directory).

      Edit the Makefile before installation if you want to change the
      target directories /usr/bin and /usr/man/cat1 or if you want
      file copies in stead of symbolic links.

   b) It will create the configuration directory /etc/tbackup if it
      doesn't already exist, then put the files `Config' and
      `Compr.ext' in /etc/tbackup unless these files already exist.
      If a pre-V0.8 `Config' file exists some new fields are added.
      It also puts some standard `.arg' files in /etc/tbackup if they
      don't already exist.

   c) Next `make install' will create a directory ~/idx (idx under
      roots home directory) for storage of index files, unless you
      have one already.  You can move this directory later if you want
      to store the index files somewhere else, see the section
      `SELECTIVE RESTORE' subsection `Creating index files'.

   d) Finally you will be asked about your floppy drive configuration.

      You will also be asked if you want to disable the extended
      floppy density option. 

      Extended floppy densities fit more data on a floppy by going
      outside the `factory specifications' of the floppies and drives.
      For example, on 3.5" DD floppies, the normal double density
      format uses 80 tracks and 9 sectors per track, the `extended
      double density' format uses 82 tracks and 10 sectors per track,
      which fits 820 Kb. on the floppy.

      Not all floppies and/or floppy drives are able to go outside
      their `factory specifications' and hold extra data.  Also, due
      to limitations in the Linux kernel, reading and writing 1.7
      Mb. on 3.5" HD floppies is extremely slow.

      You need to have the `setfdprm' program to use extended
      densities.  See the section on `extended floppy densities' for
      more information.

      If you disable the option tbackup will not bother you with
      questions about extended densities when running.  If you change
      your mind about using extended densities later, you can delete
      the file /etc/tbackup/Floppy and run `make install' again.

7) If you have a tape drive and want to use it with tbackup, you need
   to set the tape device name and optionally some other things in the
   /etc/tbackup/Config file.  The configuration instructions are in
   the section about tape drives.


*** _5_ SPACE FOR TEMPORARY FILES

Tbackup needs free space on your harddisk to store temporary files. If
you're backing up to floppy disks, tbackup needs space to store two
floppy contents. (E.g. 2.88Mb if you're backing up to 1.44Mb disks.)
This space is used to store the disk image that is currently being
written to floppy, and as a working area for preparing the next part
of the archive.

If there is insufficient free space, tbackup will refuse to run.

By default, tbackup creates a directory `bulkdir' under /tmp for
temporary storage of floppy contents.  If you don't have much free
space on the filesystem carrying /tmp you may want to locate `bulkdir'
on another partition.

Suppose you have two partitions and the `df' command shows:
  #df
  Filesystem         1024-blocks  Used Available Capacity Mounted on
  /dev/hda1              54315   53028     1287     98%   /
  /dev/hda2              44004   32562    11442     74%   /usr
  #
(/tmp is on /dev/hda1 in this case).
In this situation, you will probably want to use the /usr filesystem
for temporary storage.  To do this, create a directory /usr/tmp, and
edit the file `/etc/tbackup/Config'.  Change the line

    export bulkdir=/tmp/bulkdir
 to
    export bulkdir=/usr/tmp/bulkdir
                    ^^^^^^^ 
There is no need create the directory bulkdir under /usr/tmp, this
directory is created and deleted automatically by tbackup.

In addition to the bulkdir for large temporary files, tbackup always
needs some 15Kb free in /tmp (on the /tmp partition) to create small
temporary files.  This should pose no problem as far as disk space
requirements are concerned.  However, beware of running a program that
completely fills the /tmp partition during a backup or a restore: if
it can't make small files under /tmp tbackup may fail or crash in
unexpected ways.


*** _6_ TBACKUP RESCUE DISK

This completes the tbackup installation on your harddisk.  If your
tbackup copy on harddisk is destroyed, you need to reinstall it from
floppy.  This can be done in an easy way by using a `tbackup rescue
disk'.  The directory /usr/lib/tbackup/rescue contains tools for
making a rescue disk.  It also contains documentation files on on
rescue disks and restoring a system from scratch.

There is no need to make a tbackup rescue disk right now, you should
make it once you start using tbackup to back up your whole system.


*** _7_ BASIC OPERATION: RUNNING TBACKUP

tbackup can only be run by root. Type `su' first if you are not logged in
as the root user.

To make a backup, just type `tbackup' at the command line.  The program
will ask for parameters. 

Tbackup more or less explains itself when running. Below is a
transcript of a sample session.  All lines starting with
  |  
are output, the rest is comments. User input is indicated with `^^^^^^'.

It is probably a good idea to run tbackup yourself while reading this
transcript. 

[Note: this transcript is made with an earlier tbackup version, the
output sometimes differs slightly from what you would get running this
version.]

  -------------------------------------------------------------
  | #tbackup
     ^^^^^^^
  | 

First, tbackup needs to know which files you want to backup.  There are
several ways of choosing these files, each way is called a `choose
method'.

A `method' is really a small program that is run by tbackup.  Apart
from the `choose method', which selects the files to backup, there is
also the `pack method' that packs these files into an archive, and
the `write method' that writes the archive to a backup medium (usually
floppy disks).  Thus tbackup runs three methods to do the work.  This
is where the name `three backup' comes from.

Currently tbackup has three different choose methods: `set', `setrm',
and `one':

  |Available choose methods:
  |  1 set   -- back up a directory set
  |  2 setrm -- back up a directory set, then remove its contents
  |  3 one   -- one user specified file or directory and below
  |Enter method name or number ---> [set] one
                                          ^^^

tbackup always uses `--->' to indicate a question. 
The string between [ and ] is the default answer, you can select this
by just pressing enter.

For this demonstration, we use the `one' choose method.  For making
real backups you will probably use the `set' method.  The `set' method
is treated in a separate section below.

If you discover that you gave the wrong answer to a question, you can
always safely abort tbackup by typing ctrl-C. 

If all questions are answered and the backup begins you can still
abort tbackup, but this will be a bit more messy; after pressing
ctrl-C you may have to kill some processes that were created by
tbackup, and clean up some temporary files by hand.

Back to our transcript. Choose method `one' now asks:
  | 
  | Type the name of the file or directory to back up. (e.g. /usr/src)
  | This must be a full pathname starting with / .
  |   All subdirectories will also be backed up.
  |   Symbolic links are not followed but will be stored as symbolic links.
  | 
  | file/directory name ---> /usr/src/txt
                             ^^^^^^^^^^^^ 
This will make a backup of the files in /usr/src/txt and every
directory below.

  | 
  |  Size in kilobytes: 2023        /usr/src/txt
  | 
  | 

Now you need to choose a pack method.  This method packs all the files
selected by the choose method into one archive. There are currently
four pack methods supplied, see the list below.  Note that you need
to install the `afio' program for using methods `afio' and `afio0',
`tar' for the `tar' method, and `cpio' for the `tarcpio' method.

You will probably always want to use the `afio' method, this is the
one that produces fault tolerant compressed archives.  `tar' and
`tarcpio' are only there to make tbackup look more powerful.  No `tar
+ compress' method is provided, as partially damaged compressed tar
files are (almost?) impossible to recover.

  | Available pack methods:
  |   1 afio  -- pack files with afio, compress with gzip
  |   2 afio0 -- pack files with afio, do not compress
  |   3 tar     -- pack files with tar (no compress)
  |   4 tarcpio -- make a tar archive with cpio (no compress)
  | Enter method name or number ---> [afio] 
                                            ^(we just press enter)

The write method writes the archive created by the pack method to a
backup medium. The main purpose of tbackup is to make backups
to floppy disks, and this is done by the `floppy' method. 
  | 
  | Available write methods:
  |    1 filedev -- write archive to file or device.
  |    2 floppy  -- write to msdos floppy disks with mtools
  |    3 tape    -- write to tape
  |    4 null    -- copy archive to the bit bucket.
  | Enter method name or number ---> [floppy] 2
                                              ^
  | 
  | Write to floppy drive number (0/1) ---> [1] 1
                                                ^
  | 
  | Use double or high density floppies (d/h) ---> [d] 
                                                       ^(we just press enter)

Depending on you answers in step g) of the installation, you may also
get the options `xd' and `xh' (extended double and extended high) in
the above question.  See the section on `extended floppy densities'
for more information.

  | Use error correcting codes?---> (y/n) [y] 
                                              ^(we press enter)
  |
  | 
  |   Part size is 672 Kb.

In this case drive 1 is a 3.5" drive, so the backup is written to
720Kb floppies.  Each floppy will hold 672 Kb of data, the remaining
48 Kb is used for an msdos directory structure, to store information
about the backup set, and for the error correcting codes.

  | 
  | Always format inserted disks?---> (y/n) [n] 
                                                 ^(we just press enter)

  | 
  | Verify after write to disk?---> (y/n) [n] 
                                               ^(we just press enter)
  |

Now tbackup asks if it should make an index file of this backup.
Making index files is useful if you plan on doing selective restores.

For the moment, we don't make an index file, we will cover them in the
section on selective restores.

  | Make a backup index file ---> (y/n) [y] n
                                            ^
  |
  |   [The backup progress will be displayed on virtual console 6]

If you are running tbackup under X or from a remote site, it creates
am X window for showing the progress messages of the pack method.  You
can edit the /etc/tbackup/Config file to change the place where the
progress is printed.

All parameters are set, and the backup begins. 

  |  [Creating directory /tmp/bulkdir for temporary storage.]
  |  [Allocating space for temporary storage of floppy contents.]
  |
  | Backup description:
  |     Backup of directory /usr/src/txt.
  |     (Filenames in archive relative to /usr/src)
  |     Made with tbackup (V0.8)
  |     by root@noodle, on Sat Jul 17 22:25:40  1993
  |
  | 
  | Backup in progress...
  | 
  |
  |                         [waiting for part 1 to be completed]

Tbackup begins creating the first part of the backup set, and also
prompts for a floppy:

  | --Insert an empty, formatted disk for part 1
  |   in drive 1 and press enter. (Type o for options.) 
                                                         ^(we press enter)

Insert the first floppy disk and press enter. Typing `o' and enter
will get you to the `options menu'. More about that later.

Tbackup will check if the disk you inserted in drive 1 is formatted in
the right way, and wait until part 1 is ready to be written:

  |                             [waiting for part 1 to be completed]

If part 1 is ready, you see

  | Writing part 1 to drive 1 (i interrupts) ....
  | VFS: Disk change detected on device 2/1
  | VFS: Disk change detected on device 2/1

The two "VFS: Disk change detected on device 2/1" messages above are
printed by the kernel.  Tbackup interprets all floppy related kernel
messages for you, so don't worry if you don't understand them.

If you are running tbackup under X, you won't see the `Disk change
detected' messages: tbackup suppresses these particular ones because
their information content is zero.  If you are running tbackup on a
virtual console, you will see all kernel messages unless you have
switched off kernel message printing with an external program such as
colog(8).  More about kernel messages later.

While part 1 is written to floppy, tbackup is also preparing part 2.
If your computer is fast enough (read the section on performance
tuning if it isn't) part two is ready by the time part one is
completely written.

  | Finished writing part 1.
  | 
  | --Insert an empty, formatted disk for part 2
  |   in drive 1 and press enter. (Type o for options.) 
                                                         ^(we press enter)
  | Writing part 2 to drive 1 (i interrupts) .....

You can type `i' to interrupt the write operation and try again.  If
you press `i' it will take a few seconds for it to be `noticed'.  In
rare cases, manually interrupting a failing write operation is much
faster than waiting for the automatic error recovery.

  | VFS: Disk change detected on device 2/1
  | VFS: Disk change detected on device 2/1
  | Finished writing part 2.  [you can remove the floppy]
  | 
  | Cleaning up...
  | 
  | Backup completed.
  | #
  ----------------------------------------------------


*** _8_ ERROR RECOVERY AND THE OPTIONS MENU

Here is a list of errors that are detected by the floppy write method.

- formatting failed
- writing failed
- verify failed
- kernel reports floppy errors

- no disk inserted
- disk not preformatted, or formatted with the wrong density
- expected unformatted disk or disk formatted with the wrong density,
  but this disk is correctly formatted

- disk already contains a part of the current backup set 
  (i.e. forgot to insert a new disk)


If an error occurs when writing a backup, tbackup prints an error
message and then displays the `options menu', so that you can choose
what to do next.  With some floppy-related errors, it can take a long
time for the error to be `noticed'.  This is caused by the Linux
floppy driver, it may spend large amounts of time retrying an
operation if it fails.

Here is what happens if you forget to insert a floppy disk:
 
  ------------------------------------------------------------------
  | --Insert an empty, formatted disk for part 1
  |   in drive 1 and press enter. (Type o for options.) 
                                                         ^(we press enter)
  | Writing part 1 to drive 1 (i interrupts) .......

Not having a floppy in the drive first causes the kernel to complain:
  | floppy I/O error
  | dev 0211, sector 0
  | floppy I/O error
  | dev 0211, sector 0
  | floppy I/O error
  | dev 0211, sector 2
  | floppy I/O error
  | dev 0211, sector 4
  | floppy I/O error
  | dev 0211, sector 6

  | head: /dev/fd1H720: I/O error
  | 
Tbackup prints a more meaningful description of the error:

  | **ERROR**: no disk inserted, disk is not formatted, or disk is not
  |            formatted with the right density.

Now comes the options menu:
  | 
  |  Enter one of the following commands:
  |                            (You may want to insert a new disk first.)
  |  r  : retry writing part 1 to drive 1
  |  f  : do a low-level format on the disk in drive 1
  |  fn : do a low-level format without verify on the disk in drive 1
  |  c  : change format/verify options
  |  q! : quit trying to write this part, continue with the next part
  | -->

The options should be pretty much self-explanatory. 

There is no option for aborting the entire backup.  You can do this by
pressing ctrl-C if you want to.  After pressing ctrl-C you may have to
kill some processes and clean up some temporary files left by tbackup.


You can also get to the `options menu' by typing `o' and enter if you
are asked to insert a new disk.  Suppose that you run out of
preformatted disks when doing a backup. You can then type:

  | --Insert an empty, formatted disk for part 5
  |   in drive 1 and press enter. (Type o for options.) o
                                                        ^
And use the `c' command to change the settings for formatting and
verifying disks. 

  |  Enter one of the following commands:
  |                            (You may want to insert a new disk first.)
  |  r  : retry writing part 5 to drive 1
  |  f  : do a low-level format on the disk in drive 1
  |  fn : do a low-level format without verify on the disk in drive 1
  |  p  : write the previous part (number 4) again
  |  c  : change format/verify options
  |  q! : quit trying to write this part, continue with the next part
  |  -->c
        ^
  |  Always format inserted disks?---> (y/n) [n] y
                                                 ^
  |  Verify format?---> (y/n) [y] y
                                  ^
  |  Check if disk is already formatted (with the right density)
  |  before formatting?---> (y/n) [n] n
                                      ^
  |  Verify after write to disk?---> (y/n) [n] n
                                               ^
After this we get back to our options menu.  Note the change in
the `r' command and the disappearance of the `f' and `fn' commands. 
  |  
  |   Enter one of the following commands:
  |                             (You may want to insert a new disk first.)
  |  r  : retry format and writing part 5 to drive 1
  |  p  : write the previous part (number 4) again
  |  c  : change format/verify options
  |  q! : quit trying to write this part, continue with the next part


  |  -->r
        ^
Use the `r' command to write part 5.  Note that this is not really a
`retry', as tbackup has not yet tried to write part 5.

  |_____________________________________________________________

tbackup will now proceed to format the blank disk you have inserted,
and write part 5 to it.  If all goes well, you will be asked to insert
the next disk for part 6.  If there is another error, you will get
back to the options menu.


Suppose the floppy with part 5 on it self-destructs the moment you
eject if from the drive.  If this happens, you can use the `p' command
in the options menu to make a new part 5:

  | --Insert an empty, formatted disk for part 6
  |   in drive 1 and press enter. (Type o for options.) o
                                                        ^
  |
  |   Enter one of the following commands:
  |                             (You may want to insert a new disk first.)
  |  r  : retry format and writing part 6 to drive 1
  |  p  : write the previous part (number 5) again
  |  c  : change format/verify options
  |  q! : quit trying to write this part, continue with the next part
  |  -->p
        ^ 
I doubt you will need the `p' command very often.  Apart from self
destructing floppies, you can use it if you have second thoughts about
the quality of the floppy a part was just written to.  Tbackup has no
perfect way of detecting bad floppies, and sometimes you may know
something that tbackup doesn't.

The `p' command is only enabled just after part 5 is written, once you
start writing part 6 there is no way to go back to part 5.


*** _9_ THE `SET' CHOOSE METHOD

The `set' choose method is the main device for making backups of your
hard drive.  With this method you can make a backup of a so-called
`directory set'.

A directory set is basically a list of directory names. All files,
symbolic links, etc. in these directories and in their subdirectories
are part of the set.

You can make a full backup of a set, meaning a backup of everything in
it, and you can make two types of incremental backups.  An incremental
backup only backs up all the files in the set that were changed after
the last (full or incremental) backup.  More about incremental backups
later.

It is usually convenient to divide your hard disk into several
directory sets, for example one set for the `system' directories and
one for the `user' directories, or one set for each hard disk
partition.

Tbackup does not offer a standard directory set setup, you have to
configure the sets yourself.  Directory sets are specified by putting
files in /etc/tbackup, this will be the subject of the next section.

Here is a sample transcript to show how the `set' method works.

  -------------------------------------------------------
  |#tbackup
    ^^^^^^^
  |
  | Available choose methods:
  |    1 one -- one user specified file or directory and below
  |    2 set -- back up a directory set
  | Enter method name or number ---> [set] set
                                           ^^^
On my machine, I have divided the hard disk into 3 directory sets.
  | 
  | Available directory sets:
  | 1    ext        # extended filesystem
  | 2    mystuff    # my stuff in the /root directory
  | 3    system     # root filesystem, without /root dir.
  | Enter set name or number ---> mystuff
                                  ^^^^^^^
Tbackup now lists the dates of the previous backups of mystuff. 

  | The last full backup of mystuff was made on Mon Aug 16 04:14:36 1993,
  | that was approx. 5 days ago.
  | 
  | The last incremental backup of mystuff was made on Tue Aug 17 
  |                                                          22:57:47 1993,
  | that was approx. 4 days ago.
  | 
  | Choose a backup level.
  |  f  = full backup of all files
  |  if = incremental backup of all files changed since last full backup
  |  ii = incremental backup of all files changed since last incremental backup
  | ---> [f] ii
             ^^ 
There are two styles of making an incremental backup.  Which one works
best for you depends on the circumstances.  `ii' generally produces
smaller archives, but `if' allows you to recycle all disks used for
previous incremental backups.

If you have never made a full backup before, the `if' and `ii' options
won't be available.  If you have never made an incremental backup,
`ii' won't be available.

After choosing the backup level tbackup proceeds as usual:
  | 
  | Available pack methods:
 [etc.]
  -------------------------------------------------------------------

Information about backups of the `mystuff' directory set is recorded
by tbackup in the file /etc/tbackup/mystuff.log .  This information is
is used when an incremental backup is made.

If the backup is completed tbackup asks:

  -----------------------------------------------------------------------
  | Please confirm that this backup was successful.
  |
  | Add information about this backup to the mystuff
  | backup history logfile?---> (y/n) [y] 
                          (we press enter)^
  | Backup log file /etc/tbackup/mystuff.log updated.
  ------------------------------------------------------------------------

Answering 'y' causes the logfile to be updated, so that the current
backup is logged as the last (incremental) backup made.  You should
answer `n' if something went wrong and tbackup didn't notice it.

In case you are curious, here is what such a log file looks like:
(All lines in the file are preceded by ` >'.)

  > # Backup history log of the mystuff directory set.
  > f  081817171993 727675 --- Full / Mon Aug 18 17:17:18 1993
  > if 081917271993 727676 --- Incremental (on full) / Thu Aug 19 17:27:14 1993

Do not change these log files by hand, tbackup does all updating
itself.


*** _10_ CONFIGURING DIRECTORY SETS

A directory set named `foo' is made by putting a configuration file
called `foo.set' in the directory /etc/tbackup.  This file can be made
with a normal editor, but also with the `tdir' command.  The tdir
command is mainly useful for making temporary directory set files, we
will cover it in the `freeing harddisk space' section below.

The presence of a file /etc/tbackup/foo.set causes the directory set
to be listed as

  | 1    foo        # a set called foo

Where the `# a set called foo' part is the first line of the foo.set
file.

A *.set file may contain:

- lines starting with a #, these are comments
- empty lines

- lines that contain a directory or file name starting with a /.

- lines that contain `not' instructions which cause files to be
  excluded from the backup process.

Here is an example foo.set file:

  > # a set called foo
  > 
  > #these are the directories in the set
  > /usr/foo
  > /usr/src/foo
  > 
  > # some things not to back up
  > not name core
  > not ext .o
  > not ext .aux .dvi
  >
  > #don't back up this subdirectory
  > not dir /usr/foo/pux/tmp
  >
  > #don't back up .a files below /usr/src/linux
  > not fullname /usr/src/linux/*.a

A backup set consists of all files in the specified directories and
their subdirectories, minus the files that are excluded by `not'
instructions.

A full backup of a directory set backs up all files in the set.
Symbolic links are not followed but stored as symbolic links.  Don't
name one directory twice in a .set file, this will cause it to be
included twice in the archive.


* `Not' instructions 

Below are the available `not' instructions for excluding files from a
directory set.

not name n1 n2 ...
  Exclude all files named n1, n2, ... from the set.
  Example: `not name core' excludes all core files.
           `not name a*.log' excludes all files matching this shell
           pattern.

not ext e1 e2 ...
  Exclude all files whose name end with e1, e2, ... from the set.
  (ext stands for extension)
  Examples: `not ext .o' excludes all object files.
            `not ext o' excludes all files ending with an o
            `not ext .aux .log .dvi' excludes temporary TeX files.
  WARNING: Do not use `not ext .o' for the directory set containing
           the /usr/lib directory: this would cause the files
           /usr/lib/crt0.o and /usr/lib/gcrt0.o to be excluded from
           the backup. If these files are missing gcc(1) can't compile
           any programs.
           
not dir d1 d2 ...
  Exclude all files in the directories d1, d2, ... and the directories
  below from the set, where d1, d2, ... are full directory names
  starting with a `/'.   
  Examples: `not dir /usr/tmp' excludes this temporary directory.
            `not dir /usr/man/man[567] /usr/man/cat[567]' excludes all manual
            pages from sections 5,6, and 7.
  Note: The set
           > /usr/src
           > not dir /usr/src/linux
           > /usr/src/linux/boot
  does NOT contain the files in /usr/src/linux/boot!!!! The `not dir'
  instruction is applied to ALL (sub)directories in the set, so the
  /usr/src/linux/boot line has no effect at all.

not fullname f1 f2 ...
  Exclude the files f1, f2, ... from the set, where f1, f2, ... are
  full pathnames starting with a `/'.
  Examples: `not fullname /usr/lib/dutch.hash' excludes this file
            from the set.
            `not fullname /usr/src/linux/*.[oa]' excludes all .o and .a
            files in the /usr/src/linux directory and below    

The above `not' instructions all take shell patterns as arguments.
These patterns can contain the following special constructs:

   * : matches any sequence of zero or more characters
       (the sequence may contain / characters)
   ? : matches any single character

   [...]  : matches any of the enclosed characters, also
   [b-f]  : matches the range b, c, d, e, and f, and
   [^...] : matches all characters but ...

   \c : matches the character c, mainly used to specify the special
        characters * ? [ ] ^  \ and - if it appears between [ ].


* Trying out a directory set

If you have made a directory set, a fast way to `test' it is by
running the tguess command on it (tguess is covered later on in this
manual).  Tguess can also produce a `guess index' listing all files in
the directory set.

Some examples of .set files can be found in /usr/lib/tbackup/examples.

Here is the file system.set from this directory:

  > # root filesystem, without /root dir.
  > 
  > /
  > 
  > #these are in other sets
  > not dir /c
  > not dir /ext
  > not dir /root
  > 
  > #exclude (own) temporary files and the /proc filesystem.
  > not dir /tmp
  > not dir /proc
  > 
  > not name core
  > 

Note that the other filesystems (/c, /ext, and /proc) are excluded
with `not dir' instructions.


* Potential problems

Make sure that your backup sets don't contain the tbackup temporary
directory /tmp/tbup or the proc filesystem /proc.  Backing up the proc
filesystem wastes space, and a restore of an archive including
/tmp/tbup will almost certainly fail.

The contents of the directory /tmp/bulkdir (or whatever `bulkdir' you
have configured in the /etc/tbackup/Config file) are always excluded
automatically from all backup sets.  This way you won't have to edit
directory set files if you relocate this temporary directory by
editing /etc/tbackup/Config.


The implementation of the `not dir' command is a bit inefficient. When
backing up the set

  > /usr/X386
  > not dir /usr/X386/lib/fonts

tbackup will look at all files in /usr/X386/lib/fonts before deciding
it doesn't need them.  This takes a few seconds, but as tbackup is
usually waiting for the floppy drive anyway this doesn't matter much.

There is however a special optimization to prevent tbackup 
from looking at all files on (say) a cdrom filesystem mounted
as /cdrom, as that would take too much time.

This optimization only works
 - for `not dir' directories directly under /
 - if `/' is the only entry in the set file apart from `not' instructions.

For example, with the following set the files on the cdrom drive and the
dos partitions will not be looked at:

  > #back up everything under / except the cdrom and dos partitions
  > /
  >
  > #but don't back up the cdrom drive
  > not dir /cdrom
  > #and the msdos partitions
  > not dir /dosc /dosd
  >
  > #other stuff not to include
  > not dir /tmp /proc
  > not dir /usr/*/fonts
  > not name core

Note that, for example, a cdrom mounted under `/mountpoints/cdrom5'
will be traversed, unless of course there is a `not dir /mountpoints'
instruction.


* Expert directory set commands

In addition to the normal `not' instructions, there are two more
instructions for influencing the contents of a directory set.  With
them you can `hack' the directory set mechanism to do special things.

These are called `expert commands' because tbackup does not provide
any error checking or recovery for them.

Tbackup uses the find(1) command internally to make directory sets.
With the two expert commands you can influence this find(1) directly.

The commands are:

not findtest <find test>
  excludes all files that make the <find test> true.  See the
  find(1) manual page for a description of valid predicates.
  Example: `not findtest -type p' excludes all files with type p,
           that is named pipes, from the backup.

findtest <find test>
  only include files that make the <find test> true.
  Examples: `findtest -user root' only backs up the files owned by root
            `findtest -cnewer /etc/installdate' only includes the files
            that were created or last modified after the `ls -l' date
            of /etc/installdate. (I made /etc/installdate just after
            installing a slackware distribution from scratch. This command
            thus excludes all files that came from the slackware floppies.)
            `findtest -xdev' causes find not to cross filesystem borders
            when descending into subdirectories.
   
Always test a directory set with a `findtest' command: the command may
sometimes have unexpected results.

If you use wildcards in find tests, you must quote them as you would
in the shell, e.g.
    >  not findtest -lname '/tmp/*'

 
*** _11_ INCREMENTAL BACKUPS

An incremental backup backs up all `new things', that is all files and
symbolic links in a set that were created or modified after the
previous (full or incremental) backup.

This includes the files that were `installed' (e.g. extracted from a
tar archive) after the last backup.  If you do an `ls -l <filename>'
on such a file you usually don't get the date it was created
(installed) on your system, but the date it was last modified by the
author of the tar file.  To see the `created/last modified' or
`last status change' date that is used for incremental backups, do `ls
-cl <filename>.

* Potential problems

An incremental backups usually cost much less time than a full backup,
but incremental backups do have their shortcomings.

An incremental backup does not record
 - information about files that were deleted since the last backup.
 - information about files and directories that were moved (renamed)
   since the last backup.

Thus a restore of a full backup followed by a restore of (one or
more) incremental backups will usually not reproduce an exact copy of
the backed up filesystem: files may have the wrong name, and files
that were deleted after the full backup will be present again.

So if you do a drastic reorganization of your filesystem, it is best
to make a new full backup.

Another problem with incremental backups is that files moved *between*
directory sets may never get recorded in any incremental backup.  This
is possible because moving a file does not affect its `created/modified'
date.

Consider the following scenario:
 1) you create a file `important' in a directory belonging to the `system' set
 2) you make a full backup of the `user' set
 3) you move the `important' file to the `user' set
 4) you make an incremental backup of the `system' set
 5) you make an incremental backup of the `user' set

Step 2) and 4) did not back up `important', but step 5) didn't either
because `important' was created/modified before, and not after, the
previous backup of `user'.

If you want to move files or directories between directory sets, first
`synchronize' the sets by making an (incremental) backup of both.
After you have done that, you can safely move the files without
worrying about the above scenario.

Of course, the same kind of synchronization needs to be done if you
move directory names from one directory set description (.set) file to
another.


*** _12_ COMMAND LINE ARGUMENTS

Tbackup comes with four preconfigured command line arguments.  These
are useful for reducing the number of questions tbackup asks before
beginning the backup.  They cause tbackup to set various parameters
to `standard' values without asking.

The arguments are are `0d', `0h', `1d' and `1h'.  You can invoke
the argument `0d' by typing either `tbackup -0d' or `tbackup 0d'.

The arguments have the following meaning:
 0d: write to drive 0, use the `d' (double) density
 0h: write to drive 0, use the `h' (high)  density
 1d: write to drive 1, use the `d' (double) density
 1h: write to drive 1, use the `h' (high)  density

In addition to setting drive and density, all four arguments cause
tbackup to use the following settings for various parameters without
asking:
 choosemethod: set
 packmethod: afio
 writemethod: floppy
 format inserted disks: no
 verify after write: no
 make error correcting codes: yes
 create an index file: yes; automatically generate the index file name.
         
So if you want to back up a directory set to formatted HD floppies in
drive one, you can type

 tbackup 1h

and will only be asked for the directory set name and backup level.

You can change the settings made by these arguments by editing the
*.arg files in /etc/tbackup.  See the `customization' section for more
information.


*** _13_ KNOWN BUGS


1) Archives packed with the `tar' method do not contain the owner and
   permission information of the stored directories.

   If such an archive is restored, the directories created by tar will
   get some default permissions.  The permissions of the files in the
   directories will be restored correctly.  Thus bug makes the `tar'
   method useless for creating real backups.

   To circumvent this bug, install cpio(1) and use the `tarcpio' pack
   method instead of `tar'.


2) In some cases it may take an EXTREMELY long time (a couple of
   minutes) for tbackup to notice and recover from an error.  This is
   mainly due to the weirdness of the Linux floppy drivers. Pressing
   `i' during a failing write will shorten this time, but it may
   remain extremely long nevertheless.

   In particular, be careful using floppy disks that were formatted by
   the MS-DOS `Central Point Backup' program.  If these floppies were
   formatted in the `CPS floppy format' in stead of the normal `DOS
   standard format', they are not suitable for direct use by tbackup
   but need to reformatted first by switching on the `always format
   inserted disks' option.

   Tbackup has no way of detecting the `CPS floppy format' if such a
   disk is inserted: it will think that it is a correctly formatted
   disk and start writing to it.  Of course this write operation will
   fail, but tbackup (the floppy drivers) will take an insane amount
   of time (probably hours, I have not checked this) retrying the
   write before reporting failure.
 

--------------------------------------------------------------------------

          The sections below can be skipped at first reading.

	  Some of these sections cover very specific situations,
          there is no need to read them unless such a situatuion
	  occurs.

--------------------------------------------------------------------------


*** _14_ RUNNING TRESTORE

To restore a backup you can use the command trestore.

Here is a sample transcript.

  --------------------------------------------------------
  | #trestore
     ^^^^^^^^
  | 
  | Available read methods:
  |   1 filedev    -- read archive from file or device.
  |   2 floppy     -- read from msdos floppy disks with mtools.
  |   3 floppypart -- read a part of a floppy archive
  | Enter method name or number ---> [floppy] 2
                                              ^
  | 
  | Read from floppy drive number (0/1) ---> [1] 1
                                                 ^
  | 
  | Insert one disk from the backup set in drive 1 and press enter.
                                                    (we press enter)^
  | 

Now the floppy method reads a description of the backup set from the
disk and shows it to you. 

  | Backup description:
  | 
  |   Backup of directory /usr/doc.
  |   (Filenames in archive relative to /usr)
  |   Made with tbackup (V0.1 alpha prerelease)
  |   by root@noodle, on Mon Jul 19 10:54:07  1993
  | 
  | Restore parameters of this backup set:
  |   dea_choosemethod=one
  |   dea_onename=/usr/doc
  |   dea_packhome=/usr
  |   dea_unpackmethod=afio
  |   dea_readmethod=floppy
  |   pre_mdens=d
  |   pre_eccmethod=blexta08
  |   pre_psize=727040

Don't worry if you don't understand the meaning of the `restore
parameters' above, they are mainly there for internal use. 

  | 

Note that the `filedev' and `tape' methods do not offer the above
information.  If you write archives with `filedev' or `tape' you must
record the necessary information (contents, pack method used, ...)
yourself.

  | Press enter to use this backup set, ^C to abort.
                                                     ^(we press enter)
  | 
  |  Part size is 710 Kb.
  |
  |  [Creating directory /tmp/bulkdir for temporary storage.]
  |  [Space free in /tmp/bulkdir: 1746 Kb.]

Trestore needs enough temporary space to store one part, plus a safety
margin of 100 Kb.  If there is insufficient space, trestore will refuse
to run.

  |
  | This archive was packed with the `afio' method.
  | 
  | Available unpack methods:
  |   1 afio     -- unpack files with afio, uncompress files with gzip
  |   2 afio0    -- unpack files with afio. (no uncompress)
  |   3 selafio  -- selectively unpack files with afio, uncompress with gzip
  |   4 selafio0 -- selectively unpack files with afio. (no uncompress)
  |   5 seltar   -- selectively unpack files with tar (no uncompress)
  |   6 tar      -- unpack files with tar (no uncompress)
  | Enter method name or number ---> [afio] 
                                            ^(we press enter)
  | 

Now the unpack method asks where you want to put the contents of the
archive.  You usually want to put them where they came from:

  | This archive was packed relative to the directory /usr.
  | Unpack archive in directory ---> [/usr] 
                                            ^(we press enter) 

If you are using tbackup to transport files from one machine to another,
you may want to unpack the files in another directory, say `/var/incoming'.
This way, the files
       /var/incoming/doc/first.document
         ...
       /var/incoming/doc/last.document
are created. 

  | 
  | Protect newer files from overwriting?---> (y/n) [y] y
                                                        ^
  |

A restore index file lists all files that are restored, plus possible
unpacking error messages.  There's not much sense in making such a
file in most cases.

  | Make a restore index file ---> (y/n) [n] n
                                             ^
  |   [Creating restore progress window]
  | 
  | Restore in progress...
  | 
  | --Insert the disk with part 1 in drive 1
  |   and press enter. (Type o for options.) 

To show the error handling, we remove the disk from drive 1,
and then press enter.
                                              ^(we press enter)

  | Reading part 1 from floppy 1..........
  | floppy I/O error
  | dev 0211, sector 0
  | floppy I/O error
  | dev 0211, sector 0
  | floppy I/O error
  | dev 0211, sector 2
  | floppy I/O error
  | dev 0211, sector 4
  | floppy I/O error
  | dev 0211, sector 6
  | head: /dev/fd1H720: I/O error
  | 
  | **ERROR**: no disk inserted, disk is not formatted, or disk is not
  |            formatted with the right density.
  | 

Now we get to the trestore options menu:

  |  Enter one of the following commands:
  |                            (You may want to insert a new disk first.)
  |  r  : retry reading part 1 from drive 1
  |  s  : salvage readable data of part 1 from drive 1 and try to correct 
  |                                                                    errors
  |  q! : quit trying to read this part, continue with the next part
  |  e! : indicate end of archive

The `s' command recovers all readable data from a disk with read
errors and uses the error correcting codes to reconstruct as much
unreadable data as possible.

The `q!' command can be used if a disk from the backup set is lost.
`e!' can be used to abort the restore process.  You need to use `e!'
if the last disk of the backup set can't be read normally.

We reinsert the disk and type:
  | -->r
       ^
  | 
  | Reading part 1 from floppy 1..........
  | Finished reading part 1.  [you can remove the floppy]
  | 
  | Cleaning up...
  |
  | Restore completed.
  | #
  | ________________________________________________________


Here is a list of errors that are detected by the floppy read method.

- reading failed

- no disk inserted
- disk not formatted or formatted with the wrong density
- not a backup disk made with the floppy method

- disk belongs to a different backup set
- disk does not contain the right part 



*** _15_ SELECTIVE RESTORE

In a selective restore of a backup, not all files in the archive are
extracted, but only those whose names match a particular pattern.

Some examples of patterns are
  */Xconfig
for extracting all files named Xconfig from the archive and
  usr/src/gzip-0.7/*
for extracting the source tree of an old gzip version stored in the
archive.

The first step to making up selection patterns is knowing what files
are in the archive.  Tbackup has the capability to make `index files'
that supply this information.

* Creating index files

If you are making an archive with tbackup and are planning to do
selective restores on it later, you should switch on the `Make a
backup index file' option of tbackup.  Here is a sample transcript:

  ----------------------------------------------------------------------
  | #tbackup
     ^^^^^^^
 ...entering parameters for a full backup of directory set `ext'...
    
  |
  | Make a backup index file ---> (y/n) [n] y
                                            ^
  |
Tbackup will suggest an appropriate name for the index file. Assuming
there already is an index file named /root/idx/ext.idx.1, we get

  | backup index file name---> [/root/idx/ext.idx.2] 
                                                     ^(we press enter)

By default, tbackup will suggest making an index file in your home
directory (/root on my machine), but this can be changed to a more
appropriate place by editing the file /etc/tbackup/Config. Change the
line

    export indexdir=~/idx

to something else, e.g. 

    export indexdir=/var/index

  |
  |

Tbackup tries to ensure that there is enough room to write the index
file by watching the available disk space:

  |  [Note: the backup will be paused if the free disk space for
  |   /root/idx/ext.idx.2 drops below 50Kb. If this happens, free some
  |   disk space to continue the backup.]

Note however that this protection mechanism isn't failsafe: while
tbackup is working another program may fill up the hard disk so fast
that tbackup doesn't notice it until it is too late.

  |
  |
  |  [Creating backup progress window]
  [etc.]
  ----------------------------------------------------------------------

If you don't plan to do selective restores, the is no need for making
an index file.  If you change your mind later, you can create an index
of an existing archive by running the `tlist' command.


* Interpreting index files

An index file contains a description of the backup, the values you
entered for the parameters, and a list of all files in the archive.
Here is what a backup index file looks like:

      > Full backup of directory set ext.
      > (Filenames in archive relative to /)
      > Made with tbackup (V0.7 beta)
      > by root@noodle, on Mon Dec 27 12:47:50  1993
      > ----parameters:-----
      > export pre_choosemethod=set
      > export dea_setname=ext

    [...part deleted...]

      > export dea_indexfilename=/root/ext.idx.2
      > -------------backup index---------------------
      > ext -- okay
      > ext/tmp -- okay
      > ext/X386 -- okay
      > ext/X386/bin -- okay
      > ********** Approx. start of part 1 **********
      > ext/X386/bin/listres.z -- (26%)

    [...part deleted...]

      > ext/src/txt/megastruct.z -- (38%)
      > ext/src/txt/fscomp.z -- (43%)
      > ********** Approx. start of part 8 **********
      > ext/src/txt/khg.taz -- okay
      > ext/src/txt/lsm-05SEP93.z -- okay
      > ext/src/txt/tls.z -- (37%)
      > ext/src/txt/happynet.z -- (48%)

    [...etc....]

The file list itself is actually just the `backup progress' output
made by the pack method (in this case the afio archiver), interspersed
with `**Approx. start of part ..**' messages that tell you on which
disk a particular file can be found.

Note that afio puts a `.z' after the filename when it stores the
compressed version of a file in the archive.  For example, the last
index line shown above means that the archive contains the file
`ext/src/txt/happynet' in a compressed form, not that it contains the
compressed file `ext/src/txt/happynet.z'.

However, the `.z' in `ext/src/txt/lsm-05SEP93.z' above was not added
by afio, this file had a `.z' already (is was gzipped with an older
gzip version).  Notice that afio shows no compression percentage after
this file.

The `**Approx. start of part**' messages usually appear a little too
late in the index; for the index above the exact start of part 8 is
probably somewhere in the middle of the file ext/src/txt/fscomp or
even ext/src/txt/megastruct.


* Performing a partial restore

Suppose that the file /ext/src/txt/happynet was accidentally deleted.
Below is a transcript of the partial restore operation for this file.

  ----------------------------------------------------------------
  | # trestore
      ^^^^^^^^
  | 
  | Available read methods:
  |    1 filedev    -- read archive from file or device.
  |    2 floppy     -- read from msdos floppy disks with mtools.
  |    3 floppypart -- read a part of a floppy archive
  | Enter method name or number ---> [floppy] floppypart
                                              ^^^^^^^^^^
  | 

From reading the index file we know we can start at part 8.

  | Number of first part to read from the backup set ---> 8
                                                          ^
  | 
  |   [Note: you can end the partial read by entering `o'
  |    and then `e!' when asked to insert the next part.]
  | 
  | Read from floppy drive number (0/1) ---> [1] 
                                                 ^(we press enter)
  | 
  |
  | Insert a disk from the backup set in drive 1 and press enter.
  | 
  | Backup description:
  | 
  |   Full backup of directory set ext.
  |   (Filenames in archive relative to /)
  |   Made with tbackup (V0.7 beta)
  |   by root@noodle, on Mon Dec 27 12:47:50  1993
   [..part deleted..] 

Note that this backup description is also at the top of the index file
listing our ext/src/txt/happynet file.  This way we can verify that we
got the right floppies.

  | 
  | This archive specifies `afio' as the default unpack method.
  | 
  | Available unpack methods:
  |    1 afio     -- unpack files with afio, uncompress files with gzip
  |    2 afio0    -- unpack files with afio. (no uncompress)
  |    3 selafio  -- selectively unpack files with afio, uncompress with gzip
  |    4 selafio0 -- selectively unpack files with afio. (no uncompress)
  |    5 seltar   -- selectively unpack files with tar (no uncompress)
  |    6 tar      -- unpack files with tar (no uncompress)

We're using a selective unpacking method here:

  | Enter method name or number ---> [selafio] 
                                               ^(we press enter)
  | 
  | This archive was packed relative to the directory /.
  | Selectively unpack archive in directory ---> [/] 
                                                     ^(we press enter)
  | 

We need only one pattern to select the ext/src/txt/happynet file, but
in general you can supply as many patterns as you want for a selective
restore.

If you want to use many complicated patterns, it is best to put them
in a file beforehand.  Simple patterns can also be entered directly,
which is what we will do now:

  | Read selection patterns from file?---> (y/n) [y] n
                                                     ^
  | 
  | Enter patterns (one per line), end with a "." on a line.
  | Examples:   /usr/src/linux/*
  |             usr/src/linux/*
  |              (a leading / is ignored in pattern matches)
  |             *Xconfig
  | 
  | ---> ext/src/txt/happynet
         ^^^^^^^^^^^^^^^^^^^^
  | ---> .
         ^
  | 
  | 
  | Protect newer files from overwriting?---> (y/n) [y] 
                                                        ^(we press enter)
  | 

A restore index file will (among other things) list all files that
were selectively restored.  This information can be useful for
checking if a complicated selective restore actually worked.
There's not much sense in making an index of this restore:

  | Make a restore index file ---> (y/n) [n] 
                                             ^(we press enter)
  | 
  |   [Creating restore progress window]
  | 
  | Restore in progress...
  | 
  | --Insert the disk with part 8 in drive 1
  |   and press enter. (Type o for options.)
                                             ^(we press enter)
  | Reading part 8 from floppy 1..........
  | Finished reading part 8.  [you can remove the floppy]

Watching the backup progress messages on virtual console 6 (or in the
progress window if we're running X), we see that the archiver produces
some warnings about not being at the start of an archive.  Here is what
it looks like:

  || afio: "/tmp/tbup/archive" [offset 0]: Bad magic number
  || afio: "/tmp/tbup/archive" [offset 6k+396]: Bad swapped pathname length
  || afio: "/tmp/tbup/archive" [offset 49k+410]: Bad swapped pathname length
  || afio: "/tmp/tbup/archive" [offset 124k+56]: Apparently resynchronized
  || afio: "ext/src/txt/lsm-05SEP93.z": Continuing

After that it extracts the happynet file:

  || ext/src/txt/happynet -- uncompressed

Tbackup has no way of knowing that all selected files are restored
from the archive; it just prompts for the next floppy, leaving it up
to the user to end the restore when the last selected file has been
extracted.

The progress messages tell us that ext/src/txt/happynet has been restored,
so we end the restore via the options menu:
  | 
  | --Insert the disk with part 9 in drive 1
  |   and press enter. (Type o for options.) o
                                             ^
  | 
  |  Enter one of the following commands:
  |                            (You may want to insert a new disk first.)
  |  r  : retry reading part 9 from drive 1
  |  s  : salvage readable data for part 9 from drive 1
  |  q! : quit trying to read this part, continue with the next part
  |  e! : indicate end of archive
  | -->e!
       ^^
  | Indicate end of archive? (y/n) ---> y
                                        ^
  | End of archive indicated.
  | Watch the restore log for possible archiver errors.

Since the archive ended prematurely, we get two archiver errors:

  || afio: "/tmp/tbup/archive" [offset 710k+0]: Input EOF
  || afio: "/dev/tty": Unavailable

  | Cleaning up...
  | 
  | Restore completed.
  | # 
  ------------------------------------------------------------------


* Managing index files

Apart from suggesting an appropriate name, tbackup provides no
mechanism for managing, archiving or searching index files. In
particular, the selective restore methods do not read or interpret
index files directly: they leave this task to to the user.  (Future
versions of tbackup may provide some specialized tools for searching
index files.)

Index files can easily be managed with standard unix tools.  Index
files compress (gzip) very well, so there is usually no problem in
keeping a collection on your hard disk.

Index files (compressed or not) can be searched with the `zgrep'
command, e.g.

  | # cd ~/idx
  | # zgrep frommel.tex *idx*
      ^^^^^^^^^^^^^^^^^^^^^^^
  | mystuff.idx.1.gz:/root/tex/wuxta/frommel.tex.z -- (32%)
  | mystuff.idx.2:/root/tex/wuxta/frommel.tex.z -- (31%)
  | mystuff.idx.3.gz:/root/tex/wuxta/frommel.tex.z -- (38%)

tells you that there are three index files containing frommel.tex.
Assuming that the last one is from the most recent backup, the
following command tells us which floppies we need:

  | # zcat mystuff.idx.2.gz |head -4
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | Incremental (on full) backup of directory set mystuff.
  | (Filenames in archive relative to /)
  | Made with tbackup (V0.7 beta)
  | by root@noodle, on Fri Dec 10 13:21:45  1993
  | Broken pipe

So we should look for the floppies labeled `mystuff inc 10 dec 1993'
or somesuch.

By looking at mystuff.idx.2.gz with zmore, we can also determine the
part number of the floppy disk containing frommel.tex.


* Complicated patterns

Usually these tree kinds of patterns are all what you need:

1) full pathname, e.g. `ext/src/txt/happynet'
2) filename, e.g. `*/happynet', also useful if you don't remember the 
   exact path
3) directory prefix, e.g. `usr/src/gzip-0.7/*'.

For completeness, here are the rules for pattern matching:

- A filename must completely match a pattern to be selected, e.g. the
  file `etc/passwd' is selected by the patterns `*/passwd' and `etc/*',
  but not by `passwd' or `etc/'.

- Selection patterns differ from shell patterns in that slashes have
  no special significance: the file `usr/src/prog/utl/blah.c' will be
  selected by the pattern `usr/*/blah.c', the `*' matches the whole
  substring `src/prog/utl' in the filename.

- Afio (as used in selafio and selafio0) ignores a leading slash when
  matching patterns. Thus, the pattern `/etc/*' selects the same files
  as `etc/*'.  Tar does not ignore a leading slash, using the pattern
  `/etc/*' in seltar never selects any files, as tar always removes
  the leading slashes from the filenames in an archive.

- Patterns can contain the following special constructs:

   * : matches any sequence of zero or more characters
       (the sequence may contain / characters)
   ? : matches any single character

   [...]  : matches any of the enclosed characters, also
   [b-f]  : matches the range b, c, d, e, and f, and
   [^...] : matches all characters but ...

   \c : matches the character c, mainly used to specify the special
        characters * ? [ ] ^  \ and - if it appears between [ ].  


* Pattern files

If you want to use many complicated selection patterns for a restore,
it is best to put them in a file beforehand and use the `Read
selection patterns from file' option of trestore.

Such pattern files may contain:

- lines starting with a #, these are comments
- empty lines
- lines containing one pattern, the pattern may be surrounded by
  whitespace.

An example of a valid pattern file:

   > #a valid pattern file
   >
   > */Xconfig
   >    usr/src/xlife/*[Mm]akefile
   >
   >  usr/bin/?roff
   > *emacs*

Trestore will warn you if you supply suspicious patterns.  It can
produce the following warnings:

 Regex in stead of selection pattern
   (if you use a pattern beginning with ^ or ending in $)

 Leading / in tar pattern
   (if you use a leading slash in a pattern with the seltar unpack method)

 Control character in line <n>
 Text after pattern on line <n>
   (usually caused by an editing error in a pattern file)


*** _16_ RUNNING TLIST

The tlist command can be used to make an index of all files in an
existing archive.  

Making an index file with tlist usually takes about as much time as a
complete restore of the archive.  The best way to create index files
is by using the `Make a backup index file' option of tbackup while the
archive itself is created.  Tlist should be seen as a means of
recreating the index file if the index made by tbackup is damaged or
lost.

Running tlist goes along the same lines as running trestore, except
that you are asked for a `list method' in stead of an `unpack method'.


*** _17_ RUNNING TGUESS

The tguess command can be used to guess the number of floppy disks
needed for a backup before making it.  Here is a sample transcript:

   -------------------------------------------------------------------
   | #tguess
      ^^^^^^
   | 
   | Available choose methods:
   |    1 one -- one user specified file or directory and below
   |    2 set -- back up a directory set
   | Enter method name or number ---> [set] set
                                            ^^^
   | 
   | Available directory sets:
   | 1    ext	# extended filesystem.
   | 2    mystuff	# my stuff in the /root directory.
   | 3    system	# root filesystem, without /root dir.
   | Enter set name or number ---> mystuff
                                   ^^^^^^^
   | 
   | The last full backup of mystuff was made on Sun Dec 19 11:42:33 1993,
   | that was approx. 11 days ago.
   | 
   | Choose a backup level.
   |  f  = full backup of all files
   |  if = incremental backup of all files changed since last full backup
   | ---> [f] f
              ^
   | 

A guess index file can be useful for testing if a newly made directory
set really contains the intended files.

   | Make a guess index file ---> (y/n) [n] n
                                            ^
   | 
   |   [Creating guess progress window]
   | 
   | Guess in progress...
   | 
   |  Files and directories in archive: 2013
   | 
   |  Data in archive:  23239454 bytes  =  22694.78 Kb  =  22.16 Mb.
   | 
   |  Guess of floppy disks needed:
   |                                 compression factor
   |                               none    70%    50%    30%
   |          5.25" d   (360Kb):     70     49     35     22
   |          5.25" h  (1.20Mb):     21     15     11      7
   |
   |          3.5"  d   (720Kb):     35     24     18     11
   |          3.5" xd   (820Kb):     30     21     16     10
   |          3.5"  h  (1.44Mb):     17     12      9      6
   |          3.5" xh  (1.72Mb):     15     10      8      5
   |
   | Cleaning up...
   | 
   | Guess completed.
   | # 
   -------------------------------------------------------------------

The `mystuff' directory set contains mostly text files, which compress
to about 30% of the original size.  In the table, we can see that
about 10 720K floppy disks are needed for a full backup of this set.


*** _18_ FREEING HARDDISK SPACE: SELECT, BACKUP, DELETE

If you don't have enough room on your harddisk for new things, you
often want to backup some old stuff (e.g. a program source tree) and
delete it to get more free space.  Tbackup supports this with the
`tdir' command and the `setrm' method.

You can use the tdir command to interactively make a temporary
directory set called `tdir', containing the things you want to back up
and delete.

You can then back up the `tdir' set with the `setrm' choose method,
that will delete the directories in the set once the backup is finished.

If you need some of the old stuff again, you can easily grep the
backup index files for it and do a partial restore.

* Running tdir

tdir is a command line tool for making directory sets.  You can use
it in the following ways:

 tdir

  Without any options tdir will list the contents of the `tdir'
  directory set.

 tdir [-s setname] -c|-create <description>

  With the -c or -create option you can create a new direcrory set.
  If you don't specify a set name with the -s option, tdir will work
  on the `tdir' directory set.

 tdir [-s setname] <dirname/filename> [...]

  If you give file or directory names to tdir they are added to the
  directory set.

 tdir [-s setname] not name|ext|dir|fullname <string/"pattern"> [...]

  `Not' instructions can be added in the same way.  Patterns must be
  quoted to prevent expansion by the shell.
  
 tdir [-s setname] -d|-delete|-e|-edit|-l|-list|-u|-undoadd

  Other directory set operations:

  -d or -delete  deletes tdir.set (or setname.set).
  -e or -edit    runs the editor vi on the directory set.  If you don't
                 like vi you can configure another editor in the file
                 /etc/tbackup/Config.
  -l or -list    lists the contents of the directory set.
  -u or -undoadd removes the last line from the directory set, thus
                 undoing a previous tdir command.


Here is a sample transcript:

First we look at the tdir help information:

   | ~# tdir -h
        ^^^^^^^
   | tdir usage:
   |   tdir [-s setname] -c|-create <description>
   |   tdir [-s setname] <dirname/filename> [...]
   |   tdir [-s setname] not name|ext|dir|fullname <string/"pattern"> [...]
   |   tdir [-s setname] -d|-delete|-e|-edit|-l|-list|-u|-undoadd
   | man tdir gives more help.

We want to select some old stuff from /usr/src:
   | ~# cd /usr/src
        ^^^^^^^^^^^
We create the directory set tdir:

   | /usr/src# tdir -c some source files
               ^^^^^^^^^^^^^^^^^^^^^^^^^
   | Created directory set /etc/tbackup/tdir.set.
   | Current contents of /etc/tbackup/tdir.set:
   |      1  # some source files

Then we use `ls' so see what old stuff we can throw away:

   | /usr/src# ls
               ^^
   | cpio-2.3           defrag             perl-4.036         util-linux-1.5
   | gzip-0.7           mtools             traceroute         xview
   | gzip-1.2.3         ncurses-1.8.1      utree

We don't need the defrag source tree anytime soon, so we put it in tdir.set:
   | /usr/src# tdir defrag
               ^^^^^^^^^^^
   | Added directory "/usr/src/defrag" to tdir.set.

Just typing tdir lists the current contents of the directory set:
   | /usr/src# tdir
               ^^^^
   | Current contents of /etc/tbackup/tdir.set:
   |      1  # some source files
   |      2  /usr/src/defrag

Now we add some more stuff.  Note that you can use wildcards:
   | /usr/src# tdir gzip* utree
               ^^^^^^^^^^^^^^^^
   | Added directory "/usr/src/gzip-0.7" to tdir.set.
   | Added directory "/usr/src/gzip-1.2.3" to tdir.set.
   | Added directory "/usr/src/utree" to tdir.set.

You can also use full path names:
   | /usr/src# tdir /usr/lib/*hash
               ^^^^^^^^^^^^^^^^^^^
   | Added file "/usr/lib/dutch.hash" to tdir.set.
   | Added file "/usr/lib/english.hash" to tdir.set.

We exclude object files from the backup:
   | /usr/src# tdir not ext .o
               ^^^^^^^^^^^^^^^
   | Added predicate "not ext .o" to tdir.set.

Before running tbackup we list the directory set again:
   | /usr/src# tdir
   |           ^^^^
   | Current contents of /etc/tbackup/tdir.set:
   |      1  # some source files
   |      2  /usr/src/defrag
   |      3  /usr/src/gzip-0.7
   |      4  /usr/src/gzip-1.2.3
   |      5  /usr/src/utree
   |      7  /usr/lib/dutch.hash
   |      8  /usr/lib/english.hash
   |      9  not ext .o
   | tdir -h gives help


* The setrm method

With the setrm method we can now back up and remove the contents 
of the tdir directory set. Here is a transcript:

   | /usr/src# tbackup
               ^^^^^^^      
   | 
   | Available choose methods:
   |   1 set   -- back up a directory set
   |   2 setrm -- back up a directory set, then remove its contents
   |   3 one   -- one user specified file or directory and below
   | Enter method name or number ---> [set] setrm
                                            ^^^^^
   | 
   | Available directory sets:
   | 1    mystuff    # my stuff in the /root directory.
   | 2    system     # system files
   | 3    tdir       # some source files
   | 4    test       # test set 
   | Enter set name or number ---> [tdir] 
                                          ^(we press enter)
   | 
   | WARNING: The `not' instructions in this directory set, i.e.
   |      1  not ext .o
   | exclude files and/or directories from the archive, but they do not exclude
   | them from deletion.
   | The deletion phase of setrm removes ALL files and directories in the
   | set, i.e.
   |         /usr/lib/dutch.hash    /usr/lib/english.hash  /usr/src/defrag
   |         /usr/src/gzip-0.7      /usr/src/gzip-1.2.3    /usr/src/utree
   | and everything below, regardless of `not' instructions.
   | 
   | Press ^C to abort, enter to continue.
                                          ^(we press enter)
   | 
   | Available pack methods:

  [....]

  | Make a backup index file ---> (y/n) [y] y
                                            ^
  [....]

   | Backup in progress...

  [....]

   | Please confirm that this backup was successful.
   | 
   | Ready to remove all files and directories in tdir, i.e.
   |         /usr/lib/dutch.hash    /usr/lib/english.hash  /usr/src/defrag
   |         /usr/src/gzip-0.7      /usr/src/gzip-1.2.3    /usr/src/utree
   | and everything below.
   | 
   | Remove all these files and directories?---> (y/n) [y] y
                                                           ^
   | Removing.....
   | 
   | Cleaning up...
   | 
   | Backup completed.
   |

We no longer need tdir.set:

   | /usr/src# tdir -d
               ^^^^^^^
   | Set /etc/tbackup/tdir.set deleted.
   | /usr/src# 



*** _19_ EXTENDED FLOPPY DENSITIES

The extended floppy densities are called `xd' (extended double) and
`xh' (extended high).  They fit more data on a floppy by going outside
the `factory specifications' of the floppies and drives.  For example,
on 3.5" DD floppies, the normal `d' (double) density format uses 80
tracks and 9 sectors per track, the `xd' format uses 82 tracks and 10
sectors per track, which fits 820 Kb. on the floppy.

Tbackup needs the `setfdprm' program to use extended densities.  I
have no idea when this program first appeared in Linux, so you may
not have it.

Not all floppies and/or floppy drives are able to go outside their
`factory specifications' and hold extra data.  For example, my 5.25"
drive can't read and write `xd' density.  Most modern 3.5" HD drives
however have no problems reading and writing extended densities.

To find out which extended densities your drives support, just
experiment with tbackup and trestore.

Due to limitations in the Linux kernel, reading and writing 1.7 Mb. on
3.5" HD floppies is extremely slow (some 8 times slower as normal
floppy access).  This makes the `xd' density for 3.5" floppies
virtually useless.  There seems to be some non trivial patch needed to
the Linux kernel to allow fast _writing_ of the 3.5" xd format.

Currently you can not use the `xd' density if you have a DD floppy
drive, you can only use `xd' and `xh' on a HD drive.  I haven't found
the `xd' formatting parameters needed for DD drives, if you have them
please let me know.

* extended densities under MS-DOS

If you are actually going to make backups with extended densities,
make sure that you have access to at least one other machine that
supports them.  This other computer need not be a Linux machine, there
are also MS-DOS packages that support extended densities.  I have one
called fdformat, archive name FDFORM17.ZIP.

* using extended densities under Linux

If you want to read or write extended density floppies under Linux
outside tbackup, you will have to edit some configuration files first.

- Append the following 5 lines to /etc/fdprm:

   #Extended densities used by tbackup  (5.25 xd and xh, 3.5 xd and xh)
   410/1200        820    10   2  41    1 0x25 0x02 0xDF     0x2E 
   1476/1200       2952    18   2  82    0 0x25 0x00 0xDF     0x02
   820/1440        1640    10   2  82    0 0x25 0x01 0xDF     0x2E
   1722/1440       3444    21   2  82    0 0x25 0x00 0xDF     0x0C

- Make sure your /etc/mtools file contains the lines
   A /dev/fd0 12 0 0 0
   B /dev/fd1 12 0 0 0  
  for the a: and b: drives.

Before accessing an extended density floppy, type the command 

   setfdprm -p /dev/fd<nr> <name>

where <nr> is the floppy drive number, 0 or 1, and <name> is 
   410/1200  for the xd format in a 5.25" drive
   1476/1200 for the xh format in a 5.25" drive
   820/1440  for the xd format in a 3.5"  drive
   1722/1440 for the xh format in a 3.5"  drive.

Now you can use
   mdir a:  (or b:), 
and other mtools commands to access the floppy.

You can also mount it with
   mount -t msdos /dev/fd<nr> <mountpoint>

Formatting is possible with
   fdformat -n /dev/fd<nr>
where the -n option switches off the verify pass as verification of
extended floppy formats won't work with the current version of fdformat.
You can do a simple verify by hand by typing `cp /dev/fd<nr> /dev/null' 
and looking if cp gives an error.

If you are finished with the floppy, you sometimes may have to type
   setfdprm -c /dev/fd<nr>
to get the drive back in `autodetect mode' so that it will accept
normal floppies again.


*** _20_ TAPE DRIVES

The main thing to note about tbackup and tape drives is that tbackup
is not a tape device driver.  Tbackup assumes you have a device driver
that can reliably access your tape drive under Linux.  If you can't
use tour tape drive with normal tools like tar and cpio, you won't be
able to do so with tbackup either. 

As I don't have a tape drive myself, I can't answer many questions
about tape drives or tape drivers.  Only ask me if your tape drive
works with tools like tar and cpio, but fails to work with tbackup.

Tbackup provides no error recovery for tape write errors, and can't do
a tape verify pass.

Tbackup isn't better at low level tape handling plain tar, cpio and
afio are.  It does however add high level directory set and
incremental backup facilities.  This makes tbackup very convenient to
use in a hybrid situation where you do full backups to tapes and
incremental backups to floppies.

* Configuration

Before you can use your tape drive, the tape device name and
optionally some other parameters need to be configured by editing the
file /etc/tbackup/Config.  The comments in /etc/tbackup/Config explain
the configuration.

* Using the tape method

Tbackup provides the `tape' method for reading and writing to the tape
drive.  Note that this method does not write any information about the
backup (contents, pack method used, creation date, ...) on the tape,
you will have to record this information yourself.

Here is a sample transcript:

   | Available write methods:
   |   1 floppy  -- write to msdos floppy disks with mtools
   |   2 filedev -- write archive to file or device
   |   3 tape    -- write to tape
   |   4 null    -- copy archive to the bit bucket
   | Enter method name or number ---> [floppy] tape
   |                                           ^^^^
   | 
     [....]
   | 
   | Tape size is 250m.
   | 
   | Is the tape /dev/tape1 ready for writing?  Press enter to begin.

Tbackup knows nothing about rewinding or positioning of tapes.  You
will have to prepare the tape drive for writing yourself (e.g. be
using a program like mt(1)) before running tbackup (or at least
before pressing enter).

If the archive length exceeds the tape size you will be prompted for a
second tape:
  
   | afio: "/dev/tape1" [offset 0]: Output limit reached
   | afio: Ready for disk 2 on /dev/tape1
   | afio: quit to abort, anything else to proceed.

(Please ignore the part about being ready for /disk/ 2.)  You will see
`afio:' in this prompt even if you use the `tar' pack method.

Tbackup will report tape write errors to you *if the tape driver
reports them to tbackup*.  With some tape drivers a tape write error
may only produce a kernel message, not a message to tbackup.  It may
be necessary to watch the kernel messages for errors when writing to
tape. (Consult your tape driver documentation, if it exists.)  Unlike
the floppy method, the tape method does not watch the kernel messages
itself, nor does it print them to your xterm if you are running under X.

* No error correcting codes for tapes.

Tbackup does not provide error correcting codes for tapes.  Some tape
drives and/or drivers automatically provide error correcting codes
independent of the backup program used.  These is a GNU program called
`ecc' that can make error correcting codes for tapes that do not have
them built in, but tbackup does not use this program.  Mail me if you
want an `ecc' option for tapes in tbackup.

* No noninteractive use.

Currently, tbackup can't be used as an at(1) or crontab(1) job to do a
backup to tape at a specified time.  This is because of the
interactive nature of the tbackup user interface.

If you would like to use tbackup in a noninteractive mode, please mail
me.  Implementing such a mode wouldn't be too hard to do (see the
section about configuring command line arguments below), the main
things that need to be changed are the progress window and the `press
enter to begin' messages.


*** _21_ TBACKUP ON A MULTIUSER SYSTEM

Tbackup can only be run by the root user.  This is done to avoid
difficulties in accessing restricted resources like devices and home
directories of other users.

* User access while running tbackup.

You may want to prevent other users from using the system while
tbackup is running, especially when it is backing up the user
directories.

When a (user) directory is altered while being backed up, tbackup may
get confused and fail to back up some of the files in the directory.
Files that are missed in this way will also not get backed up in an
incremental backup made some time later.

If preventing user access during a backup is unacceptable, you may
want to show a warning message while the backup is being made, e.g.

                        ** WARNING **
          A backup of the user files is in progress.

  -  Do not move or rename any of your files or directories,
     this may cause them not to be backed up.
  -  Do not edit or change files, this may cause the contents of
     these files to be backed up incorrectly.
  -  It is however safe to create new files.

See the wall(1) and motd(5) manual pages for information on showing
messages.


* Security and privacy considerations.

If other users can use the system while a backup is being made, they
may interfere with the backup process if they can access the backup
device.  This can lead to a corrupted backup without you getting any
error message.

Check the write permissions on your floppy and/or tape drive. While
the backup is being made the backup device should only be accessible
by the root user, i.e. have mode 600 (rw-------) or 660 (rw-rw---).

Tbackup automatically makes its working directories inaccessible to
non-root users to prevent them from reading secret or private
information from its temporary files.

You should ensure that index files produced by tbackup are unreadable
for non-root users; a backup index can reveal the file names in
private (non world readable) directories to other users.


* Restoring backups you didn't make yourself.

In general you should only restore or list archives made by people you
trust.  Using an archive made by someone else can compromise your
system in two ways:

1) An archive can contain SUID programs or trojan horses that can be
used by crackers to get root privileges when restored.  Whether this
is the case may be hard to detect, even if you have extensive UNIX
knowledge.

2) The `restore parameters' in the DESCR2K files on tbackup floppies
are actually shell commands. These are executed by the floppy method
when doing a trestore or tlist.  Thus, restoring or listing an archive
with a sabotaged DESCR2K file can have serious consequences.

Before executing the restore parameters trestore and tlist first print
them on the screen.  You are asked to

  | Press enter to use this backup set, ^C to abort.

If the restore parameters on an outside disk set look suspicious (you
need to know shell programming to see this), press ^C.


*** _22_ CUSTOMIZATION

Apart from editing the /etc/tbackup/Config file and creating directory
sets, it also possible to change the (default) behavior of tbackup,
trestore, tlist and tguess by placing some files in /etc/tbackup.

* Changing the default values for the parameters.

You can change the default answers for the questions that tbackup
asks you by supplying new default values in the configuration file
`/etc/tbackup/tbackup.dea'.

The easiest way of making such a file is to run the command 
`tbackup pars'.  The `pars' option changes the behavior of tbackup:
after asking for parameters it does not begin with a backup, but in
stead writes the supplied parameters to a file.

  ------------------------------------------------------------------
  | #tbackup pars
     ^^^^^^^^^^^^
  | 
  |Available choose methods:
  |    1 one -- one user specified file or directory and below

 [etc.]

  | Verify after write to disk?---> (y/n) [n] y
                                              ^
  | 
  | Make a backup index file ---> (y/n) [y] n
                                            ^
  | 
  | The supplied parameters are recorded in the file /etc/tbackup/allpars.
  | 
  | Cleaning up....
  | #
  -----------------------------------------------------------------

The file /etc/tbackup/allpars now contains a description of all the
answers you gave.  Each line contains one parameter setting, the
parameters occur in the order in which you answered the questions.

  > #Result of running tbackup pars.
  > #
  > #  Lines are of the form "export <prefix><parametername>=<value>".
  > #  Prefixes: dea_  means set default value of parameter .. to ..
  > #            pre_  means preset parameter .. to ..
  > #       checkpre_  means preset parameter .. to .. and check if valid.
  > 
  > export dea_choosemethod=set
  > export dea_setname=mystuff
  > export dea_level=f
  > export dea_packmethod=afio
  > export dea_writemethod=floppy
  > export dea_mnr=0
  > export dea_mdens=d
  > export dea_formatalways=n
  > export dea_verifywrite=y
  > export dea_makeindexfile=y

The file is actually a shell script.  The line

  export dea_verifywrite=y

is an instruction to tbackup, that tells it to use `y' as the default
(dea_) answer to the `verifywrite' question, so that it becomes

    Verify after write to disk?---> (y/n) [y]
                                          ^^^ 
In order for tbackup to read these instructions when it starts, you
have to copy the contents of /etc/tbackup/allpars to the
/etc/tbackup/tbackup.dea (tbackup defaults) file.  So just type

 cd /etc/tbackup
 cp allpars tbackup.dea 

and the parameters you have supplied on running `tbackup pars' will be
used as default values.  It usually makes sense to do some further
editing on the tbackup.dea file, e.g. to delete the line

      export dea_setname=mystuff

so that tbackup does not supply a default directory set name.
Sometimes it is useful to combine the results of running `tbackup
pars' several times in one tbackup.dea file.

All the above also goes for the trestore, tlist and tguess commands.
To change the default parameters of trestore, you need to

 trestore pars
  [supply new default parameters]
 cd /etc/tbackup
 cp allpars trestore.dea
             ^^^^^^^
and optionally delete some lines in the trestore.dea file.


* Presetting parameters.

If you always find yourself giving the same answer to some of the
questions tbackup asks, you may want to configure tbackup to skip these
questions by `presetting' the answers. 

This can be done by editing the tbackup.dea (trestore.dea) file you
get by the procedure above.

To preset the answer to the `verify after write to disk' question to
'y', simply edit the line 

  export dea_verifywrite=y

in /etc/tbackup/tbackup.dea to read

  export pre_verifywrite=y
         ^^^ 
A `pre_' (preset) line instructs tbackup to skip the corresponding
question and use the value after the = sign for the parameter.

My own tbackup.dea file is

  > export pre_mnr=1
  > export pre_formatalways=n
  > export pre_verifywrite=n

as I always make my backups with drive 1, and never want to format and
verify the disks. (The format and verify settings can of course still
be changed with the `options' menu while making a backup.)  I don't
have any `dea_' lines in my tbackup.dea file, as the standard default
parameters work fine for me :-)

Again, presetting parameters also works for trestore, tlist and tguess.


* Checked presets.

If you specify an impossible value in a preset, e.g.

  export pre_verifywrite=42 

you will get unpredictable results when running tbackup.  By default
tbackup does not check if preset values are legal, it assumes that you
know what you are doing.  If you want a preset to be checked for
validity, set it with the command

 export checkpre_setname=systme
        ^^^^^^^^
A checkpre_ argument is processed in a verbose way that makes it easy
to interactively correct it if it is illegal:

   | Available directory sets:
   | 1    mystuff    # my stuff in the /root directory.
   | 2    system     # system files
   | 3    test       # test set 
   | Enter set name or number -[preset as]-> systme
   | No such directory set.
   | Enter set name or number ---> system
   |                               ^^^^^^

Checked presets are mainly useful when you run tbackup from another
script, and in command line argument scripts.


* Making command line arguments.

If you supply command line arguments (other than `pars') to tbackup, e.g
  tbackup aaa bbb
or
  tbackup -aaa -bbb 
then tbackup will process these arguments by executing the files
  /etc/tbackup/aaa.arg
and
  /etc/tbackup/bbb.arg
before asking any questions. (If these files don't exist you will get
an error message.)

These files can contain customized parameter defaults or presets, just
like tbackup.dea. For example, there is a file /etc/tbackup/1d.arg which
contains

  > #Argument file (used by `tbackup 1d').
  > #Backs up directory set to disk 1, double density format.
  > #Presets various other parameters.
  > export pre_choosemethod=set
  > export pre_packmethod=afio
  > export pre_writemethod=floppy
  > export pre_mnr=1
  > export pre_mdens=d
  > export pre_errcorrect=y
  > export pre_formatalways=n
  > export pre_verifywrite=n
  > export pre_makeindexfile=y
  > export pre_autoindexname=y

so that typing 
   tbackup 1d
makes a backup (with methods set, afio, and floppy) to DS/DD disks in
drive 1.  The `pre_' instructions in 1d.arg preset various parameters
to their appropriate value.

The standard tbackup installation comes with the 4 preconfigured
argument files in /etc/tbackup: 0d.arg, 0h.arg, 1d.arg and 1h.arg.
More examples of arguments can be found in /usr/lib/tbackup/examples.

You can make you own arguments by copying the result of `tbackup pars'
to a .arg file in /etc/tbackup and editing it.  Again, such .arg files
can also be made for trestore, tlist and tguess.

The .arg files are read after the /etc/tbackup.dea file (if present)
so that `pre_' instructions in the argument file take precedence over
the `pre_' or `dea_' instructions in the tbackup.dea file.  

You can not `unpreset' a parameter by putting an
  export dea_verifywrite=n
line in an .arg file. To turn off a preset made in tbackup.dea
you need to add a line reading
  unset pre_verifywrite
  ^^^^^^^^^
to the .arg file.
       

*** _23_ FILES AND DIRECTORIES

Here are the files and directories used by tbackup.

/usr/lib/tbackup/          : tbackup directory
/usr/lib/tbackup/README    : this manual
/usr/lib/tbackup/examples/ : contains sample configuration files
/usr/lib/tbackup/bin/      : contains files for internal use by tbackup

/etc/tbackup/     : contains configuration files

/tmp/tbup/        : directory for small temporary files
/tmp/bulkdir/     : default directory for temporary storage of floppy 
                    contents

/etc/tbackup/Config : tbackup configuration file. Contains location of
                      directory for temporary storage of floppy
                      contents, default directory for index files, and
                      compression factor parameters for afio.

/etc/tbackup/Floppy : holds floppy drive configuration

/etc/tbackup/Compr.ext : holds compressed file extensions for afio

/etc/tbackup/*.set : directory set descriptions
/etc/tbackup/*.log : directory set logfiles that contain the dates of
                     previous backups of a set.

/etc/tbackup/allpars      : output file of `tbackup pars', `trestore
                            pars', `tlist pars' and `tguess pars'
/etc/tbackup/tbackup.dea  : default settings for tbackup
/etc/tbackup/trestore.dea : default settings for trestore
/etc/tbackup/tlist.dea    : default settings for tlist
/etc/tbackup/tguess.dea   : default settings for tguess

/etc/tbackup/*.arg : scripts implementing command line arguments.

<indexdir>/backup.idx.<number>    : default index file name for backups
<indexdir>/<setname>.idx.<number> : default index file name for backups
                                    of a directory set.
                                    <number> is determined by looking
                                    for existing (compressed) index
                                    files in <indexdir>, taking the
                                    highest existing number plus one.


*** _24_ PERFORMANCE TUNING

On most machines (most here means machines with 8 Mb RAM or more and a
clock speed of 20 MHz or more), the speed of the floppy drive will be
the limiting factor for the speed of tbackup.

Tbackup usually has the next part of a compressed archive ready before
the writing of the current part is finished, except maybe when working
on a directory containing a lot of small files.

On slower machines however, the time it takes to make a part may be
(much) longer than than the time needed to write it. 

If you find that part creation is too slow on your machine, you can
speed up the compression process by reconfiguring tbackup.  Of course,
if tbackup spends less CPU time compressing it will produce larger
archives, so there is a tradeoff to be made here.

First, here are some general comments about performance:

- Compressing files takes a lot of CPU time.  Tbackup will happily
  multitask with other programs (as long as these programs don't
  change the directories it is backing up), but becomes slow if these
  other programs are cpu-intensive.

- To save time, afio won't try to compress files that are already
  compressed, that is files having one of the following extensions:
     .z .gz .Z
     .taz .tgz .tzg
     .gif .arc .zip .zoo .lha .jpeg .jpg
  A list of these extensions is kept in the /etc/tbackup/Compr.ext
  configuration file.  If you have many compressed files with an
  extension not listed above, (e.g. .mpeg), it may pay off to add this
  extension to the /etc/tbackup/Compr.ext file.

- Tbackup is disk intensive: apart from reading the contents of the
  files to be archived, it causes disk traffic by reading and writing
  temporary files.  
  - Don't run programs that use much memory (X386, emacs, ...), so that
    there is a lot of free RAM to do disk caching.  A small amount of
    extra disk cache can sometimes increase performance by a large
    factor.
  - Don't run other disk intensive programs along with tbackup.
  Trestore, tguess and tlist are not as disk intensive as tbackup,
  free RAM is much less a problem with these programs.
   

To decrease the time tbackup spends creating the parts of a compressed
archive, you can change two configuration parameters in the file
/etc/tbackup/Config.  This file should be pretty much
self-explanatory, the relevant part is included below:

  > #The lines below determine the computation time/compression ratio
  > #tradeoff factor, as used in the `afio' pack method.
  > #
  > #Select the appropriate two lines or alternatively experiment until you
  > #have found a setting you like.  
  > #The two parameters are the values for the afio -G and -T options,
  > #see the afio manual page for more information.
  > 
  > #Uncomment these settings for machines like a 486/33 with 8 Mb or better:
  > export gzipspeed=6
  > export threshold=512
  > 
  > #Uncomment these settings for machines like a 486/16 or 386/33 with 8 Mb:
  > #export gzipspeed=5
  > #export threshold=3k
  > 
  > #Uncomment these settings for machines like a 486/16 or 386/33 with 4 Mb:
  > #export gzipspeed=2
  > #export threshold=5k
  > 
  > #Uncomment these settings for slow machines without much memory.
  > #export gzipspeed=1
  > #export threshold=8k

Note that I have no magical way of guessing the i/o speed / CPU speed
ratio on your machine.  It may well be the case that you need to
uncomment the last entry to get a good backup time/compression ratio
tradeoff on your 386/33 8Mb machine.

Experiments on my machine show the following tradeoff figures:

 gzipspeed  threshold  cpu time used   archive size 
     6         512        100%           100%
     5         3k          60%           106%
     2         5k          40%           113%
     1         8k          25%           125%

  <no compress, afio0>     14%           178%

These figures are probably completely meaningless in the general case:
they do not only depend on the i/o speed / CPU speed ratio but also on
the kind of files in the archive.


*** _25_ KERNEL MESSAGES

Tbackup interprets all floppy related kernel messages for you, so
there is no need to read this section unless you are curious.

* Kernel messages when running tbackup under X

If you are making a backup to floppies under X, tbackup relays all
kernel messages except the `Disk change detected' messages to the
window in which tbackup is running.

* Kernel messages when running tbackup on a virtual console

If you are running tbackup on a virtual console, tbackup does not
relay the kernel messages to your screen, as the kernel already
performs this task.  Note that kernel messages always appear on the
current virtual console.  The kernel messages will not appear if you
have switched off message printing with an external program such as
colog(8).  Switching off printing does not affect tbackup, it will
still be able to capture the messages.

* Read/write error message

These is only one kernel message for reporting a floppy error condition:

  floppy I/O error
  dev <devnumber>, sector <secnumber>

If you see this message, you know that a read or write operation
failed.  If it was a read error, the kernel will signal it to the
reading program.  Write errors are not signalled.

* Read/write problem messages

There are many messages for indicating problems with the floppy drive.
If there is a problem, the floppy driver generally will retry the
operation.  If an operation does not succeeded within N retries, the
kernel produces a real floppy I/O error.

Here is a list of the most frequent messages (most are accompanied
by floppy drive and/or sector numbers):

   Drive <nr> is write protected
   data CRC error
   CRC error
   sector not found
   probe failed...
   wrong cylinder
   bad cylinder
   seek failed
   Reset-floppy called

Apart from the first message, all these messages roughly translate to

   Warning: cheap floppies


*** _26_ ERROR CORRECTING CODES AND BAD FLOPPIES

The error correcting codes take approximately 5% of the available
space on a floppy disk.  They allow complete reconstruction of the
data on disks that are only slightly damaged (say 4 or less sectors
are unreadable, or one complete track is unreadable), and often allow
partial reconstruction when there are much more unreadable sectors.

Of course there are limits to what error correcting codes can do. If
6% of the data on a floppy is lost, there is not way to reconstruct it
all.  But with proper floppy disk care, the chance of this happening
is very small, much smaller than the chance that a few unreadable
sectors will develop.  And even if 6% is unreadable, the tbackup
compressed archive format will keep the effects localized.

Note that the availability of error correcting codes does not mean that
tbackup can safely write to floppies that are known to have bad spots.
Some backup programs know how to avoid bad spots when they detect
them, but tbackup does not do this, mainly due to limitations in the
Linux floppy drivers.

Tbackup will try to detect bad spots when writing a backup, and will
reject a bad floppy if it finds one.  Many kinds of bad floppies
cannot be detected while writing, allowing some floppies that are
known to be bad to be used for backups.  Doing this is not
recommended.  Using bad floppies will always make an archive less
safe, regardless of error correcting codes.


*** _27_ RESTORING ARCHIVES WITH MISSING DATA

An archive will miss data if there were read errors during a restore,
and in case error correcting codes were used, if there were too many
read errors to correct them all.

When restoring archives with missing data, two things can happen
depending on how large the `holes' in the archive are:

 a) large hole: some files in the archive are not restored or only
                partially restored
 b) small hole: some of the data in one file is restored incorrectly.

Unfortunately, trestore cannot produce a listing of exactly those
files that are affected by the missing data.  Determining the damages
can be tricky.  Here are some general rules:

There are three sources of information about missing data:

1) The index file of the archive made during the backup.
2) The index file made during the restore.
3) The contents of the restored files.

By comparing the backup and restore indexes (assuming both are
available), you can see what files are missing completely from in the
restore.

Also, the restore index file usually contains archiver error messages
that were produced when processing the `holes' in the archive.  The
files listed immediately before or after an error message may be
damaged or missing completely. For example, with the restore log

 > usr/src/utils/Makefile -- uncompressed
 > usr/src/utils/proto.h -- okay
 > gzip: stdin: invalid compressed data--crc error
 > afio: "inentry xwait()": Exit 1
 > usr/src/utils/utils.c -- uncompressed
 > usr/src/utils/utils.h -- uncompressed

the files proto.h and utils.c may have been restored incorrectly.
Also, the original archive may have contained a one or more files
`between' proto.h and utils.c that are now lost completely.

Unfortunately, the archiver can sometimes restore a file incorrectly
because of a hole *without* giving any error message.  However, if a
file is listed as `-- uncompressed' and is not immediately preceded by
an error message, you can safely assume it was restored correctly.

Another way of checking if files were restored right is by looking at
their contents.  If a file was listed as `-- okay' in a restore of an
archive with a hole in it, and the hole was in the file, the archiver
will in general not produce an error message in the restore log: it
will say `-- okay' but it is wrong.  The hole in the restored file
will however always be `filled' by trestore with ????? characters.  A
quick way of checking all files in a directory for this is by running
the command
  grep -l "????????????" * 
in this directory. This will list all the names of files with
sequences of ????? characters in them.

If a floppy is missing completely from a backup set, and you use the
`q!' command of trestore to go on to the next floppy, in rare cases
the holes in restored files will be filled with the string `This is
dummy data' in stead of ?????.


*** _28_ TBACKUP FLOPPY FORMAT

Here is how tbackup stores an archive on floppies.  This information
may be useful in some cases.

Tbackup uses a normal MS-DOS file structure on its floppies. This has
several advantages:
-you can use standard software to read a tbackup floppy
-you won't mistake a backup floppy for a preformatted empty floppy if
 you're running MS-DOS
-you can try MS-DOS based file recovery tools if a floppy has read errors.

Each tbackup floppy contains a file called DESCR2K.  It contains a
description of the backup set the floppy belongs to and some
parameters that are used when restoring the archive.  All floppies in
a backup set have the same DESCR2K file, which is either 2 Kb or 1.5
Kb big.  Here is an example:

  > Full backup of directory set mystuff.
  > (Filenames in archive relative to /)
  > Made with tbackup (V0.8)
  > by root@noodle.nl, on Sun Apr 17 14:06:14  1994
  > ---
  > export dea_choosemethod=set
  > export dea_setname=mystuff
  > export dea_packhome=/
  > export dea_level=f
  > export dea_backupdate=041714051994
  > export dea_unpackmethod=afio
  > export dea_readmethod=floppy
  > export pre_mdens=h
  > export pre_eccmethod=blexta08
  > export pre_psize=1381376
  > export dea_indexfilename=/root/idx/mystuff.idx.5
  > <<<
  > This is a filler string to get a 1.5k file.................
  > This is a filler string to get a 1.5k file.................
  >  [etc.]

In general you can use a command like
  mtype b:descr2k |head
to see the relevant part.

Tbackup puts files called PARTAA.001, PARTAB.002, PARTAC.003, etc. on
the consecutive floppies of the backup set.  These files hold the real
archive information, if you put them all together you get an afio (or
tar) format archive file.

The error correcting codes for each floppy are in files called
ECORAA.001, ECORAB.002, etc.  The codes are made with the blexta
(Block Level EXor Trace Assembly) algorithm, see the
/usr/lib/tbackup/bin/ecor.c source code for more information.

On the last disk of the backup set, a file called `PART}' is written
indicate the end of the set.


*** _29_ RESTORING FILES WITHOUT TBACKUP

In one particular disaster scenario, you don't have a Linux machine
with tbackup and trestore at your disposal, but need to get files from
your backups anyway.

The following text describes how to unpack a `floppy' backup set if
you don't have tbackup. 

1) get access to a unix machine with a large amount of free disk space.
2) find a way to copy the contents of MS-DOS disks to this unix machine.
3) copy the files named PARTAA.001, PARTAB.002, ... from the backup disks.
4) `cat' them all together in one large file, i.e.
        cat `ls part* | sort` >archive
5) look at the `export dea_unpackmethod=...' line in a DESCR2K file on
   one of the disks to find out if the archive is a `tar'
   or an `afio' archive.
6a) if the archive was packed with tar, use tar to unpack it.
6b) if the archive was packed with afio, unpack it with afio or cpio.
    (Afio produces cpio-format archives.)

If you don't have access to a unix machine you may have some luck with
`tar', `afio' or `cpio' ports to other operating systems (MS-DOS, VMS,
OS/2, amigaOS, ....), provided the ported versions have a sensible way
of dealing with filename limitations.


--------------------------------------------------------------------------

There is more documentation (covering the tbackup rescue disk) in the
/usr/lib/tbackup/rescue directory.

--------------------------------------------------------------------------
