                          **************
                          * CIV_DE.TXT *
                          **************

                 About CIV_DE.EXE  and  CIV_DE.LIB

Version:      Beta 1
Release date: 24 September 1993

-------------------  BORING STUFF  ------------------------

Both CIV_DE.EXE and CIV_DE.LIB are copyright of BoGaRT sas. Florence - 
Italy. They are provided 'as they are', without any warranty, either 
expressed or implied.... (open any software manual, at random, and complete 
the previous sentence).

CIV_DE.EXE and CIV_DE.LIB are FREE.  If, this statement notwithstanding, 
you think my work is worth of some remuneration, I'll greedily accept any 
amount you may like to send...

I would like to thank Steve Cotellesse and Holger Eichmann for their advice 
and for the very useful utilities they made available.

Send suggestions, corrections, complaints, money... to:

Maurizio M. Gavioli
Via Mino 1
I-50122 FLORENCE
ITALY

Internet:         vincigli@vm.idg.fi.cnr.it
CompuServe:       INTERNET:vincigli@vm.idg.fi.cnr.it

-------------------  DISCLAIMER  --------------------------

CIV_DE is the result of days and days of hard decoding and file analysis 
work.  It is not distributed with an eye towards enabling anyone to rob the 
original authors of any rightful profits and should be used only for the 
common good of the CIV community and to encourage availability and sales of 
CIVILIZATIONS and related products and utilities.

-------------------  SUMMARY DESCRIPTION  -----------------

CIV_DE.EXE is a simple exercise in using the decoding/encoding library 
CIV_DE.LIB.  The program has two major functions:
1) to read, decode, and save a CIV encoded file (.MAP, .PIC)
2) to read, encode, and save back a decoded .MAP or .PIC file

CIV_DE is particularly suited for dealing with .MAP files (decoding and re-
encoding), but it can be used also for decoding and encoding .PIC files; 
however, given that .PIC files may contain color palettes in addition to 
the bitmap, some care is required to regenerate a proper .PIC after 
decoding (see below).

CIV_DE.LIB is a small C library intended as a basic decoding-encoding tool 
for CIV encrypted files; CIV_DE.EXE is an example of its usage.

FILE LIST:

CIV_DE.EXE   CIV's Decoder/Encoder
CIV_DE.C     the source of CIV.EXE
CIV_DE.LIB   the C library
CIV_DE.H     the library header
CIV_DE.TXT   this file

-------------------  CIV_DE.EXE  --------------------------

DECODING

The program can generate three kinds of decoded outputs:

1) binary image(default)
2) textual hex dump
3) Windows bitmap (.BMP)

CIV's .MAP files contains only the encrypted 'bitmap', while .PIC files may 
contain (and usually do) also a 'short' palette (256 items of 1 byte each) 
and/or a 'long' palette (256 items of 3 bytes each): text and binary 
outputs only include the decrypted bitmap, BMP adds also the 'long' 
palette.

1) BINARY OUTPUT is a binary image of the bitmap: to read and edit it a 
binary editor is needed.
2) TEXT OUTPUT represents binary bytes of the bitmap with hex digits.  Two 
points are worth noting about text outputs:
* digits 0 are replaced by periods to make empty areas more visible.
* text ouputs are made of separated vertical "slices" 40 bytes (=120 
characters) wide: for instance, a 320x200 bitmap will be rendered with 8 
"slices" (covering bytes 0-39, 40-79, 80-119... of each row) each 200 rows 
long.  Each slice is terminated by a FormFeed character.  Once printed, 
slices can be "glued" together side by side to recontruct to entire image.
3) WINDOWS BITMAP OUTPUT generates a Windows compatible .BMP file.  This 
format contains also a color palette; if the file to be decoded contains a 
palette (as many .PIC file do), this palette is added to the .BMP; if the 
file did not contain a palette (like .MAP file), a default palette is 
supplied by the program.  In both case, an optional palette source file can 
be specified, and the program will insert in the .BMP the palette found in 
that file.

The syntax for calling CIV_DE for DECODING is:

CIV_DE inputfile[.MAP] [outputfile] [/I|/T|/B] [/Wnn] [/Ppalettesourcefile]

where:
inputfile   is the CIV file to read; may contain a path; a .MAP extension 
            is supplied by the program if none is entered
outputfile  is the name of the output file to create; may contain a path
            and an extension; if no extension is entered, the program
            supplies .BIN for binary outputs, .LST for text outputs and
            .BMP for bitmap output.  This parameter is optional, if it is
		omitted, the program creates an output file with the same file
		name as the input file and extension as above.

/I          generates binary image output (this is always the default and 
            this switch has been added only for the sake of clarity)
/T          generates text output (optional; default is binary)
/B          generates a Windows bitmap (optional; default is binary)

/Wnn        assumes the bitmap is nn bytes wide (0 < nn < 321);
            with text and BMP output, adjusts the dimensions of the bitmap; 
            with binary output, this option is ignored
            There is no default for this option: if it is not used, the 
            program will use the width encoded in the input file itself.

/Pfilename  with BMP output, inserts into the BMP the palette found in 
            filename (which MUST be a CIV .PIC or .PAL file); with binary
            and text output, this option is ignored.

The first file name is always the input file and the second, if present, is 
the output file; options, indicated by the '/' character, can appear 
anywhere and in any order.

Examples:

	CIV_DE CIVIL0
decodes CIVIL0.MAP into CIVIL0.BIN in binary format

      CIV_DE CIVIL0 MAP_0
decodes CIVIL0.MAP into MAP_0.BIN in binary format.

      CIV_DE \MPS\CIV\TER257.PIC TER257 /B
decodes TER257.PIC into TER257.BMP with BMP format.

      CIV_DE \MPS\CIV\CIVIL1 CIVIL1 /B /P\mps\civ\sp257.pic
decodes CIVIL1.MAP into CIVIL1.BMP with BMP format using the color palette 
found in SP257.PIC (you are encouraged to try this example and to look at 
the resulting BMP, for instance, with the Windows PaintBrush).

ENCODING

The program can re-encode only BINARY or BMP decoded file, not TEXTUAL.
No input format has enough data to reconstruct a full 3-sections ('short' 
palette, 'long' palette, bitmap) .PIC file: binary contains only the bitmap 
and the Windows BMP format has roon for only one palette: by default, it 
NEVER inserts palette(s) in the re-encoded file (this is well-suited for 
.MAP files); when re-encoding a .PIC file, an option palette source file 
may (and has to) be specified.

The syntax for calling CIV_DE for ENCODING is:

CIV_DE infile [outfile[.MAP]] /E [/I|/B] [/Wnn] [/Ppalettesourcefile]

where:
infile      is the decoded file to read; may contain a path; a .BIN or a 
            .BMP extension is supplied by the program, if none was entered.
outfile     is the name of the CIV file to create; may contain a path;
            if no extension was entered, the program supplies .MAP.  This
		parameter is optional, if it is omitted, the program creates an
		output file with the same file name as the input file and
		extension .MAP.

/E          specifies ENCODING

/I          assumes the input file is a binary image (default)
/B          assumes the input file is a BMP

/Wnn        assumes the bitmap is nn bytes wide and adjusts the dimensions 
            of the bitmap; with binary input the default is 320; with BMP 
            input, the default is the width of the BMP bimap.

/Pfilename  add to the output file the palette found in filename (which 
            MUST be a CIV .PIC or .PAL file).

The first file name is always the input file and the second, if present, is 
the output file; options, indicated by the '/' character, can appear 
anywhere and in any order.

Examples:

      CIV_DE \MYMAPS\CIVIL0 CIVIL0 /E
encodes CIVIL0.BIN (binary format) into CIVIL0.MAP without any palette.

      CIV_DE TER257 \MPS\CIV\TER257.PIC /E /B /P\mps\civ\ter257.pic
encodes TER257.BMP (BMP format) into TER257.PIC using the palette found in 
TER257.PIC itself (the palette is read before the output file get 
overwritten!).

      CIV_DE CIVIL1 \MPS\CIV\CIVIL1 CIVIL1 /E /B
encodes CIVIL1.BMP into CIVIL1.MAP without any palette.

EDITING OF DECODED FILES

Theoretically, .PIC's and .MAP's can be decoded, edited and re-encoded, 
provided that the editing step do not break the integrity of the file 
contents.
Given that text files cannot be re-encoded, text editing is of no rele-
vance.  Binary editing requires an hex editor and is very cumbersome (many 
CIV files have rows 320 bytes wide and this clashes with the usual 16-byte 
line of many hex editors).
The solution would be BMP editing: there are many programs which can read 
and manipulate BMP.  Unfortunately, all the programs I tried play with the 
palette and change the coding of the colors: the .PIC looks fine while in 
the Windows environment, but once back in CIV all colors are mixed up.

If there is someone who knows a workaround, is willing to write a special 
BMP editor or to add, for instance, a PCX compatibility to CIV_DE, he would 
be vvvery welcome.

-------------------  CIV_DE.LIB  --------------------------

The library supplies 4 global variables and 2 accessible functions (plus 
some useful declarations and #define's).

GLOBAL VARIABLES

int GridWidth     the width of a CIV (bit)map
int GridHeight    the height of a CIV (bit)map

These variables serve both for "input" and "output": they are set by 
ReadCIVFile(), after a CIV file has been decoded, to the width and height 
of the decoded bitmap.
Also they MUST be set to proper values before calling WriteCIVFile() to 
encode a file.

CIVPALETTE1 CIVPalette1   a CIV 'short' palette
CIVPALETTE3 CIVPalette3   a CIV 'long' palette

These two structs (defined in the CIV_DE.H header) are used by 
ReadCIVFile() to store the palettes (if any) found in the decoded file: 
after ReadCIVFile() returns, if CIVPalette1.Length and/or 
CIVPalette3.Length are not 0, the decoded file contained one and/or the 
other palette.
The two structure MUST be filled and the Length members set to their proper 
values (CIVPalette1.Length = 0x102; CIVPalette3.Length = 0x302) for 
WriteCIVFile() to include the palettes in the encoded file.

FUNCTIONS

int ReadCIVFile(char * FileName, char far * CIVBuffer, int Flags)

reads and decodes the given file (the name can include a path) into 
CIVBuffer, filling GridWidth, GridHeight, CIVPalette1 and CIVPalette3.
CIVBuffer is assumed to be at least 64,000 bytes long and must be allocated 
beforehand by the calling application.
Flags: CIV_GETBITMAP = get palettes (if present) and bitmap
       0             = get only the palettes

Return: 0 on disk error
        1 if successful


int WriteCIVFile(char * FileName, char far * CIVBuffer)

encodes and saves to the given file (the name can include a path) the 
'bitmap' present into CIVBuffer.  It includes in the output file the 
contents of CIVPalette1 if CIVPalette1.Length is not 0, and of CIVPalette3 
if CIVPalette3.Length is not 0.

Return: 0 on disk error
        1 if successful

----------------------------------------------------------------

                    A SHORT NOTE ON CIVILIZATION'S
                         .MAP, .PIC and PAL
                           FILE STRUCTURES

To my surprise, I found .MAP, and .PIC files made of the same stuff: both 
contain a 'bitmap' which is encrypted with the same algorithm.  It is for 
this reason that the same code can be used for decode and encode both types 
of files.

In general, .MAP, .PIC and .PAL files are made of sections.  Three kinds of 
sections actually occur in current CIV files, which can be recognized by 
their first byte.  All sections have a common 4-byte header:

byte 0      section code:  45h = 'short' palette
                           4Dh = 'long' palette
                           58h = encrypted bitmap
byte 1      coding type: 30h or 31h
byte 2-3    section length (header excluded)

The purpose of the 'short' palette is not yet clear.  They are always 0102h 
bytes long (header excluded) and have the following structure:
byte 4      first color
byte 5      last color
byte 6-261  256 single-byte entries

'Long' palettes are actual palettes, containing VGA color definitions.  
They are always 0302h bytes long (header excluded) with the structure:
byte 4      first color
byte 5      last color
byte 6-773  256 3-byte entries, containing Red, Green, Blue (in that order)
            components of 256 colors.  Components are in VGA range 00-3F

Encrypted bitmaps have the following structures:
byte 4-5    bitmap width
byte 6-7    bitmap height
byte 8      always 0Bh (?)
byte 9      a value used as a 'seed' for the encryption algorithm
byte 10-... encrypted bitmap

Coding type seems to be relevant only for section 58h: type 30h is 
'normal', type 31h is used for bitmap where color codes are always less 
than 10h (then taking only a nibble): before encoding, the two lower 
nibbles of two near bytes are packed into a single byte, halving the size 
of the bitmap, which is afterward encrypted normally.  At decoding, each 
decrypted byte is split back into two.  The only files with coding 31h are: 
KINK00.PIC, KINK03.PIC, MAP.PIC, SPRITES.PIC and TORCH.PIC.
CIV_DE is able to decode both coding types; when rencoding, it always uses 
normal type (30h).

.PAL files contain only a 'long' palette section.
.PIC files usually contain a 'short' palette, a 'long' palette and an 
encrypted bitmap sections (in that order), but one or both palettes may 
sometime lack.

.MAP files contain only a section 58h, with the encrypted map.  Maps are in 
fact 320x200 bitmaps and can be easily divided in 4x4 sub-squares of 80x50 
pixels each. Six of these squares contain some components of CIV graphics 
(city improv. and worder icons, minister pictures...), but the other 10 
depict 10 parameters of the 80x50 game map:

    0    --   79 80   --  159 160  --  239 240  --   319
  0 *-----------*------------*------------*------------*
    |  terrain  | as below + | units and  |  small map |
    |    map    | city areas | un. trips  | (upp.left) |
 49 *-----------*------------*------------*------------*
    |  island   |  tactical  |                         |
    |  coding   |   values?  |                         |
 99 *-----------*------------*          C I V          |
    | all field |visib. field|                         |
    | improvmts | improvmts  |        graphics         |
149 *-----------*------------*                         |
    |    ???    |    ???     |                         |
    |    ???    |    ???     |                         |
199 *-----------*------------*-------------------------*

Thanks to Holger Eichman for his work in recognizing some of the above, 
without the help of CIV_DE!

