Understanding color IMGs A novel by Dr. Bob 27 September 1992 IMG file formats, bi-level and color. The IMG standard from DRI is composed of a file header and encoded (or not encoded) bit-image data. Bi-level, or monochrome, IMGs have a very straight forward and efficient storage method. In fact, the compression ratio is about the best around for non-LZW compression (GIFs and some TIFFs use LZW to achieve quite a great compression ratio). Bi-level IMGs have been in widespread use for quite a while but with the advent of color video systems, the IMG standard has become bogged down. This is due, primarily, to the vague- ness in the description of the IMG file format concerning storage of the color data (both the color palette and the color bitimage itself). Since GEM has taken a rather backseat position in the computing world today, it is doubtful that DRI will assist in clarifying the issue. And since it can be said that ATARI is the last real GEM strong hold in the computing world (being that the ST's operating system is designed in its entirety around GEM), it would seem a rather natural step that they (Atari) take some step or steps to either publish a standard or at least a suggestion for a standard for color IMG graphics. Alas, this has not happened. In all the seven years since the ST came into being, no color IMG format has gelled into a stan- dard. Several vendors have designed both legal and illegal variations of the IMG standard in order to support color but in the end, all that has come into being is incompatibility. This document will describe four different renditions of color IMG formats (variations on a theme, you might say). A fifth, which has been discovered but not yet disected, will be appended at a later date. Names will be given to discern one version from another. These names are not intended to detract from anyones rights or give any privileges to anyone, but simply to keep some clarity amist the confusion. First we'll examine the normal bi-level IMG format to give us a basis for later comparison. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - GLOSSARY insert: BI-LEVEL: Two colors. Usually meant to be black and white (B/W). This is often called monochrome although 'monochrome' can also imply shades of grey. Bi-Level is a more accurate description of the black-n-white imagery we're concerned with in this document. TOKEN: Used in uncompressing a file. A code, usually only a byte, that indicates the start of a compression scheme. For IMGs, there are four different tokens: $80=Bit-string, $00=Pattern-run, $00+$ff=VRC (note: two bytes) and there is Solid-run which is any other value not listed above. WORD: A 16-bit value, taking up two bytes of space. The order is Motorola Hi-Lo. (on other systems, the order may be reversed to lo-hi) Sample: 256 = hex $0100 $01,$00 128 = hex $0080 $00,$80 DRI: Abbreviation of Digitial Research Inc., the owner of GEM (Graphic Enviornment Manager) and its parts such as AES, VDI etc - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - The standard DRI IMG file header is comprised of eight (8) words: word offset typical description -------------------------------------- 0 $00 $0001 IMG version 1 $02 $0008 Header length * 2 $04 $0001 Number of planes 3 $06 $0002 Pattern def len 4 $08 $0055 Microns width 5 $0A $0055 Microns height 6 $0C $0280 Image width 7 $0E $0190 Image height -------------------------------------- Let's examine each of these. IMG version: This denotes the version of the IMG file format. It is always one (1), by DRI's specification. No other IMG version has ever been designed (or authorized) see: XIMG also HEADER length: This is, slightly, a misnomer since it alludes to the LENGTH of the header. It is actually the number of WORDS in the header, so it may be more accurate to term this: HEADER COUNT All bi-level images have an 8 in this word, meaning that there are 8 words in the header. The value found here for color images will vary depending mainly on the size of the palette and also the particular color IMG rendition. note: since the palette is stored within the header of the IMG file, HEADER COUNT includes the palette data as well as the standard header. Close attention must be paid to this word when working with color IMG files since it is the only way we have to determine the start of the image data. With bi-level IMGs it was safe to assume that all IMG file headers were fixed at 8 words. An assumption like this can be dangerous when working with color IMGs. Always determine the header length from this word. the start of the image is found using: IMAGE_START = Filestart + (HEADER_LEN*2) Number of PLANES: This is, as it seems, the number of planes in the image. Bi-level (mono) images have, of course, only one plane. This word also dictates, as one would assume, the number of colors in the image. An image with 4 planes has 16 colors and an image with 8 planes has 256 colors. NUMBER COLORS = 2^PLANES. PATTERN DEFINITION length: This word is only of importance for one of the compression techniques in the IMG specification from DRI. Some authors may use it and some may not. It specifies the size of patterns for the token PATTERN-RUN, and is usually either one (1) or two (2) but can, in all legality, be ANY number. You'll find, however, that it is usually an EVEN number when it's higher than 1. A 1 means that the pattern to be duplicated is one byte in length or 8-bits. A 2 means the pattern is two bytes wide, 4 means it is four bytes wide and so on. MICRONS, words 4 and 5: MICRONS denote the actual size of the pixels. They can be teeny tiny dots or they can be huge. Many authors may choose to ignore this (and many do) since it is common practice to treat one dot as one video pixel. Also of interest here is the fact that both WIDTH and HEIGHT are specified. This means that the pixels may not necessarily be square (equal in width and height). This is often the case when the image is based on a particular video resolution such as Atari/ST Medium resolution or the PC's 2-color resolution or any other resolution where the aspect is not 1:1 (the TT's low rez comes to mind also). DPI = (25,500/MICRONS) MICRONS = (25,500/DPI) 85 MICRONS = 300 DPI 255 MICRONS = 100 DPI And finally, image WIDTH and HEIGHT: words 6 and 7 Width is specified in number of pixels and height, of course, is the number of lines (or rasters). Although the width is stated in number of pixels, the image only stores whole bytes. If the image WIDTH is 633 pixels then 80 bytes are stored. 79 full bytes and one last byte of which only 1 bit contains any information. The other 7 bits are not valid image data and may be be blank, filled or totally garbage. ----------------------------------------------------------------- A note on IMG compression methods: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Although this document does not go into detail on the different compression methods used, there are some details which are important and that are not mentioned or not clearly stated in the normal channels. All and any compression ends at each raster boundary. In other words: each raster is compressed individually. Pattern runs, byte strings, bit runs all end at the end of each raster. Each new raster, if compressed, starts a fresh compression sequence. There is no overrun from one raster to another. Although it up to the author which compression functions to use, it is necessary for an IMG reader to expect a VRC function (even though one particular IMG may or may not contain one). Always assume that an absence of any VRC (or VRC=0) is the same as VRC=1. This will avoid confusion. Since a VRC of 1 does NOT mean to repeat the raster 1 time but means only to write the raster once. Actually, a VRC code of 1 (one) is completely unecessary in any IMG. If this is encountered it is probably due to a fluke in the authors encoding technique and/or a lack of clarity in his/her source of IMG documentation. This is not to say that a VRC of 1 is in any way illegal. Quite the contrary, it is completely legal; just not a necessesity. ----------------------------------------------------------------- *** COLOR IMG FORMAT VARIATIONS *** It can be said that there is only one IMG format in existance. While this is technically true, it is more a case of semantics than an actual real-life truth. If there were only one IMG format then there should be no com- patibility problems with any color IMG file and any application that attempts to access that color IMG file. Sadly, that is not the case. While there may be only one FORMAT, there is certainly an abundance of color 'dialects'. Each of which is just different enough to cause woes to the end user. What can be so difficult in establishing a standard color IMG format? The main areas of contention are: 1) color palette, what type of system 2) arrangement of the bit image planes A third item has arisen due to the existance of the different 'dialects' 3) How to discern one type of color IMG file from another. - - - - - - - - - - - - - - 1) COLOR PALETTE: a) where b) what kind DRI specified no particular method for storing the color palette. Nor did they say where it should be stored. A) where Every color 'dialect' design has, quite curiously, chosen the best method as to where to store the palette data. It is placed directly after the normal file header and the HEADER LENGTH word is adjusted to include this palette data. Conclusion: no problem here. B) kind of color HOW should the palette be stored? This question arises since the ST community has for a long time used and has grown accustomed to the fixed size files of DEGAS, TINY and NEO. When authors then started to design color IMGs they naturally carried over some of their learning, namely the palette. These DEGAS, TINY and NEO files used a palette that is similar to the palettes of other computer systems but with the Atari ST specific word sized colors. This is commonly called a 'hardware' or, in the ST community, the 'XBIOS' style of palette. Since we're working with DRI's IMG file format, it is natural to assume that the color palette also be stored as a DRI standard may or might be. So, other authors decided to, instead, store the palette as the VDI portion of GEM would expect it. Both methods have their advantages. The XBIOS method lends itself to easy porting of other file formats since it is directly hardware oriented and can be efficiently and quickly converted to VDI colors. The VDI method, while portable with a little extra effort, does not require any modification for use in a VDI enviornment. Conclusion: incompatible palettes. 2) BIT IMAGE PLANES Due to DRI's vague documentation, no clear method has been established as to how to store the color bit image data and seems to be totally open to each authors interpretation. Some have chosen to store each plane of data in its entirety and separate from another, while other authors decided to interleave rasters of each plane. Once again, each method has advantages and disadvantages. Somehow, it would not be suprising to soon find yet a third method appear that stores each pixel in its entirety (like GIF files) or even a fourth method that stores the plane data in a direct ST video layout (like DEGAS, TINY, NEO). Conclusion: incompatible bit-image If it is true, then, that there exists only one IMG format then it must also be true that the IMG format is, indeed, incompatible with itself. ------------------------------------------------------------------- Here, then, are four of the color IMG dialects currently in use. We'll label them: NOSIG, HYPERPAINT, XIMG and STTT. NOSIG is an archaic dialect that is limited to 16 colors. We call it NOSIG because it contains no signature or no means by which to determine exactly what dialect this file may be. We say it is fixed to only 16 colors because, 1) no 256 color IMGs of this sort have been seen and, 2) one must _assume_ that any 8-plane form would follow the same procedures as a four plane file. SIGNATURE: none PALETTE : XBIOS (fixed at 16 colors) BITIMAGE : separate planes HYPERPAINT is an IMG format with a twist. A noted graphic editor will also create these files when used on an STe (using the STe's higher color capacity). SIGNATURE: word $0080 preceeds palette PALETTE : XBIOS BITIMAGE : interleaved raster planes XIMG is called such since it stores that ascii text, "XIMG", as a signature in the file header. note: XIMG states an img version of 2 SIGNATURE: long "XIMG" preceeds palette PALETTE : VDI style BITIMAGE : separate planes STTT is called such since it stores that ascii text, "STTT", as a signature in the file header. SIGNATURE: long "STTT" preceeds palette PALETTE : XBIOS BITIMAGE : separate planes Legend for following chart: A) NOSIG B) HYPERPAINT C) XIMG D) STTT Sample/Typical IMG file headers for 4 plane/ 16 color IMG file: offset description A B C D ----------------------------------------------------------------- 0 imgver 1 1 2 1 2 hedlen 24 25 59 27 4 planes 4 4 4 4 6 patdef 2 2 1 1 8 micwid $0294 $022C $022C $0116 10 michgt $02DF $022C $022C $0116 12 imgwid _ _ _ _ 14 imghgt _ _ _ _ - - - - - - 16 pal $0080 "XI" "ST" 18 pal "MG" "TT" 20 $0000 $0010 22 pal pal 24 ----------------------------------------------------------------- notes: the image width and height are not shown as these will be totally dependent upon the particular image in the file. 'pal' denotes where the palette begins in the header. a 256 color IMG header is very similar. PLANES will be 8 and the value in 'hedlen' will be larger to encompass the larger color palette. The value in the header's headlength will always contain at least an eight since the IMG must have at least the 8 normal header words. Additional words will be added to this sum for the palette and any signature word or long. VDI palette: 3 words per color (1 for each of R,G,B) XBIOS palette: 1 word per color. For a 16 color VDI palette : 48 words For a 16 color XBIOS palette: 16 words For a 256 color VDI palette : 768 words For a 256 color XBIOS palette: 256 words Different variations of color IMGs may also include a signature which is also counted in the HEADER LENGTH word. NOSIG off descrp A --------------------------------------------------------------- 0 imgver 1 always 1, as per DRI specs 2 hedlen 24 24 words = 8 normal + 16 color 4 planes 4 four planes 6 patdef 2 8 micwid $0294 38 DPI 10 michgt $02DF 34 DPI 12 imgwid _ 14 imghgt _ - - - - - - 16 the palette begins here and is 16 words in the XBIOS format (1 word per palette entry) immediatly following the palette is the bitimage with each plane stored in its entirety. notes: none problems: Since no signature exists, one must _assume_ that any 4-plane IMG file is actually this format. possible: solution: Check for all other variants first. If the other tests fail then assume that the IMG is this type. =============================================================== HYPERPAINT off descrp B --------------------------------------------------------------- 0 imgver 1 always 1, as per DRI specs 2 hedlen 25 8 normal + 16 colors + 1 signature 4 planes 4 four planes 6 patdef 2 8 micwid $022C 45 DPI 10 michgt $022C 45 DPI 12 imgwid _ 14 imghgt _ - - - - - - 16 $0080 (128) this is the only signature of this dialect. 18 the palette begins here and is 16 words in the XBIOS format (1 word per palette entry) immediatly following the palette is the bitimage stored as 4 rasters (one from each plane) inter- leaved. notes: The order of the rasters are inverted! Plane-0 is the last raster in each group. In a four-plane IMG, the order of the rasters is: planes 3,2,1,0 problems: The simple signature is misleading since the NOSIG variant expects the palette to begin here, may easily mistake the $0080 signature word to be the first color of the palette. Since these two dialects, NOSIG and HYPERPAINT, are very different in plane layout, you'll find that a wrong choice of dialect will result in a totally garbaged picture. possible: solution: The possibility of $0080 being the first palette entry is slim (but still probable). "Best Guess" is all that can be said here. =============================================================== XIMG off descrp C --------------------------------------------------------------- 0 imgver 2 NOTE THIS!! 2 hedlen 59 8 normal + (16 colors *3) + 3 signature 4 planes 4 four planes 6 patdef 1 8 micwid $022C 45 DPI 10 michgt $022C 45 DPI 12 imgwid _ 14 imghgt _ - - - - - - 16 "XIMG" signature (4 bytes) 20 $0000 zero word 22 the palette begins here. It holds 3 words per color in the VDI format of 0-1000. ( 16 colors = 48 words) (256 colors = 768 words) immediatly following the palette is the bitimage stored as separate planes. notes: none problems: Eight plane images may appear a bit unwieldy but innovative coding can easily clear this hurdle. possible: Keep a pointer to the image buffer start and solution: weave the image into the proper planes as you uncompress it. =============================================================== STTT off descrp D --------------------------------------------------------------- 0 imgver 1 as per DRI specs 2 hedlen 27 8 normal + 16 colors + 3 signature 4 planes 4 four planes 6 patdef 1 8 micwid $0116 90 DPI 10 michgt $0116 90 DPI 12 imgwid _ 14 imghgt _ - - - - - - 16 "STTT" signature (4 bytes) 20 $0010 palette count (or the number of colors) 22 the palette begins here and is in XBIOS form (1 word per palette entry) ( 16 colors = 16 words) (256 colors = 256 words) immediatly following the palette is the bitimage stored as separate planes. notes: the 'palette count' word is a good redundancy check problems: Eight plane images may appear a bit unwieldy but innovative coding can easily clear this hurdle. possible: Keep a pointer to the image buffer start and solution: weave the image into the proper planes as you uncompress it. ===============================================================