The Class "tc" - Token Converter
================================


int str_to_id(char* str)
------------------------
Takes a string and returns an integer which identifies this string.

const char* id_to_str(int id)
-----------------------------

Takes an integer, id, and returns a pointer to the string identified by id.
id must have been generated by a preceding call to str_to_id.  


Implementation
--------------
The strings and their associated identifiers persist in a file which is
divided into a fixed number of pages (determined by TCSize), each containing
a fixed number of characters (determined by TCPageSize).  The string is
hashed to generate a page number and it is stored in the first available
position within that page.  The identifier returned by str_to_id encodes
within the top two bytes the page on which the string is stored, and within
the bottom two bytes the offset within that page.  Thus neither TCSize nor
TCPageSize can exceed 0xFFFF, and TCSize should be prime to ensure an even
distribution of strings across the pages.


Bugs and Comments
-----------------
The size of a Token Converter is fixed at compile time using the parameters
TCSize and TCPageSize.  Thus an empty Token Converter takes up a lot of
space.  More importantly, this places a limit to the number of strings which
can be stored.  A more graceful approach is clearly called for.

The largest string that can be stored is TCPageSize.  A more flexible
strategy for long strings should be provided.

The address returned by id_to_str may be re-used on subsequent calls to both
str_to_id and id_to_str.  Therefore the string should be copied if it is
needed.
