Mini SQL 2.0

Beta

Writing Lite / W3-mSQL Modules

Please Note

The writing of modules requires a good understanding of the C language. Modules are used to extend the functionality of the language beyond what can be achieved using the Lite Library facilities. If you are not familiar with C this document may be hard to understand.

Introduction

The Lite scripting language (which incoporates W3-mSQL) provides a mechanism for accessing external C language functions from within Lite scripts. This functionality is used to provide the Standard Module (access to many standard UNIX functions) and also the mSQL Module (access to the mSQL API functions). If there is a C library that you would like to access from Lite or W3-mSQL scripts then you can write a custom module to provide this access.
The applications of this are endless. For example, you may wish to use Lite as a scripting language for network management. In this case you could easily write a module to provide access to an SNMP library from within your Lite scripts. Similalry, if you wanted to generate GIF images in real-time you could provide access to a graphics library such as GD using a simple interface module. Hughes Technologies will provide extra modules for Lite in the future.

How does it work?

Inside the Lite Virtual Machine (VM), that is the software environment that actually manages the execution of Lite code, there is a table that provides a mapping between Lite function names and external C code functions from the modules. For example, the lite function open ( ) may map to a C function in the standard module called doOpen ( ). The VM uses this table whenever your Lite script calls a function. If your script calls the open ( ) function the VM understands that it must actually call and external C function called doOpen ( ).
The other aspect of calling external functions is the passing of data to the function and returning data from the function. A function will usually require some parameters (such as a file name in the case of open ( ) ) and will also usually return some information (such as a file descriptor). Data that is being passed to the script as a parameter must be taken from the internal data structures used by the VM and provided in a useable form to the C code function in the module. Similarly, the C function must be able to take a normal C variable value and give it back to the VM in a useable form so that it can then be manipulated within the Lite script (assigned to a variable for example).
Passing data into the module function is achieved via a parameter list. A paremeter list is a linked list of symbol structures where each element of the list represents one of the parameters passed to the function from inside the Lite script. The C function can then traverse this list and access the data values contained within the symbol structures. The parameters are presented to the C function in the same order in which they were passed to the Lite function. So, if a call was made inside a lite script as follows
open ( "/tmp/test.txt" , "<" ) ;
the C function doOpen ( ) would be called with a parameter list containing 2 symbols. The first symbol would contain "/tmp/test.txt" while the second would contain "<".
Returning data to the script from the C function is achieved in a similar manner. It is the responsibility of the C function to create a valid symbol structure containing the return data. A pointer to this symbol structure is then assigned to a global, external variable called externReturn so that the VM can access this newly created symbol when execution resumes. Convenience functions are provided to simplify the creation of symbols and are covered in detail below.

Defining the module functions

Informing the VM of the functions you have defined in your module is done through a C table contained in the module's header file. Each entry in the table includes the Lite name for the function, a pointer to the C function, and details about the number and type of parameters. If details of the parameters are given, the VM will ensure that the function is called with the correct number and type of functions. This simplifies the code you write in your module as you do not need to check the validity of the data being passed as parameters.

Figure 1. The mSQL module header file

/* ** mod_msql.h - Lite / W3-mSQL module for mSQL Access ** ** ** Copyright (c) 1995 Hughes Technologies Pty Ltd. ** ** Permission to use, copy, and distribute for non-commercial purposes, ** is hereby granted without fee, providing that the above copyright ** notice appear in all copies and that both the copyright notice and this ** permission notice appear in supporting documentation. ** ** The software may be modified for your own purposes, but modified versions ** may not be distributed. ** ** This software is provided "as is" without any expressed or implied warranty. ** */ /* ** External function prototypes */ void doMsqlConnect(); void doMsqlClose(); void doMsqlSelectDB(); void doMsqlQuery(); void doMsqlStoreResult(); void doMsqlFreeResult(); void doMsqlFetchField(); void doMsqlFetchRow(); void doMsqlFieldSeek(); void doMsqlDataSeek(); void doMsqlListDBs(); void doMsqlListTables(); void doMsqlInitFieldList(); void doMsqlListField(); void doMsqlNumRows(); void doMsqlEncode(); void initModMsql(); /* ** mSQL external function definitions ** ** This table maps from internal Lite function names to the C code. ** The format is :- ** ** {FunctName, CFunction, NumArgs, { Arg1Type, Arg2Type....., 0}} ** ** Setting NumArgs to -1 indicates varargs */ efunct_t msql_efuncts[] = { { "msqlConnect", doMsqlConnect, -1, {0}}, { "msqlClose", doMsqlClose, 1, {P_INT,0}}, { "msqlSelectDB", doMsqlSelectDB, 2, {P_INT,P_TEXT, 0}}, { "msqlQuery", doMsqlQuery, 2, {P_INT,P_TEXT, 0}}, { "msqlStoreResult", doMsqlStoreResult, 0, {0}}, { "msqlFreeResult", doMsqlFreeResult, 1, {P_INT,0}}, { "msqlFetchRow", doMsqlFetchRow, 1, {P_INT,0}}, { "msqlFetchField", doMsqlFetchField, 1, {P_INT,0}}, { "msqlFieldSeek", doMsqlFieldSeek, 2, {P_INT,P_INT,0}}, { "msqlDataSeek", doMsqlDataSeek, 2, {P_INT,P_INT,0}}, { "msqlListDBs", doMsqlListDBs, 1, {P_INT,0}}, { "msqlListTables", doMsqlListTables, 1, {P_INT,0}}, { "msqlInitFieldList", doMsqlInitFieldList, 2, {P_INT,P_TEXT,0}}, { "msqlListField", doMsqlListField, 0, {0}}, { "msqlNumRows", doMsqlNumRows, 1, {P_INT}}, { "msqlEncode", doMsqlEncode, 1, {P_TEXT}}, { NULL, 0 } };

Using the above header file as an example we'll go through the process of defining the module interfaces step by step. The example is the actual module definition for the mSQL module.
The first block after the copyright notice provides simple pre-declaration of the C functions exported from the module. You must pre-declare the functions or your compiler will complain. Note that all module functions are defined as void as the data returned by the function is done so via a symbol structure as outlined above.
Once the functions are declared you can create the table defining the module interface. Each entry in the table includes

The name of the Lite function as a text string
A pointer to the C function
The number of parameters the function requires (-1 indicates a variable number of paramaters, known as varargs in the C world).
A list of parameter types, 1 for each of the parameters, terminated by a zero value

The list itself is terminated with an NULL entry as illustrated in the example. If the function has varargs you can define the types of the known parameters. As an example, a call to printf ( ) can include a variable number of parameters but the first parameter must be a character string (the format string). Defining such a function could be done using -1 for the parameter count and { P_TEXT, 0 } as the parameter type list. This will allow the VM to enforce the correct type on the format string.

Passing data to module functions

The module functions are just standard C functions that know how to understand the Lite VM's parameter list and how to return data to the VM properly. As was mentioned previously, the basic type involved in both operations is the symbol structure. The symbol structure contains several fields that are of no interest to us here so we will just outline those that are important. Below is an abreviated definition of the symbol stucture.

Figure 2. The Symbol Structure

Field Name Definition Purpose

val char * A pointer to a value of the appropriate type

type short Integer value specifying the symbol type

array char Flag indicating if the symbol is an array

As was previously mentioned, the parameter list is a link list in which each element contains a symbol structure. The field name of the symbol structure within the elements of the linked list is sym and the field name of the pointer to the next element of the list is next (where next is NULL in the last list element). So, it stands to reason that if the parameter list is called params that the value of the first parameter can be found in params->sym->val, the value of the second parameter is in params->next->sym->val and so on.
It is shown in that the val is a generic pointer to the data element (represented as a char pointer). In the case of a character value being passed as a parameter we simply have to access the value as illustrated in the previous paragraph. If the value is an integer or a real value then we have to measures to ensure it is cast properly (otherwise we will not get the correct value). Illustrated below in figure 3 is the approach required to handle the 3 basic types (note that arrays are handled in a special manner as described below). The example below is a function that is passed 3 values, an int, a real and a char. The fact that there is a cast to the correct pointer type as well as a cast of the result looks like overkill but it is the safest approach (I've seen some compilers that requires this syntax).

Figure 3. Handling Parameter Values

void someModuleFunction(params) plist_t *params; { int intVal; double realVal; char charVal; plist_t *curParam; curParam = params; intVal = (int) * (int *)curParam->sym->val; curParam = curParam->next; realVal = (double) * (double *)curParam->sym->val; curParam = curParam->next; charVal = curParam->sym->val; }

Returning data from module functions

It was previously mentioned that data is returned from a module function using a symbol structure. Once the symbol structure is created and filled with the correct value a pointer to the newly created symbol must be stored in a known location. This allows the VM to identify the return value of the function (and to determine if a value was returned at all). To do this, the C function must assign a pointer to the symbol structure to the external variable externReturn. This variable is of type sym_t and should be defined in your module as
	extern	sym_t *externReturn ;
The value assigned to externReturn must be a pointer to a dynamically allocated symbol structure (i.e. created with malloc). It cannot be the address of a statically declared or globale symbol structure (i.e. you cannot use something like
externReturn = & mySymbol; 
As a programmer you may either create a new symbol structure yourself using malloc ( ) and setting the internal fields to the correct value, or you can use the convenience functions provided within the Lite library. Each of the functions listed below creates a new dynamically allocated symbol structure, filled with the correct values, based on the type of symbol and the value desired.

Figure 4. Convenience Routines

createIntSymbol ( intValue )
Creates an Integer symbol with a value of intValue

createCharSymbol ( charValue )
Creates an Char symbol with a value of charValue

createRealSymbol ( doubleValue )
Creates an Real symbol with a value of doubleValue

The passing of data to and from a module function may be best shown with a complete example. One of the functions provided by Lite's standard module is the standard UNIX chmod ( ) function. The functionality of chmod ( ) is provided in the standard C library so the module must provide just the interface between the Lite script and the standard C library function. For those that are not familiar with the chmod ( ) function, it take 2 parameters being the path of a file as a text string and the desired mode of the file as an integer. It returns an integer value as a result code. The interface is defined in mod_std.h (the header file of the standard module) as

{ "chmod", doChmod, 2, { P_TEXT, P_INT, 0 } }

and the actual module function is written as

/************************************************************** ** _doChmod ** ** Purpose : Interface to chmod(2) ** Args : path and mode ** Returns : int status code ** Notes : */ void doChmod(params) plist_t *params; { int mode, result; char *path; path = params->sym->val; mode = (int) * (int *)params->next->sym->val; result = chmod(path,mode); externReturn = createIntSymbol(result); }

As you can see it's a very simple and straight forward process. Writing an interface to external C libraries (which is the main reason you may wish to implement your own modules) is usually a case of writing a function like the above for each of the functions provided by the C library you wish to access. You could of course write your more of your own code inside the module function to implement a new element of functionality not provided by a C library (as is the case with the generic open ( ) function provided by Lite's standard module).

Handling arrays

Until now, all the data being passed to a module function or returned from a module function has been a simple type. There will however be times when you need to pass an array into a function of return an array as a result. Arrays are still represented using the symbol structure previously mentioned alhough the val field of the struture points to a linked list of array structures rather than just a simple piece of data. Each element of the linked list represents an element in the array and contains both a value and a pointer to the structure for the next element in the list.
As with symbol structures, routines have been provided in the lite library to simplify the creation and handling of arrays. The functions are outlined in Figure 5 below. As you can see you never actually have to handle the array structures yourself.

Figure 5. Array Convenience Routines

sym_t * createArray ( )

Creates an empty array

void symSetArrayElement ( array, index, sym )
sym_t * array;
int index;
sym_t * sym;
Sets the value of element index or the array pointed to by array to the value contained in the symbol structure pointed to by sym.

sym_t symGetArrayElement ( array, index )
sym_t * array;
int index;
Returns the symbol for element index of the array pointed to by array. As it is a pointer to a symbol structure that is returned, you can access the actual value in the same way you access the symbol values passed in the parameter list.

int symGetNumArrayElements ( array )
sym_t * array;
Returns the number of elements contained in the array pointed to by array.

sym_t symArrayDup ( array )
sym_t * array;
Creates a new array identical to the array pointed to by the array parameter.

void symFreeSymbol ( sym )
sym_t * sym;
Destroys the symbol pointed to by sym. If the symbol is an array then each element of the array is destroyed and any memory allocated to the array is freed.

Copyright © 1996 Hughes Technologies Pty Ltd.