                          Program Documentation
                 CAITEST  Computer Adaptive Test Engine

Thomas B. Wiggers
528 Cliffview Drive 
Brandon, Ms. 39042

Program Setup

Required Programs:

CAITEST.EXE    Main Program
COMUTL.DOC     Documentation for Print Screen Disable & Enable
NOPSCR.COM     Disables SHIFT+PRTSCR
PSCR.COM       Enables SHIFT+PRTSCR
STARTCAI.BAT   Main Program Loader
WRITECFG.EXE   Creates Configuration File


DataFile Naming Conventions:

"Filename".INI                     Configuration files for the CAITEST engine

"QFIL"+"LD"+"N"+".MCH"             LOW difficulty question file
where "N" is the file number

"QFIL"+"N"+".MCH"                  MODERATE difficulty question file
where "N" is the file number

"QFIL"+"HD"+"N"+".MCH"             HIGH difficulty question file
where "N" is the file number

"QFIL"+"LD"+"N"+".SFL"             LOW difficulty solution file
where "N" is the file number

"QFIL"+"N"+".SFL"                  MODERATE difficulty solution file
where "N" is the file number

"QFIL"+"HD"+"N"+".SFL"             HIGH difficulty solution file
where "N" is the file number

SCORFIL.DAT                   Score File
                              The name of the associated LOG file is also listed.

"Filename".LOG                LOG files  
                              These files provide a complete record of student
                              performance on a given exam.

File Sets  

The data files for the CAITEST program may be set up in either of two ways.

When a competency-based examination is selected, each file set must consist of 3 files.  

"QFIL"+"LD"+"N"+".MCH"             LOW difficulty question file
where "N" is the file number

"QFIL"+"N"+".MCH"                  MODERATE difficulty question file
where "N" is the file number

"QFIL"+"HD"+"N"+".MCH"             HIGH difficulty question file
where "N" is the file number

Questions are read from these files according to the criteria established by the WRITECFG program, and
described below.  If a solution file is to be presented, there must be a solution file for EACH level of
question file.  The solution files are to be named as follows:

"QFIL"+"LD"+"N"+".SFL"             LOW difficulty solution file
where "N" is the file number

"QFIL"+"N"+".SFL"                  MODERATE difficulty solution file
where "N" is the file number

"QFIL"+"HD"+"N"+".SFL"             HIGH difficulty solution file
where "N" is the file number





When a percent correct examination is selected, only one question file is necessary in a file set. It should
be named as follows:

               "QFIL"+"N"+".MCH"   
               where "N" is the file number

If solution files are selected, there MUST be a solution file for each numbered question file.  The solution
file should be named as follows:

               "QFIL"+"N"+".SFL"
               where "N" is the file number


The CAITEST program does not contain a question entry, edit, or print functions.  It was designed to
work with question files created by the TEST GENERATOR program, and all entry and editing of
questions MUST be done with that program.

Follow each step exactly as explained below to set up the CAITEST for operation:

1.   Copy the program files either to a blank, formatted diskette, or to a subdirectory on your hard
     drive.

2.   Change to the diskette or subdirectory by typing "A:" or "B:" followed by pressing the "ENTER"
     key, or by the "CD" command to change directory.

3.   Once the directory containing the programs has been made the current directory, type
     "WRITECFG" and press the "ENTER" key.


The Configuration Program: "WRITECFG.EXE"

Purpose:       Provides an easy way to configure the CAITEST program by asking the configuration
               questions and then creating the initialization file (.INI suffix files).  This file is an ASCII
               text file, and may be edited with any text editor, once the format is understood.

Enter the Subject Area or Topic for the Examination
Enter here what you wish to use for the examination title.  There is a maximum length allowed, and you
will be prompted if you exceed that length, and be allowed to reenter the title.

Enter Examination Access Code:
Determines the filename of the initialization file.  Must be a legal filename of no longer than 8 characters. 
The student will be prompted to enter this access code when they begin the testing program, so that the
proper initialization of the program may be accomplished.  The initialization filename will be the
ACCESS CODE + ".INI", and will be located in the directory from which "WRITECFG" is run.

Do You Wish a Difficulty-based Exam:
A "Y" here indicates that grading of the exam and selection of questions should be based on student
competency.  A "N" indicates that question selection should be from medium-difficulty files only, and
that grade be determined as a percentage of correct answers.

Enter the Number of Questions to be randomly selected..
A "N" answer to the question above indicates that the examination is to be scored by simple percentage
scoring, not competency based.  Only one question is necessary per Qfile in this case, however a better
approach is to create a single Qfile containing all the questions for the examination, say 50.  Answering
"50" to this question means that all 50 questions will be selected and presented to the student for the
exam.  A "40" here means that 40 of the 50 would be randomly pulled.  The program automatically
calculates the total length of the examination. For example, 3 file sets (Qfiles), where 10 questions are to
be selected randomly from each file, gives a total test length of 30 questions.

How many (maximum) high difficulty questions...
Enter here the maximum number of high difficulty questions the student will be asked if they are
successful with questions of moderate difficulty.  If the number of questions in a given file is less than
this number, all questions in the file will be selected and the program will continue.

How many (maximum) medium difficulty questions...
Enter here the maximum number of medium difficulty questions the student will be asked if they are
unsuccessful with questions of high difficulty OR when a new file set is begun.  If the number of
questions in a given file is less than this number, all questions in the file will be selected and the program
will continue.

How many (maximum) low difficulty questions...
Enter here the maximum number of low difficulty questions the student will be asked if they are
unsuccessful with questions of medium difficulty.  If the number of questions in a given file is less than
this number, all questions in the file will be selected and the program will continue.

_________
NOTE: If any of the numbers entered above are LESS than the total number of questions in a file, the
total number of questions asked from a file set will be adjusted to that number.  For example:

Suppose you have entered "3" as the numbers of HIGH, MEDIUM, and LOW difficulty questions to be
selected.  There are 7 questions in QFILLD1.MCH, 22 questions in QFIL1.MCH, and 2 questions in
QFILHD1.MCH.  The total number of questions which will be selected from file set #1 is adjusted
downward to "2".  Since you have indicated that you want to ask 3 medium difficulty questions, the
student will receive 2 medium difficulty questions, and then move to the next file.

Always make sure that there are AT LEAST the number of questions in the question files that you
specify in the above settings to prevent this adjustment from occurring.
_________


What will be the minimum acceptable competency index for successful completion of the exam?
See "Program Logic & Scoring" section at the end of this documentation for an explanation of C.I.
determination.

After how many questions do you want to move to the next file..
See "Program Logic & Scoring" section 8 at the end of this documentation for an explanation of this
option.

Do you want a solution...
Solution files are ASCII text files that provide a general indication to the student as to how the questions
in a file should be solved.  For example, if a file contains molarity questions, the solution file might
contain an explanation of the formulas required for solving the problems.  A solution file is only relevant
when all questions within a file are related.  If solution files are selected, ALL files must have solution
files, even if they simply state that there is no solution file necessary for a particular group of questions. 
Solution files must be named according to the naming conventions stated at the beginning of this
documentation, and must reside in the same directory as the datafiles .

Should the Student be given immediate feedback...
If the answer to this question is "Y" then students will be told whether or not the answer they have
selected is correct or incorrect before the program moves on to the next question.  If the student's answer
is incorrect, they will also be told which was the correct response.  If no entry is made here, the program
defaults to NO FEEDBACK (N).

Enter the MAXIMUM TIME..
At the end of the allotted time the student is presented with a message that they have exceeded the allotted
time, and that they have been unsuccessful in their attempt at the examination.

Enter the PATH or location of the data files:
Type here the EXACT PATHNAME where the datafiles for a given examination reside.  They do not
have to be in the same directory as the CAITEST programs, although it is acceptable for them to be. If the
PATH is a subdirectory, a backslash "\" MUST be included in the pathname as the last character (see
examples).  If the PATH is the Root Directory, the "\" is not necessary.

Example pathnames:
     A:
     A:\CAITEST\
     C:
     C:\CAITEST\LABMATH\

PROGRAM COLORS
Allows the user to customize the colors of the program.  They do not have to be entered or altered.  Press
'ENTER' to accept all the default colors.

To Adjust Color:  Colors #2 and #3 of the colors list are used to adjust background and foreground color,
respectively.  Enter the appropriate number from the following list:

     1         Dark Blue               9         Light Blue
     2         Dark Green             10        Light Green
     3         Dark Cyan              11        Light Cyan
     4         Dark Red               12        Light Red
     5         Purple                 13        Light Purple
     6         Brown                  14        Light Yellow
     7         White                  15        Bright White
     8         Gray
          
The user is next presented with a confirmation screen.  If everything is as needed, type "Y" to store the
values to the configuration file.

4.   Type "STARTCAI" and press 'ENTER' to begin the CAITEST program.

5.   When prompted for the Access Code, enter the code exactly as specified in the WRITECFG
     program above.  The Access Code is NOT case-sensitive.

6.   The remaining activities are prompted on-screen and should be self-explanatory.


Program Logic & Scoring

1.   Low Difficulty questions:          1 point.
     Medium Difficulty questions:       2 points
     High Difficulty questions:         3 points

2.   Competency Index Calculations:

          Sum of Points Received for Correct Answers as Shown Above
     C.I.=     -----------------------------------------------------------------------------------
                    Total Questions Answered

     example:  A student has answered 2 low, 3 medium, and 1 high difficulty question
               correctly out of 15 questions presented.  Point values assigned are as
               follows:       2 low at 1 point each =  2 points
                              3 med at 2 points each=  6 points
                             1 high at 3 points each=  3 points
                                                 -------
                                                   11 points total
          11
     C.I. =  ------ or .733
          15

3.   A student is presented initially with medium difficulty questions.  After the student
     answers all questions, the computer will evaluate the C.I. of the student.  If the CI is
     above the minimum required, the computer will then present high difficulty questions to
     the student from the same file set.  If after evaluation the CI is found to be below the
     minimum required, the computer will present low difficulty questions.

4.   If, after attempting the high difficulty questions, the CI is found to be below the
     acceptable minimum, the student is presented with more medium difficulty questions
     from the same file set.

5.   If, after attempting the high difficulty questions the CI is above the acceptable minimum,
     the computer will move on to the next file set.

6.   If the student has attempted low difficulty questions, and the CI is below the minimum,
     the computer will move to the next file set and select questions of medium difficulty.

7.   If the student answers all of the low difficulty questions which are presented,  the
     computer will present additional medium difficulty questions, same file.

8.   If a student has successfully completed the medium difficulty questions, been presented
     with high difficulty questions and after the high difficulty questions has a competency
     index below the minimum acceptable, the student will receive more moderate level
     questions.  It is conceivable that the student might miss one or more of these moderate
     questions.  If, after receiving the moderate level questions, the CI is below the minimum
     acceptable, the program will present low difficulty questions for the student to attempt. 
     This could continue indefinitely, except for the value for the "Maximum Number of
     Questions per File Set" which has been entered by the instructor.  When the number of
     questions asked the student from a single file set is equal to this number, the program
     moves to the next file set, regardless of the calculated CI.

9.   Questions that are skipped may be returned to at the end of the examination.  When a
     question is skipped, the CI is calculated on the basis of the questions actually answered. 
     There is one possible "worst-case" scenario:

          If all questions on the entire examination are skipped, the CI for each file is
          calculated to be "0".  Therefore, the computer will present only medium and low
          difficulty questions.  If the minimum competency level set by the instructor is
          high enough, it might be possible that the student would not be able to pass the
          examination by answering all questions on the second pass.

          example:  20 questions are presented.  All are skipped on the first pass.  This will
          result in the presentation of  10 medium difficulty and 10 low difficulty questions. 
          (2+2 per file).  The result of the calculation of the CI, assuming all questions were
          answered correctly on the second pass would be:

               10 X 1 point =      10 points
               10 X 2 points=      20 points
                                 ----
                                   30 points
                         30
               C.I.=  ------- or 1.5
                         20
          If the instructor has stipulated 1.8 as the minimum acceptable CI, then the student
          will not receive a passing score.

10.  When the examination is completed, the student receives information concerning the
     number of questions which were missed, a list of the files that were attempted, and the CI
     for each individual file set.  There is also a cumulative CI which represents the calculated
     CI for the entire examination.  

11.  The student scorefile contains the following information:

     Time of Student Log-on
     Student Name
     Name of Examination Log-File
     Number of Questions on Examination
     Percent Score on Examination
     Cumulative CI for Entire Examination, if competency-based
     Length of Time Required to Complete Examination
     CI for Each File Set Attempted (in order, left to right), if competency-based
     CI for Questions That Were Skipped on the First Pass, if competency-based

12.  The Examination Log-File contains the following information:

     TOTQST    Total Number of Questions for Examination
     FILE      Filename of Questions Presented to the Student
     QUES      Question Number in File
     S         Student Response (* indicates that the student skipped the question)
     A         Actual Answer
     
     Tracking Values for Several Scoring Variables for use in troubleshooting:
     FS        Score for current Qfile
     TS        Total Score for all questions presented

     COR       Number of Correct Responses on Examination
     %         Percent Score on Examination
     CI        CI for File Set
     MQ        Maximum Number of questions per file -ACTUAL allowed
     QMX       Maximum Number of questions per file -REQUESTED by configuration
               program
     
Computer Adaptive Test Program
Version 2.0 Upgrade Revisions  02/17/94

Revions to the version 2.0 test driver include the following:

1.   The program now allows the selection of multiple questions from a single QFILE, when
     utilizing a non competency-based examination.  For example a 50 question test may be
     generated from a single 50 question file, instead of 50 QFILES with a single question
     each.  The program tests each question selected to determine if it has been used
     previously, and will not select the same question again within a single examination.

2.   The program allows the instructor to select whether or not students will receive
     immediate feedback when they miss a question on the examination, i.e. "No the correct
     answer should have been 'B'" ,or whether students will not be notified immediately when
     they miss a question.

3.   Students may see the time remaining for the exam, or the percent grade through the prior
     question on the exam (when % based) by selecting the appropriate menu option.

4.   Student logfile output is now formatted and tabular.

5.   Instead of being presented with only a listing of missed questions at the end of the
     examination, students are allowed to review the questions they missed.  The questions are
     presented again and the correct answer is shown.

6.   For 286 and higher processors, the following programs are provided, in which the
     compiled code is optimized for these machines.  These programs execute MUCH faster
     on newer machines:  
     
          CAIT286.EXE    CAITEST.EXE
          CATUT286.EXE   CATUTIL.EXE
          TGEN286.EXE    TESTGEN.EXE
          WRCFG286.EXE   WRITECFG.EXE

7.   Default colors have been optimized for both color and monochrome monitors.  Default
     selections will work equally well on both types of monitors.

 
Any problems or questions should be addressed to: 

Thomas Wiggers
528 Cliffview Drive, 
Brandon, Ms. 39042

tel: day       (601)984-6309 
     night     (601) 992-1373 collect.   

The programs carry a lifetime guarantee, and are NOT copy protected.  They MAY NOT be
transferred or given to another party.  Since the programs are being routinely updated and
improved, any constructive suggestions will be appreciated.



caidoc.wp5
February 21, 1994