













                         A TUTORIAL INTRODUCTION TO GCT

                                  Brian Marick
                              Testing Foundations

                     Documentation for version 1.3 of GCT.
                              Document version 1.2


          This document is a step-by-step guide to the basics of using
          the  Generic Coverage Tool (GCT).  The _G_e_n_e_r_i_c _C_o_v_e_r_a_g_e _T_o_o_l
          (_G_C_T) _U_s_e_r'_s _G_u_i_d_e has the full details.

          Please suggest improvements to this manual, to  other  manu-
          als, or to GCT itself.






























9

9










          Preface

          GCT is free software; you can redistribute it and/or  modify
          it under the terms of the GNU General Public License as pub-
          lished by the Free Software Foundation; either version 1, or
          (at your option) any later version.

          GCT is distributed in the hope that it will be  useful,  but
          WITHOUT  ANY  WARRANTY; without even the implied warranty of
          MERCHANTABILITY or FITNESS FOR A  PARTICULAR  PURPOSE.   See
          the GNU General Public License for more details.



          For more information about  GCT  or  the  other  services  I
          offer, contact:

          Brian Marick
          Testing Foundations
          809 Balboa
          Champaign, Illinois  61820

          (217) 351-7228
          Email: marick@cs.uiuc.edu

          You can join the GCT mailing list by sending  mail  to  gct-
          request@ernie.cs.uiuc.edu.



















          This document is Copyright  8c9  1992  by  Brian  Marick,  who
          hereby  permits  you  to reproduce it for personal use.  You
          may not reproduce it for profit.



9

9










                                   CHAPTER  1


                                  Introduction



          In this tutorial, you'll step through the  basics  of  using
          GCT:

          (1)  Instrumenting a program.

          (2)  Obtaining coverage data.

          (3)  Writing new tests to increase coverage.

          Most of the ideas will be introduced as they're used, but  a
          summary of what GCT does seems in order.

          A perfectly effective  test  suite  would  find  every  bug.
          Since  we don't know how many bugs there are, we can't meas-
          ure how closely a test suite approaches perfection.   Conse-
          quently, we use some approximate measure of test suite qual-
          ity:  since we can't measure what we want, we measure  some-
          thing related.

          With coverage, we estimate test suite quality  by  examining
          how thoroughly the tests exercise the code:

          (1)  Is every if statement taken in both the true and  false
               directions?   (If  it's  never taken in the true direc-
               tion, why do you believe that case works?)

          (2)  Is every case taken?  What about the default case?

          (3)  Is every while loop executed more than once?  Does some
               test  force  the  while  loop  to be skipped? (That is,
               force the loop test to be false  the  first  time  it's
               tested.)

          (4)  Is every loop executed exactly once?  (There are poten-
               tial  initialization  bugs  that can be masked by later
               iterations of the loop.)

          (5)  Do the tests probe off-by-one errors?

          GCT  answers  these  questions;  equivalently,  it  measures
          branch, loop, and relational operator coverage.  These types
          of coverage are most appropriate near the time the  code  is
          written  ("unit  testing").   Other  types  of  coverage are
          better suited for later testing ("system testing"), but they
          aren't described here.  The principles are the same.
9

9                                      -2-          GCT Tutorial







                                      -3-          GCT Tutorial


          One of these principles is that  coverage  numbers  must  be
          used   carefully.    Approximate  measures  of  quality  are
          dangerous; it's easy  to  concentrate  on  the  "making  the
          numbers",  and  somehow  losing  quality in the process.  By
          analogy, consider hiring  a  college  graduate.   A  perfect
          grade  point  average  from  the  best school in the country
          means something quite different than one from  <insert  your
          alma  mater's  traditional  rival  here>.  You wouldn't hire
          based only on GPA; don't judge test quality only  on  cover-
          age.

          In particular, it is usually unwise to generate tests solely
          to  achieve high coverage.  While it is true that good tests
          imply coverage, the converse is not necessarily true.   Many
          of the important bugs in programs are caused by omitted code
          (such as missing error handling) -- measuring how well tests
          exercise  the  code  that's  there  tells  you nothing about
          whether they exercise code that's _n_o_t there.[1]

          Because the focus of this tutorial is the mechanics of  GCT,
          little more will be said on this very important topic.















          ____________________
9             Here are some references about  the  prevalence  of  dif-
          ferent   types  of  errors.  Robert  L.  Glass,  "Persistent
          Software Errors", _T_r_a_n_s_a_c_t_i_o_n_s _o_n _S_o_f_t_w_a_r_e _E_n_g_i_n_e_e_r_i_n_g, vol.
          SE-7,  No.  2,  pp.   162-168, March, 1981. V. Basili and D.
          Weiss, "A Methodology  for  collecting  valid  software  en-
          gineering  data", _I_E_E_E _T_r_a_n_s_a_c_t_i_o_n_s _o_n _S_o_f_t_w_a_r_e _E_n_g_i_n_e_e_r_i_n_g,
          vol. SE-10, pp. 728-738, November, 1984.  Thomas J.  Ostrand
          and   Elaine   J.  Weyuker,   "Collecting  and  Categorizing
          Software Error Data in an Industrial  Environment",  _J_o_u_r_n_a_l
          _o_f _S_y_s_t_e_m_s _a_n_d _S_o_f_t_w_a_r_e, Vol. 4, 1984, pp. 289-300.  Dewayne
          E. Perry and W. Michael Evangelist, "An Empirical  Study  of
          Software Interface Errors", _P_r_o_c_e_e_d_i_n_g_s _o_f _t_h_e _I_n_t_e_r_n_a_t_i_o_n_a_l
          _S_y_m_p_o_s_i_u_m _o_n _N_e_w _D_i_r_e_c_t_i_o_n_s _i_n _C_o_m_p_u_t_i_n_g, IEEE Computer  So-
          ciety, August 1985, Trondheim, Norway, pages 32-38.



9










                                   CHAPTER  2


                           Instrumenting Your Program




          If you're impatient, you need only read the first  two  sec-
          tions.

          _1.  _U_s_i_n_g _l_c

          The first step is to copy  the  source  from  the  GCT  _d_e_m_o
          directory  to your own directory.  (The person who installed
          GCT can tell you where the demo files are.)  It is generally
          a good idea to make a copy of your source before instrument-
          ing it.  The problem is not so much GCT - which  is  careful
          not  to  destroy your source - as you.  It's not unknown for
          people to get confused, remove the wrong files, and discover
          that they've wiped out the original source.

          Take a look at the directory.  You'll see many files.   Some
          of the more important are:

          lc.c
               This is the source for the program  you'll  instrument.
               It's  a  program that counts the lines of code and com-
               ments in C programs.

          lc.1
               This is  the  manpage,  suitable  for  formatting  with
               nroff.   _l_c._1_t  is  more useful when using troff output
               for printing.

          Makefile
               This is the makefile for the program.

          run-suite
               This is a test suite driver for lc.  The test  data  (C
               programs)  are  in the files named _l_c_1, _l_c_2, and so on.
               This  test  suite  was   developed   using   techniques
               described  elsewhere[1],  but  a few tests were removed
               for this tutorial.  run-suite is actually a  poor  test
               suite  driver.  It doesn't compare the actual output of
               lc to the expected output.  It  just  sends  output  to
          ____________________
9             [1] Brian Marick, "Experience with the Cost of Test Suite
          Coverage   Measures",  _P_a_c_i_f_i_c  _N_o_r_t_h_w_e_s_t  _S_o_f_t_w_a_r_e  _Q_u_a_l_i_t_y
          _C_o_n_f_e_r_e_n_c_e, October, 1991.  Also available as  a  compressed
          postscript file in cs.uiuc.edu:/pub/testing/experience.ps.Z.



9                                      -4-          GCT Tutorial







                                      -5-          GCT Tutorial


               /dev/null, so it's useless for actually  finding  bugs.
               It's fine for this tutorial, though.

          Type

                  % make
                  % lc lc.c

          to see how lc works.  You should see this output:

          % lc lc.c
              Pure     Pure Both         Total   TotalTotal
              Code     Comment  Cod&Com  Blank   CodeCommentLinesPages

          lc.c    311  437  62  117 373  499 927 28

          "Pure Code" means lines of code with no comments, "Pure Com-
          ment"  means  lines with only comments; "Both Cod&Com" means
          lines containing both.  "Blank" lines contain  neither  code
          nor comments.

          Now type

              % make clean

          to delete the executables; we'll be building new ones.

          _2.  _I_f _Y_o_u'_r_e _I_m_p_a_t_i_e_n_t

          If you don't care about the step-by-step process of  instru-
          mentation, just type

              % make -f Makefile.gct gct

          and proceed to the next chapter.  If you  want  to  use  GCT
          regularly, you'll want to read this chapter eventually.

          _3.  _D_e_c_i_d_i_n_g _W_h_a_t _t_o _M_e_a_s_u_r_e

          The first step is to tell GCT what to measure.   That infor-
          mation is kept in the file _g_c_t-_c_t_r_l, which contains:

              (coverage branch multi loop relational)
              lc.c

          The first line tells GCT we  want  branch,  multi-condition,
          loop, and relational operator coverage.  These are the types
          of coverage I recommend for detailed testing of  code  (unit
          and subsystem testing).  We'll see what these coverages mean
          when we look at GCT's reports.  Other types of coverage  are
          possible; see the User's Manual.

9

9







                                      -6-          GCT Tutorial


          The second line tells GCT to measure coverage only for _l_c._c.

          _4.  _I_n_s_t_r_u_m_e_n_t_i_n_g _t_h_e _P_r_o_g_r_a_m

          The next step is to _i_n_s_t_r_u_m_e_n_t the program so that  it  col-
          lects data about its execution as it runs.

          To do this, we use a modifed version of the program's origi-
          nal  makefile.   For  most makefiles, the changes are minor.
          The modified makefile is  the  file  _M_a_k_e_f_i_l_e._g_c_t.   Examine
          that  file, and you'll see that there's a target called gct.
          But, rather than using make, we'll do the instrumentation by
          hand.   Each  following  section  explains what a particular
          line of the gct target does.  Type the first command in each
          section to do what make would have done.

          _4._1.  _I_n_s_t_r_u_m_e_n_t_i_n_g

          % gct-init[2]

          This program copies a number of files into  your  directory.
          You  should  never  need  to  look at the internals of these
          files, but here's what they contain:

          gct-ps-defs.h
               A makefile usually invokes the  C  compiler  separately
               for  each source file.  GCT is invoked in the same way.
               This file stores information about where  the  previous
               invocation left off.

          gct-ps-defs.c
               For speed, coverage data is stored in the memory of the
               instrumented  program.   This file defines that in-core
               log; it is linked into the final executable.

          gct-write.c
               To be useful, the log must outlive the execution of the
               program  -  it  must  be written to disk.  This program
               contains a routine that writes this log.  It also  con-
               tains a routine that reads in a previous version of the
               log, so that the log accumulates over several tests.

          gct-defs.h
               This file contains various C macros used to  manipulate
               the log.


          ____________________
9             [2] If the shell cannot find gct-init, the directory that
          GCT  was  installed  in  must  not  be  in  your search path
          ($PATH).  Ask the person who installed GCT where it is.



9







                                      -7-          GCT Tutorial


          _4._2.  _I_n_s_t_r_u_m_e_n_t_a_t_i_o_n

          In this step, we tell the makefile to compile the program in
          the normal way, but to use GCT as the "compiler".

          % make all "CC=gct -DTESTING"

          You should see something like this:

          gct -DTESTING   -c  lc.c
          gct -DTESTING -o lc lc.o

          make gives _l_c._c to GCT.  The result is an instrumented file,
          also  called _l_c._c.  The makefile then invokes GCT to produce
          an executable.  Since GCT sees no C source, it does nothing.
          The actual creation of the executable comes later.

          The TESTING define is used to make the instrumented  program
          read and write the log file.  GCT stores the original source
          in the file _g_c_t__b_a_c_k_u_p/_l_c._c.   If  you  examine  that  file,
          you'll see two uses of the define.

          (1)  The first is in this code:

               #ifdef TESTING

               myexit(status)
               {
                 gct_writelog("LOG");
                 exit(status);
               }
               #define exit myexit

               #endif

               This is a standard trick to ensure the log  is  written
               out (into the file _L_O_G) any time the program exits.

          (2)  The second is in _m_a_i_n() where

                   gct_readlog("LOG");

               is used to read the log (if it exists) when the program
               starts.

          If you're curious, look inside the instrumented  source,  in
          _l_c._c.   The  code  is  difficult  to read, because it's been
          preprocessed and because GCT doesn't  preserve  indentation.
          You  can  recognize instrumentation by the use of the string
          "_G".  For a statement like

              if (function(a))
9

9







                                      -8-          GCT Tutorial


          GCT replaces it with (in effect):

              if ( (_t_e_m_p_o_r_a_r_y=(_f_u_n_c_t_i_o_n(_a)), __G_2(_3_4, _t_e_m_p_o_r_a_r_y), _t_e_m_p_o_r_a_r_y))

          The value of the if's test is first  saved  in  a  temporary
          variable.   The  _G2  macro records whether the if was taken
          true or false. The branch is the 34th log  entry.   Finally,
          the  temporary  variable  is  used  as the value of the if's
          test, so that the program's execution is unaffected  by  the
          instrumentation.  _G  is  a macro that expands out to only a
          few instructions, so the instrumented program  is  not  much
          slower  than  the original.  (The amount of slowdown varies,
          of course, depending on the program and how much  instrumen-
          tation you turn on.)

          _4._3.  _C_o_m_p_i_l_i_n_g _S_u_p_p_o_r_t _C_o_d_e

          % cc -c gct-ps-defs.c
          % cc -c gct-write.c

          This compiles the definitions of _g_c_t__w_r_i_t_e_l_o_g,  _g_c_t__r_e_a_d_l_o_g,
          and the in-core table that holds the log.

          _4._4.  _M_a_k_i_n_g _t_h_e _E_x_e_c_u_t_a_b_l_e

          % make all "OBJ=lc.o gct-ps-defs.o gct-write.o"

          Having instrumented the program, we now need to compile  it.
          When  making  the  final executables, we need to link in the
          files we  just  compiled,  since  the  instrumented  program
          refers to their contents.  You should see:

          cc   -c  lc.c
          cc -o lc lc.o gct-ps-defs.o gct-write.o


          You now have an instrumented lc.   You'll  be  able  to  run
          tests  and get information about coverage.  This information
          will refer to the line numbers in  the  original  (uninstru-
          mented)  file.   You  certainly  do  not want to look at the
          instrumented file to figure out what the  coverage  informa-
          tion is telling you.

          _4._5.  _R_e_s_t_o_r_i_n_g _t_h_e _O_r_i_g_i_n_a_l _S_o_u_r_c_e

          % grestore

          restores the  original  source.   You  no  longer  need  the
          instrumented version.



9

9










                                   CHAPTER  3


                      A First Pass Over the Coverage Data




          To run the test suite, type

              % run-suite

          It should take about ten seconds.  The result is  a  logfile
          named _L_O_G.

          To get a first idea of how thorough the test suite is, type

              % gsummary LOG


          You should see this:

          BINARY BRANCH INSTRUMENTATION (76 conditions total)
          3 ( 3.95%) not satisfied.
          73 (96.05%) fully satisfied.

          SWITCH INSTRUMENTATION (14 conditions total)
          0 ( 0.00%) not satisfied.
          14 (100.00%) fully satisfied.

          LOOP INSTRUMENTATION (24 conditions total)
          4 (16.67%) not satisfied.
          20 (83.33%) fully satisfied.

          MULTIPLE CONDITION INSTRUMENTATION (38 conditions total)
          2 ( 5.26%) not satisfied.
          36 (94.74%) fully satisfied.

          OPERATOR INSTRUMENTATION (27 conditions total)
          1 ( 3.70%) not satisfied.
          26 (96.30%) fully satisfied.

          SUMMARY OF ALL CONDITION TYPES (179 total)
          10 ( 5.59%) not satisfied.
          169 (94.41%) fully satisfied.

          I'll explain the first two  sections;  the  other  types  of
          instrumentation are covered in the next chapter.

          Every branching statement (if, ?, or  loop  test)  generates
          two  _c_o_v_e_r_a_g_e  _c_o_n_d_i_t_i_o_n_s:  one that the true case be taken,
          and one that the false case be taken.  In lc, there  are  38
          branching  statements  generating  76 conditions.  Of these,


                                      -9-          GCT Tutorial







                                      -10-         GCT Tutorial


          three have not been satisfied;  three cases (true or  false)
          remain to be exercised.

          lc also contains some switch statements with a total  of  14
          cases  (including defaults).  All the cases have been taken.
          (Each case label is a separate coverage condition.)

          The summary coverage is high.  That's not surprising,  since
          this  test  suite was derived by removing a few tests from a
          test suite that reached 100% coverage.  However,  a  summary
          coverage  greater  than  90%  is  typical  of  thorough test
          suites.[1]

          Simple numbers  like  gsummary's  are  a  blunt  instrument.
          Don't  use  GCT to reduce your testing to a few numbers, but
          rather to expand your understanding of what your test  suite
          really does.  greport, described in the next chapter, is the
          tool you'll use.





























          ____________________
9             [1] In addition to the _E_x_p_e_r_i_e_n_c_e  paper  mentioned  ear-
          lier,  I can send an unpublished case study: the coverage of
          GCT's own test suite.



9










                                   CHAPTER  4


                               Writing New Tests




          _1.  _A_n_a_l_y_s_i_n_g _m_i_s_s_e_d _c_o_v_e_r_a_g_e

          Type

          % greport LOG

          You should see:

          "lc.c", line 255: operator > might be >=. (L==R)
          "lc.c", line 255: condition 1 (argc, 1) was taken TRUE 42, FALSE 0 times.
          "lc.c", line 257: condition 1 (<...>[...], 1) was taken TRUE 0, FALSE 14 times.
          "lc.c", line 271: if was taken TRUE 0, FALSE 28 times.
          "lc.c", line 280: loop zero times: 0, one time: 18, many times: 10.
          "lc.c", line 290: if was taken TRUE 0, FALSE 40 times.
          "lc.c", line 425: if was taken TRUE 37, FALSE 0 times.
          "lc.c", line 860: loop zero times: 0, one time: 5, many times: 19.
          "lc.c", line 872: loop zero times: 0, one time: 0, many times: 6.

          You'll notice that this looks a lot like error output from a
          C compiler.  This is intentional, for two reasons:

          (1)  Psychological.  As with a compiler's output, you  exam-
               ine  the  errors (in this case, test deficiencies), fix
               them, redo the compile  (rerun  the  test  suite),  and
               repeat  the process until you get no more output.  Just
               as with a compiler, the messages  you  see  may  prompt
               large  changes,  larger  than  is strictly necessary to
               make them go away.

          (2)  Practical.  These error messages  are  compatible  with
               the  error(1)  program  (on BSD UNIX) and the GNU Emacs
               next-error function.  (A version of next-error tailored
               to  GCT is provided with GCT.  If you're an Emacs user,
               you may want to look at Appendix A now.)

          Let's examine the output line by line.  The first two  lines
          of  greport  output  refer  to line 255, which is the option
          processing loop. The line in question is highlighted  below,
          but I recommend you edit _l_c._c; the discussion will be easier
          to follow if you don't have to constantly flip pages of this
          tutorial.

              while (--argc > 0 && (**++argv == '-'))
              {
              if (  (*argv)[1] == LCURL || (*argv)[1] == RCURL)


                                      -11-         GCT Tutorial







                                      -12-         GCT Tutorial


              {
                  white_bracket = TRUE;
              }
              else if (sscanf (*argv + 1, "%d", &page_size) == FALSE)
              {
                  fprintf (stderr, "lc: Bad page size argument: %s\n", *argv);
                  exit (BAD_FLAG);           /* Forgive me, Edsger */
              }
              }

          The first greport line,

          "lc.c", line 255: operator > might be >=. (L==R)

          is from _r_e_l_a_t_i_o_n_a_l coverage, which  checks  for  tests  that
          probe common misuses of relational operators.  In this case,
          we're worried that the programmer made the common mistake of
          using  >  instead of the correct >=.  Suppose that happened.
          We want to write the test that will have the best chance  of
          revealing  the  bug;  that is, we want a test where what the
          correct program would do is as different  as  possible  from
          what the program we have does.

          If --argc is exactly  0,  the  correct  program  (--argc>=0)
          would enter the body of the while loop, while the buggy pro-
          gram (--argc>0) won't.  For all other values  of  argc,  the
          correct and buggy program will take the same path.  It seems
          that argc==0, the boundary condition, is the  best  test  to
          find  this  particular bug; GCT is telling us we never tried
          it.

          To force this case, the call to lc must have  no  non-option
          arguments.  It must look like one of these:

          % lc < INPUT
          % lc -} < INPUT
          % lc -34 < INPUT

          If you look at the run-suite file, you'll see that there are
          no  such  tests:  lc is never tested when it takes its input
          from standard input.  This is a major omission.  Rather than
          writing  a  test  like this right away, we should write down
          "input from  standard  input"  as  a  _t_e_s_t  _c_o_n_d_i_t_i_o_n  in  a
          separate  list  of test conditions.  We may later be able to
          combine several of these test conditions into a single  test
          case, which saves us effort.[1]

          ____________________
9             [1] It's also likely to result in a better test  --  more
          complicated  test  cases  are better (up to a point) because
          they are more likely to catch bugs by chance.



9







                                      -13-         GCT Tutorial


          The next line of greport output is from the same line of the
          program.

          "lc.c", line 255: condition 1 (argc, 1) was taken TRUE 42, FALSE 0 times.

          This is from _m_u_l_t_i_c_o_n_d_i_t_i_o_n coverage, which checks that  all
          parts  of  a  logical expression are used.  Line 255's while
          loop test has two components:

             --argc > 0

          and

             **++argv == '-'

          The first of them was  always  true  in  every  test,  never
          false.   The  second component isn't mentioned, because it's
          evaluated to both true and false.  If you want  to  see  how
          many  times  it's  evaluated to each value, use greport -all
          (see the manpage).

          This condition tells us the same thing that we already knew.
          To  get  argc  equal  to  zero,  there can be only arguments
          beginning with a dash -- that is, lc  must  take  its  input
          from standard input.

          The next line,

          "lc.c", line 257: condition 1 (<...>[...], 1) was taken TRUE 0, FALSE 14 times.

          corresponds to this code:

              while (--argc > 0 && (**++argv == '-'))
              {
              if (  (*argv)[1] == LCURL || (*argv)[1] == RCURL)
              {
                  white_bracket = TRUE;
              }
              else if (sscanf (*argv + 1, "%d", &page_size) == FALSE)
              {
                  fprintf (stderr, "lc: Bad page size argument: %s\n", *argv);
                  exit (BAD_FLAG);           /* Forgive me, Edsger */
              }
              }

          It is also an example of multicondition  coverage,  but  the
          message  looks  peculiar.   What  does (<...>[...], 1) mean?
          It's there in case "condition 1" is not enough to locate the
          condition  greport  refers to.  (In this case, the condition
          is (*argv)[1] == LCURL.)

          Conditions are numbered from left to right.  But in a deeply
          nested expression, like (A && (B || C) || D), it can be hard










                                      -14-         GCT Tutorial


          to count conditions accurately.   The  parenthetical  remark
          helps  you  find  them.  The first component is the leftmost
          operand of the condition.  In this  case,  it's  (*argv[1]).
          Greport always abbreviates arrays as _a_r_r_a_y_n_a_m_e[...], because
          the expression in the array is often complex and would  make
          the  line too long.  When _a_r_r_a_y_n_a_m_e is more complicated than
          a simple identifier, GCT abbreviates it as <...>,  again  to
          save space.

          These abbreviation rules are probably a  bad  idea,  but  in
          practice it's rarely difficult to figure out what subexpres-
          sion is  meant.   As  an  additional  help,  the  number  in
          parenthesizes  is  the  nesting  depth of the subexpression,
          starting with 1.  (It's always deeply parenthesized  expres-
          sions that cause trouble.)

          What this line tells us is that  *argv[1]  is  never  LCURL.
          That  is,  we've  never  given the program the -{ option.  A
          quick check of run-suite confirms this.   We'll  write  that
          down in our test condition list.

          The next line,

          "lc.c", line 271: if was taken TRUE 0, FALSE 28 times.

          tells us that this if has  never  been  taken  in  the  true
          direction.

              if (argc == 0)
              {
                  tally_file (stdin, &file_tally);
                  show_header ();
                  show_tally ("", &file_tally);
              }

          We already knew that:  the program never reads from standard
          input.

          The next line,

          "lc.c", line 280: loop zero times: 0, one time: 18, many times: 10.











9

9







                                      -15-         GCT Tutorial


          is the first example of _l_o_o_p coverage.  This particular loop
          traverses all of lc's non-option arguments.

              for (index = 1; index <= argc; index++)
              {
                  if ((fp = fopen (*argv, "r")) == NULL)
                  {
                  status = FILE_NOT_FOUND;
                  fprintf (stderr, "lc: can't open %s\n", *argv);
                  }
                  else
                  {
                  tally_file (fp, &file_tally);
                  if (fclose (fp) == EOF)
                      panic (PANIC, "Fclose error.");
                  show_tally (*argv, &file_tally);
                  if (argc > 1)
                      make_total (&total_tally, &file_tally);
                  }
                  argv++;
              }


          A greport line for a loop tells about three types of traver-
          sals:

          (1)  ones where the loop test failed on the  first  try,  so
               the loop was never entered.

          (2)  ones where the loop test failed on the second  try,  so
               the loop body was executed exactly once.

          (3)  ones where the loop was traversed more than once.

          Some types of bugs are only detected by one of  these  cases
          (though the single-traversal case is probably the least use-
          ful).

          In this particular case, we know  that  argc>0  (because  we
          just looked at the if statement that handled the zero case).
          So it is impossible to traverse  the  while  loop  0  times.
          This  is an example of an _i_n_f_e_a_s_i_b_l_e _t_e_s_t _c_o_n_d_i_t_i_o_n.  We can
          ignore it.

          Of the two feasible cases, we see that lc was given a single
          argument 18 times and more than one argument 10 times.

          We move on to:

          "lc.c", line 290: if was taken TRUE 0, FALSE 40 times.

          which corresponds to this line:
9

9







                                      -16-         GCT Tutorial


                          if (fclose (fp) == EOF)
                              panic (PANIC, "Fclose error.");

          Evidently, the programmer thinks an EOF return from fclose()
          is  impossible  (but  is checking anyway, just in case).  Of
          course, what a programmer thinks and what is true may be two
          different  things.   We  as testers would want to think hard
          about how to generate EOF returns.   (Error  handling  is  a
          fertile  source  of  serious bugs caused by mistaken assump-
          tions.)  Let's assume that we'll fail, that this is  another
          example of an infeasible test condition.

          This line,

          "lc.c", line 425: if was taken TRUE 37, FALSE 0 times.

          is another sanity check:

                 if (line_info.blank)
                     (file_tally->blank)++;
                 else
                     panic (PANIC, "Non-empty line contains nothing.");

          It's never been taken in the false direction, and  the  pro-
          grammer  believes  it  can't  be.  In thorough testing, we'd
          think hard about this; for this tutorial, we'll move on.

          The next line,

          "lc.c", line 860: loop zero times: 0, one time: 5, many times: 19.

          is another impossible loop case (these are not uncommon):

              for (index = 0; index < 4; index++)
              {
                  c = PULL_OFF(fp);
                  if (c != QUOTE)            /* store it. */
                  chars[index] = c;
                  else              /* found what we want. */
                  break;
              }

          It is impossible for this loop not to be entered; hence  the
          "zero  times"  case  in loop coverage can never have a value
          other than zero.  (The loop can  be  entered  exactly  once,
          though,  if  the  break statement is executed the first time
          through.  This happened 5 times.)

          The final line of greport output is this:

          "lc.c", line 872: loop zero times: 0, one time: 0, many times: 6.

          which corresponds to this code:










                                      -17-         GCT Tutorial


              for (index = 3; index >= 0; index--)
              {
                  c = chars[index];    /* because chars[array] is too big to expand in PUSH_BACK. */
                  PUSH_BACK(c, fp);
              }

          This is another impossible while  loop.   It  always  counts
          down  from  3  to  0,  so  it can never be taken zero or one
          times.

          We're finished.  We generated only a short list of things to
          test.   This  is  as it should be, given a thorough starting
          test suite.  We wasted some time looking at impossible  con-
          ditions, but not very much.

          _2.  _T_h_e _n_e_w _t_e_s_t_s

          We have two new test conditions:

          1. input from standard input.
          2. Use the -{ option.

          This is easy to do with a single test, which you should just
          type to the shell:

          % echo "{" | lc -\{

          (The backslash in front of the { is  to  prevent  the  shell
          from interpreting it specially.) This is not a very thorough
          test of the -{ option, but it will do for this tutorial.  Is
          the output correct?

          Now type

          % gsummary LOG

          You'll see that  total  coverage  has  gone  up  to  96.65%.
          Greport shows:

          "lc.c", line 280: loop zero times: 0, one time: 18, many times: 10.
          "lc.c", line 290: if was taken TRUE 0, FALSE 40 times.
          "lc.c", line 425: if was taken TRUE 38, FALSE 0 times.
          "lc.c", line 860: loop zero times: 0, one time: 5, many times: 19.
          "lc.c", line 872: loop zero times: 0, one time: 0, many times: 6.

          We know that all of these are infeasible.

          _3.  _S_u_p_p_r_e_s_s_i_n_g _i_n_f_e_a_s_i_b_l_e _c_o_v_e_r_a_g_e

          It's annoying that we can't make the infeasible test  condi-
          tions  go  away.   Worse, if we're testing a large program -
          one where it may take us  several  tries  to  eliminate  all
          feasible  test  conditions  -  we  don't  want to waste time










                                      -18-         GCT Tutorial


          looking at infeasible conditions every time we  run  greport
          after a change to the test suite.

          The gedit program can help.  To use it, type

          % greport -edit LOG > edit.g

          Edit _e_d_i_t._g.  You'll see this:

          "lc.c", line 280: [25: 0 0 18 10] loop zero times: 0, one time: 18, many times: 10.
          "lc.c", line 290: [34: 0 40] if was taken TRUE 0, FALSE 40 times.
          "lc.c", line 425: [75: 38 0] if was taken TRUE 38, FALSE 0 times.
          "lc.c", line 860: [167: 0 0 5 19] loop zero times: 0, one time: 5, many times: 19.
          "lc.c", line 872: [180: 0 0 0 6] loop zero times: 0, one time: 0, many times: 6.

          The numbers in brackets are the raw  versions  of  what  the
          text says.  In the first case (line 280), the first 0 is the
          number of times the loop was skipped.  The next two  numbers
          (0  and  18)  add  up  to  the  number of times the loop was
          entered exactly once.  (The reason why there are two numbers
          is unimportant - it's due to the implementation of loop cov-
          erage.)  The final number (10) is the number  of  times  the
          loop  was  traversed  many  times.  In the case of branches,
          like line 290, the first  number  is  the  true  count,  the
          second the false count.

          Replace the zeros that are impossible with either "s",  "S",
          "0s", or "0S":[2]

          "lc.c", line 280: [25: 0S 0 18 10] loop zero times: 0, one time: 18, many times: 10.
          "lc.c", line 290: [34: S 40] if was taken TRUE 0, FALSE 40 times.
          "lc.c", line 425: [75: 38 S] if was taken TRUE 38, FALSE 0 times.
          "lc.c", line 860: [167: S 0 5 19] loop zero times: 0, one time: 5, many times: 19.
          "lc.c", line 872: [180: S 0S 0 6] loop zero times: 0, one time: 0, many times: 6.

          Notice that you only had to replace one of the zeroes in the
          "single traversal" case of the last loop.

          Now type

          % gedit edit.g
          % gsummary LOG






          ____________________
9             [2] If you're using Emacs "gedit-mode", SPC will helpful-
          ly position you at the first zero in the next line.



9







                                      -19-         GCT Tutorial


          You'll see:

          BINARY BRANCH INSTRUMENTATION (76 conditions total)
          0 ( 0.00%) not satisfied.
          76 (100.00%) fully satisfied. [2 ( 2.63%) suppressed]

          SWITCH INSTRUMENTATION (14 conditions total)
          0 ( 0.00%) not satisfied.
          14 (100.00%) fully satisfied.

          LOOP INSTRUMENTATION (24 conditions total)
          0 ( 0.00%) not satisfied.
          24 (100.00%) fully satisfied. [4 (16.67%) suppressed]

          MULTIPLE CONDITION INSTRUMENTATION (38 conditions total)
          0 ( 0.00%) not satisfied.
          38 (100.00%) fully satisfied.

          OPERATOR INSTRUMENTATION (27 conditions total)
          0 ( 0.00%) not satisfied.
          27 (100.00%) fully satisfied.

          SUMMARY OF ALL CONDITION TYPES (179 total)
          0 ( 0.00%) not satisfied.
          179 (100.00%) fully satisfied. [6 ( 3.35%) suppressed]

          If you use greport, it will show you nothing.   You're  done
          using coverage on your test suite.
























9

9










                                  APPENDIX  A


                                 Using gedit.el




          gedit.el provides a version of the Emacs  next-error  inter-
          face  that  works better for greport output.  The source for
          gedit.el is, by default, in the GCT library directory;  who-
          ever  installed  GCT  may have put it with other local emacs
          files.

          The most convenient way to use gedit.el  is  to  always  put
          greport  output  into  files  ending in .g and put something
          like

          (setq auto-mode-alist (cons (cons "\\.g$" 'gedit-mode) auto-mode-alist))

          into your ._e_m_a_c_s file.  That done, whenever you edit such  a
          file, you'll use gedit-mode.

          Either load gedit.el manually, or put something like

          (autoload 'gedit-mode (expand-file-name "~/gct/src/gedit.el"))

          in your ._e_m_a_c_s file.

          When in gedit-mode, each time you hit  SPC,  a  new  greport
          line  will  move  to the top of one window.  The appropriate
          source file will appear in the other window.  An arrow  (=>)
          will  point  to the line in question. (The arrow is not part
          of the file.)

          Like next-error, there's no convenient way to back  up.   If
          you hit e, you'll restart from the beginning.















9

9                                      -20-         GCT Tutorial



