













                         A TUTORIAL INTRODUCTION TO GCT

                                  Brian Marick
                              Testing Foundations

                     Documentation for version 1.3 of GCT.
                              Document version 1.2


          This document is a step-by-step guide to the basics of using
          the  Generic Coverage Tool (GCT).  The _G_e_n_e_r_i_c _C_o_v_e_r_a_g_e _T_o_o_l
          (_G_C_T) _U_s_e_r'_s _G_u_i_d_e has the full details.

          Please suggest improvements to this manual, to  other  manu-
          als, or to GCT itself.






























9

9










          Preface

          GCT is free software; you can redistribute it and/or  modify
          it under the terms of the GNU General Public License as pub-
          lished by the Free Software Foundation; either version 1, or
          (at your option) any later version.

          GCT is distributed in the hope that it will be  useful,  but
          WITHOUT  ANY  WARRANTY; without even the implied warranty of
          MERCHANTABILITY or FITNESS FOR A  PARTICULAR  PURPOSE.   See
          the GNU General Public License for more details.



          For more information about  GCT  or  the  other  services  I
          offer, contact:

          Brian Marick
          Testing Foundations
          809 Balboa
          Champaign, Illinois  61820

          (217) 351-7228
          Email: marick@cs.uiuc.edu

          You can join the GCT mailing list by sending  mail  to  gct-
          request@ernie.cs.uiuc.edu.



















          This document is Copyright  8c9  1992  by  Brian  Marick,  who
          hereby  permits  you  to reproduce it for personal use.  You
          may not reproduce it for profit.



9

9










                                   CHAPTER  1


                                  Introduction



          In this tutorial, you'll step through the  basics  of  using
          GCT:

          (1)  Instrumenting a program.

          (2)  Obtaining coverage data.

          (3)  Writing new tests to increase coverage.

          Most of the ideas will be introduced as they're used, but  a
          summary of what GCT does seems in order.

          A perfectly effective  test  suite  would  find  every  bug.
          Since  we don't know how many bugs there are, we can't meas-
          ure how closely a test suite approaches perfection.   Conse-
          quently, we use some approximate measure of test suite qual-
          ity:  since we can't measure what we want, we measure  some-
          thing related.

          With coverage, we estimate test suite quality  by  examining
          how thoroughly the tests exercise the code:

          (1)  Is every if statement taken in both the true and  false
               directions?   (If  it's  never taken in the true direc-
               tion, why do you believe that case works?)

          (2)  Is every case taken?  What about the default case?

          (3)  Is every while loop executed more than once?  Does some
               test  force  the  while  loop  to be skipped? (That is,
               force the loop test to be false  the  first  time  it's
               tested.)

          (4)  Is every loop executed exactly once?  (There are poten-
               tial  initialization  bugs  that can be masked by later
               iterations of the loop.)

          (5)  Do the tests probe off-by-one errors?

          GCT  answers  these  questions;  equivalently,  it  measures
          branch, loop, and relational operator coverage.  These types
          of coverage are most appropriate near the time the  code  is
          written  ("unit  testing").   Other  types  of  coverage are
          better suited for later testing ("system testing"), but they
          aren't described here.  The principles are the same.
9

9                                      -2-          GCT Tutorial







                                      -3-          GCT Tutorial


          One of these principles is that  coverage  numbers  must  be
          used   carefully.    Approximate  measures  of  quality  are
          dangerous; it's easy  to  concentrate  on  the  "making  the
          numbers",  and  somehow  losing  quality in the process.  By
          analogy, consider hiring  a  college  graduate.   A  perfect
          grade  point  average  from  the  best school in the country
          means something quite different than one from  <insert  your
          alma  mater's  traditional  rival  here>.  You wouldn't hire
          based only on GPA; don't judge test quality only  on  cover-
          age.

          In particular, it is usually unwise to generate tests solely
          to  achieve high coverage.  While it is true that good tests
          imply coverage, the converse is not necessarily true.   Many
          of the important bugs in programs are caused by omitted code
          (such as missing error handling) -- measuring how well tests
          exercise  the  code  that's  there  tells  you nothing about
          whether they exercise code that's _n_o_t there.[1]

          Because the focus of this tutorial is the mechanics of  GCT,
          little more will be said on this very important topic.















          ____________________
9             Here are some references about  the  prevalence  of  dif-
          ferent   types  of  errors.  Robert  L.  Glass,  "Persistent
          Software Errors", _T_r_a_n_s_a_c_t_i_o_n_s _o_n _S_o_f_t_w_a_r_e _E_n_g_i_n_e_e_r_i_n_g, vol.
          SE-7,  No.  2,  pp.   162-168, March, 1981. V. Basili and D.
          Weiss, "A Methodology  for  collecting  valid  software  en-
          gineering  data", _I_E_E_E _T_r_a_n_s_a_c_t_i_o_n_s _o_n _S_o_f_t_w_a_r_e _E_n_g_i_n_e_e_r_i_n_g,
          vol. SE-10, pp. 728-738, November, 1984.  Thomas J.  Ostrand
          and   Elaine   J.  Weyuker,   "Collecting  and  Categorizing
          Software Error Data in an Industrial  Environment",  _J_o_u_r_n_a_l
          _o_f _S_y_s_t_e_m_s _a_n_d _S_o_f_t_w_a_r_e, Vol. 4, 1984, pp. 289-300.  Dewayne
          E. Perry and W. Michael Evangelist, "An Empirical  Study  of
          Software Interface Errors", _P_r_o_c_e_e_d_i_n_g_s _o_f _t_h_e _I_n_t_e_r_n_a_t_i_o_n_a_l
          _S_y_m_p_o_s_i_u_m _o_n _N_e_w _D_i_r_e_c_t_i_o_n_s _i_n _C_o_m_p_u_t_i_n_g, IEEE Computer  So-
          ciety, August 1985, Trondheim, Norway, pages 32-38.



9










                                   CHAPTER  2


                           Instrumenting Your Program




          If you're impatient, you need only read the first  two  sec-
          tions.

          _1.  _U_s_i_n_g _l_c

          The first step is to copy  the  source  from  the  GCT  _d_e_m_o
          directory  to your own directory.  (The person who installed
          GCT can tell you where the demo files are.)  It is generally
          a good idea to make a copy of your source before instrument-
          ing it.  The problem is not so much GCT - which  is  careful
          not  to  destroy your source - as you.  It's not unknown for
          people to get confused, remove the wrong files, and discover
          that they've wiped out the original source.

          Take a look at the directory.  You'll see many files.   Some
          of the more important are:

          lc.c
               This is the source for the program  you'll  instrument.
               It's  a  program that counts the lines of code and com-
               ments in C programs.

          lc.1
               This is  the  manpage,  suitable  for  formatting  with
               nroff.   _l_c._1_t  is  more useful when using troff output
               for printing.

          Makefile
               This is the makefile for the program.

          run-suite
               This is a test suite driver for lc.  The test  data  (C
               programs)  are  in the files named _l_c_1, _l_c_2, and so on.
               This  test  suite  was   developed   using   techniques
               described  elsewhere[1],  but  a few tests were removed
               for this tutorial.  run-suite is actually a  poor  test
               suite  driver.  It doesn't compare the actual output of
               lc to the expected output.  It  just  sends  output  to
          ____________________
9             [1] Brian Marick, "Experience with the Cost of Test Suite
          Coverage   Measures",  _P_a_c_i_f_i_c  _N_o_r_t_h_w_e_s_t  _S_o_f_t_w_a_r_e  _Q_u_a_l_i_t_y
          _C_o_n_f_e_r_e_n_c_e, October, 1991.  Also available as  a  compressed
          postscript file in cs.uiuc.edu:/pub/testing/experience.ps.Z.



9                                      -4-          GCT Tutorial







                                      -5-          GCT Tutorial


               /dev/null, so it's useless for actually  finding  bugs.
               It's fine for this tutorial, though.

          Type

                  % make
                  % lc lc.c

          to see how lc works.  You should see this output:

          % lc lc.c
              Pure     Pure Both         Total   TotalTotal
              Code     Comment  Cod&Com  Blank   CodeCommentLinesPages

          lc.c    311  437  62  117 373  499 927 28

          "Pure Code" means lines of code with no comments, "Pure Com-
          ment"  means  lines with only comments; "Both Cod&Com" means
          lines containing both.  "Blank" lines contain  neither  code
          nor comments.

          Now type

              % make clean

          to delete the executables; we'll be building new ones.

          _2.  _I_f _Y_o_u'_r_e _I_m_p_a_t_i_e_n_t

          If you don't care about the step-by-step process of  instru-
          mentation, just type

              % make -f Makefile.gct gct

          and proceed to the next chapter.  If you  want  to  use  GCT
          regularly, you'll want to read this chapter eventually.

          _3.  _D_e_c_i_d_i_n_g _W_h_a_t _t_o _M_e_a_s_u_r_e

          The first step is to tell GCT what to measure.   That infor-
          mation is kept in the file _g_c_t-_c_t_r_l, which contains:

              (coverage branch multi loop relational)
              lc.c

          The first line tells GCT we  want  branch,  multi-condition,
          loop, and relational operator coverage.  These are the types
          of coverage I recommend for detailed testing of  code  (unit
          and subsystem testing).  We'll see what these coverages mean
          when we look at GCT's reports.  Other types of coverage  are
          possible; see the User's Manual.

9

9







                                      -6-          GCT Tutorial


          The second line tells GCT to measure coverage only for _l_c._c.

          _4.  _I_n_s_t_r_u_m_e_n_t_i_n_g _t_h_e _P_r_o_g_r_a_m

          The next step is to _i_n_s_t_r_u_m_e_n_t the program so that  it  col-
          lects data about its execution as it runs.

          To do this, we use a modifed version of the program's origi-
          nal  makefile.   For  most makefiles, the changes are minor.
          The modified makefile is  the  file  _M_a_k_e_f_i_l_e._g_c_t.   Examine
          that  file, and you'll see that there's a target called gct.
          But, rather than using make, we'll do the instrumentation by
          hand.   Each  following  section  explains what a particular
          line of the gct target does.  Type the first command in each
          section to do what make would have done.

          _4._1.  _I_n_s_t_r_u_m_e_n_t_i_n_g

          % gct-init[2]

          This program copies a number of files into  your  directory.
          You  should  never  need  to  look at the internals of these
          files, but here's what they contain:

          gct-ps-defs.h
               A makefile usually invokes the  C  compiler  separately
               for  each source file.  GCT is invoked in the same way.
               This file stores information about where  the  previous
               invocation left off.

          gct-ps-defs.c
               For speed, coverage data is stored in the memory of the
               instrumented  program.   This file defines that in-core
               log; it is linked into the final executable.

          gct-write.c
               To be useful, the log must outlive the execution of the
               program  -  it  must  be written to disk.  This program
               contains a routine that writes this log.  It also  con-
               tains a routine that reads in a previous version of the
               log, so that the log accumulates over several tests.

          gct-defs.h
               This file contains various C macros used to  manipulate
               the log.


          ____________________
9             [2] If the shell cannot find gct-init, the directory that
          GCT  was  installed  in  must  not  be  in  your search path
          ($PATH).  Ask the person who installed GCT where it is.



9







                                      -7-          GCT Tutorial


          _4._2.  _I_n_s_t_r_u_m_e_n_t_a_t_i_o_n

          In this step, we tell the makefile to compile the program in
          the normal way, but to use GCT as the "compiler".

          % make all "CC=gct -DTESTING"

          You should see something like this:

          gct -DTESTING   -c  lc.c
          gct -DTESTING -o lc lc.o

          make gives _l_c._c to GCT.  The result is an instrumented file,
          also  called _l_c._c.  The makefile then invokes GCT to produce
          an executable.  Since GCT sees no C source, it does nothing.
          The actual creation of the executable comes later.

          The TESTING define is used to make the instrumented  program
          read and write the log file.  GCT stores the original source
          in the file _g_c_t__b_a_c_k_u_p/_l_c._c.   If  you  examine  that  file,
          you'll see two uses of the define.

          (1)  The first is in this code:

               #ifdef TESTING

               myexit(status)
               {
                 gct_writelog("LOG");
                 exit(status);
               }
               #define exit myexit

               #endif

               This is a standard trick to ensure the log  is  written
               out (into the file _L_O_G) any time the program exits.

          (2)  The second is in _m_a_i_n() where

                   gct_readlog("LOG");

               is used to read the log (if it exists) when the program
               starts.

          If you're curious, look inside the instrumented  source,  in
          _l_c._c.   The  code  is  difficult  to read, because it's been
          preprocessed and because GCT doesn't  preserve  indentation.
          You  can  recognize instrumentation by the use of the string
          "_G".  For a statement like

              if (function(a))
9

9







                                      -8-          GCT Tutorial


          GCT replaces it with (in effect):

              if ( (_t_e_m_p_o_r_a_r_y=(_f_u_n_c_t_i_o_n(_a)), __G_2(_3_4, _t_e_m_p_o_r_a_r_y), _t_e_m_p_o_r_a_r_y))

          The value of the if's test is first  saved  in  a  temporary
          variable.   The  _G2  macro records whether the if was taken
          true or false. The branch is the 34th log  entry.   Finally,
          the  temporary  variable  is  used  as the value of the if's
          test, so that the program's execution is unaffected  by  the
          instrumentation.  _G  is  a macro that expands out to only a
          few instructions, so the instrumented program  is  not  much
          slower  than  the original.  (The amount of slowdown varies,
          of course, depending on the program and how much  instrumen-
          tation you turn on.)

          _4._3.  _C_o_m_p_i_l_i_n_g _S_u_p_p_o_r_t _C_o_d_e

          % cc -c gct-ps-defs.c
          % cc -c gct-write.c

          This compiles the definitions of _g_c_t__w_r_i_t_e_l_o_g,  _g_c_t__r_e_a_d_l_o_g,
          and the in-core table that holds the log.

          _4._4.  _M_a_k_i_n_g _t_h_e _E_x_e_c_u_t_a_b_l_e

          % make all "OBJ=lc.o gct-ps-defs.o gct-write.o"

          Having instrumented the program, we now need to compile  it.
          When  making  the  final executables, we need to link in the
          files we  just  compiled,  since  the  instrumented  program
          refers to their contents.  You should see:

          cc   -c  lc.c
          cc -o lc lc.o gct-ps-defs.o gct-write.o


          You now have an instrumented lc.   You'll  be  able  to  run
          tests  and get information about coverage.  This information
          will refer to the line numbers in  the  original  (uninstru-
          mented)  file.   You  certainly  do  not want to look at the
          instrumented file to figure out what the  coverage  informa-
          tion is telling you.

          _4._5.  _R_e_s_t_o_r_i_n_g _t_h_e _O_r_i_g_i_n_a_l _S_o_u_r_c_e

          % grestore

          restores the  original  source.   You  no  longer  need  the
          instrumented version.



9

9










                                   CHAPTER  3


                      A First Pass Over the Coverage Data




          To run the test suite, type

              % run-suite

          It should take about ten seconds.  The result is  a  logfile
          named _L_O_G.

          To get a first idea of how thorough the test suite is, type

              % gsummary LOG


          You should see this:

          BINARY BRANCH INSTRUMENTATION (76 conditions total)
          3 ( 3.95%) not satisfied.
          73 (96.05%) fully satisfied.

          SWITCH INSTRUMENTATION (14 conditions total)
          0 ( 0.00%) not satisfied.
          14 (100.00%) fully satisfied.

          LOOP INSTRUMENTATION (24 conditions total)
          4 (16.67%) not satisfied.
          20 (83.33%) fully satisfied.

          MULTIPLE CONDITION INSTRUMENTATION (38 conditions total)
          2 ( 5.26%) not satisfied.
          36 (94.74%) fully satisfied.

          OPERATOR INSTRUMENTATION (27 conditions total)
          1 ( 3.70%) not satisfied.
          26 (96.30%) fully satisfied.

          SUMMARY OF ALL CONDITION TYPES (179 total)
          10 ( 5.59%) not satisfied.
          169 (94.41%) fully satisfied.

          I'll explain the first two  sections;  the  other  types  of
          instrumentation are covered in the next chapter.

          Every branching statement (if, ?, or  loop  test)  generates
          two  _c_o_v_e_r_a_g_e  _c_o_n_d_i_t_i_o_n_s:  one that the true case be taken,
          and one that the false case be taken.  In lc, there  are  38
          branching  statements  generating  76 conditions.  Of these,


                                      -9-          GCT Tutorial







                                      -10-         GCT Tutorial


          three have not been satisfied;  three cases (true or  false)
          remain to be exercised.

          lc also contains some switch statements with a total  of  14
          cases  (including defaults).  All the cases have been taken.
          (Each case label is a separate coverage condition.)

          The summary coverage is high.  That's not surprising,  since
          this  test  suite was derived by removing a few tests from a
          test suite that reached 100% coverage.  However,  a  summary
          coverage  greater  than  90%  is  typical  of  thorough test
          suites.[1]

          Simple numbers  like  gsummary's  are  a  blunt  instrument.
          Don't  use  GCT to reduce your testing to a few numbers, but
          rather to expand your understanding of what your test  suite
          really does.  greport, described in the next chapter, is the
          tool you'll use.





























          ____________________
9             [1] In addition to the _E_x_p_e_r_i_e_n_c_e  paper  mentioned  ear-
          lier,  I can send an unpublished case study: the coverage of
          GCT's own test suite.



9










                                   CHAPTER  4


                               Writing New Tests




          _1.  _A_n_a_l_y_s_i_n_g _m_i_s_s_e_d _c_o_v_e_r_a_g_e

          Type

          % greport LOG

          You should see:

          "lc.c", line 255: operator > might be >=. (L==R)
          "lc.c", line 255: condition 1 (argc, 1) was taken TRUE 42, FALSE 0 times.
          "lc.c", line 257: condition 1 (<...>[...], 1) was taken TRUE 0, FALSE 14 times.
          "lc.c", line 271: if was taken TRUE 0, FALSE 28 times.
          "lc.c", line 280: loop zero times: 0, one time: 18, many times: 10.
          "lc.c", line 290: if was taken TRUE 0, FALSE 40 times.
          "lc.c", line 425: if was taken TRUE 37, FALSE 0 times.
          "lc.c", line 860: loop zero times: 0, one time: 5, many times: 19.
          "lc.c", line 872: loop zero times: 0, one time: 0, many times: 6.

          You'll notice that this looks a lot like error output from a
          C compiler.  This is intentional, for two reasons:

          (1)  Psychological.  As with a compiler's output, you  exam-
               ine  the  errors (in this case, test deficiencies), fix
               them, redo the compile  (rerun  the  test  suite),  and
               repeat  the process until you get no more output.  Just
               as with a compiler, the messages  you  see  may  prompt
               large  changes,  larger  than  is strictly necessary to
               make them go away.

          (2)  Practical.  These error messages  are  compatible  with
               the  error(1)  program  (on BSD UNIX) and the GNU Emacs
               next-error function.  (A version of next-error tailored
               to  GCT is provided with GCT.  If you're an Emacs user,
               you may want to look at Appendix A now.)

          Let's examine the output line by line.  The first two  lines
          of  greport  output  refer  to line 255, which is the option
          processing loop. The line in question is highlighted  below,
          but I recommend you edit _l_c._c; the discussion will be easier
          to follow if you don't have to constantly flip pages of this
          tutorial.

              while (--argc > 0 && (**++argv == '-'))
              {
              if (  (*argv)[1] == LCURL || (*argv)[1] == RCURL)


                                      -11-         GCT Tutorial







                                      -12-         GCT Tutorial


              {
                  white_bracket = TRUE;
              }
              else if (sscanf (*argv + 1, "%d", &page_size) == FALSE)
              {
                  fprintf (stderr, "lc: Bad page size argument: %s\n", *argv);
                  exit (BAD_FLAG);           /* Forgive me, Edsger */
              }
              }

          The first greport line,

          "lc.c", line 255: operator > might be >=. (L==R)

          is from _r_e_l_a_t_i_o_n_a_l coverage, which  checks  for  tests  that
          probe common misuses of relational operators.  In this case,
          we're worried that the programmer made the common mistake of
          using  >  instead of the correct >=.  Suppose that happened.
          We want to write the test that will have the best chance  of
          revealing  the  bug;  that is, we want a test where what the
          correct program would do is as different  as  possible  from
          what the program we have does.

          If --argc is exactly  0,  the  correct  program  (--argc>=0)
          would enter the body of the while loop, while the buggy pro-
          gram (--argc>0) won't.  For all other values  of  argc,  the
          correct and buggy program will take the same path.  It seems
          that argc==0, the boundary condition, is the  best  test  to
          find  this  particular bug; GCT is telling us we never tried
          it.

          To force this case, the call to lc must have  no  non-option
          arguments.  It must look like one of these:

          % lc < INPUT
          % lc -} < INPUT
          % lc -34 < INPUT

          If you look at the run-suite file, you'll see that there are
          no  such  tests:  lc is never tested when it takes its input
          from standard input.  This is a major omission.  Rather than
          writing  a  test  like this right away, we should write down
          "input from  standard  input"  as  a  _t_e_s_t  _c_o_n_d_i_t_i_o_n  in  a
          separate  list  of test conditions.  We may later be able to
          combine several of these test conditions into a single  test
          case, which saves us effort.[1]

          ____________________
9             [1] It's also likely to result in a better test  --  more
          complicated  test  cases  are better (up to a point) because
          they are more likely to catch bugs by chance.



9







                                      -13-         GCT Tutorial


          The next line of greport output is from the same line of the
          program.

          "lc.c", line 255: condition 1 (argc, 1) was taken TRUE 42, FALSE 0 times.

          This is from _m_u_l_t_i_c_o_n_d_i_t_i_o_n coverage, which checks that  all
          parts  of  a  logical expression are used.  Line 255's while
          loop test has two components:

             --argc > 0

          and

             **++argv == '-'

          The first of them was  always  true  in  every  test,  never
          false.   The  second component isn't mentioned, because it's
          evaluated to both true and false.  If you want  to  see  how
          many  times  it's  evaluated to each value, use greport -all
          (see the manpage).

          This condition tells us the same thing that we already knew.
          To  get  argc  equal  to  zero,  there can be only arguments
          beginning with a dash -- that is, lc  must  take  its  input
          from standard input.

          The next line,

          "lc.c", line 257: condition 1 (<...>[...], 1) was taken TRUE 0, FALSE 14 times.

          corresponds to this code:

              while (--argc > 0 && (**++argv == '-'))
              {
              if (  (*argv)[1] == LCURL || (*argv)[1] == RCURL)
              {
                  white_bracket = TRUE;
              }
              else if (sscanf (*argv + 1, "%d", &page_size) == FALSE)
              {
                  fprintf (stderr, "lc: Bad page size argument: %s\n", *argv);
                  exit (BAD_FLAG);           /* Forgive me, Edsger */
              }
              }

          It is also an example of multicondition  coverage,  but  the
          message  looks  peculiar.   What  does (<...>[...], 1) mean?
          It's there in case "condition 1" is not enough to locate the
          condition  greport  refers to.  (In this case, the condition
          is (*argv)[1] == LCURL.)

          Conditions are numbered from left to right.  But in a deeply
          nested expression, like (A && (B || C) || D), it can be hard










                                      -14-         GCT Tutorial


          to count conditions accurately.   The  parenthetical  remark
          helps  you  find  them.  The first component is the leftmost
          operand of the condition.  In this  case,  it's  (*argv[1]).
          Greport always abbreviates arrays as _a_r_r_a_y_n_a_m_e[...], because
          the expression in the array is often complex and would  make
          the  line too long.  When _a_r_r_a_y_n_a_m_e is more complicated than
          a simple identifier, GCT abbreviates it as <...>,  again  to
          save space.

          These abbreviation rules are probably a  bad  idea,  but  in
          practice it's rarely difficult to figure out what subexpres-
          sion is  meant.   As  an  additional  help,  the  number  in
          parenthesizes  is  the  nesting  depth of the subexpression,
          starting with 1.  (It's always deeply parenthesized  expres-
          sions that cause trouble.)

          What this line tells us is that  *argv[1]  is  never  LCURL.
          That  is,  we've  never  given the program the -{ option.  A
          quick check of run-suite confirms this.   We'll  write  that
          down in our test condition list.

          The next line,

          "lc.c", line 271: if was taken TRUE 0, FALSE 28 times.

          tells us that this if has  never  been  taken  in  the  true
          direction.

              if (argc == 0)
              {
                  tally_file (stdin, &file_tally);
                  show_header ();
                  show_tally ("", &file_tally);
              }

          We already knew that:  the program never reads from standard
          input.

          The next line,

          "lc.c", line 280: loop zero times: 0, one time: 18, many times: 10.











9

9







                                      -15-         GCT Tutorial


          is the first example of _l_o_o_p coverage.  This particular loop
          traverses all of lc's non-option arguments.

              for (index = 1; index <= argc; index++)
              {
                  if ((fp = fopen (*argv, "r")) == NULL)
                  {
                  status = FILE_NOT_FOUND;
                  fprintf (stderr, "lc: can't open %s\n", *argv);
                  }
                  else
                  {
                  tally_file (fp, &file_tally);
                  if (fclose (fp) == EOF)
                      panic (PANIC, "Fclose error.");
                  show_tally (*argv, &file_tally);
                  if (argc > 1)
                      make_total (&total_tally, &file_tally);
                  }
                  argv++;
              }


          A greport line for a loop tells about three types of traver-
          sals:

          (1)  ones where the loop test failed on the  first  try,  so
               the loop was never entered.

          (2)  ones where the loop test failed on the second  try,  so
               the loop body was executed exactly once.

          (3)  ones where the loop was traversed more than once.

          Some types of bugs are only detected by one of  these  cases
          (though the single-traversal case is probably the least use-
          ful).

          In this particular case, we know  that  argc>0  (because  we
          just looked at the if statement that handled the zero case).
          So it is impossible to traverse  the  while  loop  0  times.
          This  is an example of an _i_n_f_e_a_s_i_b_l_e _t_e_s_t _c_o_n_d_i_t_i_o_n.  We can
          ignore it.

          Of the two feasible cases, we see that lc was given a single
          argument 18 times and more than one argument 10 times.

          We move on to:

          "lc.c", line 290: if was taken TRUE 0, FALSE 40 times.

          which corresponds to this line:
9

9







                                      -16-         GCT Tutorial


                          if (fclose (fp) == EOF)
                              panic (PANIC, "Fclose error.");

          Evidently, the programmer thinks an EOF return from fclose()
          is  impossible  (but  is checking anyway, just in case).  Of
          course, what a programmer thinks and what is true may be two
          different  things.   We  as testers would want to think hard
          about how to generate EOF returns.   (Error  handling  is  a
          fertile  source  of  serious bugs caused by mistaken assump-
          tions.)  Let's assume that we'll fail, that this is  another
          example of an infeasible test condition.

          This line,

          "lc.c", line 425: if was taken TRUE 37, FALSE 0 times.

          is another sanity check:

                 if (line_info.blank)
                     (file_tally->blank)++;
                 else
                     panic (PANIC, "Non-empty line contains nothing.");

          It's never been taken in the false direction, and  the  pro-
          grammer  believes  it  can't  be.  In thorough testing, we'd
          think hard about this; for this tutorial, we'll move on.

          The next line,

          "lc.c", line 860: loop zero times: 0, one time: 5, many times: 19.

          is another impossible loop case (these are not uncommon):

              for (index = 0; index < 4; index++)
              {
                  c = PULL_OFF(fp);
                  if (c != QUOTE)            /* store it. */
                  chars[index] = c;
                  else              /* found what we want. */
                  break;
              }

          It is impossible for this loop not to be entered; hence  the
          "zero  times"  case  in loop coverage can never have a value
          other than zero.  (The loop can  be  entered  exactly  once,
          though,  if  the  break statement is executed the first time
          through.  This happened 5 times.)

          The final line of greport output is this:

          "lc.c", line 872: loop zero times: 0, one time: 0, many times: 6.

          which corresponds to this code:










                                      -17-         GCT Tutorial


              for (index = 3; index >= 0; index--)
              {
                  c = chars[index];    /* because chars[array] is too big to expand in PUSH_BACK. */
                  PUSH_BACK(c, fp);
              }

          This is another impossible while  loop.   It  always  counts
          down  from  3  to  0,  so  it can never be taken zero or one
          times.

          We're finished.  We generated only a short list of things to
          test.   This  is  as it should be, given a thorough starting
          test suite.  We wasted some time looking at impossible  con-
          ditions, but not very much.

          _2.  _T_h_e _n_e_w _t_e_s_t_s

          We have two new test conditions:

          1. input from standard input.
          2. Use the -{ option.

          This is easy to do with a single test, which you should just
          type to the shell:

          % echo "{" | lc -\{

          (The backslash in front of the { is  to  prevent  the  shell
          from interpreting it specially.) This is not a very thorough
          test of the -{ option, but it will do for this tutorial.  Is
          the output correct?

          Now type

          % gsummary LOG

          You'll see that  total  coverage  has  gone  up  to  96.65%.
          Greport shows:

          "lc.c", line 280: loop zero times: 0, one time: 18, many times: 10.
          "lc.c", line 290: if was taken TRUE 0, FALSE 40 times.
          "lc.c", line 425: if was taken TRUE 38, FALSE 0 times.
          "lc.c", line 860: loop zero times: 0, one time: 5, many times: 19.
          "lc.c", line 872: loop zero times: 0, one time: 0, many times: 6.

          We know that all of these are infeasible.

          _3.  _S_u_p_p_r_e_s_s_i_n_g _i_n_f_e_a_s_i_b_l_e _c_o_v_e_r_a_g_e

          It's annoying that we can't make the infeasible test  condi-
          tions  go  away.   Worse, if we're testing a large program -
          one where it may take us  several  tries  to  eliminate  all
          feasible  test  conditions  -  we  don't  want to waste time










                                      -18-         GCT Tutorial


          looking at infeasible conditions every time we  run  greport
          after a change to the test suite.

          The gedit program can help.  To use it, type

          % greport -edit LOG > edit.g

          Edit _e_d_i_t._g.  You'll see this:

          "lc.c", line 280: [25: 0 0 18 10] loop zero times: 0, one time: 18, many times: 10.
          "lc.c", line 290: [34: 0 40] if was taken TRUE 0, FALSE 40 times.
          "lc.c", line 425: [75: 38 0] if was taken TRUE 38, FALSE 0 times.
          "lc.c", line 860: [167: 0 0 5 19] loop zero times: 0, one time: 5, many times: 19.
          "lc.c", line 872: [180: 0 0 0 6] loop zero times: 0, one time: 0, many times: 6.

          The numbers in brackets are the raw  versions  of  what  the
          text says.  In the first case (line 280), the first 0 is the
          number of times the loop was skipped.  The next two  numbers
          (0  and  18)  add  up  to  the  number of times the loop was
          entered exactly once.  (The reason why there are two numbers
          is unimportant - it's due to the implementation of loop cov-
          erage.)  The final number (10) is the number  of  times  the
          loop  was  traversed  many  times.  In the case of branches,
          like line 290, the first  number  is  the  true  count,  the
          second the false count.

          Replace the zeros that are impossible with either "s",  "S",
          "0s", or "0S":[2]

          "lc.c", line 280: [25: 0S 0 18 10] loop zero times: 0, one time: 18, many times: 10.
          "lc.c", line 290: [34: S 40] if was taken TRUE 0, FALSE 40 times.
          "lc.c", line 425: [75: 38 S] if was taken TRUE 38, FALSE 0 times.
          "lc.c", line 860: [167: S 0 5 19] loop zero times: 0, one time: 5, many times: 19.
          "lc.c", line 872: [180: S 0S 0 6] loop zero times: 0, one time: 0, many times: 6.

          Notice that you only had to replace one of the zeroes in the
          "single traversal" case of the last loop.

          Now type

          % gedit edit.g
          % gsummary LOG






          ____________________
9             [2] If you're using Emacs "gedit-mode", SPC will helpful-
          ly position you at the first zero in the next line.



9







                                      -19-         GCT Tutorial


          You'll see:

          BINARY BRANCH INSTRUMENTATION (76 conditions total)
          0 ( 0.00%) not satisfied.
          76 (100.00%) fully satisfied. [2 ( 2.63%) suppressed]

          SWITCH INSTRUMENTATION (14 conditions total)
          0 ( 0.00%) not satisfied.
          14 (100.00%) fully satisfied.

          LOOP INSTRUMENTATION (24 conditions total)
          0 ( 0.00%) not satisfied.
          24 (100.00%) fully satisfied. [4 (16.67%) suppressed]

          MULTIPLE CONDITION INSTRUMENTATION (38 conditions total)
          0 ( 0.00%) not satisfied.
          38 (100.00%) fully satisfied.

          OPERATOR INSTRUMENTATION (27 conditions total)
          0 ( 0.00%) not satisfied.
          27 (100.00%) fully satisfied.

          SUMMARY OF ALL CONDITION TYPES (179 total)
          0 ( 0.00%) not satisfied.
          179 (100.00%) fully satisfied. [6 ( 3.35%) suppressed]

          If you use greport, it will show you nothing.   You're  done
          using coverage on your test suite.
























9

9










                                  APPENDIX  A


                                 Using gedit.el




          gedit.el provides a version of the Emacs  next-error  inter-
          face  that  works better for greport output.  The source for
          gedit.el is, by default, in the GCT library directory;  who-
          ever  installed  GCT  may have put it with other local emacs
          files.

          The most convenient way to use gedit.el  is  to  always  put
          greport  output  into  files  ending in .g and put something
          like

          (setq auto-mode-alist (cons (cons "\\.g$" 'gedit-mode) auto-mode-alist))

          into your ._e_m_a_c_s file.  That done, whenever you edit such  a
          file, you'll use gedit-mode.

          Either load gedit.el manually, or put something like

          (autoload 'gedit-mode (expand-file-name "~/gct/src/gedit.el"))

          in your ._e_m_a_c_s file.

          When in gedit-mode, each time you hit  SPC,  a  new  greport
          line  will  move  to the top of one window.  The appropriate
          source file will appear in the other window.  An arrow  (=>)
          will  point  to the line in question. (The arrow is not part
          of the file.)

          Like next-error, there's no convenient way to back  up.   If
          you hit e, you'll restart from the beginning.















9

9                                      -20-         GCT Tutorial



