Subject: v11i082: Watcher system monitor program, Part01/02 Newsgroups: comp.sources.unix Sender: sources Approved: rs@uunet.UU.NET Submitted-by: Kenneth Ingham Posting-number: Volume 11, Issue 82 Archive-name: watcher/part01 [ Watcher is a system-monitor utililty, described at the Phoenix Usenix. --r$ ] #! /bin/sh # This is a shell archive. Remove anything before this line, then unpack # it by saving it into a file and typing "sh file". To overwrite existing # files, type "sh file -c". You can also feed this as standard input via # unshar, or by typing "sh Docs/Abstract <<'END_OF_Docs/Abstract' X.sp 0.5i X.ce 2 XKeeping watch over the flocks Xat night (and day) X.sp 0.3i X.ce 8 XKenneth Ingham XUniversity of New Mexico Computing Center XDistributed Systems Group X2701 Campus NE XAlbuquerque, NM 87131 X(505) 277-8044 Xingham@charon.unm.edu Xucbvax!unmvax!charon!ingham X.sp 0.2i X.ce XTopic Areas: Applications, System management, Utilities X.sp 0.5i XThe computing facilities offered by the University of New Mexico XComputing Center include three microvaxen, five large vaxen (780 or Xbigger), and a Sequent B8000. In addition to these Unix/VMS machines, Xthe UNMCC Distributed Systems Group (DSG) monitors a number of the Xvarious microvaxen and sun workstations scattered across campus. This Xduty falls to the DSG Programmer designated as "DOC", or "DSG On Call", Xwho receives his beeper based on a monthly rotation schedule. X.sp XIn the past, shell scripts running every six hours reported various Xsystem statistics to DOC, who then scanned the output for signs of Xpossible trouble. As the number of machines and the number of Xpotential problems grew, the mound of output that DOC had to process, Xmost of which merely indicated normal system operation, became Xoverwhelming. Now, with several machines to monitor and only one Xperson acting in this capacity, DOC can often waste a tremendous amount Xof time wading through system status reports, time which can be better Xspent actually fixing system problems. X.sp XIn response to this situation, the author developed a tool which Xintroduces some intelligence into the machine's self-reporting, letting Xthe machine filter out messages indicating normal operation and Xforwarding to DOC only those messages which point out trouble areas. XThe result of these efforts is Watcher, a very general and extensible Xsystem self-monitor. Running more often than the set of Xshell scripts, Watcher keeps closer tabs on the system; since it Xdelivers only a summary of potential problems, however, this extra Xmonitoring produces \fIno\fR corresponding increase in the demand on Xthe system manager. No problems slip by unnoticed in the more concise Xoutput, leading to an improvement in overall system availability as well Xas the more effective utilization of the system manager's time. X.sp XWatcher was designed to be almost as flexible as DOC in deciding what Xconstitutes a problem with the system. Running at intervals specified Xin crontab, Watcher issues a number of Xuser-specified commands (each of which Xdelivers its output in a different format), parsing all or part of the Xoutput from either the left or the right. It compares this Xto the last such output obtained, checking for indications Xof a system abnormality. Such signs might take the form of a Xtoo abrupt change in a certain value (e.g. a process which suddenly Xbegins gobbling vast amounts of cpu time), Xa value which exceeds the allowable maximum or minimum (such as a Xan overly-full file system), Xor an unacceptable change in a string value X(e.g. when "up" changes to "down"). For commands such as X"ps" whose output varies considerably with each run, specific Xparts of the output can be designated as a key; successive runs of XWatcher will home in on these key areas for their comparisons. X.sp XSince the user specifies not only the commands Watcher will execute and Xthe time lapse between successive runs, but also the aforementioned Xparameters which indicate system anomalies, Watcher can easily be seen Xas a very flexible, general system monitor. Its use at UNM has provided Xa marked increase in the productivity of the system manager, which has Xled in turn to the increase in the reliability and availability of the Xsystems at UNMCC. END_OF_Docs/Abstract if test 3638 -ne `wc -c Docs/Makefile <<'END_OF_Docs/Makefile' XTROFF=xroff X Xpaper: X $(TROFF) macros Paper X Xabstract: X $(TROFF) Abstract END_OF_Docs/Makefile if test 73 -ne `wc -c Docs/README <<'END_OF_Docs/README' XIn this directory is all of the documenttion about watcher. It should Xinclude a man page, the abstract sent in to usenix, and the paper Xpublished in the proceedings of the 1987 Summer Usenix in Phoenix. X XThe Makefile shows how to print the abstract and the paper. The man Xpage is the same as all other man pages. END_OF_Docs/README if test 315 -ne `wc -c Docs/macros <<'END_OF_Docs/macros' X\" macro definitions X.de EX X.nf X.in +0.25i X.sp X.. X.de NX X.in -0.25i X.fi X.ad X.sp X.. X.de SE X.ps \\n(ps+1 X.ft B X.sp 0.2i X\\$1 \\$2 \\$3 \\$4 \\$5 \\$6 \\$7 \\$8 \\$9 X.ft X.ps \\n(ps X.sp 0.2i X.. X.de NP X'sp 0.75i X.tl ''%'' X'bp X'sp 1.0i X.. X.de TI X.ps \\n(ps+5 X.vs \\n(vs+5 X.ft B X.ce X\\$1 \\$2 \\$3 \\$4 \\$5 \\$6 \\$7 \\$8 \\$9 X.ps \\n(ps X.vs \\n(vs X.ft X.. X.de AU X.sp 0.5i X.ps \\n(ps+3 X.ft I X.ce \\$1 X.ps \\n(ps X.. X.de AB X.ll -0.5i X.in +0.5i X.sp 0.2i X.ps \\n(ps+3 X.ft B X.ce XAbstract X.ps \\n(ps X.sp 0.2i X.ft R X.. X.de BD X.in -0.5i X.ll +0.5i X.sp 0.3i X.ps \\n(ps X.ft R X.. X\" end of macro definitions X.wh -1.5i NP X.in 0.25i X.ll 6.25i X.sp 0.5i \" for the first page X.ps 11 X.vs 14 X.nr ps 11 X.nr vs 14 X\" end of macro package END_OF_Docs/macros if test 712 -ne `wc -c Ideas <<'END_OF_Ideas' X Repace number with int or float. Currently watcher treats all X numbers as floats. There is some code already there X that could handle the different types. The %d for a X float is counterintuitive. This would be fixed by X making %d int and %f float like printf. X X Use yacc's union rather than YYSTYPE default. This would X probably help portability. X X Print alias in pp output. Currently it isn't printed. No real X reason for this. X X Multiple ok values for string fields. Sometimes there may be X more than one string value which is considered "ok". X Watcher should be able to handle this. Possible X syntax: X X(/usr/ucb/ruptime | fgrep -f /usr/local/lib/watcher/UnmHosts) { ruptime } X 2 status%s 1 machine%k 7 loadav%d: X loadav 0 10; X status "up", "slow", "dead". X X Positive and/or negative deltas make a difference. Currently X watcher only notices things which increase. It would X probably be useful to watch for things that drop also. X Possible syntax: X X(df -i | /usr/ucb/tail +2 | grep -v tmp) { 'df no tmp' } X 1-9 filesystem%k 41-42 spaceused%d 64-65 inodesused%d 1-9 device%k: X spaceused -15%; X spaceused 0 89; X X Right to left parsing as option instead of always l-r. It may X be easier at times to specify that the useful info is X located in the fifth from the right column rather than X the eit`ghth from the left. END_OF_Ideas if test 1347 -ne `wc -c MANIFEST <<'END_OF_MANIFEST' X File Name Archive # Description X----------------------------------------------------------- X Docs 1 X Docs/Abstract 1 X Docs/Makefile 1 X Docs/Paper 2 X Docs/README 1 X Docs/macros 1 X Docs/watcher.1 2 X Ideas 1 X MANIFEST 1 This shipping list X Makefile 1 X README 1 X Support 1 X Support/Daemons 1 X Support/README 1 X Support/UnmHosts 1 X Support/Watcherfile 1 X Support/crontab.entry 1 X Support/syswatch 1 X baderr.c 1 X check_item.c 1 X checkline.c 1 X clean_hist.c 1 X control.y 2 X defs.h 2 X do_args.c 1 X doit.c 1 X externs.c 1 X find_of.c 1 X find_pre_cmd.c 1 X find_pre_val.c 1 X get_col_fld.c 1 X get_rel_fld.c 1 X getargv.c 1 X init.c 1 X init_sigs.c 1 X line_to_vec.c 1 X main.c 1 X open_cf.c 1 X open_hf.c 1 X pp.c 1 X pp_change.c 1 X pp_out.c 1 X read_hist.c 1 X save_key.c 1 X yyerror.c 1 X yylex.c 1 END_OF_MANIFEST if test 1494 -ne `wc -c Makefile <<'END_OF_Makefile' X# X# makefile for watcher X# X# SYS should be either BSD or SYSV XSYS = BSD X XOBJS = baderr.o check_item.o checkline.o do_args.o doit.o externs.o\ X find_of.o find_pre_cmd.o find_pre_val.o get_col_fld.o\ X get_rel_fld.o getargv.o init.o init_sigs.o line_to_vec.o main.o\ X open_cf.o open_hf.o pp.o pp_change.o pp_out.o read_hist.o\ X save_key.o y.tab.o yyerror.o yylex.o XCFLAGS = -O -D$(SYS) X XDIST = Makefile Docs Support Ideas X Xwatcher: $(OBJS) X cc $(CFLAGS) -o watcher $(OBJS) X Xclean: X rm -f a.out core watcher *.o y.tab.c y.tab.h y.output Make.out X Xlint: X lint -x *.c > Lint.out X Xshar: X shar README *.c *.h *.y $(DIST) > Watcher.shar X X# Two shar files are made because one is too large for the mailers with X# a 64K limit. Xdist: X shar README *.c > Watcher.shar1 X shar *.h *.y $(DIST) > Watcher.shar2 X Xy.tab.c y.tab.h: control.y X yacc -d control.y X Xy.tab.o: y.tab.c Xyylex.o: y.tab.h X$(OBJS): defs.h END_OF_Makefile if test 891 -ne `wc -c README <<'END_OF_README' XWatcher has been sucessfully made and run on 4.3BSD on vaxen, Ultrix 1.1 X& 1.2 on vaxen (what else?), and a sun 3/160 version 3.0 of the sun os. XThanks to Alan Silverstein for bringing up watcher on a system V.2 machine Xand pointing out the berkeleyisms (which I hope are all #ifdef'd now). X XThere have been recent problems which I cannot reproduce which appear to Xbe a pointer in a malloc'd area pointing off a bit. I have been unable Xso far to track this problem down. X XA short description of all of the files and directories for watcher. X XIdeas - changes that I need to or would like to make for watcher. Xknown bugs are listed here. Someday I will fix them. If you fix them Xbefore I do, please send the fixes to me. X XDocs - here is all of the documentation for watcher. A man page and the Xpaper that appeared in the proceedings of the summer usenix conference Xin 1987. X XMakefile - you guessed it. X XREADME - this file X XSupport - directory of useful files to use with watcher. There were Xtaken from actual use at UNM (although they were taken a while back and Xmay not be what is in use now). They may help in seeing how at least Xone site is running watcher. X XThe rest of the files should be the source. X XHappy Watching. X XBug fixes and reports should be sent to: X XKenneth Ingham, UNM Computing Center, Albuquerque, NM 87131, 505-277-8044 X XUsenet: {convex,ucbvax,gatech,csu-cs,anl-mcs}!unmvax!charon!ingham XBITNET: ingham@unmb XInternet: ingham@charon.UNM.EDU X END_OF_README if test 1471 -ne `wc -c Support/Daemons <<'END_OF_Support/Daemons' Xcron Xrwhod Xrouted Xupdate Xdaemon Xinit Xstatd Xsendmail -bd Xsyslogd Xnamed Xswapper Xcensus Xinetd END_OF_Support/Daemons if test 91 -ne `wc -c Support/README <<'END_OF_Support/README' XThese are files used at the University of New Mexico as of May 19, 1987 Xor so. Due to the evolving nature of what we determine needs to be Xwatched and the various on call people coming up with new ways to watch Xfor problems without generating more output, it is probably not what we Xare now running. However it should give an idea of how watcher is used Xat UNM. X XThe following files all live in /usr/local/lib/watcher (besides the Xsource for watcher itself): X Daemons X UnmHosts X Watcherfile X XThe following are misc. files which are also used with watcher. X crontab.entry - what we have in our crontab X syswatch - a shell script which actually runs watcher and mails X the output to the on call person. END_OF_Support/README if test 704 -ne `wc -c Support/UnmHosts <<'END_OF_Support/UnmHosts' Xariel Xcharon Xdeimos Xeuropa Xgeinah Xhydra Xizar Xunmb END_OF_Support/UnmHosts if test 50 -ne `wc -c Support/Watcherfile <<'END_OF_Support/Watcherfile' X(df -i | /usr/ucb/tail +2 | grep -v tmp) { 'df no tmp' } X 1-9 filesystem%k 41-42 spaceused%d 64-65 inodesused%d 1-9 device%k: X spaceused 15%; X spaceused 0 89; X inodesused 15%; X inodesused 0 49. X(df -i | /usr/ucb/tail +2 | grep tmp) { 'df tmp only' } X 1-9 filesystem%k 41-42 spaceused%d 64-65 inodesused%d 1-9 device%k: X spaceused 0 89; X inodesused 0 49. X(/usr/ucb/ruptime | fgrep -f /usr/local/lib/watcher/UnmHosts) { ruptime } X 2 status%s 1 machine%k 7 loadav%d: X loadav 0 10; X status "up". X(ps -aux | fgrep -v -f /usr/local/lib/watcher/Daemons | /usr/ucb/tail +2) X { 'ps with no daemons' } X 9-14 pid%k 16-19 percentcpu%d 42-45 cputime%d: X cputime 0 10. X(ps -ax | fgrep -f /usr/local/lib/watcher/Daemons | uniq -c) { 'daemon count' } X 1 daemon_count%d 6 daemon_name%k: X daemon_count 0 1. X(ps -aux | fgrep -f /usr/local/lib/watcher/Daemons ) { 'daemon ps' } X 9-14 pid%k 16-19 percentcpu%d 42-45 cputime%d: X cputime 2. END_OF_Support/Watcherfile if test 936 -ne `wc -c Support/crontab.entry <<'END_OF_Support/crontab.entry' X0 0 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 2 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 4 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 6 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 8 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 10 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 12 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 14 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 16 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 18 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 20 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog X0 22 * * * root /usr/local/lib/syswatch >> /usr/lib/cronlog END_OF_Support/crontab.entry if test 804 -ne `wc -c Support/syswatch <<'END_OF_Support/syswatch' X#! /bin/csh -f Xset FILE = /tmp/watch$$ Xset MAILTO = ( doc ) Xcd /usr/local/lib/watcher Xwatcher > $FILE Xif (-z $FILE) then X echo " " | mail -s "`hostname` had no problems at `date`" $MAILTO Xelse X mail -s "System problem report for `hostname` at `date`" $MAILTO < $FILE Xendif X Xrm -f $FILE END_OF_Support/syswatch if test 286 -ne `wc -c baderr.c <<'END_OF_baderr.c' X/* X baderr: A bad error has been caught. Print an error message explaining X why we are dying, then exit. X X Assumptions: X the signal number in sig is an actual signal number and has an X entry in sys_siglist (BSD ONLY). X X Arguments: X sig: the number of the signal which brought us here. X X Author: X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xbaderr(sig) Xint sig; X{ X extern char *sys_siglist[]; X X printf(" \n"); X printf(" \n"); X#ifdef BSD X printf(">> Unrecoverable error. %s Bye. <<\n",sys_siglist[sig]); X#else X printf(">> Unrecoverable error. Signal %d. Bye. <<\n", sig); X#endif X printf(" \n"); X printf(" \n"); X exit(1); X} END_OF_baderr.c if test 853 -ne `wc -c check_item.c <<'END_OF_check_item.c' X/* X check_item: given a value and a change format structure, make sure X that the value is in range. X X Basically, this routine is a large switch statement on the type of X change that grabs the necessary info, and checks to see if the item X is worth mentioning. X X Note that what we print out depends on whether or not something else X has been found wrong on this line. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xcheck_item(cf, value, cmd, line, prev_val) Xchar *value, *cmd, *line; Xstruct change_fmt_st *cf; Xunion all_u *prev_val; X{ X extern int line_ok, cmd_ok; X double v, pct_chg, abs_chg; X int len; X X v = atof(value); X len = strlen(value); X X switch(cf->type) { X case PERCENT: X if (prev_val == NULL) /* nothing to compare with */ X return; X if (v == 0) /* avoid divide by 0 */ X return; X pct_chg = (v - prev_val->floatval) / v; X if (pct_chg > cf->fmt.percent) { X if (line_ok) { X printf("%s had ", cmd); X printf("%s change by more than %.2f percent.\n", X cf->name, cf->fmt.percent*100); X printf("%s\n",line); X } X else { X printf("Also, it had "); X printf("%s change by more than %.2f percent.\n", X cf->name, cf->fmt.percent*100); X } X printf("Previous value %.2f; ", X prev_val->floatval); X printf("current value %.2f.\n", v); X line_ok = False; X } X break; X case ABSOLUTE: X if (prev_val == NULL) /* nothing to compare with */ X return; X abs_chg = v - prev_val->floatval; X if (abs_chg > cf->fmt.abs_amount) { X if (line_ok) { X printf("%s had ", cmd); X printf("%s change by more than %.2f.\n", X cf->name, cf->fmt.abs_amount); X printf("%s\n",line); X } X else { X printf("Also, it had "); X printf("%s change by more than %.2f.\n", X cf->name, cf->fmt.abs_amount); X } X printf("Previous value %.2f; ", X prev_val->floatval); X printf("current value %.2f.\n", v); X line_ok = False; X } X break; X case MAX_MIN: X if (v > cf->fmt.max_min.max || v < cf->fmt.max_min.min) { X if (line_ok) { X printf("%s has a ", cmd); X printf("max/min value out of range:\n"); X printf("%s\n",line); X } X else { X printf("Also, it has a "); X printf("max/min value out of range:\n"); X } X printf("where %s = %.2f; ", cf->name, v); X printf("valid range %.2f to %.2f.\n", X cf->fmt.max_min.min, cf->fmt.max_min.max); X line_ok = False; X } X break; X case STRING: X if (strncmp(cf->fmt.str_value, value, len) != 0) { X if (line_ok) { X printf("%s has a string ", cmd); X printf("value which is not valid:\n"); X printf("%s\n",line); X } X else { X printf("Also, it has a string"); X printf("value which is not valid:\n"); X } X printf("where %s = '%s'; Should be '%s'\n", X cf->name, value, cf->fmt.str_value); X line_ok = False; X } X break; X default: X printf("check_item: impossible condition\n"); X break; X } X cmd_ok = line_ok; X} END_OF_check_item.c if test 2976 -ne `wc -c checkline.c <<'END_OF_checkline.c' X/* X checkline: check the input line just read in against what we expect X to find and report any problems. Actually, most of the work is done X by check_item. We just identify what needs to be checked. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xcheckline(cmd, line, prev_res) Xstruct cmd_st *cmd; Xchar *line; Xstruct old_cmd_st *prev_res; X{ X extern int line_ok; X extern FILE *hf; X struct change_fmt_st *cf; X union out_fmt_u of; X union all_u *prev_value; X char value[MAX_STR]; X char key_val[MAX_STR]; X char *cmd_name; X X cmd_name = (cmd->alias != NULL ? cmd->alias : cmd->pipeline); X X save_key(cmd, line, key_val); /* side effect: return key value */ X /* for each change format item */ X line_ok = True; X for (cf=cmd->change_fmt; cf; cf=cf->next) { X /* find the output format entry for this item */ X if (!find_of(cmd->out_type, cf->name, cmd->out_fmt, &of)) { X fprintf(stderr, "Warning: %s appears in change list ", X cf->name); X fprintf(stderr, "but not in output format for %s\n", X cmd_name); X continue; X } X X /* X find the part of the line corresponding to check. X Also find the previous results corresponding to the X key for this line (if any). X */ X switch (cmd->out_type) { X case RELATIVE: X find_prev_value(prev_res, of.rel_fmt->name, X key_val, &prev_value); X if (get_rel_field(line, of.rel_fmt->field, value) != X NULL) X check_item(cf, value, cmd_name, line, X prev_value); X break; X case COLUMN: X find_prev_value(prev_res, of.rel_fmt->name, X key_val, &prev_value); X if (get_col_field(line, of.col_fmt->start, of.col_fmt->end, value) != NULL) X check_item(cf, value, cmd_name, line, X prev_value); X break; X } X X /* X save the value in the history file for future X comparisons. X */ X if (cmd->key.rel_fmt != NULL) X fprintf(hf, "\t\t%s %c %s\n", cf->name, X TCHAR(cf->type), value); X } X if (!line_ok) X printf("---------\n"); X} END_OF_checkline.c if test 1962 -ne `wc -c clean_hist.c <<'END_OF_clean_hist.c' X/* X clean_hist: free up all memory used by the history structure. The X main purpose of this is to use the range checking of malloc to check X for pointer problems. If you don't have source, this is of dubious X value. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xclean_hist() X{ X extern struct old_cmd_st *chead; X struct old_cmd_st *bcp; X struct val_st *vp, *bvp; X struct key_st *kp, *bkp; X X while (chead != NULL) { X printf("freeing '%s'\n",chead->pipeline); X free(chead->pipeline); X kp = chead->keys; X while (kp != NULL) { X printf("\tfreeing '%s'\n",kp->key_value); X free(kp->key_value); X vp = kp->vals; X while (vp != NULL) { X printf("\tfreeing '%s'\n",kp->vals->name); X free(vp->name); X bvp = vp; X vp = vp->next; X free(bvp); X } X bkp = kp; X kp = kp->next; X free(bkp); X } X bcp = chead; X chead = chead->next; X free(bcp); X } X printf("\nThe world is now free again! Aren't you glad?\n"); X} END_OF_clean_hist.c if test 983 -ne `wc -c do_args.c <<'END_OF_do_args.c' X/* X do_args: parse the comand line arguments and set variables related to X them. X X Copied from main: X watcher [-p] [-v] [-h histfile] [-f controlfile] X X -p : pretty print control file as a verification of parse X (default no pretty print). This option prevents X processing of control file. X -v : be verbose. X -h : file in which to save output for future compare (default X ./watcher.history). X -f : controlfile to use (default ./DEF_CONTROL{,2}). X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xdo_args(argc, argv) Xint argc; Xchar *argv[]; X{ X extern int pflag, cflag, vflag; X extern char controlname[], histfilename[]; X X register int i; X X /* defaults */ X pflag = False; X cflag = False; X vflag = False; X (void) sprintf(histfilename, "%s", DEF_HISTFILE); X X for (i=1; idoit.c <<'END_OF_doit.c' X/* X doit: here is where the real purpose of the program is actually X carried out. X X for each command, run it and look for problems. X write results from this run back out. X X need to change from popen to ps_open. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xdoit() X{ X extern struct cmd_st *clist; X extern int vflag; X extern int cmd_ok; X extern FILE *hf; X X char line[MAX_STR]; X struct cmd_st *p; X struct old_cmd_st *prev_results, *find_prev_cmd(); X FILE *ps, *popen(); X X /* run commands */ X for (p=clist; p != NULL; p=p->next) { X cmd_ok = True; X if (vflag) X printf("Executing: '%s'\n\n", p->pipeline); X if (p->key.rel_fmt != NULL) X fprintf(hf, "%s\n", p->pipeline); X /* get prev results for comparison */ X prev_results = find_prev_cmd(p->pipeline); X ps = popen(p->pipeline, "r"); X while (fgets(line, MAX_STR, ps) != NULL) { X line[strlen(line)-1] = '\0'; X if (vflag) X printf(" Read: '%s'\n",line); X checkline(p, line, prev_results); X } X if (!cmd_ok) X printf("\n"); X } X} END_OF_doit.c if test 1048 -ne `wc -c externs.c <<'END_OF_externs.c' X/* X externs: external variable declarations. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X XFILE *cf, *hf; X Xint intval, ointval; Xchar *strval, ostrval[MAX_STR]; Xchar pipeline[MAX_STR]; X Xstruct cmd_st *clist = NULL; Xstruct old_cmd_st *chead = NULL; X Xint parse_error = False; X Xint pflag, cflag, vflag; Xchar histfilename[MAX_STR]; Xchar controlname[MAX_STR]; Xint line_ok; Xint cmd_ok; END_OF_externs.c if test 429 -ne `wc -c find_of.c <<'END_OF_find_of.c' X/* X find_of: find the output format corresponding for the field whose X name we are given. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xfind_of(type, name, of_head, of) Xint type; Xunion out_fmt_u of_head, *of; Xchar *name; X{ X struct rel_out_st *rf; X struct col_out_st *cf; X X switch(type) { X case RELATIVE: X for (rf=of_head.rel_fmt; rf; rf=rf->next) X if (strcmp(name, rf->name) == 0) { X (*of).rel_fmt = rf; X return True; X } X break; X case COLUMN: X for (cf=of_head.col_fmt; cf; cf=cf->next) X if (strcmp(name, cf->name) == 0) { X (*of).col_fmt = cf; X return True; X } X break; X default: X fprintf(stderr,"internal error; unknown type in find_of\n"); X exit(1); X } X X return False; X} END_OF_find_of.c if test 749 -ne `wc -c find_pre_cmd.c <<'END_OF_find_pre_cmd.c' X/* X find_prev_cmd: find the previous results for this command (if any). If X there is no match, then we return NULL. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xstruct old_cmd_st * Xfind_prev_cmd(pipeline) Xchar *pipeline; X{ X extern struct old_cmd_st *chead; X struct old_cmd_st *cp; X X cp = chead; X while (cp != NULL && strcmp(pipeline, cp->pipeline) != 0) X cp = cp->next; X X return cp; X} END_OF_find_pre_cmd.c if test 441 -ne `wc -c find_pre_val.c <<'END_OF_find_pre_val.c' X/* X find_prev_value: given an old command structure, look through the X prior output for the output for name of type type. Return it in the X union pointed to by value or NULL if not found. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xfind_prev_value(cmd, field_name, key_val, value) Xstruct old_cmd_st *cmd; Xchar *field_name, *key_val; Xunion all_u **value; X{ X struct val_st *vp; X struct key_st *kp; X X *value = NULL; X X if (cmd == NULL) X return; X if (cmd->keys == NULL) X return; X X /* find the correct keyword */ X for (kp=cmd->keys; kp != NULL; kp=kp->next) { X if (strcmp(kp->key_value, key_val) == 0) X break; X } X X if (kp == NULL) X return; X if (kp->vals == NULL) X return; X X /* find the value under the keyword */ X for (vp=kp->vals; vp != NULL; vp=vp->next) { X if (strcmp(vp->name, field_name) == 0) { X *value = &(vp->val); X return; X } X } X} END_OF_find_pre_val.c if test 904 -ne `wc -c get_col_fld.c <<'END_OF_get_col_fld.c' X/* X get_col_field: we have a column output format and want a certain X part of it. Place the it in 'value' if it exists and return a X pointer to the string. If there are problems place a null terminated X zero length string in 'value' and return NULL as an error condition. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xchar * Xget_col_field(line, start, end, value) Xchar *line, *value; Xint start, end; X{ X int len; X X value[0] = '\0'; X len = strlen(line); X X /* error checking */ X if (start > len || end > len) { X fprintf(stderr,"'%s' has %d columns. Specified were %d to %d\n", X line, len, start, start); X return NULL; X } X len = end - start + 1; X (void) strncpy(value, &line[start-1], len); X value[len] = '\0'; X X return value; X} END_OF_get_col_fld.c if test 787 -ne `wc -c get_rel_fld.c <<'END_OF_get_rel_fld.c' X/* X get_rel_field: we have a relative output format and want a certain X field from it. Place the field in 'value' if it exists and return a X pointer to the string. If there are problems place a null terminated X zero length string in 'value' and return NULL as an error condition. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xchar * Xget_rel_field(line, field, value) Xchar *line, *value; Xint field; X{ X int i; X int nfields; X char *vec[MAX_VEC]; X X value[0] = '\0'; X X /* break up the line */ X nfields = line_to_vec(line, vec, " \t"); X X /* error checking */ X if (nfields <= 0) { X fprintf(stderr, "'%s' didn't break up into fields.\n",line); X return NULL; X } X if (nfields < field) { X fprintf(stderr, "Warning: '%s' has only %d fields, not %d\n", X line, nfields, field); X return NULL; X } X X (void) strcpy(value, vec[field-1]); X X for (i=0; igetargv.c <<'END_OF_getargv.c' X/* X getargv: get a value from argv, whether it is immediately following X the flag or is the next argument. Complain if not found and exit. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xgetargv(what, argv, i, err) Xchar *what, *argv[], *err; Xint i; X{ X if (argv[i][2]) X (void) strcpy(what, &argv[i][2]); X else { X if (argv[++i]) X (void) strcpy(what, argv[i]); X else { X fprintf(stderr,"Missing %s!\n", err); X exit(1); X } X } X return i; X} END_OF_getargv.c if test 495 -ne `wc -c init.c <<'END_OF_init.c' X/* X init: initialize any variables needing values, do any other X initialization needed. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xinit() X{ X extern char ostrval[], *strval; X X ostrval[0] = '\0'; X strval = ostrval; X init_sigs(); X open_cf(); X} END_OF_init.c if test 296 -ne `wc -c init_sigs.c <<'END_OF_init_sigs.c' X/* X init_sigs: take care of setting up the signal handling. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xinit_sigs() X{ X int exit(), baderr(); X X (void) signal(SIGINT, exit); X (void) signal(SIGHUP, exit); X (void) signal(SIGQUIT, baderr); X (void) signal(SIGSEGV, baderr); X (void) signal(SIGBUS, baderr); X (void) signal(SIGFPE, baderr); X (void) signal(SIGILL, baderr); X#ifdef BSD X (void) signal(SIGTTIN, baderr); X (void) signal(SIGTTOU, baderr); X#endif X} END_OF_init_sigs.c if test 507 -ne `wc -c line_to_vec.c <<'END_OF_line_to_vec.c' X/* X line_to_vec: take a string and separate it into a vector of strings, X splitting it at characters which are supplied in 'splits'. X Ignore any 'splits' at the beginning. Multiple 'splits' are X condensed into one. Splits are discarded. X X Assumptions: X line is null terminated. X no single word is longer than MAX_STR. X X Arguments: X line: line to split. X vec: split line. X splits: array of characters on which to split. Null terminated. X X Global data used: X none. X X Local variables: X len: length of 'word' so far. X name: current entry in the vector we are building. X word: pointer into name. X X Returns: X The number of vectors created; -1 if malloc fails. X The argument vec is left with a NULL pointer after the last word. X X Author: X Kenneth Ingham X X Date: X Thu Sep 5 13:59:21 MDT 1985 X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xline_to_vec(line, vec, splits) Xchar *line, *vec[], *splits; X{ X register int i, v, j; X register unsigned len; X int n; X char word[MAX_STR]; X X if (line == NULL || line[0] == '\0') X return 0; X X /* skip any splits in the beginning */ X for (i=0; line[i] && index(splits, line[i]) != 0; i++) X ; X X j = 0; X len = 0; X v = 0; X n = 0; X while (line[i]) { X if (index(splits, line[i]) != 0) { X word[j] = '\0'; X vec[v] = malloc(len+1); X if (vec[v] == NULL) X return -1; X strcpy(vec[v], word); X j = 0; X len = 0; X v++; X n++; X i++; X for ( ; line[i] && index(splits, line[i]) != 0; i++) X ; X } X else { X word[j++] = line[i++]; X len++; X } X } X X if (index(splits, line[i]) != 0) { X word[j] = '\0'; X vec[v] = malloc(len+1); X if (vec[v] == NULL) X return -1; X strcpy(vec[v], word); X n++; X } X X return n; X} END_OF_line_to_vec.c if test 1705 -ne `wc -c main.c <<'END_OF_main.c' X/* X main: main routine for the watcher program. X X read from a file describing commands (pipelines) to execute, formats of X the output, max changes allowed (% or absolute), and max & min X values for various fields. X problems noticed are reported. X as a side effect, be able to pretty print the description file X (originally use to verify parsing). X X format: X (command)\tformat : X \tfield\tchange\tmax\tmin X . X . X . X X See yacc file for complete grammar description of control file. X X outline of program: X parse control file and build data structures. X run each pipeline and compare output to previous output (only X save relevant fields; save directory is either default X or command line specified; no previous file or format X changed (ie we are watching different or new fields) we X create new file and next time we do compare). X differences that are not allowable are reported. X X Usage of program: X watcher [-p] [-v] [-h histfile] [-f controlfile] X X -p : pretty print control file as a verification of parse X (default no pretty print). This option prevents X processing of control file. X -v : be verbose when doing work; useful for debugging. X -h : file in which to save output for future compare (default X ./watcher.history). X -f : controlfile to use (default ./watcherfile or ./Watcherfile). X X Note that the basic data structures are all linear linked lists, with X many items in the list being heads of other lists. When problems X occur, get out the pencil and paper and start drawing the lists. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xmain(argc, argv) Xint argc; Xchar *argv[]; X{ X extern int parse_error; X extern struct cmd_st *clist; X extern int pflag; X X do_args(argc, argv); X init(); X X if (yyparse() == 1 || parse_error) { X fprintf(stderr, "%s: parse error in control file.\n", NAME); X exit(1); X } X X if (clist == NULL) { X fprintf(stderr, "No command list to execute!\n"); X exit(1); X } X X if (pflag) X pp(clist); X else { X read_hist(); X open_hf(); /* for writing our history */ X doit(); X } X} END_OF_main.c if test 2114 -ne `wc -c open_cf.c <<'END_OF_open_cf.c' X/* X open_cf: open the control file. It is either specified on the X command line or DEF_CONTROL or DEF_CONTROL2. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xopen_cf() X{ X extern char controlname[]; X extern FILE *cf; X extern int cflag; X X if (cflag) { /* specified control file */ X cf = fopen(controlname, "r"); X if (cf == NULL) { X fprintf(stderr, "Unable to open '%s'\n", X controlname); X exit(1); X } X } X else { /* try defaults */ X if ((cf = fopen(DEF_CONTROL, "r")) == NULL) { /* #1 */ X if ((cf = fopen(DEF_CONTROL2, "r")) == NULL) { /*#2*/ X fprintf(stderr, "Unable to open %s or %s\n", X DEF_CONTROL, DEF_CONTROL2); X exit(1); X } X /* if we're here #2 must have worked */ X (void) strcpy(controlname, DEF_CONTROL2); X } X else X /* if we're here #1 must have worked */ X (void) strcpy(controlname, DEF_CONTROL); X } X} END_OF_open_cf.c if test 897 -ne `wc -c open_hf.c <<'END_OF_open_hf.c' X/* X open_hf: open the history file for writing out log of what we looked X at this run. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xopen_hf() X{ X extern char histfilename[]; X extern FILE *hf; X X hf = fopen(histfilename, "w"); X if (hf == NULL) { X fprintf(stderr, "Unable to open '%s'\n", X histfilename); X exit(1); X } X} END_OF_open_hf.c if test 373 -ne `wc -c pp.c <<'END_OF_pp.c' X/* X pp: pretty print the command structure. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xpp(clist) Xstruct cmd_st *clist; X{ X extern char histfilename[]; X extern char controlname[]; X X printf("History file name: %s\n", histfilename); X printf("Control file name: %s\n", controlname); X printf("\n\n"); X X while (clist != NULL) { X printf("( %s )\n", clist->pipeline); X pp_out(clist->out_type, clist->out_fmt); X printf(" :\n"); X pp_change(clist->change_fmt); X clist = clist->next; X } X} END_OF_pp.c if test 534 -ne `wc -c pp_change.c <<'END_OF_pp_change.c' X/* X pp_change: pretty print the change format. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xpp_change(cf) Xstruct change_fmt_st *cf; X{ X while (cf != NULL) { X switch(cf->type) { X case PERCENT: X printf("\t\t%s %5.2f %%", cf->name, X cf->fmt.percent*100); X break; X case ABSOLUTE: X printf("\t\t%s %6.2f", cf->name, X cf->fmt.abs_amount); X break; X case MAX_MIN: X printf("\t\t%s %6.2f %6.2f", cf->name, X cf->fmt.max_min.min, X cf->fmt.max_min.max); X break; X case STRING: X printf("\t\t%s \"%s\"", cf->name, X cf->fmt.str_value); X break; X default: X printf("Impossible change format type: %d\n", X cf->type); X break; X } X if (cf->next != NULL) X printf(" ;\n"); X else X printf(" .\n"); X cf = cf->next; X } X} END_OF_pp_change.c if test 813 -ne `wc -c pp_out.c <<'END_OF_pp_out.c' X/* X pp_out: pretty print the output of command structure. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xpp_out(type,of) Xint type; Xunion out_fmt_u of; X{ X struct rel_out_st *rp; X struct col_out_st *cp; X X switch(type) { X case RELATIVE: X rp = of.rel_fmt; X while (rp != NULL) { X printf(" %d %s %% %c", rp->field, rp->name, X TCHAR(rp->type)); X rp = rp->next; X } X break; X case COLUMN: X cp = of.col_fmt; X while (cp != NULL) { X printf(" %d - %d %s %% %c", cp->start, X cp->end, cp->name, TCHAR(cp->type)); X cp = cp->next; X } X break; X default: X printf("Impossible value for outfmt type: %d\n", type); X break; X } X} END_OF_pp_out.c if test 696 -ne `wc -c read_hist.c <<'END_OF_read_hist.c' X/* X read_hist: open the file containing the results of our last run. If X it is not there then we assume that this is a first run and set the X head of the list to NULL. Otherwise, we read the results of the previous X run into a mess of a data structure for later use in comparisons. X X Assumed format of history file: X X command X key X name type value X X witht the following definitions: X command: pipeline that was executed X key: value of key on line X name: output field name X type: field type (same as defined in output format) X value: what was in the field. X X We create a linked list of linked lists of linked lists. Improvement X would be to change to a tree of some sort to speed up searches. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xread_hist() X{ X extern char histfilename[]; X extern FILE *hf; X extern int vflag; X extern struct old_cmd_st *chead; X X char line[MAX_STR]; X struct old_cmd_st *cp; X struct val_st *vp; X struct key_st *kp; X int len; X char *sp; X X if (vflag) X printf("Using %s for historyfile\n", histfilename); X X hf = fopen(histfilename, "r"); X if (hf == NULL) { X if (vflag) X printf("This is a first run.\n"); X chead = NULL; X return; X } X X chead = NULL; cp = NULL; kp = NULL; X while (fgets(line, MAX_STR, hf) != NULL) { X line[strlen(line)-1] = '\0'; /* kill trailing cr */ X if (line[0] != '\t') { /* command */ X if (chead == NULL) { X cp = (struct old_cmd_st *)malloc(sizeof(struct old_cmd_st)); X chead = cp; X } X else { X cp->next = (struct old_cmd_st *)malloc(sizeof(struct old_cmd_st)); X cp = cp->next; X } X X cp->pipeline = malloc((unsigned)strlen(line)+1); X (void) strcpy(cp->pipeline, line); X cp->next = NULL; X } X else if (line[0] == '\t' && line[1] != '\t') { /* key */ X if (cp == NULL) { X printf("Bad history file: keyword found "); X printf("before pipeline. Ignoring history "); X printf("file.\n"); X chead = NULL; X return; X } X if (cp->keys == NULL) { X cp->keys = (struct key_st *)malloc(sizeof(struct key_st)); X kp = cp->keys; X } X else { X kp->next = (struct key_st *)malloc(sizeof(struct key_st)); X kp = kp->next; X } X kp->next = NULL; X kp->key_value = malloc((unsigned)strlen(line)+1); X (void) strcpy(kp->key_value, &line[1]); X } X else if (line[0] == '\t' && line[1] == '\t') { /* vals */ X if (kp == NULL) { X printf("Bad history file: value found "); X printf("before keyword. Ignoring history "); X printf("file.\n"); X chead = NULL; X return; X } X if (kp->vals == NULL) { X kp->vals = (struct val_st *)malloc(sizeof(struct val_st)); X vp = kp->vals; X } X else { X vp->next = (struct val_st *)malloc(sizeof(struct val_st)); X vp = vp->next; X } X vp->next = NULL; X sp = index(&line[2], ' '); X len = sp - &line[2]; X vp->name = malloc((unsigned)len+1); X (void) strncpy(vp->name, &line[2], len); X vp->name[len] = '\0'; X sp++; X len = strlen(line) - len; X switch (*sp) { X case 's': X vp->val.strval = malloc((unsigned)len+1); X (void) strcpy(vp->val.strval, sp+2); X vp->type = STRING; X break; X case 'd': X vp->val.floatval = atof(sp+2); X vp->type = FLOAT; X break; X default: X /* bad condition */ X printf("Unknown data type in history file.\n"); X printf("Offending line: %s\n",line); X exit(1); X break; X } X } X } X (void) fclose(hf); X} END_OF_read_hist.c if test 3404 -ne `wc -c save_key.c <<'END_OF_save_key.c' X/* X save_key: save the key for this line in the history file so that X we can use it next run. X X Assume that the keyory file is opened and that the pipeline has X already been placed in the file. We place the data in the file in X the following format: X X pipeline X key_value X name value X . X . X . X . X . X . X pipeline X . X . X . X X X Side effect: return the actual key value for other parts of the X program to use. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xsave_key(cmd, line, key_val) Xstruct cmd_st *cmd; Xchar *line, *key_val; X{ X extern int vflag; X extern FILE *hf; /* assumed to be already open */ X X if (cmd->key.col_fmt == NULL) { /* no key; no reason to save. */ X if (vflag) X printf("%s has no key field.\n",cmd->pipeline); X return; X } X X switch (cmd->out_type) { X case RELATIVE: X (void) get_rel_field(line, cmd->key.rel_fmt->field, X key_val); X break; X case COLUMN: X (void) get_col_field(line, cmd->key.col_fmt->start, X cmd->key.col_fmt->end, key_val); X break; X } X X if (!key_val[0]) { X if (vflag) X printf("the key field for %s is empty\n",cmd->pipeline); X return; X } X X fprintf(hf, "\t%s\n",key_val); X} END_OF_save_key.c if test 1239 -ne `wc -c yyerror.c <<'END_OF_yyerror.c' X/* X wow. X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xyyerror(s) Xchar *s; X{ X fprintf(stderr,"%s\n", s); X} END_OF_yyerror.c if test 138 -ne `wc -c yylex.c <<'END_OF_yylex.c' X/* X yylex for watcher: this is a simple routine looking for numbers, X special characters and strings. The special chars are stored in X 'words' and represent tokens by themselves. In y.tab.h are the X values to return for the various tokens which are not listed in X 'words'. X X Kenneth Ingham X X Copyright (C) 1987 The University of New Mexico X*/ X X#include "defs.h" X Xchar words[] = "\".*|;:%@$-{}"; X Xyylex() X{ X extern int yylval, intval, ointval; X extern char *strval, ostrval[], pipeline[]; X extern FILE *cf; X int c, value, i; X static char str[MAX_STR]; X X while (isspace(c = getc(cf))) X ; X X if (c == EOF) X return EOF; X X if (c == '(') { /* aha, pipeline */ X c = getc(cf); X for (i=0; c != EOF && c != ')'; i++) { X str[i] = c; X c = getc(cf); X } X str[i] = '\0'; X if (c == EOF) { X fprintf(stderr, "Missing ')' to end pipeline.\n"); X return EOF; X } X (void) strcpy(pipeline, str); X return PIPELINE; X } X X if (c == '#') { /* comment to end of line */ X while (c != '\n' && c != EOF) X c = getc(cf); X if (c == EOF) X return EOF; X return yylex(); X } X X if (index(words, c) != 0) { X yylval = c; X return c; X } X X if (isdigit(c)) { /* a number */ X value = c - '0'; X while (isdigit(c = getc(cf))) X value = value * 10 + c - '0'; X ointval = intval; X intval = value; X (void) ungetc(c, cf); X return NUMBER; X } X X (void) strcpy(ostrval, strval); X X if (c == '\'') { /* literal string */ X c = getc(cf); X for (i=0; c != EOF && c != '\''; i++) { X str[i] = c; X c = getc(cf); X } X str[i] = '\0'; X strval = str; X return STRING; X } X X /* nothing else matched. Must be plain string (whitespace sep) */ X for (i=1, str[0]=c; c != EOF && !isspace(c) && !index(words,c); i++) { X c = getc(cf); X str[i] = c; X } X (void) ungetc(c, cf); X str[i-1] = '\0'; X strval = str; X return STRING; X} END_OF_yylex.c if test 1804 -ne `wc -c