From decwrl!sun-barr!cs.utexas.edu!uunet!allbery Sat Oct 28 16:25:51 PDT 1989 Article 1143 of comp.sources.misc: Path: decwrl!sun-barr!cs.utexas.edu!uunet!allbery From: allbery@uunet.UU.NET (Brandon S. Allbery - comp.sources.misc) Newsgroups: comp.sources.misc Subject: v08i096: libhoward portability library, part 1 of 9 (repost) Message-ID: <70934@uunet.UU.NET> Date: 28 Oct 89 18:13:01 GMT Sender: allbery@uunet.UU.NET Reply-To: howard@dahlbeck.ericsson.se (Howard Gayle) Lines: 1206 Approved: allbery@uunet.UU.NET (Brandon S. Allbery - comp.sources.misc) Posting-number: Volume 8, Issue 96 Submitted-by: howard@dahlbeck.ericsson.se (Howard Gayle) Archive-name: libhoward/part01R #! /bin/sh # This is a shell archive. Remove anything before this line, then feed it # into a shell via "sh file" or similar. To overwrite existing files, # type "sh file -c". # The tool that generated this appeared in the comp.sources.unix newsgroup; # send mail to comp-sources-unix@uunet.uu.net if you want that tool. # If this archive is complete, you will see the following message at the end: # "End of archive 1 (of 9)." # Contents: README freezePch0.b hetero.tex # Wrapped by howard@hasse on Mon Sep 25 07:08:04 1989 PATH=/bin:/usr/bin:/usr/ucb ; export PATH if test -f 'README' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'README'\" else echo shar: Extracting \"'README'\" \(10030 characters\) sed "s/^X//" >'README' <<'END_OF_FILE' X$Header: README,v 1.3 89/09/23 15:25:11 howard Exp $ X X DESCRIPTION X XThis is my C function library, collection of C include files, Xcollection of TeX and LaTeX include files, and set of useful Xlittle shell files. I use it in essentially all my software. XThe C library includes macros and functions for portability, Xwriting error messages, safely converting numbers to internal Xrepresentations, safely reading lines of text, performing Xvarious common functions with full error checking, some string Xfunctions, etc. There are shell files for supporting my scheme Xfor software sharing in a network of heterogeneous UNIX hosts. XThis is described in a LaTeX report. X X X RATIONALE X XMany people make each software system they release stand-alone, Xby duplicating code for common C functions. I decided instead Xto put the C functions I use over and over into a library, for Xthese reasons: X 1) In a library it is easier for others to use them. They X are all documented. X 2) Net bandwidth is saved by posting them only once. X 3) Now that some UNIXes support shared libraries (e.g. SunOS X 4.x), I want to use them. Shared libraries reduce the X sizes of executable files, and they make it possible to X fix bugs in library functions without recompiling all X programs that use them. X X X PREREQUISITES X XANSI-C float.h and limits.h include files must be on the C Xinclude file search path. If you don't have them, you can use XSteven Pemberton's config program to make one. It was posted to Xcomp.sources.misc on 30 April 1989 as volume 6 issue 96, archive Xname config2. X XThe makefiles use features that may only be in GNU make, version X3.54 or later. X XChris Tweed's sets command must be on the command search path. XIt was posted to comp.sources.unix on 24 February 1988 as volume X13, issue 68, archive name sets. X XSome of the documentation is in LaTeX. X XThe Bourne shell scripts use if [...] syntax instead of explicit Xcalls to test. You might need to link test to [ to make this Xwork. X X X INSTALLATION X XIt will be much easier to understand the installation if you Xfirst read my report. It is in the LaTeX source file Xhetero.tex. If you have LaTeX installed, you can format it like Xthis. X % latex hetero X % bibtex hetero X % latex hetero X % latex hetero X (print hetero.dvi) X XFigure out where you want to install various files. Here I'll Xassume a separate directory for everything, but it's easy to Xmake a different choice. I discuss an alternative example Xbelow. X XCreate a directory for compiling on this machine type. I'll Xcall it /usr/local/free/howard. X XFind out the version. Look at the Header line (typically the Xfirst line) of the FREEZE file. The version is the first Xnumber, and ends at the period. X XCreate a subdirectory for this version. e.g. X/usr/local/free/howard/1. X XCreate the following subdirectories for results: X bin Installed Bourne shell scripts. X include/howard Installed include files. X man/man1 Manual entries. X man/man3 X man/cat1 On-line formatted manual entries, X man/cat3 for systems with e.g. catman. X tex Installed TeX & LaTeX macro files. X XFor example: X % cd /usr/local/free/howard/1 X % mkdir bin include include/howard man man/{cat,man}{1,3} tex X XCreate a subdirectory for doing the actual compilations. I'll Xcall it sun34: X % mkdir sun34 X XBootstrap the Bourne shell scripts into their installation Xdirectory. Let $DD be where the master source is: X % cp $DD/*.b bin X % cd bin X % /bin/ls -1 | sed -e 's;\.b$;;' | xargs -i= -t mv =.b = X % chmod +x * X % cd .. X XIf you want to override the behavior of any of the Bourne shell Xscripts for portability or because of different local policies, Xthis is a good time to do so. I suggest putting locally changed Xcopies in a separate bin directory that is on the command search Xpath *before* the bin directory in which the scripts will be Xinstalled, e.g. create /usr/local/free/howard/1/local-bin, and Xput it on the command search path before X/usr/local/free/howard/1/bin. Here is a guide to possible local Xchanges: X X mkAr This makes the ar archive library. It is very BSDish X and must change for the `lorder $* | tsort` crowd. X X mkCIncl These both assume that calling /lib/cpp -M is the X mkLibIncl way to find out what include files a C source X file uses. Change them if that's wrong for your X system. X X mkDistI This is how files get from the master source X directory to the compilation directory. I use X symbolic links. X X mkInstF This is how non-executable files get installed. I X use hard links. Shell scripts count as X non-executable. X X mkInstX This is how executable files get installed. I X use hard links. This would be the place to strip X symbol tables. Shell scripts are *not* considered X to be executable. X X mkLaTeXPS This formats a LaTeX file, including BibTeX X processing, all the way to a printable X representation, which for me is PostScript. If X you want some other final format, I would suggest X writing a separate shell script. X X mkManPS This formats a manual entry to PostScript, using X TransScript. If you have some other way to format X manual entries, I would suggest writing a separate X shell script. X XIf you write original software with these tools, you may also Xwant to change mkDistO, which places files in the distribution Xdirectory. I use hard links for this. Also, the following are Xwritten for use with RCS, and would have to be changed for use Xwith SCCS: cia, freeze, freeze0, freezePch, freezePch0, XmkFreeze, mkUncmprs. X XSet up the command search path to pick up the Bourne shell Xscripts. Rehash if necessary. X XChange directory into the actual compilation directory, e.g. cd Xsun34. X XCreate an mk shell file. Make it executable. Here's a starting Xpoint for SunOS 4.x: X XDD=/usr/local/free-dist/howard/1/dist XSRCS=`cd $DD; echo *` XRM='rm -f' Xexport DD SRCS RM Xmake -f uMakefile Xexec make \ X CATMAN=catman \ X FMTLATEX=mkLaTeXPS \ X FMTMAN=mkManPS \ X SLT=libhoward.so.`cat FREEZE.version` \ X $* X X XIf you don't have a catman command, drop that line. FMTLATEX Xshould be the name of a command to turn LaTeX source into Xwhatever your printer prints; if you don't have one just drop Xit. Likewise, FMTMAN should be the name of a command to turn Xmanual entries into printable form. If you don't have shared Xlibraries, drop the SLT= line. Add any local changes you need Xhere. If your C compiler doesn't understand different Xoptimization levels, add CFLAGS=-O as an argument to the last Xmake. X XHere is an alternate mk file for a system using GCC, and with Xresults installed in more conventional places: X XDD=/usr/local/free-dist/howard/1/dist XSRCS=`cd $DD; echo *` XCC=gcc XCFLAGS='-g -O -traditional' XINCLUDES=-I/usr/local/include XRM='rm -f' Xexport DD SRCS CC CFLAGS INCLUDES RM Xmake -f uMakefile Xexec make \ X AID=/usr/local/lib \ X BID=/usr/local/bin \ X CATMAN=catman \ X HID=/usr/local/include/howard \ X LIBPATH='-L/usr/local/lib' \ X LNID=/usr/local/lib/lint \ X MID=/usr/local/man \ X TESTCFLAGS='-g' \ X TEXID=/usr/local/tex \ X $* X XBe sure that the parent directory of the directory in which the Xinclude files are installed (HID) is on the include file search Xpath (INCLUDES). X XGet MakeCommon and uMakefile from the master source directory, Xe.g. X % mkDistI MakeCommon $DD X % mkDistI uMakefile $DD X XRun X % mk vars XThis will produce a huge number of error messages about missing Xinclude files. Just ignore them. It may also produce C Xcompiler error messages when mk-cc-lims runs. Ignore them too: Xmk-cc-lims is just trying to figure out what your C compiler can Xand can't do. It will also echo the values of a bunch of make Xvariables. That's normal. X XCheck out all the generated include files (*.i). Make sure they Xlook reasonable. X XInstall the include files: X % mk install-include X XNow, give mk a chance to build the depend file correctly: X % rm -f depend X % mk XThis should make everything. There should *not* be any error Xmessages about missing include files. X XYou can do some testing now if you like. Each C function comes Xwith a built-in test driver. For example, you can test a2ul() Xlike this: X % mk a2ul-t X % a2ul-t 10 XRunning "mk test" will compile all the test drivers. A few of Xthe test drivers even do marginally useful things. X XRun "mk install" and you're finished. X X X ANOMALIES X XGNU make is still (at version 3.54) somewhat buggy. It does not Xalways pick up environment variables, so it is sometimes Xnecessary to rewrite mk to pass them as command line arguments Xinstead. Run "mk vars" to check that everything is all right. X XGNU make also pollutes the environment with all make variables. XThis can be too much for some of the commands it runs. One way Xaround this is to replace command foo by "env - foo". I needed Xto do this for ranlib in SunOS 4.0.1. It was OK in 4.0.3. XAlso, the Bourne shell sometimes seems to suffer from an excess Xof environment variables. Using the C shell instead is a Xpossible workaround, e.g. pass SHELL=/usr/bin/csh on the command Xline. (IMHO, GNU make needs some way to indicate which Xvariables should and should not be exported, e.g. something like X"export" in the Bourne shell.) X XThe documentation for atof(3) says errno is set to ERANGE on Xoverflow and underflow. In SunOS 4.0.1, and perhaps other Xsystems, errno is also sometimes set even without overflow, e.g. Xon an input of 0. As a workaround, pass DEFS=-DBADATOF to the Xlast make in mk. This skips the check of errno. Unfortunately, Xthis means overflow is *never* detected. This bug *seems* to Xhave been fixed in SunOS 4.0.3. X-- XHoward Gayle XTN/ETX/T/BG XEricsson Telecom AB XS-126 25 Stockholm XSweden Xhoward@ericsson.se Xuunet!ericsson.se!howard XPhone: +46 8 719 5565 XFAX : +46 8 719 9598 END_OF_FILE if test 10030 -ne `wc -c <'README'`; then echo shar: \"'README'\" unpacked with wrong size! fi # end of 'README' fi if test -f 'freezePch0.b' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'freezePch0.b'\" else echo shar: Extracting \"'freezePch0.b'\" \(1076 characters\) sed "s/^X//" >'freezePch0.b' <<'END_OF_FILE' X X# freezePch0.b - auxiliary shell script for freezePch X# X# $Header: freezePch0.b,v 1.1 89/09/23 10:39:47 howard Exp $ X# X# Copyright 1989 Howard Lee Gayle X# This file is written in the ISO 8859/1 character set. X# X# This program is free software; you can redistribute it and/or modify X# it under the terms of the GNU General Public License version 1, X# as published by the Free Software Foundation. X# X# This program is distributed in the hope that it will be useful, X# but WITHOUT ANY WARRANTY; without even the implied warranty of X# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the X# GNU General Public License for more details. X# X# You should have received a copy of the GNU General Public License X# along with this program; if not, write to the Free Software X# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. X# X# Shell variables: X# u - usage string X Xu="Usage: freezePch0 file old-revision new-revision" Xif [ $# -ne 3 ] Xthen X echo "$u" 1>&2 X exit 1 Xfi Xecho '' Xecho "patch -c $1 << 'EOF'" Xrcsdiff -c -r"$2" -r"$3" "$1" | sed -e '1,2d' Xecho EOF END_OF_FILE if test 1076 -ne `wc -c <'freezePch0.b'`; then echo shar: \"'freezePch0.b'\" unpacked with wrong size! fi # end of 'freezePch0.b' fi if test -f 'hetero.tex' -a "${1}" != "-c" ; then echo shar: Will not clobber existing file \"'hetero.tex'\" else echo shar: Extracting \"'hetero.tex'\" \(31934 characters\) sed "s/^X//" >'hetero.tex' <<'END_OF_FILE' X% hetero.tex - configuration management in a heterogeous net X% X% Copyright 1989 Howard Lee Gayle X% This file is written in the ISO 8859/1 character set. X% X% $Header: hetero.tex,v 1.2 89/09/20 13:03:18 howard Exp $ X% X% This program is free software; you can redistribute it and/or modify X% it under the terms of the GNU General Public License version 1, X% as published by the Free Software Foundation. X% X% This program is distributed in the hope that it will be useful, X% but WITHOUT ANY WARRANTY; without even the implied warranty of X% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the X% GNU General Public License for more details. X% X% You should have received a copy of the GNU General Public License X% along with this program; if not, write to the Free Software X% Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. X\documentstyle[11pt,openbib]{report} X\input gold-lt X\input latin-lt X\input unix-lt X\global \def \today {$Revision: 1.2 $} X\raggedright X\arraycolsep 0pt X\parskip 1.5ex X\parindent 2em X\title{Sharing software in a network of heterogeneous UNIX hosts} X\author{Copyright \copyright\ 1989 Howard Lee Gayle X\thanks{ XTN/ETX/T/BG, XEricsson Telecom~AB, XS-126~25 Stockholm, XSweden, Xhoward@ericsson.se, Xuunet!ericsson.se!howard, XPhone: +46~8~719~5565, XFAX: +46~8~719~9598, XTelex: 14910~ERIC~S.}} X\begin{document} X\maketitle X\begin{abstract} XMany organizations now have networks of heterogeneous XUNIX hosts, Xbut most UNIX software comes in a form Xmost suited to installing on a single, stand-alone machine. XThis report describes mechanisms and tools for managing XUNIX software in a network of heterogeneous UNIX hosts. XAll the tools are available as free software. XI address both ``making the best of'' existing software as it Xis distributed, and writing new software. X\end{abstract} X\tableofcontents X X\chapter*{License} XThis program is free software; you can redistribute it and/or modify Xit under the terms of the GNU General Public License version~1, Xas published by the Free Software Foundation. X XThis program is distributed in the hope that it will be useful, Xbut WITHOUT ANY WARRANTY; without even the implied warranty of XMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the XGNU General Public License for more details. X XYou should have received a copy of the GNU General Public License Xalong with this program; if not, write to the Free Software XFoundation, Inc., 675~Mass~Ave, Cambridge, MA~02139, USA. X X\chapter{Introduction} XMany organizations now have networks of heterogeneous X\UNIX\ hosts.\footnote{Operating systems other than UNIX are Xoutside the scope of this report, but I conjecture that similar Xtechniques could be used for some of them.} XBy heterogeneous, I mean the hosts have different computer Xarchitectures, run different versions of the \UNIX\ system, or Xboth. XFor example, an organization may have on its network Sun-3s Xand Sun-4s running SunOS~4.0, Xand VAXen running BSD~4.3. X XMost \UNIX\ software, as distributed, comes in a form Xmost suited to installing on a single, stand-alone machine. XThis report describes mechanisms and tools for managing X\UNIX\ software in a network of heterogeneous \UNIX\ hosts. XAll the tools are available as free software. XI address both ``making the best of'' existing software as it Xis distributed, and writing new software. XThis report is not about how to write portable software; Xit {\em is} about what to do with portable software once you Xhave it. X(For an excellent treatment of how to write portable \UNIX\ Xsoftware, see X\cite{Lapin}.) X X\chapter{Notation} XIn this report, a X{\bf system type}, Xor often simply X{\bf type}, Xis a specific computer Xarchitecture\footnote{Exactly what constitutes an architecture Xdepends on one's purpose. XHere I mean the architecture as seen by the compiler. XThus, for user software, a Sun~3/60 and a Sun~3/80 have the Xsame architecture.} Xrunning a specific version of \UNIX. XIn my examples I will use abbreviations for types such as Xthese: X\begin{description} X\item[sun34] A Sun-3 running SunOS~4.$x$. X\item[sun44] A Sun-4 running SunOS~4.$x$. X\item[vax43] A DEC VAX-11 running BSD~4.3. X\end{description} XThese are merely for illustration; each organization must Xchoose suitable abbreviations. X XA X{\bf software system}, Xor often simply X{\bf system}, Xis a set of software that is managed as a single unit, Xis installed as one package, Xand is supported by one X{\bf team}. XA system could be anything from a one-line shell file on up. XA team has one or more individual members. X XA X{\bf version} Xis one particular implementation of a system. XDifferent versions can be the result of changes over time, but Xparallel versions can also exist. XFor example, GNU Emacs has gone through versions 18.52, 18.53, X18.54, X\etcP. XAt the same time, one could have parallel versions, say for XJapanese, denoted 18.52-ja, 18.53-ja, 18.54-ja, X\etcP. X XA X{\bf virtual host} Xis an alias for some host, based on a unique service that host Xprovides. XVirtual hosts are implemented by normal host aliasing Xmechanisms, X\eg\ CNAME resource records in a domain name server. XFor example, a virtual host named ``dist'' might be used for Xmaster source distribution. X X\chapter{Goals} XMy goals are: X\begin{enumerate} X\item There should be a single master copy of the source for Xeach system. X\item It should be possible for a different team to support Xeach system. X\item The team supporting a system should have easy access to Xthe master source. X\item Subject to organizational policy and contractual obligations, Xall users Xshould have easy read-only access to the master source. X(See X\cite{Weiser:Source} Xfor an excellent discussion of how important this is.) X\item Only the team supporting a system and the superuser Xshould be able to change files associated with that system. XTeams should not be able (inadvertently) to damage files in Xsystems they do not support. X\item No version should need to be compiled more than once per Xtype. XThis means that once a version has been compiled for a type, Xall other hosts of that type should be able to share the Xbinaries.\footnote{Sharing can be by copying or by network X(\eg\ NFS) access to a single copy, or both. XEach organization must trade off disk space, execution time, Xand availability in the face of network faults. XA discussion of these tradeoffs is outside the scope of this Xreport. XThe mechanisms described in this report do not depend on any Xparticular kind of sharing.} X\item For each system, it should be easy to find out Xwhat support, if any, is provided. X\item For supported systems, it should be easy to Xcontact the support teams. X\item When feasible, it should be possible to have more than Xone version of a system installed simultaneously. XWhen a new version is installed, users should be able to Xcontinue to use an older version during a transition period. XUsers should be able to choose when to switch to a new version. X\end{enumerate} X X\chapter{Mechanisms} X\section{Master source virtual host(s)} XThe master source for each system is placed on exactly one Xvirtual host. XThe simplest scheme would be to put all systems on one virtual Xhost. X XAt Ericsson we use two virtual hosts: Xfree-dist for free software and public domain software, Xand unfree-dist for other software. XThis allows us to impose appropriate access controls to Xunfree-dist and yet give everyone easy access to free-dist. X X\section{Type virtual hosts} XEach type gets one or more virtual hosts for compiled versions Xfor that type. XAt Ericsson we use two virtual hosts per type: Xfree-{\it type\/} for free software and public domain software, Xand unfree-{\it type\/} for other software, Xwhere X{\it type\/} Xis one of the type abbreviations above. XFor example, the free-sun34 virtual host has compiled versions for XSun-3s running SunOS~4.$x$. X X\section{File system} X\subsection{Path to master source} XThe master source for each version gets its own Xdirectory on the appropriate master source virtual Xhost.\footnote{Many of the files in a version directory Xand its subdirectories may be links to files in other versions.} XThe absolute path to any version is X{\it source-prefix\/}/{\it system\/}/{\it version}, Xwhere X{\it source-prefix\/} is a fixed Xprefix, X{\it system\/} Xis the name of the software system, Xand X{\it version\/} Xis the version identifier. X XAt Ericsson there are actually two source-prefixes: X/usr/local/free-dist and /usr/local/unfree-dist. XFor example, the master source for GNU Emacs version~18.54 Xis on the free-dist virtual host in X/usr/local/free-dist/gnu-emacs/18.54. X X\subsection{Path to compiled version} XEach compiled version also gets its own directory on the Xappropriate type virtual hosts. XOn every host, the absolute path is X{\it compiled-prefix\/}/{\it system\/}/{\it version}. X XAt Ericsson, X{\it compiled-prefix\/} Xis either /usr/local/free or /usr/local/unfree. XFor example, the compiled GNU Emacs version~18.54 is in X/usr/local/free/gnu-emacs/18.54 Xon X{\em every} Xhost. X X\subsection{Currently recommended version} XCompiled version~0 is an abbreviation for the currently Xrecommended version on a given host. XThis can be implemented by a symbolic link. XFor example, a new version of GNU Emacs might be installed as Xfollows. XInitially, version~0 is the same as version~18.54. XVersion~18.55 is installed, but version~0 remains the same as Xversion~18.54. XUsers who wish to do so can now test version~18.55. XAfter a testing period, version~0 is changed to be the same as Xversion~18.55. XNow most users get version~18.55 by default. XHowever, version~18.54 is still in place, and users can use it Xif they wish. XEventually it will go away, to make room for a still later Xversion. X XThe version~0 abbreviation can also be used in master source to Xindicate the currently recommended version. X X\subsection{Contents of compiled version directory} XEach compiled version directory contains a complete set of Xfiles, including a bin subdirectory for executables and a man Xsubdirectory for on-line manual entries. XThis gives system administrators complete control over which Xcompiled versions they install on the hosts they support. XIt also gives users complete control over which versions they Xuse. X X\subsection{Default version selection} XIn principle, users can choose exactly which versions to use. XHowever, used alone, this mechanism can lead to long command Xsearch paths, long MANPATH environment variable values, Xlong .login files, X\etcP. XTherefore, most sites set up local bin, man, include, X\etc\ directories, X\eg\ /usr/local/bin, /usr/local/man, and /usr/local/include. XThe system administrators for a site place in these local Xdirectories the versions they expect most users will want. XThis can be done with symbolic or hard links, or by copying. XFor example, a site could have a symbolic link from X/usr/local/bin/emacs Xto X/usr/local/free/gnu-emacs/0/bin/emacs, Xand a symbolic link from X/usr/local/man/man1/emacs.1 Xto X/usr/local/free/gnu-emacs/0/man/emacs.1. XThen users who wish to use the versions selected by the system Xadministrators need only add /usr/local/bin to their command Xsearch paths and /usr/local/man to the values of their MANPATH Xenvironment variables. XTypically, the system Xadministrators would provide default shell Xinitialization files X(\eg\ .login) Xto accomplish this. X X\section{File distribution} XMaster source files must be made available to the hosts on Xwhich they are to be compiled. XAlso, compiled versions must be made available from the type Xvirtual hosts to all other hosts of that type. XThere are essentially three mechanisms available: X\begin{enumerate} X\item local file system sharing, X\item network copying, Xand X\item distributed file systems. X\end{enumerate} X XLocal file system sharing is possible when several virtual Xhosts are implemented by one physical host. XFor example, if the physical host that implements a master source Xvirtual host is a sun44, then the same physical host can Ximplement a sun44 type virtual host. XIn addition, the compiled versions the host makes available are Xexecutable by the host's own users. X XNetwork copying uses command such as rdist or ftp to copy files Xover a network.\footnote{Copying by other means, \eg\ tape, Xcould also be used, but is outside the scope of this report.} X XDistributed file systems make files on one host appear to be in Xthe file system of another host. XExamples include Sun's Network File System (NFS) and AT\&T's XRemote File System (RFS). X XAt Ericsson all three file distribution mechanisms are used. XThe same physical host currently implements the free-dist, Xunfree-dist, free-sun34, and unfree-sun34 virtual hosts. XNFS distribution is used to other hosts with NFS. XHosts without NFS use rdist. XSites are encouraged to have local copies of executables, not Xto NFS-mount them from type virtual machines. XThe reasons for this policy are to decrease response time, Xprovide better availability, decrease net traffic, and decrease Xload on the type hosts. X XOn every host with NFS, /net/free-dist and /net/unfree-dist Xaccess the master source, and /net/free and /net/unfree access Xthe appropriate type virtual hosts. XFor example, the master source for GNU Emacs version~18.44 is Xaccessible from all hosts with NFS Xin /net/free-dist/gnu-emacs/18.44; Xthe executables for hosts of Xthat type are in X/net/free/gnu-emacs/18.44, Xand the executables for the currently recommended version are Xin X/net/free/gnu-emacs/0. XHosts running Sun's automounter system use that for /net. XOther hosts with NFS but without automounter simply NFS-mount Xon subdirectories of /net. X X\section{Permissions} XEach system can be supported by a different team. XIn fact, each each (system,~type) pair can be supported by a Xdifferent team. XTeams can be implemented by \UNIX\ groups. X XAt Ericsson we use \UNIX\ groups only to implement multi-person Xteams, since the number of different groups to which one user can Xbelong is limited on many \UNIX\ systems. XSingle-person teams are not implemented in any special way. X X\section{Obtaining information about a system} XBefore picking up a compiled version of a software system, Xsystem administrators often want information such as: X\begin{enumerate} X\item What does the software system do? X\item What kind of support is provided? X\item How does one contact the support team? X\end{enumerate} XAt Ericsson, the support team places this information in a XREADME file directly under the system directory. XFor example, on any host with NFS, a system administrator can Xget this information about GNU Emacs by reading X/net/free-dist/gnu-emacs/README. XLack of a README file indicates total lack of support. X XOnce system administrators decide to pick up a version, they Xmust still know what files to copy where. XAt Ericsson, for each type virtual host, the support team Xplaces this information in a README file directly under the Xsystem directory. XFor example, on any host with NFS, a system administrator can Xget this information about GNU Emacs by reading X/net/free/gnu-emacs/README. XWe also use the type README files to give hints to system Xadministrators about what to tell end users. XOften these hints take the form of a news article for end users Xabout what the system does, what has changed since the previous Xversion, and how to use the system. XAgain, lack of a type README file indicates total lack of Xsupport for that system on that type. X X\chapter{Installing existing single-host software} XToday, most software systems are distributed for installation on a Xsingle host. XThe systems are often portable, but their installation tools X(\eg\ makefiles), instructions, and directory layouts assume a Xmodel of installation on a single, stand-alone host. XFor example, GNU Emacs is distributed with an etc subdirectory Xcontaining both source and auxiliary files. XThe source files are only needed for compilation, but the auxiliary Xfiles must be present on every host. XBecause they are in the same directory, it is inconvenient to Xcopy only the needed Xfiles.\footnote{Of course, all users should have X{\em access} to all GNU sources, but in many cases network Xaccess is perfectly adequate for source but not for executables.} X XThis chapter describes how to make the best of existing systems. X X\section{Master source for a new system} X\begin{enumerate} X\item Choose a software system name. XFor software distributed in moderated USENET source news groups Xsuch as comp.sources.misc, the archive name is often a good Xchoice. XIf any of the types are limited to 14~character file name Xsegments, then the system name should not exceed 14~characters. X\item This step is performed by the superuser. XFor a multi-person support team, create a corresponding X\UNIX\ group. XIt is also often convenient to create a mail alias to all Xmembers of the team, both for internal communication among team Xmembers, and to Xmake sure that all team members see problem reports. XFor example, the \TeX\ support team might have a bugs-tex mail Xalias. X\item This step is (usually) performed by the superuser. XCreate the master source top level directory. XFor a single-person team, make the owner of the directory the Xuser who will perform the support. XFor a multi-person team, make the group of the directory the Xgroup corresponding to the team, and make the directory Xgroup-writeable. XOn \UNIX\ versions that require the set-group-id bit to be set Xfor the directory in order that created files have the group of Xthe directory, set this bit. X\item Set the umask. XFor a single-person team, this would typically be~22; Xfor a multi-person team,~2. X\item Create a version directory. XIf no version is specified in the distributed software then Xversion~1 can be used. X\item If the installation requires any special actions, it is Xoften helpful to create a log file that can be used as a guide Xfor subsequent installations. XIt can be a file named ``log'' in the version directory. XI regard log files as ``revisionistic'' history: Xthey contain the commands I X{\em should} Xhave executed, not necessarily the commands I actually executed. X\item In the version directory, create a subdirectory to hold Xthe distributed source, if necessary. X(At Ericsson we use dist.) XLarger systems often already have source subdirectories. X\item Place the source in the directory. XIf necessary, uncompress compressed files, decode encoded Xfiles, X\etcP. X\item Remove write permission from all ordinary files, X\eg\ by running X\begin{verbatim} Xfind . -type f -print | xargs chmod a-w X\end{verbatim} XThis will reduce the risk of changing master source when Xediting in local changes. XIt also makes it easier to see which files have been changed locally. X\item If necessary, create a README file directly under the Xsystem directory, \eg\ /usr/local/free-dist/gnu-emacs/README. X\item If desired, set up version~0 to indicate the currently Xrecommended master source version. X\end{enumerate} X X\section{Patching a master source to a new version} XNew versions are often distributed as Xpatches\footnote{The word ``patch'' is misleading here. XThe term generally means changes made directly to object or Xmachine code, but in this case refers to source differences, Xusually the output of the diff command with the -c option for Xcontext diffs. XThese are usually installed using Larry Wall's patch program.} Xto older versions. X\begin{enumerate} X\item Set the umask. X\item Create a version directory for the new version. X\item Copy the file system hierarchy from the old version to Xthe new. XUsually, both versions will be on the same file system, so Xordinary files can be hard-linked (shallow copy). XMy lnR command can be used for this. X\item The old version may have been the result of applying Xpatches to a still older version. XIf so, the .orig and .rej files left from the previous patching Xprocess should be removed, X\eg\ by running X\begin{verbatim} Xfind . '(' -name '*.orig' -o -name '*.rej' ')' -print | xargs rm -f X\end{verbatim} X\item Apply the patches. XUsually, this is done with Larry Wall's patch program. XSince no local changes should ever be made to master source, Xall patches should succeed. X\item If desired, set up version~0 to indicate the currently Xrecommended master source version. XThis would typically be done after some testing. X\end{enumerate} X X\section{Compiling a version on a type virtual host} X\begin{enumerate} X\item This step is (usually) performed by the superuser. XCreate a top-level system directory with the appropriate Xowner, group, and mode. X\item Set the umask. X\item Copy the file system hierarchy from the master source Xversion to the version for this type. XIf hard or symbolic links can be used, my lnR command can be Xused. X\item Create a log file under the version directory to record Xdetails of the installation for future use. XIf there was already a log file in the master version, remove Xit. XIf this is not the first installation of a system, it is often Xconvenient to copy the log file from the previous installation, Xand then edit Xit.\footnote{I find GNU Emacs shell mode very convenient for Xthis operation. X(Actually I use Olin Shiver's cmushell mode.) XI just copy commands from the edited log file window to the Xshell window and execute them.} X\item Create any subdirectories needed, \eg\ bin, man, Xman/man1, man/cat1, \etcP. X\item Make any local changes needed. XIf the files are (hard or soft) links from the master source Xdirectory, the links first need to be broken. XMy copy-self command can be used for this. XIn any case, files to be changed must be made writeable before Xthey can be edited. XFor multi-person teams, they should be made group-writeable. XThe names of all changed files should be logged. XIt may also be useful to log the changes made. X\item If a new version of a previously installed system is Xbeing installed, the local changes to Xa file are often exactly the same as when a previous version Xwas installed. XIn this case, the diff command can be used to determine what Xthe previous changes were, and the patch command can be used to Xapply them. XThe patch command automatically breaks links. X\item When the only changes to a Makefile are to variable Xdefinitions, it is often more convenient to leave the Makefile Xunchanged and instead override the definitions with command Xline arguments to make. XThe exact command should be logged for future use. X\item Perform the system-specific installation procedures. XIt is often desirable to bypass the actual installation phase X(\eg\ make install), and instead do this part by hand. X\item If necessary, create a README file directly under the Xsystem directory. X\item After testing, set up version~0. X\end{enumerate} X X\chapter{New software} XIn this chapter I describe how I write software from scratch Xfor installation in a network of heterogeneous \UNIX\ hosts. XI first try to make the software itself as portable as possible. XThen, I use the same techniques as described above for Xexisting software, but much more automated. XAlso, I try to concentrate all type-specific information into a Xsingle file. XMy approach uses GNU make, but cake could be used Xinstead.\footnote{I chose GNU make over cake because GNU make Xhas more built-in functionality. XDoing the same things with cake would require forking child Xprocesses, and I thought that would be less efficient. XUnfortunately, GNU make has been rather buggy up to now, Xalthough version~3.54 seems {\em much} improved. XIf I were starting over, I might well choose cake instead. XI have used cake for years, and have only found two bugs during Xthat time.} X XIn essence, each version has a makefile that is identical Xacross all types. XFor development, the makefile retrieves source files from an XRCS Xsubdirectory.\footnote{SCCS could also be used. XThis would Xrequire changing some auxiliary shell files but should not Xaffect the makefiles. XI chose RCS because I like the user interface more than SCCS, Xand because RCS is free software whereas SCCS is not.} XAt release time, files are copied to a distribution directory. XFor installation on each type, files are automatically copied Xfrom the distribution directory. XThe makefile is identical for development and installation on Xall types. XThe only differences are in a small shell file that passes Xarguments to make. X XIn the rest of this chapter I discuss my approach in detail. X X\section{Files} XThe main makefile (Makefile) contains rules for creating most targets. XIt is assisted by a micro-makefile (uMakefile) Xthat makes any files needed before make can be run using the Xmain makefile, X\eg\ the depend file that lists which C executables depend on Xwhich include files. XThe micro-makefile also makes the main makefile, if necessary. XBoth the micro-makefile and the main makefile include a few Xcommon definitions from the MakeCommon file. XMakefile, uMakefile, and MakeCommon are stored under RCS, and are identical for Xdevelopment and for installation on all types. XA shell file named mk runs make twice, first using uMakefile, Xthen Makefile. XThere is a different mk for development and for installation on Xeach type. XAll type-specific information is in mk. X X\section{Development} XI do development in a directory named X{\it source-prefix\/}/{\it system\/}/{\it version}/work, Xwith RCS files in a subdirectory named RCS. X XHere is the mk file I use for development for cz: X\begin{verbatim} XDD='../dist' XSRCS=`cd RCS; echo * | sed -e 's;$; ;' -e 's;\.Z ; ;g' -e 's;,v ; ;g'` XINCLUDES='-I/usr/local/local-include -I../../../howard/0' XWORK=RCS Xexport DD SRCS INCLUDES WORK Xmake -f uMakefile Xexec make -k \ X CFLAGS='-g -Bstatic' \ X FMTLATEX=mkLaTeXPS \ X FMTMAN=mkManPS \ X LIBPATHS='-L/usr/local/free-dist/howard/0/work' \ X LL=../../../howard/0/work/llib-lhoward.ln \ X RCID=/usr/local/free-dist/cz/2/work \ X DEFS='-DDEBUG' \ X $* X\end{verbatim} XThe first line sets the distribution directory (see below) to ../dist. XThe second line sets the SRCS environment variable to a list of all Xthe source files in the RCS subdirectory, with RCS's ,v Xsuffixes stripped. XThe RCS files may be Xcompressed.\footnote{If disk space gets tight, it is convenient Xto be able to compress all the RCS files of master source that Xhas not been accessed for a while. XThe .DEFAULT rules in uMakefile and Makefile call the mkUncmprs Xshell script, which uncompresses RCS files if necessary and Xthen checks out the latest revisions. XThe .DEFAULT feature was another reason I chose GNU make Xinstead of cake.} XThe third line sets the INCLUDES environment variable to Xoverride the default value in MakeCommon. XThe fourth line sets the WORK environment variable, indicating Xdevelopment rather than installation. XThese environment variables are exported, and GNU make is run Xusing uMakefile. XThis will update the depend file if necessary. XThen make is run again using the main makefile. XIt is given a number of command-line variable definitions for Xdevelopment use, X\eg\ the -Bstatic flag is passed to the C compiler to produce Xstatically linked executables for easier debugging. X XThis approach is very dynamic: each call to mk Xfigures out what source files are available, Xrebuilds depend if necessary, Xuncompresses and checks out source files and Makefile if Xnecessary, X\etcP. XThe makefiles do not need to be changed when a new source file Xis added, nor do developers ever have to remember to run a X``make depend'' command. X XOf course, all this automation has its price, and on slow Xhardware the price may be too high. XIn that case, mk can be written Xsimply to exec make with the appropriate arguments, Xincluding SRCS with the names of all source files hard-coded. XThe developers must modify mk whenever adding a new source Xfile, and must run some other shell script to make depend when Xnecessary. XAutomation can be retained in the mk scripts used for Xinstallations, since they are only executed once. X X\section{Capturing version information} XA software system may consist of many master source files, but Xit is convenient to be able to identify a version uniquely with Xa short string, \eg\ 1.3. XMy freeze command captures the RCS revision and date of every Xfile in an RCS subdirectory, then places this information into Xan RCS file named FREEZE,v. XThe latest revision of FREEZE,v can then be used to identify a Xversion uniquely. XBy running freeze just before each release, developers can Xeasily determine the RCS revision of every file in every Xrelease, no matter how old. X X\section{Release numbering convention} XAt present, I use a very simple release numbering convention. XEach internal release is numbered by the RCS revision of the XFREEZE,v file, \eg\ 1.1, 1.2, 1.3, \etcP. XWhen an internal release passes its tests, it is released as an Xexternal release. XThe next internal release then has its first number Xincremented. XFor example, if 1.3 is the first internal release also to be Xreleased externally, then the next internal release is 2.1. XFrom an external view, only the first number is significant; Xthe second number may be ignored. XThe FREEZE file is distributed with each release, so the dates Xof the release itself and of all components are easily Xavailable. X XA drawback of this scheme is that there is no notation to Xdistinguish major from minor releases. XExternal release 3 may fix a tiny bug in external release 2, or may be a Xtotal rewrite. XA three-number format for internal releases would be Xpreferable. XThe first number would be incremented for major external Xreleases, the second for minor external releases, Xand the third for internal releases. XUnfortunately, a three-number scheme is difficult to implement Xusing RCS, so I have not (yet) done so. X X\section{Release mechanics} XWhen developers are ready for an internal release, they make Xsure all RCS files are checked in X(\eg\ with X\verb+rlog -L -R RCS/*,v+). XThey run freeze to record all RCS revisions. XThey then run mk~dist. XThis places each release file in a distribution directory. XThe makefile calls the mkDistO shell file for each file; XmkDistO can be written to make a hard link, symbolic link, or Xcopy. XIt could also run rcp, rdist, or other Xcommands.\footnote{The distribution directory must appear Xlocal, since the files in it depend on the files in the working Xdirectory. XThe makefile does not run mkDistO on files that have not Xchanged. XHard links are thus usually the best choice.} X X\section{Type installation} XAfter a release, it is necessary to recompile the Xversion on one or more types. XOn each type, there is a directory for each version. XInitially, it is only necessary to create this compilation Xdirectory and any other directories needed for results, X\eg\ bin, man, man/man1, man/cat1, \etcP. XIn the compilation directory, three files are needed initially: XMakeCommon, uMakefile, and mk. XTypically, MakeCommon and uMakefile are created as symbolic Xlinks into the distribution directory, Xbut mk must be unique for each type, since mk contains all Xtype-specific information. XFor example, here is an mk shell file for a sun34 type for cz: X\begin{verbatim} XDD='/net/free-dist/cz/2/dist' XSRCS=`cd $DD; echo *` Xexport DD SRCS Xmake -f uMakefile Xexec make \ X $* X\end{verbatim} XHere, the value of the SRCS environment variable is a list of Xall files in the distribution directory. XThe micro-makefile sets up the main makefile and the depend Xfile. XFiles are brought in from the distribution directory to the Xcompilation directory by running mkDistI, which again can be Xcustomized to make hard links, symbolic links, or Xcopies.\footnote{I use symbolic links since the distribution Xdirectory is often on a different file system, typically XNFS-mounted.} X XAfter running mk, ``mk~install'' performs the installation for the Xtype. XTwo shell files are used for installing a file in a directory: XmkInstX for executable files and mkInstF for other files X(including shell scripts). XAgain, mkInstF and mkInstX can be customized to make hard Xlinks, symbolic links, or copies, and to execute any other Xcommands needed. XFor example, mkInstX can run strip. X X\bibliography{b} X\bibliographystyle{alpha} X X\chapter*{Colophon} XThis report was written in \LaTeX\ version~2.09. XIt was typeset in 11~point Computer Modern. X X\end{document} END_OF_FILE if test 31934 -ne `wc -c <'hetero.tex'`; then echo shar: \"'hetero.tex'\" unpacked with wrong size! fi # end of 'hetero.tex' fi echo shar: End of archive 1 \(of 9\). cp /dev/null ark1isdone MISSING="" for I in 1 2 3 4 5 6 7 8 9 ; do if test ! -f ark${I}isdone ; then MISSING="${MISSING} ${I}" fi done if test "${MISSING}" = "" ; then echo You have unpacked all 9 archives. rm -f ark[1-9]isdone ark[1-9][0-9]isdone else echo You still need to unpack the following archives: echo " " ${MISSING} fi ## End of shell archive. exit 0