








        Installing ``C News'' Network News Software


                       Geoff Collyer

                  Department of Statistics
                   University of Toronto


                          _A_B_S_T_R_A_C_T

          This document describes the flow  of  network
     news  within  and between machines, what each com-
     ponent of C news does, and how to install C news.



_I_n_t_r_o_d_u_c_t_i_o_n

     Network news (or _n_e_t_n_e_w_s for short) consists of a  col-
lection of messages formatted similarly to ARPAnet mail (see
ARPA Internet RFC 1036 for  details),  widely  spread.   The
logical  network,  imposed  on top of various real networks,
formed by the set of  all  interconnected  sites  exchanging
network  news  is  called ``Usenet'' and was formed in 1979,
radiating out from Duke University.  Netnews  is  propagated
between  cooperating  machines by a flooding algorithm, with
some loop prevention  heuristics:  each  machine  sends  its
neighbours  news that the neighbours have (probably) not yet
seen.

     Flow of netnews between machines may be achieved by use
of any network which can transmit an arbitrary stream of (at
least 7-bit) ASCII code, unmodified.  If  a  network  cannot
transmit  ASCII  intact  (e.g.  BITNET),  it  is possible to
encode netnews before transmission across  the  network  and
decode it upon reception from the network.  Since one cannot
be certain that one's network  neighbours  will  be  up  and
reachable  at all times, outgoing netnews must be queued, at
least in the case of network trouble.   To  date,  at  least
these  networks,  protocols  and  media  have  been  used to
transmit netnews correctly: UUCP, RS232, NNTP,  Ethernet(R),
the  ARPA Internet, Datakit(R), ACSnet, magnetic tape, SMTP,
and BITNET, though at least the last two require  some  form
of  encapsulation  to  avoid  corruption  of  articles; SMTP
because some common  implementations  get  the  newline-CRLF
mappings  wrong,  thus  throwing off byte counts in batches,
and BITNET because of its Procrustean  chopping,  expanding,
mapping,  bashing  and  smashing of all data sent through it
(sending lines of 80  or  fewer  characters  of  letters  of
either  case  and  digits  and  plus and minus appears to be



                      January 10, 1992





                           - 2 -


safe).

     Netnews arrives via some network, which causes  a  com-
mand to be executed upon arrival (e.g.  _r_n_e_w_s).  _r_n_e_w_s typi-
cally spools  the  inbound  netnews  for  later  processing.
Eventually (often within the hour or at the end of the busi-
ness day), the input queue is run and the netnews is  stored
locally,  typically  under  /_u_s_r/_s_p_o_o_l/_n_e_w_s,  and queued for
transmission to netnews neighbours.  Once  stored  on  disk,
netnews  may  be read by any of a collection of unprivileged
news readers, including _c_a_t(1).   _E_x_p_i_r_e  is  run  typically
each night to remove old netnews from disk.

     C News was originally written to provide a much  faster
and  smaller,  more robust, reliable and correct implementa-
tion of netnews software than B 2.11 news.  We believe  that
C  News  is  also faster, smaller and more robust than B 3.0
news.  B 3.0 news has many more features; take that  as  you
will.

_C _N_e_w_s _C_o_m_p_o_n_e_n_t_s.

     _R_n_e_w_s invokes the input subsystem, which spools  incom-
ing  netnews in its original form, as received, typically in
compressed batches.  Periodically, the input queue should be
run  by  invoking _n_e_w_s_r_u_n, thus uncompressing any compressed
input and feeding it to _r_e_l_a_y_n_e_w_s.

     _R_e_l_a_y_n_e_w_s files incoming netnews as  articles  on  disk
and  initiates spooled transmission to other machines, often
by simply writing the names of the disk files containing the
articles  on the ends of `batch' files of file names.  Quite
a bit of policy from RFC  1036  is  embedded  in  _r_e_l_a_y_n_e_w_s.
_i_n_e_w_s  is a complex front-end for _r_e_l_a_y_n_e_w_s which implements
much of the per-site policy on news  posting.   _i_n_e_w_s  is  a
fairly-slow  shell  script which is adequately fast for most
sites, but it will need to be  replaced  by  something  more
streamlined  for sites which gateway mailing lists into net-
news, for example.

     The output _b_a_t_c_h_e_r reads lists of file names  and  gen-
erates news batches (see RFC 1036 or _n_e_w_s(5) for the format)
of prescribed sizes and queues the batches for  transmission
to  other  sites.   The  batcher  is run asynchronously with
_r_e_l_a_y_n_e_w_s, typically once an hour outside of business hours.

     _E_x_p_i_r_e is generally run once per night to  remove  from
disk  news  articles  older than a few days.  _E_x_p_i_r_e can use
different expiry criterion for different newsgroups and  can
archive articles instead of removing them.  _E_x_p_i_r_e also runs
asynchronously with _r_e_l_a_y_n_e_w_s.

     There are many news readers.  C News comes with a  lim-
ited  news reader (_r_e_a_d_n_e_w_s by Michael Rourke) sufficient to



                      January 10, 1992





                           - 3 -


replace long-winded /_e_t_c/_m_o_t_ds but you will  want  a  heavy-
duty  news  reader  if you plan to read more than local news
groups.  We recommend _r_n by Larry Wall, available from  your
netnews  neighbours or your nearby _c_o_m_p._s_o_u_r_c_e_s._u_n_i_x archive
site.  There are others: _v_n and _v_n_e_w_s are two.

_P_r_e_p_a_r_a_t_i_o_n _f_o_r _I_n_s_t_a_l_l_a_t_i_o_n

     Netnews consumes a lot of disk space and often a lot of
transmission  time.   Here  are  some  relevant measurements
regarding a full netnews feed as  of  the  time  of  writing
(January  1989),  taken  from _n_e_w_s._l_i_s_t_s: a day's netnews is
about 3Mb and 1,400 articles in 450 newsgroups.  Years  ago,
sites  often  kept  14 days of netnews on disk, but now many
sites keep news for 3 to 5 days, thus allowing for the occa-
sional  long weekend.  Thus a full news feed expired after 4
days will consume about 12Mb.  Some people  feel  that  news
volume  is  doubling  roughly  every  16 months.  If this is
true, we can expect volume to increase by a factor of 10  in
about  4  years  to 30Mb per day or 115Mb for 4 days.  It is
thus wise planning to set aside a lot of disk space for net-
news.   There  are  two  ways  to  cope with ever-increasing
volumes of netnews: refuse to  accept  more  newsgroups,  or
expire news after shorter intervals on disk.  A current full
feed takes just over 7 hours to transmit at  1200  baud,  so
for dial-up connections one clearly wants the fastest modems
one can afford.

     Clearly, transmitting a full news feed is a non-trivial
commitment  of resources, so you may have some difficulty in
finding a site willing to supply one.  Such a  site  may  in
turn  expect  you to feed yet other sites.  You will need to
agree  with  your  prospective  netnews  neighbour(s)   upon
transfer media, protocols and networks.

     Before proceeding to install C News,  you  should  read
this  document through to the end, probably read the compan-
ion document _T_h_e _I_n_t_e_r_f_a_c_e _B_e_t_w_e_e_n _C _N_e_w_s  _A_n_d  _T_h_e  _O_u_t_s_i_d_e
_W_o_r_l_d,  and  possibly  read  selected  items  in  the _C _N_e_w_s
_I_m_p_l_e_m_e_n_t_o_r'_s _N_o_t_e_b_o_o_k and the manual pages.

     You will need to assign a user id and group id to  net-
news  (often  new  ids  called  ``news'');  initialise these
directories:  NEWSCTL  (typically  /_u_s_r/_l_i_b/_n_e_w_s),   NEWSBIN
(/_u_s_r/_l_i_b/_n_e_w_s_b_i_n),   and  NEWSARTS  (/_u_s_r/_s_p_o_o_l/_n_e_w_s);  and
install each subsystem.  NEWSCTL and NEWSARTS are  logically
one  subtree,  defining  a news data base, but are split for
backward compatibility with older  news  software,  particu-
larly old news readers.  NEWSBIN contains programs and shell
scripts which might be common  amongst  machines  sharing  a
common architecture (e.g. Sun 3's); NEWSCTL/bin may override
these.  The goal is to  install  the  subsystems,  integrate
them into a working news system, and configure the news sys-
tem to communicate with other news systems.



                      January 10, 1992





                           - 4 -


     There are a few key files that must  exist  before  any
serious  attempt  may  be made to operate the news software.
NEWSCTL/active is the list  of  newsgroups  that  this  site
knows  (is  willing  to  accept or individually reject), and
must be owned by the  NEWS  userid  (the  userid  that  owns
NEWSBIN/relay/relaynews, typically _n_e_w_s).  You will probably
want to get your initial active file from your upstream feed
and  edit  it to suit the set of groups you wish to receive.
Be sure to make the second field more than five digits wide,
by  adding  leading  zeroes  (ten  digits  is a conservative
width).  NEWSCTL/sys defines the newsgroups that  this  site
is  willing  to accept and describes how they are to be for-
warded to its neighbours.  NEWSCTL/server contains the host-
name  of  your  file  server,  if you have multiple machines
sharing news via  a  network  file  system.   NEWSCTL/whoami
similarly  contains  the name by which a cluster of machines
sharing  news  is  to  be  known  for  purposes   of   news.
NEWSCTL/mailname is optional and contains the full (possibly
dotted) name by which your cluster is known for purposes  of
mail.   NEWSCTL/organisation (or NEWSCTL/organization if you
insist) contains the name of your organisation,  which  will
be  copied  into the _O_r_g_a_n_i_z_a_t_i_o_n: header of articles posted
locally, by default.  NEWSCTL/mailpaths defines mail  routes
to  machines which contain aliases for postings to moderated
newsgroups.  NEWSCTL/log,  NEWSCTL/errlog,  NEWSCTL/history,
NEWSCTL/history.dir,  and NEWSCTL/history.pag must exist and
be owned by the NEWS  userid.   Tentative  versions  of  all
these files are built by the installation procedures, but it
is quite likely that you'll have to edit some of them.

_C _N_e_w_s _I_n_s_t_a_l_l_a_t_i_o_n

     Proceed to the  conf  directory  of  the  distribution.
There  is  a  major  shell file here, named build, that will
interrogate you at length and construct shell  files  to  do
the real work.  You may need to do ``chmod +x build'' before
running it, to make it executable.

     You will probably need your system's manuals on hand to
answer build's questions.  Another terminal (or another win-
dow, on a bitmap display) would also be useful.  You'd  best
be  prepared to take notes, also, as build will occasionally
suggest that something be checked when you're done.

     Build itself does not alter any files  or  perform  any
installation  chores:   all it does is create shell files in
the conf directory.  If you  already  know  something  about
news  software,  or  are merely suspicious that build hasn't
done everything right, you should probably  read  the  shell
files   before  running  them.   There  are  four  of  them:
doit.root, doit.bin, doit.news, and again.root.

     Doit.root sets up the major directories for  news,  and
sets  their ownerships correctly.  It typically will have to



                      January 10, 1992





                           - 5 -


be run as _r_o_o_t to have adequate permissions for all of this.
It is brief.

     Doit.bin does most of the work of building and  instal-
ling  the  programs.   It should be run as the user who owns
the distribution directories and  will  own  the  executable
programs under NEWSBIN.

     Doit.news does some other  small  chores  and  installs
control  files.   If any of the control files already exist,
it will complain and refuse to overwrite them, as  a  safety
precaution.   It  should  be  run  as  the owner of the news
files.  Since many of the files it is installing  are  built
by doit.bin, that should be run first.

     Finally, again.root tends to ownership  and  permission
changes  on  a few programs that need to run set-userid.  It
requires the ability to change ownerships and to set permis-
sions  on  the files afterwards, which usually means running
it as _r_o_o_t.  It too is brief.

     There are undoubtedly strange systems  out  there  that
build  and  friends  are  not smart enough to cope with.  In
such cases it will be necessary  to  edit  the  shell  files
before  running  them,  or  to use them as guides and do the
work by hand.  In particular, systems that  require  strange
options  in  Makefiles  will  need to have those inserted by
hand.

     If you want to test pieces of C News without installing
them,  some  (not  all)  of the subsystems have a ``make r''
feature that runs a regression test.  Note:  almost  all  of
these   require   that   NEWSCTL/bin/config,  or  its  local
equivalent, be in place already so that shell files can find
out what PATH (etc.) they should be using.

     Note that it is easy to build a test news system  which
is  completely independent of other existing news systems on
the same machine, or to build one which shares  its  NEWSBIN
with  another  C  news  system,  or shares input of articles
(e.g. running an old B news system and a new C  news  system
off  the  same  stream of input until you are confident that
your new C news system is operating to  your  satisfaction).
See  _s_u_b_s_t(1) for the mechanism which permits quickly chang-
ing   all   the   places   that   know   the   location   of
NEWSCTL/bin/config.   After  that,  edit  this news system's
NEWSCTL/bin/config and things should be set up for  separate
existence.

_F_i_r_s_t _N_e_w_s

     When you arrange to get a news feed from your neighbor,
you should also ask him to send you the current set of arti-
cles in the newsgroup  news.announce.newusers.   Several  of



                      January 10, 1992





                           - 6 -


these  are  very important reading for people who are new to
the net.

_U_n_u_s_u_a_l _S_y_s_t_e_m_s

     We believe that C News runs fine  on  16-bit  machines,
but  it  hasn't  been tested very thoroughly on them lately.
It will not perform quite as  well  with  the  more  limited
space.

     Machines with very old compilers  can  be  a  headache.
There  are some hooks in h/news.h for doing without ``void''
and ``unsigned long'', two  particular  problem  areas,  but
they have to be arranged manually; build does not know about
them.

     Some very  old  systems  cannot  do  _s_e_t_u_i_d(_g_e_t_e_u_i_d()),
which  makes  it impossible for a daemon to make directories
and get the ownership right.  We provide  a  small  program,
setnewsids,  to  run  setuid-root.  Relaynews knows about it
and invokes it if _s_e_t_u_i_d(_g_e_t_e_u_i_d()) fails; it then sets per-
missions  correctly  and  re-invokes relaynews.  The code is
short enough to be read and understood in full, so that  the
suspicious  system  administrator  can  be  sure  that  this
setuid-root program is not going to do something awful.
































                      January 10, 1992


