An FTP Primer I: The File Transfer Protocol

Ray Beausoleil
beausol@u.washington.edu

   Some of the most capable software packages currently available for  
microcomputers are publicly issued as either "freeware" or "shareware."  
Freeware is generously released by the author without charge, while shareware  
is distributed under a "try before you buy" agreement. Copies of many of these  
programs are archived on mini and mainframe computers throughout the world. If  
your local Unix site is connected to the INTERNET, then you may transfer these  
files to your site and download them to your microcomputer.

   FTP is the INTERNET standard File Transfer Protocol. The program "ftp" is  
the Unix user interface to the Internet protocol, and allows you to transfer  
files to and from a remote network host. This article necessarily cannot serve  
as an exhaustive description of the "anonymous FTP" process. (In fact, the Unix  
"ftp" program is only one of many user interfaces to the INTERNET FTP.)  
However, in Part I I will give an example of a typical Unix FTP session, and I  
will list the addresses of a few archive sites for Macintosh and MS-DOS  
software. Next month, in Part II, I will describe some of the techniques for  
decoding and downloading this software to your microcomputer.

   FTP is a privilege granted to users of the INTERNET by the administrators of 
sites containing these archives. Please try to restrict your file transfers to  
non-business hours, because you and other FTP users can cause the host computer 
to slow down significantly. In addition, please minimize the time that you are  
connected to the remote site so that others may take advantage of the  
opportunity.

   A screen capture of a typical (brief) FTP session is shown in Fig. 1. From a 
Unix machine at the University of Washington ("milton"), I connected to sumex-
aim.stanford.edu, where a large collection of Macintosh software is archived.  
The numbers in parentheses located on the left margin of Fig. 1 correspond to  
the annotations listed below.

(1)  At your Unix prompt, enter "ftp" followed by a space and either the name  
or the numerical INTERNET address of the remote host. Occasionally, a host will 
accept only the INTERNET address. If the remote site is active, then you will  
be connected to the remote FTP server.

(2)  When the remote machine asks for your name, enter "anonymous." Some hosts  
limit the number of off-site FTP users; if the number of connections has  
already reached the maximum, you will be asked to try again later.

(3)  For your password, you are generally asked to enter your e-mail address,  
such as "beausol@u.washington.edu," followed by a return. However, some systems 
will permit you to enter virtually any text instead.

(4)  After these preliminaries, you will enter your local Unix "ftp" command  
interpreter, which will provide you with the "ftp>" prompt. Usually, you will  
first be logged in to a "root directory" that has been set aside for FTP  
purposes by the system administrators; you must then navigate to the archive  
itself. At sumex-aim.stanford.edu, the Macintosh software archive is located in 
the directory "info-mac." You can change the current working directory on the  
remote host using the "cd" command. In the example of Fig. 1, I have also moved 
to the subdirectory "util" on the info-mac directory. Whenever you have  
finished with a particular subdirectory, enter "cdup" at the "ftp>" prompt to  
move up one directory level.

(5)  Enter "get" followed by a space and the name of the file to be  
transferred. The file will then be copied to your local Unix directory. In the  
example of Fig. 1, the ".hqx" extension indicates that the Macintosh file has  
been encoded as purely ASCII characters using BinHex (discussed in more detail  
in Part II.) Usually, ASCII is the default FTP transfer mode. However, if the  
file has been archived in binary format, then at the "ftp>" prompt, and prior  
to transferring the file, you must enter "binary." Sometimes, if the file is  
large and the remote system is busy, the transfer may take considerable time.  
To keep an eye on the progress of the transfer, enter "hash" before getting the 
file; then as each 1024-byte block of data is transferred, a "#" hash mark will 
be printed on your monitor. If you would like to see an abbreviated list of the 
files contained in the current directory, enter "ls"; for a more verbose list  
that includes information about each file's size and the date that it was  
archived, enter "dir" followed by a carriage return.

(6)  When you are finished transferring the files you need, be courteous and  
log off of the remote machine as quickly as possible. At the "ftp>" prompt,  
enter either "by" or "qui."

   These are the most basic commands that you will use during any FTP session.  
You can usually find a complete description of the FTP command interpreter used 
at your Unix site by viewing the "manual pages" (i.e., by entering "man ftp").

   There are a number of archives of software for the Macintosh, IBM PC and  
compatibles, and other popular personal computers. In Table I, I have listed  
the name of the host, the Internet address, the directory containing the  
archive, the microcomputer, and the appropriate transfer mode for some of the  
most popular archives. (Of course, virtually all documentation files are stored 
as pure ASCII.) One of the largest and most frequently updated Macintosh  
software archives is maintained at sumex-aim.stanford.edu. I recommend that the 
inexperienced user explore the subdirectory /info-mac/help. OzTeX, the free  
Macintosh implementation of TeX mentioned by Tony Siegman in last month's  
column, may be found at midway.uchicago.edu (University of Chicago); it is  
quite large, so be prepared! The MS-DOS archive at wsmr-simtel20.army.mil  
(White Sands Missile Range) is enormous, but it is difficult to navigate  
because the host does not use Unix. Nevertheless, there are a number of text  
documents in the root directory that describe the FTP commands for the simtel20 
server. A Unix mirror of the simtel20 archive is maintained at  
wuarchive.wustl.edu (Washington University at St. Louis); if you are new to FTP 
on the IBM PC, then an excellent place to begin is the wuarchive subdirectory  
/mirrors/msdos/starter.

   The best way to learn more about FTP and other microcomputer archives is to  
subscribe to the appropriate INTERNET news groups. A full description of  
INTERNET news groups is beyond the scope of this article; you'll need to list  
the manual pages for your system news reader or to contact your local system  
administrator to get started. However, news groups of interest to microcomputer 
users have names that begin with prefixes like "comp.sys.mac..." and  
"comp.sys.ibm.pc..." In particular, the news groups "comp.sys.mac.digest" and  
"comp.sys.ibm.pc.d" describe in detail new additions to the archives at sumex-
aim and simtel20, respectively.

   The precise implementation of FTP is site-dependent, so "your mileage may  
vary." But the basics of file transfer are usually easy to master and are well  
worth the effort. Next month, I will describe the basics of decoding and  
downloading files obtained via FTP.

(I am grateful to Geoff Kinnel, Betsy Richards, Geoff Allen, and Jim Matthews  
for their generous assistance in gathering information for this article.)

(C) Copyright 1991 Raymond G. Beausoleil




Figure 1: Screen Capture of an ftp session.

(1)   milton> ftp 36.44.0.6
      Connected to 36.44.0.6.
      220 sumex-aim FTP server (SunOS 4.0/3) ready.
(2)   Name (36.44.0.6:beausol): anonymous
      331 Guest login ok, send mail address (user@host) as password.
(3)   Password:
      230 Guest connection 7 (of 25) accepted, access restrictions apply.
(4)   ftp> cd /info-mac/util
      250 CWD command successful.
(5)   ftp> get unstuffit-deluxe-20.hqx
      200 PORT command successful.
      150 ASCII data connection for unstuffit-deluxe-20.hqx
      (128.95.136.1,11969) (172867 bytes).
      226 ASCII Transfer complete.
      local: unstuffit-deluxe-20.hqx remote: unstuffit-deluxe-20.hqx
      175534 bytes received in 7.2 seconds (24 Kbytes/s)
(6)   ftp> by
      221 Goodbye.


Table I:

Site Name               IP Address      Directory       Software   Mode
------------------------------------------------------------------------
sumex-aim.stanford.edu  36.44.0.6       /info-mac        Mac      ASCII
midway.uchicago.edu     128.135.12.73   /pub/OzTeX/oztex Mac      Binary
wsmr-simtel20.army.mil  26.2.0.74       Non-Unix!        MS-DOS   Binary
wuarchive.wustl.edu     128.252.135.4   /mirrors/msdos   MS-DOS   Binary

--------------------------------- End of Part I ------------------------------

----------------------------- Beginning of Part II ---------------------------

An FTP Primer II: Downloading and Decoding  

Ray Beausoleil
beausol@u.washington.edu

   Last month, in Part I of this article, I briefly described the INTERNET File  
Transfer Protocol (FTP). In particular, I demonstrated how to use the Unix  
"ftp" interface to transfer personal computer software packages from a remote  
archive to your local Unix mainframe or minicomputer. These programs have  
usually been compressed and/or encoded by the archive administrators. This  
month, I will provide a brief introduction to the process of first downloading  
this software to your microcomputer, and then decoding it using special utility 
programs.  

   The method that you must use to download software from your local mainframe  
computer to your microcomputer depends on the operating system and the software 
installed on the mainframe, and on the configuration of your network (e.g.,  
Ethernet or modem). You should begin by asking your system administrator for a  
list of information transfer protocols that have been installed at your site  
and the commands that invoke these protocols. Some of the most popular 
protocols include XModem, YModem, ZModem, and Kermit. Paul Kelley has described 
XModem and Kermit in his article "Information transfer for personal computers," 
published in the February 1991 edition of this column.

   In Tables I and II, I have listed freeware and shareware telecommunications  
and utility software packages for the Macintosh and the IBM PC, respectively.  
For Macintosh software, I recommend the archive at sumex-aim.stanford.edu  
(INTERNET address 36.44.0.6), located in the directory "/info-mac". ZTerm, an  
excellent shareware telecommunications application program, can be found in the 
directory "/info-mac/comm". ZTerm not only uses the XModem and YModem data  
transfer protocols, but also offers one of the best implementations of the  
ZModem protocol for any personal computer. When you are connected through ZTerm 
to a Unix mainframe that uses the ZModem protocol, you may download a file  
simply by entering "sz" followed by a space and the name of the file. ZTerm  
detects that a file is being sent, identifies the file type, and then chooses  
the correct transfer mode! For local systems which have implemented the Kermit  
protocol, the program MacKermit (also available at sumex-aim.stanford.edu in  
the "/info-mac/comm" directory) provides a robust Macintosh interface.

   A huge archive of IBM PC software is maintained at wuarchive.wustl.edu  
(INTERNET address 128.252.135.4), in the directory "/mirrors/msdos". The most  
popular shareware telecommunications package for the IBM PC, ProComm, may be  
found there in the subdirectory named (oddly enough) "procomm". ProComm uses  
the XModem, YModem, and Kermit transfer protocols, and can also follow the  
ZModem protocol through an external command shell using the transfer program  
"DSZ.COM", which should be placed in the same MS-DOS directory as the ProComm  
program itself. After connecting to your local Unix mainframe through ProComm,  
enter "sz" followed by a space and the name of the file to be downloaded. Then, 
through ProComm, enter the MS-DOS gateway (the "DOS shell") and input "dsz rz"  
at the MS-DOS prompt. After the file has been written to your personal  
computer's disk, simply enter "exit" at the DOS prompt to return to ProComm.

   As Paul Kelley mentioned in his article, the simplest way to download files  
to your microcomputer is to display the file on your monitor. Both ZTerm and  
Procomm allow you to capture incoming text and write the text to a disk file.  
As described below, you may then decode the files for later use. Almost all of  
the files stored in the large archive of Macintosh software maintained at  
sumex-aim.stanford.edu have been encoded as ASCII (text) characters using a  
format known as BinHex 4. These files have names that end with the extension  
".hqx", and can be captured directly from your monitor after you have  
transferred them to your local Unix host. You may direct a Unix machine to  
write a ".hqx" file to your monitor by entering the command "cat", followed by  
a space and the name of the file to be transferred. However, almost all of the  
files stored in the IBM PC software archive wuarchive.wustl.edu have been  
stored in binary format. After you have transferred one of these files (e.g.,  
"file.arc") to your local Unix mainframe or minicomputer, you can use the  
command "uuencode file.arc file.arc" to ASCII-encode "file.arc" and then write  
it to your monitor. In both cases, you must decode the ASCII file on your  
microcomputer using one of the utility programs listed in Tables I and II;  
since you will not be able to decode these utilities themselves until after you 
have the utilities, you must obtain copies of their binary executable files  
from a colleague or user group.

   A typical binary file can be up to 50% smaller than the corresponding  
encoded ASCII file, and can be transferred to your personal computer in  
proportionally less time. Therefore, if at all possible, you should try to  
download using the binary transfer mode. This is relatively straightforward for 
IBM PC files transferred from the wuarchive.wustl.edu archive and then  
downloaded to a microcomputer using ProComm Plus.

   You may decode BinHex Macintosh files into MacBinary files on your local  
Unix host prior to downloading them using a utility such as "mcvert." You may  
build your own copy of "mcvert" by following these steps: (1) transfer the file 
"mcvert-15.shar" from the "/info-mac/unix" subdirectory at sumex-
aim.stanford.edu using "ftp"; (2) enter "sh mcvert-15.shar" at your local Unix  
prompt, and ignore any error messages caused by the file header; and (3) enter  
"make" at the Unix prompt. This process will create a number of files on your  
directory; you may delete all but "mcvert" itself. Now, to decode any files  
that have the extension ".hqx" (including files that consist of more than one  
part), simply enter "mcvert *.hqx" at the Unix prompt. Your Unix directory will 
then contain files with the extension ".bin", which can be downloaded using the 
MacBinary transfer mode.

   Both the Macintosh and the IBM PC binary files that you have downloaded to  
your personal computer are actually miniature archives that contain one or more 
compressed executable and/or data files. These archives can be easily and  
rapidly expanded using the appropriate utility files listed in Tables I and II. 
(The label "sea" designates self-extracting archives which will decompress  
themselves automatically upon launching.) On the Macintosh, files obtained from 
sumex-aim.stanford.edu will usually have extensions such as ".sit" (Stuffit  
Classic), ".cpt" (Compact Pro), or ".sea" (Compact Pro self-extracting  
archive). On the IBM PC, files transferred and downloaded from  
wuarchive.wustl.edu will have extensions such as ".ARC" (PKUNPAK.EXE), ".ZIP"  
(PKUNZIP.EXE), ".LHZ" (LHARC.EXE), ".ZOO" (LOOZ210.EXE), or ".EXE" (self-
extracting archive or simple executable file).

   At first, the downloading and decoding process may seem daunting to you,  
particularly if either the system architecture of your local host is not Unix,  
or the software installed on that system does not follow one of the few  
examples that I have given in this article. Rest assured that others have  
almost certainly solved your problems already; one of your most dependable  
resources is the knowledge of your colleagues. Alternatively, as I mentioned  
briefly in Part I, you may learn more about personal computer  
telecommunications by subscribing to the appropriate INTERNET news groups. If  
you ask the right people the right questions, you might find that while you may 
not be able stand upon the shoulders of giants, you may be able to stand upon  
their feet.

(I am grateful to John McGarvey, Matt Derstine, Geoff Kinnel, Betsy Richards,  
David Faust, and Geoff Allen for their generous assistance in gathering  
information for this article.)

(C) Copyright 1991 Raymond G. Beausoleil

Table I: Files obtained from sumex-aim.stanford.edu (36.44.0.6) in the
         directory /info-mac (ASCII Transfer Mode)

File Name              Location            Description
-----------------------------------------------------------------------
file-types.txt           help      Brief list of sumex-aim file types
zterm-085.hqx            comm      ZTerm (X,Y,ZModem; no Kermit)
mackermit-098.hqx        comm      MacKermit (No X,Y,ZModem)
stuffit-16-part1.hqx     util      Stuffit Classic w/BinHex 4 (sea)
stuffit-16-part2.hqx     util
compact-pro-13.hqx       util      Compact Pro w/BinHex 4 (sea)
mcvert-15.shar           unix      Unix shell archive for mcvert



Table II: Files obtained from wuarchive.wustl.edu (128.252.135.4) in the
          directory /mirrors/msdos (Binary Transfer Mode)

File Name           Location            Description
-----------------------------------------------------------------------
00-files.doc        starter   Description of wuarchive file types
00-index.txt        starter   Descriptions of files in /starter
prcm243.arc         procomm   ProComm shareware communications program
prcm243d.arc        procomm   ProComm documentation
prcm243d.arc        procomm   ProComm utilities
dsz0404.zip         zmodem    DSZ.COM (ZModem for ProComm)
pk361.exe           starter   PKPAK.EXE/PKUNPAK.EXE (sea)
pkz110eu.exe        starter   PKZIP.EXE/PKUNZIP.EXE (sea)
lh113c.exe          starter   LHARC.EXE (sea)
looz210.com         starter   ZOO archive extracter/lister
uudecode.com        starter   Convert ASCII uuencoded files to binary

--------------------------------- End of Part II -----------------------------.
