To move files between your account on the DIVMS Unix network and the Beowulf,
use Secure FTP (sftp).  It has a regular FTP command syntax:

   Command          Action
   sftp beowulf     Contact remote host; initiate password checking.
   cd <directory>   Change directories on remote host
   put <filename>   Copy a file from current directory on local host to
                           current directory on remote host
   mput <wildcard name>   Copy multiple files
   get <filename>   Copy file from current directory on remote host to
                           current directory on local host
   mget <wildcard name>   Get multiple files
   quit             Terminate connection to remote host.

   Example:  I am logged into my Unix workstation ("mouse") in a directory
   containing 3 C programs that I want to upload to Pacheco subdirectory on
   beowulf master.

   kcowles@mouse [112]% sftp beowulf
   Connecting to beowulf...
   kcowles@beowulf's password: 
   sftp> cd Pacheco
   sftp> put get_data.c
   Uploading get_data.c to /usr/home/kcowles/Pacheco/get_data.c
   sftp> mput *.c
   Uploading get_data.c to /usr/home/kcowles/Pacheco/get_data.c
   Uploading serial.c to /usr/home/kcowles/Pacheco/serial.c
   Uploading trap.c to /usr/home/kcowles/Pacheco/trap.c
   sftp> cd ../markers
   sftp> get Rplots.ps
   Fetching /usr/home/kcowles/markers/Rplots.ps to Rplots.ps
   sftp> quit

Some setup is required in order to compile and run programs using MPI on
the Beowulf.

1. Add the following three lines to your .bashrc file on the master:

   PVM_ROOT=/usr/pvm3; export PVM_ROOT
   LM_LICENSE_FILE=/usr/pgi/license.dat; export LM_LICENSE_FILE
   PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/usr/contrib/bin:/MPI/bin:/usr/pgi/linux86/bin; export PATH

   The first one actually is for using Luke's "snow" package with PVM.  The
   latter two enable MPI to find the files it needs.  

   Note that what look like 2 lines beginning with "PATH=..."  is actually
   one long line with no carriage returns.

   Then copy this .bashrc file to all the nodes:

           [kcowles@master kcowles]$ rcpf .bashrc $HOME/.bashrc

   You may need to log out of the master and back in to make these settings
   take effect.


2. Place the file "conf" (which I am placing in an attachment to this
   e-mail message) in a directory called ".gmpi" in your home directory on
   the master and on ALL the nodes (except node 5 for the moment, as it is
   dead).  You can do this with the following steps:

   a.  Log into the beowulf master.  Use the cluster command "rcom" to 
       issue the "mkdir" command to create the ".gmpi" subdirectory 
       on all the nodes simultaneously:

       [kcowles@master kcowles]$ rcom mkdir .gmpi

       (You will get the following error message, but the command will
        work on the master and the other nodes.
              node5.cl.cs.uiowa.edu: No route to host )

       You can log into any node by name to check this out, for example:

       [kcowles@master kcowles]$ ssh node2

   b.  Use "sftp" as described at the beginning of this message to upload the 
       "conf" file to the ".gmpi" subdirectory on the master.

   c.  From the ".gmpi" directory on the master, use the cluster "rcpf" command
       to copy the file to the .gmpi directory on all the other nodes:

       [kcowles@master .gmpi]$ rcpf conf $HOME/.gmpi/conf 

       (You'll get another error msg regarding node 5, but the rest should
        work.)
   
The above setup should have to be done only once.   Thereafter, you can
compile MPI programs written in C with the "mpicc" command.  (I will
leave it to Grace to verify that you can compile Fortran programs with
"mpif77".)  For example, to compile the program "trap.c" and name the 
executable file "trap":

    [kcowles@master Pacheco]$ mpicc trap.c -v -o trap -lm

Now you must copy the executable to all the nodes.  I had already created
a subdirectory called "Pacheco" on all the nodes, and that is where I want
to copy it:

    [kcowles@master Pacheco]$  rcpf trap $HOME/Pacheco/trap

To run the program, use 
   mpirun.ch_gm -np <number of requested processes> <executable name>
For example, to invoke 5 processes and run "trap":

    [kcowles@master Pacheco]$ mpirun.ch_gm -np 5 trap

I get the following output:
    With n = 1024 trapezoids, our estimate
    of the integral from 0.000000 to 1.000000 = 0.329442