INDRA Note 753
IEN 99
May 3rd 1979










                     NI FTP: Summary and Assessment

                            P. L. Higginson
                             C. J. Bennett










               ABSTRACT: This note is a brief summary  of
               the  NI FTP, its design aims, and the ways
               in  which  those  aims  are  achieved.   A
               comparison  of  the NI FTP and the DIN FTP
               proposed for AUTODIN II  is  contained  in
               IEN 100.

                                     2



1.  The NI FTP Protocol

     This section is a brief summary of the important features of the NI
FTP protocol.

     The NI FTP  was  written  by  a  committee  of  people  from  eight
different  organisations  in the UK who between them used a wide variety
of computer systems and hardware.  Computers  used  by  the  group  were
manufactured,  among  others,  by  DEC,  IBM,  ICL,  GEC, PRIME and CTL.
Drafts were widely circulated and discussed outside  the  design  group.
While  the  initial aim was to produce a protocol for transferring files
across EPSS, "network independence" became an overriding aim.  This  was
because  it  was  realised  that  EPSS  would  shortly  be replaced by a
different,  X25-based  network,  and   because   most   of   the   eight
organisations  had multiple systems or their own local networks, and the
ability to use the FTP in all these areas was desired.  There are  about
15  implementations in progress, of which 3 have so far exchanged files.
UCL has a TOPS20/TENEX implementation which has transferred files across
the  ARPANET,  and tests have begun on interworking with implementations
in EPSS.

     The NI FTP is a two-party file  transfer  protocol.   The  transfer
occurs  in  two  phases.   In the first, the transfer is defined and the
attributes of the data to be transferred are negotiated.  In the second,
the  data  is  actually  transferred.   Three  levels  of  protocol  are
recognised: level 0, which  covers  the  negotiation  of  file  transfer
parameters;  level  1,  which  handles error and flow control during the
transfer, along with any data or time dependent changes in the file; and
level 2, which is the actual transfer of data.

     The negotiation phase is simply structured to  be  an  exchange  of
file attributes between the two sides.  These attributes define a common
definition of the file and the transfer within a "conceptual  filestore"
supported  by the two sides.  During this phase, agreement is reached on
such things as character code, file size, file name and  access  rights,
all  of  which  are  regarded  as  attributes  of  the  file  within the
conceptual file store.  It is up to the two ends to map these attributes
into  locally  acceptable  forms  for  their  actual filestores; if this
cannot be done, the transfer may be rejected.

     The currently defined attributes cover a basic set of properties of
sequential  files.   This set can be easily extended, and facilities are
provided to enable implementations to  grow  in  an  upwards  compatible
manner.    Every   attribute  has  a  default  value,  which  allows  an
implementor to construct a transfer facility as simple or as complex  as
desired,  in  the  knowledge  that his facility will be able to interact
easily with more complex ones.  This  is  possible  because  quoting  an
unknown  attribute  or  unacceptable  value for an attribute need not of
itself cause a protocol error.  During the negotiation phase,  the  more
complex  implementation  will reduce its attribute set to one acceptable
to the simpler one.

                                     3



     The data transfer phase can treat the data  totally  transparently,
but  does provide a user record structure for text files if desired.  It
allows for data compression as an option, which can significantly reduce
the  amount of data actually transferred.  Even if this is not used, the
protocol overhead can be as low as 1 byte in 64.  For  text  files,  the
existence   of   different   degrees  of  formatting  sophistication  is
recognised by allowing a set of common format effectors to be agreed  in
the negotiation phase.  The data can be in a variety of codes, as agreed
during the negotiation phase, and the particular  code  in  use  can  be
changed  during the transfer phase, thus allowing the protocol to handle
special files such as job output files,  graphics  files  with  embedded
text,  or  symbolic files produced by a compiler for debugging purposes.
Binary files can use byte sizes of  any  arbitrary  size  as  agreed  in
advance.

     Error control is provided by the use of  data  checkpoints.   These
are inserted by the sender at suitable points, and may be matched to the
characteristics of the file source or destination storage devices.   The
acknowledgement  of  a data mark implies that the data has been securely
stored by the receiver.  The use of a data window of unacknowledged data
marks  enables  the sender to detect problems at the receiving end which
will cause it to suspend the flow of  data.   On  recovery  (or  in  any
appropriate circumstance), the receiver can ask the sender to resume the
transfer starting from any mark after the last mark acknowledged.   This
facility  also  protects  the  transfer  against loss of data within the
network, and can even be used to resume an  interrupted  transfer  in  a
separate session.

     Flow control is provided by allowing the receiver  to  request  the
sender to suspend the flow of data, and to tell it when to resume.  This
is primarily intended to  cover  device  interruptions  (eg  mounting  a
magnetic  tape).   The  use  of  these options is again subject to prior
agreement during the initial negotiation phase, as with  all  the  other
attributes.

     The mechanics of  the  data  transfer  is  left  to  the  transport
mechanism.   The  protocol  is defined so as to make minimal assumptions
about the transport service.  These  are:  the  transport  service  will
provide  a synchronised sequential bytestream between the two sides with
an acceptably low rate of error, and that if an unrecoverable error does
occur,  there  exists  some  way for the transport service to notify the
application process.  Recovery  would  then  be  handled  by  the  error
control mechanism described above.



                                     4



2.  Assessment of the NI FTP.

The design of the NI FTP was governed by four basic objectives:

     (i) The protocol should be independent of the communication medium.

     (ii) It should be flexible enough to satisfy a diverse  variety  of
     applications.

     (iii) It should be able to interface easily  to  a  wide  range  of
     operating systems.

     (iv) It should allow "escape routes" such that special features  of
     a particular operating system can be exploited where necessary.

The following features of the  protocol  were  designed  to  meet  these
objectives:

     (i) The minimal requirements necessary  from  a  transport  service
     were  identified  and  adhered  to, along with the recognition that
     unrecoverable errors (at transport level) may occur.

     (ii) The definition of the file is in terms of  a  conceptual  file
     store.   No  restrictions  were  placed on the mapping of this file
     store  to  the  local  file  store,  either   in   the   attributes
     specification or in the definition of the negotiation process.  The
     inclusion of option sets and relational operators in the  attribute
     specification  allows  extensions  to  be  easily incorporated into
     existing implementations and into the protocol itself.  A number of
     parameters  provide  the  required  "escape routes" for areas where
     operating system dependent peculiarities are anticipated.

     (iii) Complete transparency for naming files and for data  transfer
     is possible.

     (iv) The "records" of data transfer used within the protocol enable
     mixing  of  control and data within a single connection.  They need
     have no relation to the record structure (if any) used  within  the
     file, unless the user so desires.  The only restriction is that the
     communication medium should deliver eight  bit  bytes  of  data  in
     sequence.

     (v) The formats of control commands and attributes are standardised
     in  a machine-oriented form.  The action of the protocol is defined
     as a single file transfer transaction.  Hence the dominant need for
     flexible  command  formats occurs in the negotiation phase, and the
     level 0 commands are very loosely structured  to  allow  indefinite
     extension.   Only a very small number of commands must be exchanged
     during the data transfer; for this reason, a more  rigid  structure
     was adopted for level 1 commands to ease processing overhead.