for the ARPA Computer Network

14 Pages·2015·0.89 MB·English

Chapter 24 messagecarries an address. In this domain the project portends a
possible major change in the character of data communication
services in the United States.
The Interface Message Processor In a nationwide computer network, economic considerations
for the ARPA Computer Network^ also mitigate against a wideband leased line configuration that is
topologically fully connected. In a non-fiilly connected network,
messages must normally traverse several network nodes in going
F. E. Heart / R. E. Kahn / S. M. Omstein / from sourcetodestination. TheARPANetworkisdesignedonthis
W. R. Crowther / D. C. Walden principleand, ateachnode, acopyofthemessageisstoreduntilit
is safely received at the following node. The network is thus a
store and forward system and as such must dealwith problems of
Introduction routing, buffering, synchronization, errorcontrol, reliability, and
otherrelated issues. To insulate the computercenters from these
For many years, small groups ofcomputers have been intercon- problems, and to insulate the network from the problems ofthe
nected in various ways. Only recently, however, has the interac- computer centers, ARPA decided to place identical small proces-
tionofcomputersandcommunicationsbecomean importanttopic sors ateach networknode, to interconnect these small processors
initsownright.^ In 1968,afterconsiderablepreliminaryinvestiga- with leased common-carrier circuits to form a subnet, and to
tion and discussion, the Advanced Research Projects Agency of connect each research computer center into the net via the local
the DepartmentofDefense(ARPA)embarkedontheimplementa- small processor. In this arrangement the research computer
tionofanewkindofnationwidecomputerinterconnectionknown centers are called Hosts and the small processors are called
as the ARPA Network. This network will initially interconnect Interface Message Processors, or /MPs. (See Fig. 1.) This
many dissimilar computers at ten ARPA-supported research approachdividesthegenesisoftheARPA Networkintotwoparts:
centerswith50-kilobitcommon-carriercircuits. Thenetworkmay (1) design and implementation ofthe IMPsubnet, and (2) design
beextendedtoincludemanyotherlocationsandcircuitsofhigher
bandwidth.
The primary goal ofthe ARPA project is to permit persons and
programs at one research center to access data and use interac-
tively programs that exist and run in other computers of the
network. This goal may represent a major step down the path HOST
taken by computer time-sharing in the sense that the computer
resources of the various research centers are thus pooled and
directly accessible to the entire community of network partici-
pants.
Study ofthe technologyand tariffs ofavailable communications
facilities showed that use ofconventional line switching facilities
wouldbe economicallyandtechnicallyinefficient. The traditional
method of routing information through the common-carrier
switched network establishes adedicated path foreach conversa- HOST
tion. Withpresenttechnology, thetimerequiredforthistaskison
the order of seconds. For voice communication, that overhead
time is negligible, but in the case ofmany short transmissions,
such as may occur between computers, that time is excessive.
Therefore, ARPAdecidedto buildanewkindofdigitalcommuni-
cation system employing wideband leased lines and message
switching, wherein a path is not established in advance and each
'Proc. AFIPS SJCC, 1970, pp. 551-567.
*Abibliographyofrelevantreferencesisincludedattheendofthispaper;
amore extensive list may be found in Cuadra [1968].
Chapter24 The Interface Message ProcessorfortheARPAComputerNetwork 403
and implementation ofprotocols and techniques for the sensible Network Design
utilization ofthe network by the Hosts.
Implementation of the subnet involves two major technical The design ofthe networkisdiscussed intwoparts. The firstpart
activities: providing 50-kilobit common-carrier circuits and the concerns therelationsbetweenthe Hostsandthesubnet, andthe
associated modems; and providingIMPs, alongwith softwareand secondpart concerns the design ofthe subnet itself
interfaces to modems and Host computers. For reasons of
economic and political convenience, ARPA obtained common- Host-Subnet Considerations
carriercircuitsdirectlythroughgovernmentpurchasingchannels;
AT&T (Long Lines) is the central coordinator, although the The basicnotion ofasubnet leads directlyto aseries ofquestions
General Telephone Company is participating at some sites and about the relationship between the Hosts and the subnet: What
other common carriers may eventually become involved. In tasks shall be performed by each? What constraints shall each
January 1969, Bolt Beranekand Newman Inc. (BEN) began work place on the other? What dependence shall the subnet have on
on the design and implementation of IMPs; a four-node test the Hosts?Inconsideringthesequestions, wewereguidedbythe
network was scheduled for completion by the end of 1969 and followingprinciples: (1)Thesubnetshouldfunctionasacommuni-
plans were formulated to include a total often sites by mid-1970. cationssystemwhose essentialtaskis totransferbits reliablyfrom
This paper discusses the design ofthe subnet and describes the a source location to a specified destination. Bit transmission
hardware, the software, and the predicted performance of the should be sufficiently reliable and error free to obviate the need
IMP. The issues ofHost-to-Host protocol and network utilization for special precautions (such as storage for retransmission) on the
are barely touched upon; these problems are currently being partofthe Hosts; (2)The averagetransittimethrough the subnet
consideredbytheparticipatingHostsandmaybeexpectedtobea should be under a halfsecond to provide for convenient interac-
subject oftechnical interest for many years to come. tive use ofremote computers; (3)The subnet operation shouldbe
At this time, in late 1969, the test network has become an completelyautonomous. Sincethesubnetmustfunctionasastore
oapnedraitmipnlgemreeanltiatyt.ioInMoPfsIhMaPvsefaolrrseiaxdayddbieteinonailnsstiatlelsedisaptrfoocueredsiitnegs., aHnosdt.forTwhaerdIsMyPstemmu,satncIoMntPinmuuesttonootpebreadteepwehnedtehnetrupthoeniHtossltocails
The common carriers have installed 50-kilobit leased service functioningproperlyornotand must notdepend upon a Hostfor
connecting the first four sites and are preparing to install circuits buffer storage or other logical assistance such as program reload-
at six additional sites. ing. The Hostcomputermustnotinanywaybeabletochangethe
The design of the network allows for the connection of logical characteristics of the subnet; this restriction avoids the
additional Hostsites. Amapofaprojectedeleven-nodenetworkis mischievous or inadvertent modification of the communication
shown in Fig, 2. The connections between the first four sites are system by an individual Host user; (4) Establishment ofHost-to-
indicated by solid lines. Dotted lines indicate planned connec- Host protocol and the enormous problem ofplanning to commu-
tions. nicate between different computers should be an issue separated
from the subnet design.
Messages, Links, and RFNMs In principle, a single transmis-
sionfrom one Hosttoanothermayrangefromafewbits, aswitha
single teletype character, up to arbitrarily many bits, as in avery
long file. Because ofbufferinglimitations in the subnet, an upper
limit was placed on the size ofan individual Host transmission;
8095 bits was chosen for the maximum transmission size. This
Hostunitoftransmission iscalledamessage. Thesubnetdoesnot
impose any pattern restrictions on messages; binary text may be
transmitted. Messages may be ofvariable length; thus, a source
Host must indicate the end ofa message to the subnet.
A major hazard in a message switched network is congestion,
whichcanariseeitherduetosystemfailuresortopeaktrafficflow.
Congestion typically occurs when a destination IMP becomes
flooded with incoming messages for its Host. If the flow of
messages to this destination is not regulated, the congestion will
Fig. 2. Network map (from the ARPANET Directory). back up into the network, affectingotherIMPs and degradingor
404 Part2 RegionsofComputerSpace Section5 ' Networics
I
even completely clogging the communication service. To solve present, this delay is set to 10 ^.sec. Anydelay introducedbythe
this problem we developed a quenching scheme that limits the Host in the handshake procedure further slows the rate.
flowofmessages to agivendestinationwhen congestionbegins to
occur or, more generally, when messages are simply not getting System Failure Considerable attention has been given to the
through. possible efiects on a Host ofsystem failures in the subnet. Minor
Thesubnettransmitsmessagesoverunidirectionallogicalpaths system failures (e.g., temporary line failures) will appear to the
between Hosts known as links. (A link is a conceptual path that Hosts only in the form ofreduced rate ofservice. Catastrophic
has no physical reality; the term merely identifies a message failuresmay, however, resultinthelossofmessagesoreveninthe
sequence.) The subnet accepts only one message at a time on a loss ofsubnet communication. IMPs inform a Host ofall relevant
given link. Ensuing messages on that link will be blocked from system failures. Additionally, should a Host computer go down,
enteringthe subnetuntilthe source IMPlearns thatthe previous the information is propagated throughout the subnet to all IMPS
messagehasarrivedatthedestinationHost. WhenaUnkbecomes sotheymaynotifytheirlocal Host ifitattemptstosendamessage
unblocked, the subnet notifies the source Host by sending it a to that Host.
special control message known as Ready for Next Message (or
RFNM), which identifies the newly unblocked link. The source Specific Subnet Design
Host may utilize its connection into the subnet to transmit
messages overother links, whilewaitingto send messages on the The overriding consideration that guided the subnet design was
blocked links. Up to 63 separate outgoing links may exist at any reliability. Each IMP must operate unattended and reliably over
tHhoestdessittei.naWtihoennHogsitviangndthaelsinukbnneutmabemresisnagteh,e tfihrestH3o2stbitsspeocfiftihees WloengwpeerrieodcsonwviitnhcmeidnithmaatlidtowwanstiimmpeofrotranmtaifnotreneaacnhceIaMnPdrienpatihr.e
message (known as the leader). The IMPs then attend to route subnettooperateautonomously, notonlyindependentlyofHosts,
selection, delivery, and notification ofreceipt. This use oflinks but insofar as possible from other IMPs as well; any dependency
and RFNMs also provides for IMP-to-Host deliveryofsequences between one IMP and another would merely broaden the area
ofmessages in proper order. Because the subnet allows only one jeopardized by one IMF's failure. The need for reliability and
message at a time on a given link, Hosts never receive messages autonomybears directly upon theform ofsubnetcommunication.
out ofsequence. This section describes the process of message communication
within the subnet.
Host-IMF Interfacing Each IMP will initially service a single
Host. However, we have made provision (both in the hardware Message Handling Hosts communicate with each other via a
and software) for the IMP to service up to four Hosts, with a sequence ofmessages. An IMP takes in a message from its Host
corresponding reduction in the number ofpermitted phone line computer in segments, forms these segments into packets (whose
connections. Connecting an IMP to a wide variety of difierent maximum size is approximately 1000 bits), and ships the packets
Hosts requires ahardware interface, some part ofwhich must be separatelyintothenetwork. ThedestinationIMPreassemblesthe
custom tailored toeach Host. We decided, therefore, to partition packetsanddelivers them in sequencetothe receivingHost, who
the interface such that a standard portion would be built into the obtains them as a single unit. This segmentation ofa message
IMP, andwouldbe identical forall Hosts, while aspecialportion during transmission is completely invisible to the Host comput-
ofthe interface would be unique to each Host. The interface is ers. Figures 3, 4, and 5 illustrate aspects ofmessage handling.
designed to allow messages to flow in both directions at once. A The transmitting Host attaches an identifying leader to the
bit serial interface was designed partly because it requiredfewer beginning ofeach message. The IMP forms a header by adding
lines for electrical interfacingand was, therefore, less expensive, further information for network use and attaches this header to
and partly to accommodate conveniently the variety of word each packet ofthe message.
lengths inthedifierent Hostcomputers. Thebitraterequirement Each packet is individually routed from IMP-to-IMP through
on the Host line is suSiciently low that parallel transfers are not the network toward the destination. At each IMPalong the way,
necessary. the transmitting hardware generates initial and terminal framing
TheHostinterfeceoperatesasynchronously, eachdatabitbeing charactersandparitycheckdigitsthatareshippedwiththepacket
passed across the interface viaa Ready ForNext Bit/There's Your and are used forerrordetectionbythe receivinghardwareofthe
Bit handshake procedure. This technique permits the bit rate to next IMP.
adjust to the rate of the slower member ofthe pair and allows Errors in transmission can afiect a packet by destroying the
necessary interruptions, when words must be stored into or framing and/or by modifying the data content. Ifthe framing is
retrieved from memory. The IMP introduces between bits a disturbed in anyway, the packet eitherwill not be recognizedor
(manually) adjustable delay that Hmits the maximum data rate; at will be rejected by the receiver. In addition, the check digits
Chapter24 ' TheInterfaceMessageProcessorfortheARPAComputerNetwork 405
406 Part2 RegionsofComputerSpace Section5 Networks
I
tablebycombiningitsneighbors'estimateswithitsownestimates Messages intended for dead Hosts (which are not the same as
of the delay to that neighbor. The estimated delay to each dead IMPs) cannot be delivered; therefore, these messages
neighbor is based upon both queue lengths and the recent require special handling to avoid indefinite circulation in the
performance ofthe connecting communication circuit. For each network and spurious arrival at a later time. Such messages are
destination, the table is then made to specifythat selectedoutput purged from the network either at the source IMP or at the
lineforwhich thesumoftheestimateddelaytotheneighborplus destination IMP. Dead Host information is regularly transmitted
the neighbor's delay to the destination is smallest. with the routing information. A Host computer is notified about
The routing table is consistently and dynamically updated to anotherdeadHostonlywhenattemptingtosendamessagetothat
adjust for changing conditions in the network. The system is Host.
adaptive to the ups and downs oflines, IMPs, and congestion; it An IMP may detect a major failure in one ofthree ways: (1) A
does not require theIMPto know the topologyofthenetwork. In packetexpectedforreassemblyofamultiple packetmessagedoes
particular, an IMP need not even know the identity of its notarrive. Ifamessageisnotfullyreassembledin 15minutes, the
immediate neighbors. Thus, the leased circuits could be reconfi- system presumes a failure. The message is discarded by the
gured to a new topology without requiring any changes to the destination IMPandboththesourceIMPandthesourceHostare
IMPs. notifiedviaaspecial RFNM. (2)TheHostdoesnottakeamessage
from its IMP. If the Host has not taken a message after 15
SubnetFailures Thenetworkisdesignedtobelargelyinvulner- minutes, thesystem presumesthatitwillnevertakethemessage.
abletocircuitorIMPfailureaswellastooutagesformaintenance. Therefore, as in thepreviouscase, themessage isdiscardedanda
Special status and test messages are employed to help cope with special RFNM is returned to the source Host. (3) A link is never
variousfailures. Intheabsenceofregularpacketsfortransmission unblocked. Ifa Unk remains blocked for longer than 20 minutes,
over a line, the IMP program transmits special hello packets at the system again presumes a failure; the link is then unblocked
half-secondintervals. Theacknowledgmentforahellopacketisan and an error message is sent to the source Host. (This last time
/ heard you packet. interval is slightly longer than the others so that the failure
Adeadline is defined by the sustained absence (approximately mechanisms for the first two situations will have a chance to
2.5 seconds) on that line of either received regular packets or operate and unblock the link.)
acknowledgments; no regular packets will be routed into a dead
line, and any packets awaiting transmission will be rerouted. Reliability and Recovery Procedures For higher system re-
Routingtables inthenetworkareadjustedautomaticallytoreflect liability, special attention was placed on intrinsic reliability,
the loss. We require acknowledgment ofthirty consecutive hello hardware test capabilities, hardware/software failure recovery
packets (an event which consumes at least 15 seconds), before a techniques, and proper administrative mechanisms for &ilure
dead line is defined to be alive once again. management.
A dead line may reflect trouble either in the communication To improve intrinsic reliability, we decided to ruggedize the
facilities or in the neighboring IMP itself. Normal line errors IMP hardware, thus incurring an approximately ten percent
causedbydropouts, impulsenoise, orotherconditions shouldnot hardware cost penalty. For ease in maintenance, debugging,
result in adead line, because such errors typically lastonlyafew program revision, and analysis ofperformance, all IMPs are as
miliseconds, and only occasionally as long as a few tenths ofa similaras possible; theoperationalprogram andthehardwareare
second. Therefore, we expect that a line will be defined as dead nearly identical in all IMPs.
only when serious trouble conditions occur. Ifdead lines elimi- To improve hardware test capabilities, we built special cross-
nate all routes between two IMPs, the IMPs are said to be patching features into the IMP's interface hardware; these
disconnected and each of these IMPs will discard messages features allow program-controlled connection ofoutput lines to
destined for the other. Disconnected IMPs cannot be rapidly corresponding input lines. These crosspatching features have
detected from the delay estimates that arrive from neighboring been invaluable in testing IMPs before and during field installa-
IMPs. Consequently, additional information is transmitted be- tion, and they should continue to be very useful when troubles
tween neighboringIMPs to helpdetectthiscondition. Each IMP occur in the operatingnetwork. These hardware test featuresare
transmits to its neighbors the length ofthe shortest existingpath employed by a special hardware test program and may also be
(i.e., number of IMPs) from itself to each destination. To the employed by the operational program when a line difiBculty
smallest such received number per destination, the IMP adds occurs.
one. This incremented number is the length ofthe shortest path The IMP includes a 512-word block ofprotected memory that
from that IMP to the destination. Ifthe length ever exceeds the secures special recovery programs. An IMP can recover from an
number ofnetwork nodes, the destination IMP is assumed to be IMP failure in two ways: (1) In the event of power failure, a
unreachable and therefore disconnected. power-fail interruptpermits the IMP to reachaclean stopbefore
Chapter24 TheInterface Message ProcessorfortheARPAComputerNetwork 407
|
the program is destroyed. When power returns, a special selected Host every ten seconds. In addition, a record ofactual
automatic restart feature turns the IMP back on and restarts the packet arrival times on modem lines allows for the modeling of
program. (We considered several possibilities for handling the line traffic. (As part ofits research activity, the group at UCLA is
packets found in an IMP during a power failure and concluded acting as a network measurement center; thus, statistics for
that no plan to salvage the packets was both practical and analysis will normally be routed to the UCLA Host.)
foolproof For example, we cannot know whether the packet in Perhaps the most powerfulcapabilityfornetworkintrospection
transmission at the time offailure successfully left the machine is tracing. Any Host message sent into the network may have a
before the powerfailed. Therefore, we decided simply to discard "tracebit" set in the leader. Wheneveritprocesses apacketfrom
all the packets and restart the program.) (2) The second recovery suchamess—age, theIMPkeepsspecial recordsofwhathappensto
mechanism is a "watchdog timer," which transfers control to thatpacket e.g., howlongthepacketisonvariousqueues,when
protected memory whenever the program neglects this timerfor it comes in and leaves, etc. Each IMP that handles the traced
about one minute. In the event ofsuch transfer, the program in packet generates special trace report messages that are sent to a
unprotected memoryispresumedtobedestroyed(eitherthrough specified Host; thus, acompleteanalysis ofwhat hashappenedto
a hardware transient or a software failure). The program in that message can be made. When used in an orderly way, this
protected memory sends a reload request down a phone line tracing facility will aid in understanding at a very detailed level
selectedat random. The neighboring IMP responds by sendinga the behavior of routing algorithms and the behavior of the
copyofitswhole program backon the phone line. Anormal IMP network under changing load conditions.
would discard this message because it is too long, but the
recovering IMP can use it to reload its program. Flexibility Flexibility for modifications in IMP usage has been
Everything unique to a particular IMP must thus reside in its provided by several built-in arrangements: (1) provision within
protected memory. Only one register (containing the IMP the existingcabinetforanadditional4Kcorebank; (2) modularity
number) currently diflFers from IMP-to-IMP. The process of ofthe hardware interfaces; (3) provision for operation with data
reloading, which requires a few seconds, can be tried repeatedly circuits of widely different rates; (4) a program organization
untilsuccessful; however, ifafterseveralminutestheprogramhas involving many nearly self-contained subprograms in the IMP
not resumed operation, a laterphase ofthewatchdogtimer shuts program structure.
ofiFall power to the IMP. This last aspect offlexibility presents asomewhat controversial
Inadditiontoprovidingrecoverymechanismsforbothnetwork design choice. There are many advantages to keeping all IMP
and IMP failures, we have incorporated into the subnetacontrol software nearly identical. Because ofthe experimental nature of
center that monitors network status and handles trouble reports. thenetwork, however, wedonotyetknowwhetherthisluxuryof
The control center, located at a network node, initiates and identical programs will be an optimal arrangement. Several
follows up any corrective actions necessary for proper subnet potential ap—plications of"Host-unique" IMP software have been
functioning. Furthermore, this centercontrols and schedules any considered e.g., using ASCII conversion routines in each IMP
modifications to the subnet. to establish a "Network ASCII" and possibly to simplify the
protocol problems ofeach Host. As ofnow, the operational IMP
Introspection Because the network is experimental in nature, programincludesastructurethatpermitsuniquesoftwareplug-in
considerableefforthasbeenallocatedtodevelopingtoolswhereby packages at each Host site, but no plug-ins have yet been
the network can supply measures of its own performance. The constructed.
operational IMPprogram is capable oftaking statisticson its own
performance on a regular basis; this function may be turned on
and offremotely. The various kinds ofresulting statistics, which The Hardware
are sent via the network to a selected Host for analysis, include
"snapshots," ten-second summaries and packet arrival times. We selected a Honeywell DDP-516 for the IMP processor
Snapshots are summaries ofthe internal status ofqueue lengths because we wanted a machine that could easily handle currently
and routing information. A synchronization procedure allows anticipated maximum traffic and that had already been proven in
these snapshots, which are taken every halfsecond, to occur at the field. We considered only economic machines with fast cycle
roughlythe same time inallnetworkIMPs;a Hostreceivingsuch times and good instruction sets. Furthermore, we needed a
snapshot messages could presumably build up an instantaneous machine with a particularly good I/O capability and that was
picture ofoverall network status. Ten-second summaries include available in a ruggedized version. The geographical proximity of
such IMP-generatedstatisticsasthenumberofprocessedmessag- the supplier to BBN was also a consideration.
es ofeach kind, the number ofretransmissions, the traffic to and The basic machine has a 16-bit word length and a 0.96-jjisec
from the local Host, and so forth; this statistical data is sent to a memory cycle. The IMP version is packaged in a single cabinet.
408 Part2 RegionsofComputerSpace SectionS Networlcs
I
and includes a 12K memory, a set of 16 multiplexed channels
(which implement a 4-cycle data break), a set of 16 priority
interrupts, a 100-^Jlsec clock, and a set of programmable status
lights. Also packaged within this cabinet are special modular
interfaces for connecting the IMP to phone line modems and to
Hostcomputers; these interfaces usethe samekindof1 MHzand
5 MHz DTL packs from which the main machine is constructed.
In addition, a number of features that have been incorporated
make the IMP somewhat resilient to a variety offailures.
Teletypes and high-speed paper tape readers which are at-
tached to the IMPs are used only for maintenance, debugging,
and system modification; in normal operation, the IMP runs
without any moving parts except fans. Within the cabinet, space
has been reserved for an additional 4K memory. Figure 6 is a
picture ofan IMP, and Figure 7 shows its configuration.
Ruggedization ofcomputer hardware for use in friendly envi-
ronments is somewhat unusual; however, we felt that the consid-
erable difficulty that IMP failures can cause the networkjustified
this step. Although the ruggedized unit is notfully "qualified" to
MIL specs, it does have greater resistance to temperature
variance, mechanical shock and vibration, radio frequency inter-
Chapter24 TheInterface MessageProcessorfortheARPAComputerNetwork 409
]
sync pattern that keeps them in character sync. Bit sync is (512 words per page). These programs communicate primarily
maintained by the modems themselves, which provide both throughcommonregistersthatreside inpagezeroofthemachine
transmit and receive clockingsignals to the interfaces. When the andthataredirectlyaddressablefromallpagesofmemory. Amap
program initiates transmission, thehardware firsttransmitsapair ofcore storage is shown in Fig. 8. Seven ofthe twelve programs
of initial framing characters (DLE, STX). Next, the text ofthe are directly involved in the flow ofpackets through the IMP: the
packet is taken word by word from the memory and shifted taskprogramperformsthemajorportionofthepacketprocessing,
seriallyonto the phone line. Atthe end ofthedata, thehardware includingthe reassemblyofHost messages; themodemprograms
generates a pair ofterminal framing characters (DLE, ETX) and (IMP-to-Modem and Modem-to-IMP) handle interrupts and
shifts them onto the phone line. After the terminal framing resetting ofbufiers for the modem channels; the Host programs
characters, the hardware generates and transmits 24 check bits. (IMP-to-Hostand Host-to-IMP)handleinterruptsandresettingof
Finally, the interface returns to idle (sync) mode. bufiers for the Hostchannels, build packetheaders during input,
The hardware doubles any DLE characters within the binary andconstructRFNMs thatarereturnedtothesource Hostduring
data train (that is, transmits them twice), thereby permitting the output;thetime-outprogrammaintainsasoftwareclock, timesout
receivinginterface hardware to distinguish them from the termi- unacknowledged packets for retransmission,^ and attends to
nal framing characters and to remove the duplicate. Transmitted infrequent events; the link program assigns and verifies message
packets are ofa known maximum size; therefore, any overflowof numbersandkeeps trackoflinks. Abackgroundloopcontains the
inputbufierlength is evidenceoferroneous transmission. Format remaining five programs anddealswith initialization, debugging.
errors in the framing also register as errors. Check bits are
computedfrom thereceiveddataandcomparedwiththereceived
checkbitstodetecterrorsinthetext. Anyoftheseerrorssetaflag COMMON STORAGE
and cause a program interrupt. Before processing a packet, the RESTART^
program checks the error flag to determine whether the packet INITIALIZATION BACKGROUND
was received correctly. TASK
MODEM TO IMP
IMP Software IMP TO MODEM
HOST TO IMP
Implementation of the IMPs required the development of a IMP TO HOST
sophisticatedoperationalcomputerprogramandthedevelopment TIMEOUT
of several auxiliary programs for hardware tests, program con-
struction, and debugging. This section discusses in detail the LINK (with table)
24 PAGES
design of the operational program and briefly describes the
auxiliary software. TABLES
DEBUG
OperationalProgram
The principal function ofthe operational program is the process-
ing of packets. This processing includes segmentation of Host
messages into packets for routing and transmission, building of
headers, receiving, routingand transmittingofstore and forward
packets, retransmittingofunacknowledgedpackets, reassembling
received packets into messages for transmission to the Host, and
generating of RFNMs and acknowledgments. The program also
monitors network status, gathers statistics, and performs on-line
testing. This real-time program is an efficient, interrupt-driven,
involute machine language program that occupies about 6000 1 PAGE = 512 WORDS
BUFFER SPACE
words ofmemory. It was designed, constructed, and debugged I
over a period ofabout a year by three programmers. PROTECTED PAGE
The entire program is composed oftwelve iunctionally distinct
pieces;eachpieceoccupiesnomorethanoneortwopagesofcore Fig. 8. Map of core storage.
410 Part2 RegionsofComputerSpace Section5 Networl(s
I
testing, statisticsgatheringandtracing. Afterabriefdescriptionof Output: A separate output queue is constructed for each
datastructures, wewill discuss packet processingin some detail. modem channel and each Host channel. Each modem output
queue issRubFdNivMided intoanacknowledgmentqueue, apriority
Buffer Allocation, Queues, and Tables The major system data queue, a queue, and a regular message queue, which
structures (see Table 1) consist of bufiFers and tables. The are serviced in that order. Each Host output queue is
buffer-storage space is partitioned into about 70 fixed length subdividedintoacontrolmessagequeue, apriorityqueue, and
buffers, each of which is used for storing a single packet. An aindriecgautleadrormdeesrs.age queue, which are also serviced in the
unused buffer is chained onto a free buffer list and is removed
from this list when it is needed to store an incoming packet. A Sent: A separate queue for each modem channel contains
packet, oncestoredinabuffer, isnevermoved. Afterapackethas packets thathavealreadybeen transmitted on that linebutfor
been successfiilly passed along to its Host or to another IMP, its which no acknowledgment has yet been received.
bufferis returnedtothefreelist. Thebufferspaceispartitionedin Reassembly:The reassemblyqueuecontainsthosepacketsthat
such a way that each process (store and forward traffic. Host are being reassembled into messages for the Host.
traffic, etc.) is always guaranteed some buffers. For the sake of
program speedand simplicity, no attemptis made to retrievethe Tablesincoreareallocatedforthestorageofqueuepointers,for
space wasted by partially filled buffers. trace blocks, for reassembly information, for statistics, and for
In handling store and forward traffic, all processing is on a per links. Most noteworthy ofthese is the link table, which is usedat
packet basis. Further, although traffic to and from Hosts is the source IMP for assignment of message numbers and for
composed olmessages, the IMP rapidly converts to dealingwith blocking and unblocking links, and at the destination IMP to
packets;the HosttransmitsamessageasasingleunitbuttheIMP verify message numbers for sequence control.
takes it one bufferat a time. As each buffer is filled, the program
selects another bufferfor input until theentire message has been Packet Flow and Program Structure Figure 9 is a schematic
provided for. These successive buffers will, in general, be drawing of packet processing; the processing programs are
scattered throughout the memory. An equivalent inverse process described below.
occurson output to the Hostafterallpackets ofthe message have The Host-to-IMP routine (H -^ I) handles messages being
arrivedatthedestinationIMP. Noattemptisevermadetocollect transmitted from the local site. The routine uses the leader to
thepacketsofamessageintoacontiguousportionofthememory. construct aheader that is prefixed to each packet ofthe message.
Buffers currently in use are eitherdedicated to an incomingor It also creates alink forthe message ifnecessary, blocks the link,
an outgoing packet, chained on a queue awaiting processing by puts thepacketsofthemessageonthe Hosttaskqueueforfurther
the program, or being processed. Occasionally, a buffer may be processing by the task routine, and triggers the programmable
simultaneously found on two queues; this situation can occur task interrupt. The routine thenacquiresafreebufferandsets up
when a packet is waiting on one queue to be forwarded and on a new input. The routine tests a hardware trouble indicator,
another to be acknowledged.
There are four principal types ofqueues:
10 <*f)S' / ~*"« \
Task: Packetsreceivedon Hostchannelsareplacedonthe Host rr-
task queue. All received acknowledgments, dead Host and
routing information, / heard you and hello packets are placed ^'- •'^' P kit
on the system task queue; all otherpackets from the modems
are placed on the modem task queue. The program services — ^''t*^"«r"
the system task queue first, then the Host task queue, and ^OtBuG« BilL—
finally the modem task queue. .^^ •OOctmtINHCfB(- / UuBAtlSNir
BAcTGrOuND
Table 1 Program Data Structures
—
5010200 wwoorrddss——qmueesuseagpeoibnutfefresrstorage MHQDfM OUrl-J ^ y»
300 words—trace blocks
100 words—reassembly blocks
150 words—routing tables I I • OUIUl 1 J- fOV
400 words—lini< tables
300 words statistics tables
Fig. 9. Internal packetflow.
Chapter24 TheInterface Message ProcessorfortheARPAComputerNetwork 411
verifies the message format, and checks whether or not the The time-out routine is started every 25.6 msec (called the
destination is dead, the linktable is full, or the linkblocked. The time-out period) by a clock interrupt. The routine has three
routine is serially reentrant and services all Hosts connected to sections: thefast time-out routine, which "wakes up" any Hostor
the IMP. modem interrupt routine that has languished (for example, when
The Modem-to-IMP routine (M -^ I) handles inputs from the the Hostinputcouldnotimmediatelystartanewinputbecauseof
modems. This routine consists ofseveral identical routines, one a shortage in buffer space); the middle time-out routine, which
for each modem channel. (Such duplication is useful to obtain retransmitsany packets that havebeen toolongonamodem sent
higher speed.) This routine sets up an input buffer (normally queue; and the slow time-out routine, which marks lines as alive
obtained from the free Ust), places the received packet on the or dead, updates the routing tables and does long term garbage
appropriate task queue, and triggers the programmable task collection ofqueues and other data structures. (For example, it
interrupt. Should nofree bufiersbeavailableforinput, thebuflFer protectsthesystemfrom thecumulativeeffectofsuchfailuresasa
attheheadofthemodemtaskqueueispreempted. Ifthemodem lostpacketofamultiplepacketmessage, wherebuffersaretiedup
task queue is also empty, the received packet is discarded by inmessage reassembly.) Italsodeleteslinksautomaticallyalter15
setting up its buffer for input. However, a sufficient number of seconds ofdisuse, after 20 minutes ofblocking, orwhen an IMP
free buffers are specifically reserved to assure that received goes down.
acknowledgments, routing packets, and the like are rarely These three routines are executed in the followingpattern:
discarded. FFFF FFFF FFFF FFFF FFFF FFFF
The task routine uses the header information to direct packets M M M M M . . .
to theirproperdestination. The task routine is drivenby the task
interrupt, which is setwheneverapacket is put on a task queue. and, althoughtheyrunoffacommoninterrupt, areconstructedto
The task routine routes packets from the Host taskqueueontoan allow faster routines to interrupt slower ones should a slower
output queue determined from the routing algorithm. routine not complete execution before the next time-out period.
Foreachpacketonthemodem taskqueue, thetaskroutinefirst The link routineenters, examines, anddeletesentriesfromthe
determineswhethersufficientbufferspaceisavailable. IftheIMP linktable. Atablecontainingaseparatemessagenumberentryfor
has a shortage of store and forward buffers, the buffers on the many links to every possible Host would be prohibitively large.
modem task queue are simply returned to the free list without Therefore, the table contains entries only for each of 63 total
further processing. Normally, however, an acknowledgment outgoinglinksatany Hostsite. Hashingisusedtospeedaccessing
packet is constructed and put near the front ofthe appropriate ofthis table, butthelinkprogram isstillquitecostly; itusesabout
modem output queue. The destination of the packet is then tenpercentofbothspeedandspaceinaconceptuallytrivialtask.
inspected. If the packet is not for the local site, the routing
algorithm selects a modem output queue for the packet. If a Initialization and Background Loop The IMPprogram starts in
packet for the local site is a RFNM, the corresponding link is an initialization section that builds the initial data structures,
unblocked and the RFNM is put on a queue to the Host. Ifthe preparesforinputsfrom modemand Hostchannels, andresetsall
packetisnotaRFNM, itisjoinedwithothersofthesamemessage program switches to their nominal state. The program then falls
on the reassembly queue. Whenver a message is completely into thebackground loop, whichisanendlesslyrepeatedseriesof
reassembled, the packets of the message are put on an output low-priority subroutines that are interrupted to handle normal
queue to the Host for processingby the IMP-to-Host routine. traffic.
Inprocessingthe system taskqueue, thetaskroutine returnsto The programs in the IMPbackgroundloopperformavarietyof
the free list those buffers from the sent queue that have been functions: TTY is used to handle the IMP Teletype traffic;
referenced by acknowledgments. Anypackets skippedoverbyan DEBUG, to inspect or change IMP core memory; TRACE, to
acknowledgment are designated for retransmission. Routing, 7 transmitcollected informationabouttracedpackets;STATISTICS,
heard you, and hello packets are processed in a straightforward to take and transmit network and IMP statistics; PARAMETER-
fashion. CHANGE, to alter the values ofselected IMP parameters; and
—
The IMP-to-Modem routine (I > M) transmits successful pack- DISCARD, to throw away packets. Selected Hosts and IMPs,
ets from the Modem output queue. Aftercompleting the output, particularly the Network Measurement Center and the Network
this routine places any packet requiring acknowledgment on the Control Center, will find it necessary or useful to communicate
sent queue. with one or more of these background loop programs. So that
—
The IMP-to-Host routine (I » H) sets up successive outputs of theseprogramsmaysendandreceivemessagesfromthenetwork,
packets on the Host output queues and constructs a RFNM for they are treatedas"fake Hosts." Ratherthanduplicatingportions
eachnon-controlmessagedeliveredtoaHost. RFNM packetsare of the large IMP-to-Host and Host-to-IMP routines, the back-
returned to the system via the Host task queue. ground loop programs are treatedas fftheywere Hosts, and they