User:Wjw1961/ServerNet

ServerNet is a switched fabric communications link primarily used in high-performance computing. Its features include good scalability, clean fault containment, error detection and failover. The ServerNet architecture specification defines a connection between nodes, either processor or high performance I/O nodes such as storage devices. Tandem Computers developed the original ServerNet architecture and protocols for use in its own proprietary computer systems starting in 1992, and released the first ServerNet systems in 1995. Early attempts to license the technology and interface chips to other companies failed, due in part to a disconnect between the culture of selling complete hardware / software / middleware computer systems and that needed for selling and supporting chips and licensing technology. A follow-on development effort ported the Virtual Interface Architecture to ServerNet, and Infiniband directly inherited many ServerNet features. After 25 years, systems still ship today based on the ServerNet architecture.

Description

Like Fibre Channel, PCI Express, Serial ATA, and many other modern interconnects, ServerNet uses point-to-point bidirectional serial links for the connection nodes, like processors and high speed peripherals such as disks. Different releases supported different signaling and data transfer rates.

OSI_model

Signaling rate

The initial release of ServerNet (a.k.a. ServerNet-I) used a source-clocked parallel interface with a 50 megabit per second signaling rate in each direction. Each data bit in a ServerNet-I link used NRZ coding, a connection carried a separate clock, and used a proprietary 8B/9B encoding on the data lines. Between chips on a single circuit board, each data bit and clock used a single wire. Connections through cables connecting different equipment cabinets used differential driver / receiver interface chips and pairs of signal wires.

ServerNet-II changed to use 1.25 (Gbit/s) fully serial links with 8B/10B encoding — every 10 bits sent carry 8bits of data — so that the useful data transmission rate is four-fifths the raw rate or 1 Gbit/sec. This coding scheme also removed the need for a separate clock line, and allowed the use of commodity transceiver chips and cables.

ServerNet-III doubled the bit rate to 2.5 Gbit/sec (raw) / 2.0 Gbit/sec (useful).

Latency

ServerNet-I switch chips have a latency of about 300 nanoseconds. ServerNet-II switch chips have a latency of about 200 nanoseconds.

Topology

ServerNet uses a switched fabric topology, as opposed to a network with a shared medium like Ethernet.

Like the channel model used in most mainframe computers, all transmissions begin or end at a channel adapter. Each processor contains a host channel adapter (HCA) and each peripheral has a target channel adapter (TCA). These adapters can also exchange information for security or quality of service. Unlike many such systems, however, all nodes in a ServerNet system operate as peers in the network. Some nodes, particularly processors, have more capabilities, while others, like disk or WAN interfaces, have fewer.

Messages

Data transmission uses packets carrying up to 64 data bytes that together to form messages. A message can be:

a direct memory access read from or, write to, a remote node (RDMA)
a channel send or receive
a transaction-based operation (that can be reversed)
a multicast transmission.
an atomic operation

Programming

ServerNet has no standard programming interface.

The standard only lists a set of "verbs"; functions that must exist. The syntax of these functions is left to the vendors. The most common to date has been the syntax developed by the OpenFabrics Alliance, which was adopted by most of the InfiniBand vendors, both for Linux and Windows.

History

The effort that produced ServerNet started in 1992. It merged two competing designs, the very limited "TBus" packetized interconnect used in some early abandoned efforts by the Tandem Integrity system developers in Austin, and the FOX-II, Dynabus, and T/16 IO channel interfaces developed in Cupertino and used in Tandem's Guardian system.

The cross-site development team used the K.I.S.S. principle as a major motivator, in two different senses. First, having a single interface to ServerNet allowed Tandem to concentrate its efforts and "do one thing well", in contrast to the Guardian systems' FOX, Dynabus, and IO channel interfaces, and the Integrity Unix systems' Reliable System Bus (RSB), IO bus, and V+ buses. Second, when facing individual design decisions, the team attempted to choose simple alternatives rather than complex ones (having all packets equal in priority, for example).

The development team envisioned ServerNet as a comprehensive "system area network", both connecting CPUs to each other within a system (replacing Dynabus) and between systems (replacing FOX) and providing all high speed I/O for "back-office" applications. The team, however, had no goal of replacing all other interconnects. The initial IO interface "simply" translated VMEbus transactions from IO controllers to ServerNet transactions. Disks still connected to SCSI buses, and standard LAN, WAN, and serial interfaces provided connections to distant networks and users.

The first public disclosure of ServerNet came in a paper ("A Flexible ServerNet-Based Fault-Tolerant Architecture") presented at the Twenty-Fifth International Symposium on Fault-Tolerant Computing, 1995.

Also in 1995, Tandem introduced the Integrity S4000, the first system to use the ServerNet interconnect. In 1997 Tandem introduced the NonStop Himalaya S-Series, its first ServerNet system running the Guardian operating system.

External links

US Patents

5,991,518 - Nov 23, 1999, Jardine et al: Method and apparatus for split-brain avoidance in a multi-processor
6,249,756 - Jun 19, 2001, Bunton et al: Hybrid flow control
6,374,282 - Apr 16, 2002, Bunton et al: Method and Apparatus for Tracking Multi-Threaded System Area Network (SAN) Traffic
6,487,628 - Nov 26, 2002, Duong et al: Peripheral component interface with multiple data channels and reduced latency over a system area network
6,493,343 - Dec 10, 2002, Garcia et al: System and method for implementing multi-pathing data transfers in a system area network
6,545,981 - Apr. 8, 2003, Garcia et al: System and Method for Implementing Error Detection and Recovery in a System Area Network
6,646,984 - Nov 11, 2003, Mehra et al: Network topology with asymmetric fabrics
6,724,762 - Apr 20, 2004, Garcia et al: System and method for implementing multi-pathing data transfers in a system area network
6,765,922 - Jul 20, 2004, Bunton et al: Speculative transmit for system area network latency reduction
6,738,344 - May 18, 2004, Bunton et al: Link extenders with link alive propagation
6,870,814 - Mar 22, 2005, Bunton et al: Link extenders with error propagation and reporting
6,882,656 - Apr 19, 2005, Bunton et al: Speculative transmit for system area network latency reduction
6,950,428 - Sep 27, 2005, Horst et al: System and method for configuring adaptive sets of links between routers in a system area network (SAN)
6,959,337 - Oct 25, 2005, McLaughlin et al: Networked system for assuring synchronous access to critical facilities

Category:Serial buses Category:Computer buses Category:Supercomputers Category:Computer networks