InfiniBand

What is InfiniBand?

InfiniBand is an industry standard communications specification the InfiniBand Trade Association (IBTA) developed. The specification defines a switched fabric architecture for interconnecting servers, communications infrastructure equipment, storage and embedded systems in the data center.

InfiniBand is known for its high performance and low latency. A connection that supports a 4x link width can deliver up to 400 gigabits per second throughput. There are plans for faster speeds in the future. InfiniBand is also highly scalable. It can support tens of thousands of nodes in a single subnet. It provides quality of service (QoS) and failover capabilities that make it well suited to high-performance computing (HPC) environments.

InfiniBand is one of the network fabrics that the non-volatile memory express over fabrics (NVMe-oF) storage protocol uses. NVMe-oF also runs over Ethernet, Fibre Channel (FC) and TCP/IP networks.

How does InfiniBand work?

InfiniBand is a channel-based fabric that facilitates high-speed communications between interconnected nodes. An InfiniBand network is typically made up of processor nodes, such as PCs, servers, storage appliances and peripheral devices. It also has network switches, routers, cables and connectors.

Each processor node is configured with a host channel adapter, and each peripheral device has a target channel adapter. InfiniBand adheres to a channel model similar to that used by many mainframe computers. All transmissions begin or end with a channel adapter. These adapters can exchange information that ensures security or works with a given QoS level.

In an InfiniBand network, data is transmitted in packets that are organized into messages. InfiniBand is a messaging service that can be used by a range of high-performing applications, such as data mining, deep learning and predictive analytics. InfiniBand supports the following types of communication messages:

• remote direct memory access read or write operations;
• channel send or receive messages;
• multicast transmissions; and
• reversible transaction-based operations.

The InfiniBand architecture specification was developed by merging two competing designs:

1. Future I/O, developed by Compaq, Hewlett-Packard and IBM; and
2. Next Generation I/O, developed by Intel, Microsoft and Sun Microsystems.