Skip to content

The OFED package

5.00 avg. rating (97% score) - 1 vote

The OpenFabrics Enterprise Distribution (OFED) is a package for Linux that includes all of the needed software (libraries, utilities and drivers) in order to work with RDMA enabled protocols (InfiniBand and iWARP).

OFED content

This package includes many components to allow the support of RDMA. Here is a diagram, published by the OpenFabrics, that describes the software layers:


Applications and utilities

  • Diagnostics tools - Tools for debugging and diagnostics in an InfiniBand fabric
  • Open Subnet Manager (OpenSM) - Program that manages and configures an InfiniBand fabric
  • Various Message Passing Interface (MPI) packages - API for development of applications that can be executed in parallel on machine machines. Those packages support many interconnect technologies, including InfiniBand

User level APIs

APIs that allow development of RDMA enabled applications at the user level.

  • RDMA verbs - API of the verbs interface that was defined in the InfiniBand and iWARP specifications
  • uDAPL - API that was defined by the DAT collaborative
  • RDMA Communication Manager (RDMA-CM) - libraries for setting up RDMA connections

Upper Level Protocols (ULPs)

Modules that allow upper level and legacy applications to work over RDMA.

  • IP over InfiniBand (IPoIB) - A module that encapsulates IP over InfiniBand
  • Socket Direct Protocol (SDP) - A module that implements TCP sockets over InfiniBand
  • Reliable Datagram Sockets (RDS) - A module that allows high-performance, low latency reliable connectionless protocol for delivering datagrams
  • SCSI RDMA Protocol (SRP) - A module that allows one computer to access SCSI devices attaches to another computer over RDMA
  • iSCSI Extensions for RDMA (iSER) - A modules that allows one computer to access iSCSI devices attaches to another computer over RDMA

Mid layers

Kernel modules that enables the RDMA support for both user and kernel level.

  • IB core - The RDMA core module for the RDMA implementation in the kernel
  • CMA - The communication manager module
  • MAnagement Datagram (MAD) - Module for handling MADs in the InfiniBand fabric
  • Subnet Administrator (SA) client - Module for handling communication with the SA


Low level driver for the RDMA devices (HCAs and RNICs) of various vendors, for example:

  • Mellanox Technologies
  • Qlogic
  • Chelsio Communications
  • IBM
  • Intel


OFED is being developed, tested and released as a joint effort of many companies and organizations. It being released every six months and supports various RDMA devices and Linux distributions.


Share Our Posts

Share this post through social bookmarks.

  • Delicious
  • Digg
  • Newsvine
  • RSS
  • StumbleUpon
  • Technorati


Tell us what do you think.

  1. jjoba says: April 29, 2014

    I am playing around with iSER for my university project. I use OFED as we also have a QLogic Infiniband card on this machine.
    I noticed that using iSER from OFED does not go well with the Mellanox cards. On the other hand, using MLNX_OFED 1.5.3-3 , it seems to work really well. But I cannot get my qlogic card to speak.

    Which brings me to a question: How well is iSER really tested in "generic" OFED?

    • Dotan Barak says: April 29, 2014

      OFED is considered quite old (it was released on Jul 25, 2011),
      and MLNX_OFED 1.5.3-3 is more updated but I would suggest you to always use the latest MLNX-OFED/OFED that is available for your OS.

      Like in any other SW package, bugs may happen and latest releases usually fix issues that appeared in earlier ones.

      AFAIK, the "generic" OFED releases are a joint effort of multiple companies in the OpenFabrics community and all modules should be tested.


Add a Comment

Fill in the form and submit.

Time limit is exhausted. Please reload CAPTCHA.