SCSI RDMA Protocol

The Linux SCSI Target Wiki

(Difference between revisions)
Jump to: navigation, search
(Results)
(Results)
Line 47: Line 47:
Please see [[SCSI RDMA Protocol/configFS]] for the running layout of a /sys/kernel/config/target/srpt configuration
Please see [[SCSI RDMA Protocol/configFS]] for the running layout of a /sys/kernel/config/target/srpt configuration
 +
 +
=== lsmod output ===
 +
 +
<code>
 +
 +
  Module                  Size  Used by
 +
  ib_srpt                26028  6
 +
  ib_umad                9422  8
 +
  iscsi_target_mod      201391  1
 +
  mlx4_ib                38241  0
 +
  target_core_pscsi      11045  0
 +
  target_core_file        6307  1
 +
  target_core_iblock      6606  1
 +
  target_core_mod      206442  28 ib_srpt,iscsi_target_mod,target_core_pscsi,target_core_file,target_core_iblock
 +
  configfs              19153  2 target_core_mod
 +
  ib_iser                23172  0
 +
  rdma_cm                23833  1 ib_iser
 +
  ib_cm                  24878  2 ib_srpt,rdma_cm
 +
  iw_cm                  6019  1 rdma_cm
 +
  ib_sa                  15537  2 rdma_cm,ib_cm
 +
  ib_mad                30330  5 ib_srpt,ib_umad,mlx4_ib,ib_cm,ib_sa
 +
  ib_core                39716  9 ib_srpt,ib_umad,mlx4_ib,ib_iser,rdma_cm,ib_cm,iw_cm,ib_sa,ib_mad
 +
  ib_addr                3982  1 rdma_cm
 +
  ipv6                  242427  21 ib_addr
 +
  iscsi_tcp              7729  0
 +
  libiscsi_tcp          10485  1 iscsi_tcp
 +
  libiscsi              29466  3 ib_iser,iscsi_tcp,libiscsi_tcp
 +
  scsi_transport_iscsi    24624  4 ib_iser,iscsi_tcp,libiscsi
 +
  dm_multipath          14016  0
 +
  scsi_dh                4656  1 dm_multipath
 +
 +
</code>
=== Interrupt stats ===
=== Interrupt stats ===
Line 52: Line 84:
MSI-X interrupt stats for target side operation:
MSI-X interrupt stats for target side operation:
-
<pre>
+
<code>
lenny64guest0:/usr/src/lio-core-2.6.git# cat /proc/interrupts | grep mlx4
lenny64guest0:/usr/src/lio-core-2.6.git# cat /proc/interrupts | grep mlx4
Line 61: Line 93:
   47:      1377      1308      1338      1322  PCI-MSI-edge      mlx4-async@pci:0000:00:05.0
   47:      1377      1308      1338      1322  PCI-MSI-edge      mlx4-async@pci:0000:00:05.0
-
</pre>
+
</code>
== Timeline ==
== Timeline ==

Revision as of 02:42, 27 March 2011

LIO Target
Logo
Mellanox Technologies, Ltd.
Mellanox Infiniband SRP fabric module
Original author(s) Vu Pham
Developer(s) Mellanox Technologies, Ltd.
Initial release March 18, 2012 (2012-03-18)
Stable release 4.1.0 / June 20, 2012;
8 years ago
 (2012-06-20)
Preview release 4.2.0-rc5 / June 28, 2012;
8 years ago
 (2012-06-28)
Development status Beta
Written in C
Operating system Linux
Type Fabric module
License GNU General Public License
Website www.mellanox.com
See Target for a complete overview over all fabric modules.

An SCSI RDMA Protocol (SRP, aka SCSI Remote Protocol) fabric module (tcm_ib_srpt) for Mellanox HCAs is available upon request. SRP defines a SCSI protocol mapping onto the Infiniband architecture and/or functionally similar cluster protocols.

The use of RDMA generally allows higher throughput and lower latency than TCP/IP based communication. RDMA is only possible with network adapters that support RDMA in hardware. Examples of such network adapters are InfiniBand HCAs and 10 GbE network adapters with iWARP support. While the SRP protocol has been designed to use RDMA networks efficiently, it is also possible to implement the SRP protocol over networks that do not support RDMA.

SRP is easier to implement than iSER, but iSER offers more management functionality, e.g. the target discovery infrastructure enabled by the iSCSI protocol.

Contents

Status

The SRP driver is currently being tested and optimized.

Specifications

SRP was not approved as an official standard. The following specifications are available as available as T10 Working Drafts:

Results

configfs

Please see SCSI RDMA Protocol/configFS for the running layout of a /sys/kernel/config/target/srpt configuration

lsmod output

 Module                  Size  Used by
 ib_srpt                26028  6 
 ib_umad                 9422  8 
 iscsi_target_mod      201391  1 
 mlx4_ib                38241  0 
 target_core_pscsi      11045  0 
 target_core_file        6307  1 
 target_core_iblock      6606  1 
 target_core_mod       206442  28 ib_srpt,iscsi_target_mod,target_core_pscsi,target_core_file,target_core_iblock
 configfs               19153  2 target_core_mod
 ib_iser                23172  0 
 rdma_cm                23833  1 ib_iser
 ib_cm                  24878  2 ib_srpt,rdma_cm
 iw_cm                   6019  1 rdma_cm
 ib_sa                  15537  2 rdma_cm,ib_cm
 ib_mad                 30330  5 ib_srpt,ib_umad,mlx4_ib,ib_cm,ib_sa
 ib_core                39716  9 ib_srpt,ib_umad,mlx4_ib,ib_iser,rdma_cm,ib_cm,iw_cm,ib_sa,ib_mad
 ib_addr                 3982  1 rdma_cm
 ipv6                  242427  21 ib_addr
 iscsi_tcp               7729  0 
 libiscsi_tcp           10485  1 iscsi_tcp
 libiscsi               29466  3 ib_iser,iscsi_tcp,libiscsi_tcp
 scsi_transport_iscsi    24624  4 ib_iser,iscsi_tcp,libiscsi
 dm_multipath           14016  0 
 scsi_dh                 4656  1 dm_multipath

Interrupt stats

MSI-X interrupt stats for target side operation:

lenny64guest0:/usr/src/lio-core-2.6.git# cat /proc/interrupts | grep mlx4

 43:   16864937   16864950   16864795   16864897   PCI-MSI-edge      mlx4-comp-0@pci:0000:00:05.0
 44:          0          0          0          0   PCI-MSI-edge      mlx4-comp-1@pci:0000:00:05.0
 45:          0          0          0          0   PCI-MSI-edge      mlx4-comp-2@pci:0000:00:05.0
 46:          0          0          0          0   PCI-MSI-edge      mlx4-comp-3@pci:0000:00:05.0
 47:       1377       1308       1338       1322   PCI-MSI-edge      mlx4-async@pci:0000:00:05.0

Timeline

Timeline of the LinuxIO
Release Details 2011 2012 2013 2014 2015
123456789101112 123456789101112 123456789101112 123456789101112 123456789101112
4.x Version 4.0 4.1
Feature LIO Core Loop back FCoE iSCSI Perf SRP
CM WQ FC
USB
1394
vHost Perf Misc 16 GFC iSER Misc VAAI Misc DIF Core
NPIV
DIF iSER DIF FC vhost TCMU Xen Misc Misc virtio 1.0 Misc NVMe OF
Linux 2.6.38 2.6.39 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22

See also

Notes

External links

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Google AdSense