QLogic

The Linux SCSI Target Wiki

Revision as of 19:21, 4 July 2012

See Target for a complete overview over all fabric modules.

TCM_QLA2xxx running at line rate HW target mode from with PCIe device passthrough with MSI-X polled interrupts across Linux/SCSI qla2xxx LLD request and response rings

TCM v4.0 QLAXXX prototype using PCIe device passthrough for target mode MSI-X interrupts, and QEMU Megasas HBA emulation into Windows7 64-bit guest.

QLogic qla2xxx series Fibre Channel ASICs and HBAs are supported.

Overview

The latest version of the tcm_qla2xxx fabric module (master). The qla2xxx LLD target mode logic was refactored from the SCST qla2x00t driver, and contains a number of fixes and performance optimizations, including a modern LLD port (qla_target.c) of the qla2x00t target logic.

The QLogic Fiber Channel fabric module (tcm_qla2xxx.c) and a complementary qla2xxx target mode LLD went upstream into the Linux 3.5 kernel on 5/31/2012.^[1]

Configuration

Use targetcli from Datera, Inc. to configure QLogic targets, as described in Fibre Channel.

Setup

Getting the sources

Initial build: git setup

git clone lio-core from git://git.kernel.org/pub/scm/linux/kernel/git/nab/lio-core.git as follows:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/nab/lio-core.git 
cd lio-core
cp <your_working_config_file> .config
make menuconfig

In menuconfig:

• Select device drivers
• Select the TCM module
• Select the modules FILEIO, IBLOCK, PSCSI, Linux-iSCSI, then return to device drivers
• Select the SCSI device drivers, then SCSI low-level drivers.
• Select QLogic QLA2XXX Fibre Channel Support to build qla2xxx.ko, and TCM_QLA2XXX fabric modul to build tcm_qla2xxx.ko
• Save and return

Verify the following entries in the .config file (or make the corresponding changes):

CONFIG_SCSI_QLA_FC=m
CONFIG_TCM_QLA2XXX=m
CONFIG_INET=y
CONFIG_SCSI_FC_ATTRS=m
CONFIGFS_FS=m

If CONFIGFS_FS=y is set, a manual mount is required after the kernel boots:

mount -t configfs configfs /sys/kernel/config

Subsequent builds: git pull

Go to the directory where your lio-core kernel is located, and:

cd <lio-core-2.6>
git pull origin tcm_qla2xxx
make clean
make oldconfig

Building

Kernel and modules:

make
make modules
make modules_install
make install

Installing

New Linux kernel

Use lio-core-2.6 kernel:

• Change /boot/grub/menu.lst to select your lio-core-2.6 kernel.
• In menu.lst, make sure that vmalloc=256m is in the kernel line.

Normally, the latter is optional. But if /proc/meminfo shows vmalloc <128k, it fixes an issue with fcport_lport_map allocation seen by dmesg or in /var/log/messages. Note that 128m may be ok but too low, and 512m may be too high.

Then reboot:

reboot

and verify that the life kernel is indeed your new lio-core kernel:

uname -a 

Loading LIO Target

Verify the presence of the following required modules with modprobe: tcm_qla2xxx, target_core_mod, qla2xxx, iscsi_target_mod, configfs, e.g.:

modprobe tcm_qla2xxx

Normally all these modules should be loaded. If one is missing, you can load it as follows:

insmod fs/configfs/configfs.ko
# Or do the mount command as shown above (if .config file has CONFIGFS_FS=y)
insmod /drivers/target/target_core_mod.ko
insmod /drivers/target/iscsi_target_mod.ko
modprobe tcm_qla2xxx

Starting LIO Target

The target will be automatically started if targetcli-frozen is installed. Otherwise, start the target manually:

/etc/init.d/target start

To stop target:

/etc/init.d/target stop

To check the target status:

/etc/init.d/target status

Results

ConfigFS

See QLogic/configFS for the ConfigFS and kernel ringbuffer output of a working qla2xxx driver.

Modules

lsmod output:

Module                  Size  Used by
tcm_qla2xxx            16822  8 
qla2xxx               327379  1 tcm_qla2xxx
iscsi_target_mod      201391  1 
target_core_pscsi      11045  0 
target_core_file        6307  1 
target_core_iblock      6606  0 
target_core_mod       207359  26 tcm_qla2xxx,iscsi_target_mod,target_core_pscsi,target_core_file,target_core_iblock
configfs               19153  2 target_core_mod
ipv6                  242427  20 

Interrupt stats

MSI-X interrupt stats for target side operation:

lenny64guest0:/usr/src/lio-core-2.6.git# cat /proc/interrupts | head -n 1 ; cat /proc/interrupts | grep qla2xxx
            CPU0       CPU1       CPU2       CPU3       
  43:      11697      11610      11951      11906   PCI-MSI-edge      qla2xxx (default)
  44:      14836      14922      14582      14628   PCI-MSI-edge      qla2xxx (rsp_q)

Initiator

Kernel ring buffer

Initiator side ring buffer output during initiator login:

scsi34 : qla2xxx
qla2xxx 0000:02:00.0: LIP reset occurred (f8f7).
qla2xxx 0000:02:00.0: LIP occurred (f8f7).
qla2xxx 0000:02:00.0: LOOP UP detected (8 Gbps).
scsi 34:0:0:0: Direct-Access     LIO-ORG  RAMDISK-MCP      4.0  PQ: 0 ANSI: 5
sd 34:0:0:0: Attached scsi generic sg2 type 0
sd 34:0:0:0: [sdb] 819200 512-byte logical blocks: (419 MB/400 MiB)
sd 34:0:0:0: [sdb] Write Protect is off
sd 34:0:0:0: [sdb] Mode Sense: 2f 00 00 00
sd 34:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sdb: unknown partition table
sd 34:0:0:0: [sdb] Attached SCSI disk

Device identifiers

sg_inq -i initiator side output for EVPD=0x83 device identifiers:

truelife:/usr/src/qemu-kvm.git# sg_inq -i /dev/sdb
VPD INQUIRY: Device Identification page
  Designation descriptor number 1, descriptor length: 20
    id_type: NAA,  code_set: Binary
    associated with the addressed logical unit
      NAA 6, IEEE Company_id: 0x1405
      Vendor Specific Identifier: 0x7bb168f6d
      Vendor Specific Identifier Extension: 0x32edd42b3d9e26df
      [0x60014057bb168f6d32edd42b3d9e26df]
  Designation descriptor number 2, descriptor length: 61
    id_type: T10 vendor identification,  code_set: ASCII
    associated with the addressed logical unit
      vendor id: LIO-ORG
      vendor specific: RAMDISK-MCP:7bb168f6-32ed-42b3-9e26-fe6aab35460a
  Designation descriptor number 3, descriptor length: 8
    transport: Fibre Channel (FCP-2)
    id_type: Relative target port,  code_set: Binary
    associated with the target port
      Relative target port: 0x1
  Designation descriptor number 4, descriptor length: 8
    transport: Fibre Channel (FCP-2)
    id_type: Target port group,  code_set: Binary
    associated with the target port
      Target port group: 0x0
  Designation descriptor number 5, descriptor length: 8
    id_type: Logical unit group,  code_set: Binary
      associated with the addressed logical unit
      Logical unit group: 0x0
  Designation descriptor number 6, descriptor length: 40
    transport: Fibre Channel (FCP-2)
    id_type: SCSI name string,  code_set: UTF-8
    associated with the target port
      SCSI name string:
      21:00:00:24:ff:31:4c:48,t,0x0001

Acknowledgements

Kudos to the QLogic team, in particular Madhu Iyengar, for their excellent support and guidance with the qla2xxx driver.

See also

• RTS OS, targetcli
• Target
• Fabric modules: iSCSI, Fibre Channel, FCoE, InfiniBand, IBM vSCSI, tcm_loop
• ConfigFS: QLogic/configFS (low level kernel API)

Notes

External links

• QLogic Wikipedia entry
• Fibre Channel Wikipedia entry
• QLogic official website