QLogic

The Linux SCSI Target Wiki

Revision as of 19:53, 15 December 2010 by Admin (Talk | contribs)
Jump to: navigation, search

The tcm_qla2xxx.ko is a hardware Fibre Channel fabric module for qla2xxx series hardware from Qlogic ASICs using the target core infrastructure. Template:AdSense right

Contents

Overview

LIO Target
Logo
RisingTide Systems
tcm_qla2xxx Fibre Channel fabric module
Original author(s) Nicholas Bellinger
Developer(s) RisingTide Systems LLC
Development status Development
Written in C
Operating system Linux
Type Fabric module
License GNU General Public License
Website risingtidesystems.com
TCM_QLA2xxx running at line rate HW target mode from with PCIe device passthrough with MSI-X polled interrupts across Linux/SCSI qla2xxx LLD request and response rings
TCM v4.0 QLAXXX prototype using PCIe device passthrough for target mode MSI-X interrupts, and QEMU Megasas HBA emulation into Windows7 64-bit guest.

The QLA2XXX LLD target mode logic and accompanying tcm_qla2xxx fabric module are now up and running with v2.6.37-rc3 code in the lio-core-2.6.git/tcm_qla2xxx branch.

The QLA2XXX LLD target mode logic is based on qla2x00t from the SCST project, and contains a modern LLD port of qla2x00t-target logic in qla2x_target.c.

Status

At this point the LLD changes and tcm_qla2xxx should be considered ALPHA -> BETA status, and thus far has been run exclusively using PCIe device passthrough with MSI-X interrupts into KVM guest. Large block I/O is stable, and is able to run for sustained at line-rate. The FC nexus handling in qla2x_target.c is stable and able to pass my initial testing (shutdown, cable pull, etc). This also includes being able to shutdown tcm_qla2xx and disable target mode on an individual struct qla_hw_data, and then reload tcm_qla2xxx and recreate the configfs groups, re-enable target mode on the same qla_hw_data and everything work as expected w/o shutting down qla2xxx LLD.

Howto

Follow the instructions in Resources#LIO_4.0_for_released_Linux_tip-of-tree_kernels to 'git clone' the upstream lio-core-2.6.git source tree from kernel.org

Then from within the freshly cloned lio-core-2.6.git, switch to the tcm_qla2xxx branch:

    git checkout --track -b tcm_qla2xxx

From here, run a:

    make menuconfig

and enable the following:

    Device Drivers  --->
           <M> Generic Target Core Mod (TCM) and ConfigFS Infrastructure  ---> 
                        <M>   TCM_QLA2XXX fabric module for Qlogic 2xxx series target mode HBAs

and compile and install the qla2xxx.ko and tcm_qla2xxx.ko (along with target_core_mod.ko and backend plugin) modules using:

    make ; make modules ; make modules_install ; make install

and then build the initrd or initramfs depending upon the distribution in question, and then reboot into the >= v2.6.37-rc3 kernel from lio-core-2.6.git/tcm_qla2xxx.

Now, it's time to configure the configfs layout to enable TPG demo-mode access.. Here is a sample start script that uses a target//fileio LUN from /sys/kernel/config/target/core/fileio_0/sync_fileio as qla2xxx_port symlink for LUN=0.

  modprobe tcm_qla2xxx
  mkdir -p /sys/kernel/config/target/qla2xxx/21:00:00:24:ff:31:4c:48/tpgt_1/lun/lun_0/
  ln -s /sys/kernel/config/target/core/fileio_0/sync_fileio/ /sys/kernel/config/target/qla2xxx/21:00:00:24:ff:31:4c:48/tpgt_1/lun/lun_0/qla2xxx_port
  echo 1 > /sys/kernel/config/target/qla2xxx/21:00:00:24:ff:31:4c:48/tpgt_1/enable

Note the qla2xx FC world wide port name (WWPN) used above of 21:00:00:24:ff:31:4c:48 is coming from the hardware, and should be changed to reflect the physical FC world wide port name of the 2xxx series adapter running in target mode.

The code in action

  lenny64guest0:/usr/src/lio-core-2.6.git# tree /sys/kernel/config/target/qla2xxx/
  /sys/kernel/config/target/qla2xxx/
  |-- 21:00:00:24:ff:31:4c:48
  |   `-- tpgt_1
  |       |-- acls
  |       |-- attrib
  |       |-- enable
  |       |-- lun
  |       |   `-- lun_0
  |       |       |-- alua_tg_pt_gp
  |       |       |-- alua_tg_pt_offline
  |       |       |-- alua_tg_pt_status
  |       |       |-- alua_tg_pt_write_md
  |       |       `-- qla2xxx_port -> ../../../../../../target/core/fileio_0/sync_fileio
  |       |-- np
  |       `-- param
  |-- discovery_auth
  `-- version
  10 directories, 6 files
  Module                  Size  Used by
  tcm_qla2xxx            14850  4 
  qla2xxx               328141  1 tcm_qla2xxx
  target_core_stgt        6549  0 
  target_core_pscsi      11530  0 
  target_core_file        6533  1 
  target_core_iblock      6772  0 
  target_core_mod       199609  17 tcm_qla2xxx,target_core_stgt,target_core_pscsi,target_core_file,target_core_iblock
  configfs               19657  2 target_core_mod
  ipv6                  241067  18 
  lenny64guest0:/usr/src/lio-core-2.6.git# cat /proc/interrupts | head -n 1 ; cat /proc/interrupts | grep qla2xxx
              CPU0       CPU1       CPU2       CPU3       
    43:      11697      11610      11951      11906   PCI-MSI-edge      qla2xxx (default)
    44:      14836      14922      14582      14628   PCI-MSI-edge      qla2xxx (rsp_q)
  scsi34 : qla2xxx
  qla2xxx 0000:02:00.0: LIP reset occurred (f8f7).
  qla2xxx 0000:02:00.0: LIP occurred (f8f7).
  qla2xxx 0000:02:00.0: LOOP UP detected (8 Gbps).
  scsi 34:0:0:0: Direct-Access     LIO-ORG  RAMDISK-MCP      4.0  PQ: 0 ANSI: 5
  sd 34:0:0:0: Attached scsi generic sg2 type 0
  sd 34:0:0:0: [sdb] 819200 512-byte logical blocks: (419 MB/400 MiB)
  sd 34:0:0:0: [sdb] Write Protect is off
  sd 34:0:0:0: [sdb] Mode Sense: 2f 00 00 00
  sd 34:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
   sdb: unknown partition table
  sd 34:0:0:0: [sdb] Attached SCSI disk
  truelife:/usr/src/qemu-kvm.git# sg_inq -i /dev/sdb
  VPD INQUIRY: Device Identification page
    Designation descriptor number 1, descriptor length: 20
      id_type: NAA,  code_set: Binary
      associated with the addressed logical unit
        NAA 6, IEEE Company_id: 0x1405
        Vendor Specific Identifier: 0x7bb168f6d
        Vendor Specific Identifier Extension: 0x32edd42b3d9e26df
        [0x60014057bb168f6d32edd42b3d9e26df]
    Designation descriptor number 2, descriptor length: 61
      id_type: T10 vendor identification,  code_set: ASCII
      associated with the addressed logical unit
        vendor id: LIO-ORG
        vendor specific: RAMDISK-MCP:7bb168f6-32ed-42b3-9e26-fe6aab35460a
    Designation descriptor number 3, descriptor length: 8
      transport: Fibre Channel (FCP-2)
      id_type: Relative target port,  code_set: Binary
      associated with the target port
        Relative target port: 0x1
    Designation descriptor number 4, descriptor length: 8
      transport: Fibre Channel (FCP-2)
      id_type: Target port group,  code_set: Binary
      associated with the target port
        Target port group: 0x0
    Designation descriptor number 5, descriptor length: 8
      id_type: Logical unit group,  code_set: Binary
        associated with the addressed logical unit
        Logical unit group: 0x0
    Designation descriptor number 6, descriptor length: 40
      transport: Fibre Channel (FCP-2)
      id_type: SCSI name string,  code_set: UTF-8
      associated with the target port
        SCSI name string:
        21:00:00:24:ff:31:4c:48,t,0x0001
  [  167.034825] QLogic Fibre Channel HBA Driver: 8.03.04-k0
  [  167.035816] qla2xxx 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
  [  167.036928] Setting enable_class_2: 0
  [  167.036928] qla2xxx 0000:00:05.0: Found an ISP2532, irq 10, iobase 0xffffc90004c72000
  [  167.101442] qla2xxx 0000:00:05.0: Configuring PCI space...
  [  167.115561] qla2xxx 0000:00:05.0: Configure NVRAM parameters...
  [  167.124361] qla2xxx 0000:00:05.0: Verifying loaded RISC code...
  [  167.234641] qla2xxx 0000:00:05.0: FW: Loading via request-firmware...
  [  167.267188] qla2xxx 0000:00:05.0: Allocated (64 KB) for FCE...
  [  167.268313] qla2xxx 0000:00:05.0: Allocated (64 KB) for EFT...
  [  167.269494] qla2xxx 0000:00:05.0: Allocated (1350 KB) for firmware dump...
  [  167.275122] scsi1 : qla2xxx
  [  167.279926] qla2xxx 0000:00:05.0: 
  [  167.279928]  QLogic Fibre Channel HBA Driver: 8.03.04-k0
  [  167.279930]   QLogic QLE2562 - PCI-Express Dual Channel 8Gb Fibre Channel HBA
  [  167.279932]   ISP2532: PCI (33 MHz) @ 0000:00:05.0 hdma-, host#=1, fw=5.03.07 (d5)
  [  167.283755] qla2x00t(0): using 64 Bit PCI addressing
  [  167.551805] qla2xxx 0000:00:05.0: LIP reset occurred (f8e8).
  [  168.442731] qla2xxx 0000:00:05.0: LIP occurred (f8e8).
  [  168.445616] qla2xxx 0000:00:05.0: LOOP UP detected (8 Gbps).
  [  168.789395] qla2x00_reg_remote_port() fcport->node_name: 20 00 00 24 ff 31 4c 4c 
  [  168.790858] qla2x00_reg_remote_port() fcport->port_name: 21 00 00 24 ff 31 4c 4c 
  [  168.792300] qla2x00_reg_remote_port() s_id: 0x00 00 e8
  [  168.793178] qla2x00_reg_remote_port() loop_id: 0x0000
  [  168.793193] qla2x00_reg_remote_port() Using vha: ffff88003746ed60 vp_idx: 0x0000
  [  172.562210] TCM QLOGIC QLA2XXX fabric module v0.1 on Linux/x86_64 on 2.6.37-rc3+
  [  172.563480] Setup generic discovery
  [  172.564078] Setup generic wwn
  [  172.564593] Setup generic tpg
  [  172.564593] Setup generic tpg_base
  [  172.564593] Setup generic tpg_port
  [  172.564593] Setup generic tpg_lun
  [  172.564593] Setup generic tpg_np
  [  172.564593] Setup generic tpg_np_base
  [  172.564593] Setup generic tpg_attrib
  [  172.568735] Setup generic tpg_param
  [  172.569715] Setup generic tpg_nacl
  [  172.570439] Setup generic tpg_nacl_base
  [  172.571094] Setup generic tpg_nacl_attrib
  [  172.571775] Setup generic tpg_nacl_auth
  [  172.572427] Setup generic tpg_nacl_param
  [  172.572698] Setup generic tpg_mappedlun
  [  172.572698] <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API >>>>>>>>>>>>>>>>>>>>>>
  [  172.572698] Initialized struct target_fabric_configfs: ffff88007c001830 for qla2xxx
  [  172.572698] <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>>
  [  172.577439] TCM_QLA2XXX[0] - Set fabric -> tcm_qla2xxx_fabric_configfs
  [  172.578929] Setup generic discovery
  [  172.579518] Setup generic wwn
  [  172.580021] Setup generic tpg
  [  172.580524] Setup generic tpg_base
  [  172.581131] Setup generic tpg_port
  [  172.581425] Setup generic tpg_lun
  [  172.581425] Setup generic tpg_np
  [  172.581425] Setup generic tpg_np_base
  [  172.581425] Setup generic tpg_attrib
  [  172.581425] Setup generic tpg_param
  [  172.584712] Setup generic tpg_nacl
  [  172.585675] Setup generic tpg_nacl_base
  [  172.586476] Setup generic tpg_nacl_attrib
  [  172.587150] Setup generic tpg_nacl_auth
  [  172.587793] Setup generic tpg_nacl_param
  [  172.588451] Setup generic tpg_mappedlun
  [  172.588698] <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API >>>>>>>>>>>>>>>>>>>>>>
  [  172.588698] Initialized struct target_fabric_configfs: ffff88007c001020 for qla2xxx_npiv
  [  172.588698] <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>>
  [  172.592805] TCM_QLA2XXX[0] - Set fabric -> tcm_qla2xxx_npiv_fabric_configfs
  [  172.595943] Target_Core_ConfigFS: REGISTER -> group: ffffffffa03d4480 name: qla2xxx
  [  172.597254] Target_Core_ConfigFS: REGISTER -> Located fabric: qla2xxx
  [  172.598335] Target_Core_ConfigFS: REGISTER tfc_wwn_cit -> ffff88007c001b38
  [  172.598849] Target_Core_ConfigFS: REGISTER -> Allocated Fabric: qla2xxx
  [  172.600651] Target_Core_ConfigFS: REGISTER -> Set tf->tf_fabric for qla2xxx
  [  172.602066] qla2xxx HW vha->node_name: 20 00 00 24 ff 31 4c 48 
  [  172.603263] qla2xxx HW vha->port_name: 21 00 00 24 ff 31 4c 48 
  [  172.604448] qla2xxx passed configfs WWPN: 21 00 00 24 ff 31 4c 48 
  [  172.604636] qla2xxx: Found matching HW WWPN: 21:00:00:24:ff:31:4c:48 for lport
  [  172.683924] qla2xxx: Allocated lport_fcport_map of 134217728 bytes
  [  172.684641] qla2xxx: Allocated lport_loopid_map of 524288 bytes
  [  172.684641] TARGET_CORE[qla2xxx]: Allocated Normal struct se_portal_group for endpoint: 21:00:00:24:ff:31:4c:48, Portal Tag: 1
  [  172.688218] rd_mcp/qla2xxx: Adding to default ALUA Target Port Group: alua/default_tg_pt_gp
  [  172.689334] qla2xxx_TPG[1]_LUN[0] - Activated qla2xxx Logical Unit from CORE HBA: 1
  [  173.696888] qla2xxx 0000:00:05.0: Performing ISP error recovery - ha= ffff88005bab8000.
  [  173.709879] qla2xxx 0000:00:05.0: FW: Loading via request-firmware...
  [  174.060183] qla2xxx 0000:00:05.0: LIP occurred (f8f7).
  [  174.062347] qla2xxx 0000:00:05.0: LOOP UP detected (8 Gbps).
  [  193.713842] Unable to locate se_nacl by loop_id: 0x0000
  [  193.747641] qla2xxx 0000:00:05.0: LOOP DOWN detected (0 3 0 0).
  [  193.751999] qla2x00t(0): Link failure detected
  [  194.029097] qla2xxx 0000:00:05.0: LIP occurred (f8f7).
  [  194.031248] qla2xxx 0000:00:05.0: LOOP UP detected (8 Gbps).
  [  194.032396] Unable to locate se_nacl by loop_id: 0x0000
  [  194.033473] Unable to locate se_nacl by loop_id: 0x0000
  [  194.355625] qla2x00_reg_remote_port() fcport->node_name: 20 00 00 24 ff 31 4c 4c 
  [  194.357142] qla2x00_reg_remote_port() fcport->port_name: 21 00 00 24 ff 31 4c 4c 
  [  194.358645] qla2x00_reg_remote_port() s_id: 0x00 00 e8
  [  194.359603] qla2x00_reg_remote_port() loop_id: 0x0000
  [  194.360491] qla2x00_reg_remote_port() Using vha: ffff88003746ed60 vp_idx: 0x0000
  [  194.361826] TARGET_CORE[qla2xxx]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
  [  194.363429] qla2xxx_TPG[1] - Added DYNAMIC ACL with TCQ Depth: 1 for qla2xxx Initiator Node: 00:00:00:00:00:00:00:00
  [  194.365460] TARGET_CORE[qla2xxx]: Registered fabric_sess_ptr: ffff8800378c69a0
  [  194.366713] qla2x00t(0): session for wwn 21:00:00:24:ff:31:4c:4c (loop_id 0, s_id 0:0:e8, confirmed completion supported) added

TODO

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Google AdSense