Persistent Reservations

The Linux SCSI Target Wiki

Revision as of 00:03, 20 March 2010 by Admin (Talk | contribs)
Jump to: navigation, search

SCSI-3 and SCSI-4 persistent reservations are used for a number of cluster storage configurations for I/O fencing and failover/retakeover cases using iSCSI and other SCSI fabrics like Fibre Channel and Serial Attached SCSI.

Our Linux-iSCSI Target Mode SPC-3 defined Persistent Reservations is supported in a SCSI fabric module independent method in target_core_mod in the lio-core-2.6.git source tree, you can have a look at the latest revision of the source code at drivers/target/target_core_pr.c using git.

As of August 10th 2009 with commit and commit on v2.6.31-rc5, the TCM PR code in lio-core-2.6.git/drivers/target/target_core_pr.c is now %100 persistent reservations feature complete for SPC-4!

Also, using Asymmetric Logical Unit Assignment ALUA with Persistent Reservation in TCM/LIO v3.1 is fully supported!

As of March 14th with the patch series link, all >= SPC-3 PR logic for multi-fabric and inter-fabric operation has been included into lio-core-2.6.git master. This means As of LIO v3.4.0-rc1, 100% of TCM Persistent Reservations logic is up and running across LIO-Target TCM_Loop and OpenFCoE fabric modules!


What Cluster Clients that use Linux-iSCSI target mode PR have been tested..?

RHEL/CentOS uses sg3_util's sg_persist CLI operation for Write Exclusive, Registrants Only reservations and PREEMPT_AND_ABORT takeover using ext3 mounts. Testing with GFS (eg: with multiple writers) is underway.

See RHEL/Linux_Persistent_Reservations for more information about Linux/iSCSI Initiators using LIO-Target v3.0 Mode

All target_core_mod/LIO-Target v3.0 capable storage objects from IBLOCK, FILEIO, and RAMDISK_MCP/DR pass fence_scsi_test.

SPC-3 PR emulation is disabled on top of the passthrough SCSI subsystem plugin (this is a WIP for existing SCSI devices)

See MSFT/Persistent_Reservations for more information

Testing LIO-Target v3.0 against other SPC-3 PR compliant systems

What has been implemented..?

These two spc4 defined PR features been now been committed in lio-core-2.6.git running v2.6.31-rc5!

APTPL=1 (Activate Persist across Target Power Loss) is using key=value /var/target/pr/aptpl_$T10_UNIT_SERIAL using sg_persist See link for more information

SP_I_PT=1 (Specify Initiator Ports Capable) bit is supported and tested using sg_persist See link


Write Exclusive, Exclusive Access, Write Exclusive Registrants Only, Exclusive Access Registrants Only Write Exclusive All Registrants, Exclusive Access All Registrants


Compatible reservation handling for SPC-2 needs to be enabled for legacy environments.

ConfigFS layout for target_core_mod

  tree /sys/kernel/config/target/core
  |-- iblock_0
  |   |-- hba_info
  |   `-- lvm_test0
  |       |-- alua_lu_gp
  |       |-- attrib
  |       |   |-- block_size
  |       |   |-- emulate_tas
  |       |   |-- emulate_ua_intlck_ctrl
  |       |   |-- hw_block_size
  |       |   |-- hw_max_sectors
  |       |   |-- hw_queue_depth
  |       |   |-- max_sectors
  |       |   |-- queue_depth
  |       |   `-- task_timeout
  |       |-- control
  |       |-- enable
  |       |-- fd
  |       |-- info
  |       |-- pr
  |       |   |-- res_holder
  |       |   |-- res_pr_all_tgt_pts
  |       |   |-- res_pr_generation
  |       |   |-- res_pr_holder_tg_port
  |       |   |-- res_pr_registered_i_pts
  |       |   |-- res_pr_type
  |       |   `-- res_type
  |       `-- wwn
  |           |-- vpd_assoc_logical_unit
  |           |-- vpd_assoc_scsi_target_device
  |           |-- vpd_assoc_target_port
  |           |-- vpd_protocol_identifier
  |           `-- vpd_unit_serial
  initiator# sg_persist --in --report-capabilities -v /dev/sde
      inquiry cdb: 12 00 00 00 24 00 
    LIO-ORG  IBLOCK  3.0
    Peripheral device type: disk
      Persistent Reservation In cmd: 5e 02 00 00 00 00 00 20 00 00 
  Report capabilities response:
    Compatible Reservation Handling(CRH): 0
    Specify Initiator Ports Capable(SIP_C): 0
    All Target Ports Capable(ATP_C): 1
    Persist Through Power Loss Capable(PTPL_C): 0
    Type Mask Valid(TMV): 1
    Allow commands: 1
    Persist Through Power Loss Active(PTPL_A): 0
      Support indicated in Type mask:
        Write Exclusive, all registrants: 1
        Exclusive Access, registrants only: 1
        Write Exclusive, registrants only: 1
        Exclusive Access: 1
        Write Exclusive: 1
        Exclusive Access, all registrants: 1

for different SCSI (including LIO-Target v3.0) I_T nexuses:

  cat /sys/kernel/config/target/core/iblock_0/lvm_test0/pr/res_*
  SPC-3 Reservation: iSCSI Initiator:
  SPC-3 Reservation: All Target Ports registration 0x00000002
  SPC-3 Reservation: iSCSI Target Node Endpoint:
  SPC-3 Reservation: Relative Port Identifer Tag: 2 iSCSI Portal Group Tag: 1 iSCSI Logical Unit: 0
  SPC-3 PR Registrations:
     iSCSI Node: Key: 0x000000001234abcd PRgen: 0x00000000
     iSCSI Node: Key: 0x000000004567ffff PRgen: 0x00000001
  SPC-3 Reservation Type: Write Exclusive Access, Registrants Only

real from the SCSI device, or emulated using a admin present value into iblock_0/lvm_test0/wwn/evp_unit_serial. This is required in order to use the NAA VPD device identifers.

  cat /sys/kernel/config/target/core/iblock_0/lvm_test0/wwn/*   
  T10 VPD Identifier Association: addressed logical unit
  T10 VPD Identifier Type: NAA
  T10 VPD Binary Device Identifier: 36001405a97e4ce21c0711de829b000c2
  T10 VPD Identifier Association: addressed logical unit
  T10 VPD Identifier Type: T10 Vendor ID based
  T10 VPD ASCII Device Identifier: LIO-ORG
  T10 VPD Unit Serial Number: a97e4ce21c0711de829b000c2943d57b
Personal tools
Google AdSense