iSCSI

The Linux SCSI Target Wiki

Revision as of 19:19, 4 July 2012

LIO Target

iSCSI fabric module
Original author(s)	Nicholas Bellinger
Developer(s)	Datera, Inc.
Initial release	October 24, 2011 (2011-10-24)
Stable release	4.1.0 / June 20, 2012; 7 years ago (2012-06-20)
Preview release	4.2.0-rc5 / June 28, 2012; 7 years ago (2012-06-28)
Development status	Production
Written in	C
Operating system	Linux
Type	Fabric module
License	GNU General Public License
Website	datera.io

See Target for a complete overview over all fabric modules.

Architecture diagram of network storage types.

iSCSI login flow with CHAP authentication.

iSCSI (Internet Small Computer System Interface) provides SCSI access over IP networks.

Overview

iSCSI is an Internet Protocol (IP)-based storage networking standard for linking data storage facilities. By carrying SCSI commands over IP networks, iSCSI is used to facilitate data transfers over intranets and to manage storage over long distances. iSCSI can be used to transmit data over local area networks (LANs), wide area networks (WANs), or the Internet, and can enable location-independent data storage and retrieval.

iSCSI allows clients (called Initiators) to send SCSI commands (CDBs) to SCSI storage devices (Targets) on remote servers. It is a popular SAN protocol, allowing organizations to consolidate storage into data center storage arrays while providing hosts (such as database and web servers) with the illusion of locally-attached disks. Unlike traditional Fibre Channel, which requires special-purpose cabling, iSCSI can be run over long distances using existing network infrastructure.

iSCSI is the fabric module (iscsi_target_mod.ko) for the Unified Target engine. It went upstream into the Linux 3.1 kernel on 7/27/2011.^[1]

targetcli

targetcli from Datera, Inc. is used to configure iSCSI targets. targetcli aggregates service modules via a core library, and exports them through an API to the Unified Target, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).

Startup

targetcli is invoked by running targetcli as root from the command prompt of the underlying OS shell.

# targetcli
Welcome to targetcli:

 Copyright (c) 2012 by RisingTide Systems LLC.
 All rights reserved.

Visit us at http://www.risingtidesystems.com.

Using ib_srpt fabric module.
Using qla2xxx fabric module.
Using iscsi fabric module.
Using loopback fabric module.

/> iscsi/ info
Fabric module name: iscsi
ConfigFS path: /sys/kernel/config/target/iscsi
Allowed WWNs list type: iqn
Fabric module specfile: /var/target/fabric/iscsi.spec
Fabric module features: discovery_auth, acls, acls_auth, nps, tpgts
Corresponding kernel module: iscsi_target_mod
/>

Upon targetcli initialization, the underlying RTSlib loads the installed fabric modules, and creates the corresponding ConfigFS mount points (at /sys/kernel/config/target/<fabric>), as specified by the associated spec files (located in /var/target/fabric/fabric.spec).

Display the object hierarchy

Use ls to list the object hierarchy, which is initially empty:

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [0 Storage Object]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Note: Per default, auto_cd_after_create=true, which automatically changes the object context (or working directory) to new objects after their creation.

Set auto_cd_after_create=false to prevent RTSadmin from automatically changing the object context to new objects after their creation:

/> set global auto_cd_after_create=false
Parameter auto_cd_after_create is now 'false'.
/>

Create a backstore

Enter the top-level backstore object, and create one (storage object) using IBLOCK or FILEIO type devices.

For instance, create an IBLOCK backstore from a /dev/sdb block device. Note that this device can be any TYPE_DISK block-device, and it can also use /dev/disk/by-id/ symlinks:

/> cd backstores/
/backstores> iblock/ create name=my_disk dev=/dev/sdb
Generating a wwn serial.
Created iblock storage object my_disk using /dev/sdb.
Entering new node /backstores/iblock/my_disk.
/backstores/iblock/my_disk>

Note that RTSadmin automatically creates a WWN serial ID for the backstore device and then changes the working context to it.

The resulting object hierarchy looks as follows (displayed from the root object):

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [1 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Instantiate a target

Instantiate an iSCSI target on the existing backstore my_disk:

/backstores/iblock/my_disk> /iscsi create
Created target iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11.
Selected TPG Tag 1.
Successfully created TPG 1.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

targetcli automatically creates the Target Portal Group (TPG) and per default assigns a sequentially increasing TPG tag, starting from '1', thereby creating a TPG1.

Export LUNs

Add LUNs to the iSCSI target:

/iscsi/iqn.20...a0e4a11/tpgt1> luns/ create /backstores/iblock/my_disk
Selected LUN 0.
Successfully created LUN 0.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1/luns/lun0.
/iscsi/iqn.20...gt1/luns/lun0>

targetcli per default automatically assigns sequentially increasing LUN IDs, starting from '0', thereby exporting /dev/sdb as LUN 0 in the example above.

Return to the underlying TPG as the working context, as no attributes need to be set or modified for standard LUNs:

/iscsi/iqn.20...gt1/luns/lun0> cd <
Taking you back to /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

Create a network portal

Assign an active IP address (here the IPv4 address 192.168.1.139) to the iSCSI TPG to form a valid iSCSI Endpoint:

/iscsi/iqn.20...a0e4a11/tpgt1> portals/ create 192.168.1.139
Using default IP port 3260
Successfully created network portal 192.168.1.139:3260.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1/portals/192.168.1.139:3260.
/iscsi/iqn.20...68.1.139:3260>
/iscsi/iqn.20...68.1.139:3260> cd <
Taking you back to /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

For iSCSI Network Portals, targetcli automatically uses the iSCSI default port number of '3260', thereby forming a valid new iSCSI Endpoint. The iSCSI Endpoint makes the TPG discoverable by iSCSI initiators.

Define access rights

Configure the access rights to allow logins from initiators. The three basic setups for LUN authentication are discussed below.

Demo mode

For simple demo setups, "open" access can be granted to all initiators so that they can access all LUNs in the TPG without further authentication. To enable that so-called "demo mode" TPG operation, disable all authentication for the corresponding Endpoint:

/iscsi/iqn.20...a0e4a11/tpgt1> set attribute authentication=0
demo_mode_write_protect=0 generate_node_acls=1 cache_dynamic_acls=1.
Parameter demo_mode_write_protect is now '0'.
Parameter authentication is now '0'.
Parameter generate_node_acls is now '1'.
Parameter cache_dynamic_acls is now '1'.
/iscsi/iqn.20...a0e4a11/tpgt1> cd /
/>

This exports the IBLOCK backstore as LUN0 to initiators without any access restrictions.

Warning: Exporting "open" LUNs with no authentication requirements create significant security and data integrity hazards. Do not do this for production setups, unless you are certain of what you are doing.

Only use "demo mode" under the following circumstances:

• You have established physical security through a closed, controlled SAN environment.
• You are using your SAN in conjunction with a clustered filesystem that guarantees coherence across multiple initiators, typically via distributed locking.
• You have carefully analyzed your ACL setup with regard to its security and data integrity requirements and risks.

CHAP Initiator Authentication

Enable secure sessions for the initiator with the IQN "iqn.1991-05.com.microsoft:ibm-t410s":

/iscsi/iqn.20...70c82fb/tpgt1> acls/ create iqn.1991-05.com.microsoft:ibm-t410s
Successfully created Node ACL for iqn.1991-05.com.microsoft:ibm-t410s
Created mapped LUN 0.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1/acls/iqn.1991-05.com.microsoft:ibm-t410s/mapped_lun0.
/iscsi/iqn.20...s/mapped_lun0> cd <
Taking you back to /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

This creates an iSCSI Node ACL with a mapped LUN0.

Node ACLs allow mappings of actual LUN IDs onto arbitrary Mapped_LUN IDs, which are the LUN IDs presented to initiators. These mappings can match preferred LUN IDs for particular initiators, so for instance, a LUN1 can be mapped onto Mapped_LUN0 to make LUN1 appear as iSCSI LUN0 on the initiator. Usually, LUNs are identically mapped, however.

Setup the CHAP logon information for an initiator, which consists of the userid (login name) and password (target secret) from the initiator:

/iscsi/iqn.20...70c82fb/tpgt1> cd acls/iqn.1991-05.com.microsoft:ibm-t410s
/iscsi/iqn.20...oft:ibm-t410s> set auth userid=iqn.1991-05.com.microsoft:ibm-t410s
Parameter userid is now 'iqn.1991-05.com.microsoft:ibm-t410s'.
/iscsi/iqn.20...oft:ibm-t410s> set auth password=mytargetsecret
Parameter password is now 'mytargetsecret'.
/iscsi/iqn.20...oft:ibm-t410s> get auth
AUTH CONFIG GROUP
  mutual_password=
    The mutual_password auth parameter.

  mutual_userid=
    The mutual_userid auth parameter.

  password=mytargetsecret
    The password auth parameter.

  userid=iqn.1991-05.com.microsoft:ibm-t410
    The userid auth parameter.
/iscsi/iqn.20...oft:ibm-t410s> cd /iscsi
/iscsi>

The iSCSI Endpoint is now ready for secure logins from the specified iSCSI initiator.

Note: The Microsoft Windows iSCSI Initiator uses its IQN as a default login name. For CHAP Initiator Authentication, it requires the password length to be at least 12 bytes (96 bits), and it rejects passwords that are too simple. It allows changing the default login name through the Targets tab → Connect button → Connect To Target dialog → Advanced… button → Advanced Settings dialog → Enable CHAP log on checkbox.

Mutual CHAP Authentication

Enable secure sessions for the initiator with the IQN "iqn.1991-05.com.microsoft:ibm-t410s":

/iscsi/iqn.20...70c82fb/tpgt1> acls/ create iqn.1991-05.com.microsoft:ibm-t410s
Successfully created Node ACL for iqn.1991-05.com.microsoft:ibm-t410s
Created mapped LUN 0.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1/acls/iqn.1991-05.com.microsoft:ibm-t410s/mapped_lun0.
/iscsi/iqn.20...s/mapped_lun0> cd <
Taking you back to /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

This creates an iSCSI Node ACL with a mapped LUN0.

Node ACLs allow mappings of actual LUN IDs onto arbitrary Mapped_LUN IDs, which are the LUN IDs presented to initiators. These mappings can match preferred LUN IDs for particular initiators, so for instance, a LUN1 can be mapped onto Mapped_LUN0 to make LUN1 appear as iSCSI LUN0 on the initiator. Usually, LUNs are identically mapped, however.

Setup the mutual CHAP logon information for an initiator, which consists of:

• The userid (login name) and password (target secret) for the target.
• The mutual_userid (login name) and mutual_password (initiatir secret) for the initiator.

/iscsi/iqn.20...70c82fb/tpgt1> cd acls/iqn.1991-05.com.microsoft:ibm-t410s
/iscsi/iqn.20...oft:ibm-t410s> set auth userid=iqn.1991-05.com.microsoft:ibm-t410s password=mytargetsecret mutual_userid=iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11 mutual_password=mymutualsecret
Parameter userid is now 'iqn.1991-05.com.microsoft:ibm-t410s'.
Parameter password is now 'mytargetsecret'.
Parameter mutual_userid is now 'iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11'.
Parameter password is now 'mymutualsecret'.
/iscsi/iqn.20...oft:ibm-t410s> get auth
AUTH CONFIG GROUP
  mutual_password=mymutualsecret
    The mutual_password auth parameter.

  mutual_userid=iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11
    The mutual_userid auth parameter.

  password=mytargetsecret
    The password auth parameter.

  userid=iqn.1991-05.com.microsoft:ibm-t410
    The userid auth parameter.
/iscsi/iqn.20...oft:ibm-t410s> cd /iscsi
/iscsi>

The iSCSI Endpoint is now ready for secure logins from the specified iSCSI initiator.

Note: The Microsoft Windows iSCSI Initiator uses its IQN as the default login name. For Mutual CHAP Authentication, it requires the password length to be between 12 bytes (96 bits) and 16 bytes (128 bits), and it rejects passwords that are too simple. It allows changing the default login name through the Targets tab → Connect button → Connect To Target dialog → Advanced… button → Advanced Settings dialog → Enable CHAP log on checkbox.

Enable discovery control

Optionally, the iSCSI protocol can also control the visibility of iSCSI targets for discovery by iSCSI initiators.

CHAP Initiator Discovery Authentication

Enable CHAP initiator discovery authentication for all initiators by setting up a CHAP userid (login name) and password (target secret) in the global discovery_auth group:

/iscsi> set discovery_auth enable=1 userid=mytargetuid password=mytargetsecret
Parameter enable is now '1'.
Parameter password is now 'mytargetsecret'.
Parameter userid is now 'mytargetuid'.
/iscsi>

Only iSCSI initiators that can authenticate themselves with a user id of "mytargetuid" and a password of "mytargetsecret" can now discover this iSCSI target.

Mutual CHAP Discovery Authentication

Enable Mutual CHAP discovery authentication for all initiators by setting up Mutual CHAP information in the global discovery_auth group:

• The userid (login name) and password (target secret) for the target.
• The mutual_userid (login name) and mutual_password (initiator secret) for initiators.

/iscsi> set discovery_auth enable=1 userid=mytargetuid password=mytargetsecret
mutual_userid=mymutualuid mutual_password=mymutualsecret
Parameter password is now 'mytargetsecret'.
Parameter userid is now 'mytargetuid'.
Parameter mutual_password is now 'mymutualsecret'.
Parameter mutual_userid is now 'mymutualuid'.
Parameter enable is now '1'.
/iscsi> get discovery_auth
DISCOVERY_AUTH CONFIG GROUP
  enable=1
    The enable discovery_auth parameter.

  mutual_password=mymutualsecret
    The mutual_password discovery_auth parameter.

  mutual_userid=mymutualuid
    The mutual_userid discovery_auth parameter.

  password=mytargetsecret
    The password discovery_auth parameter.

  userid=mytargetuid
    The userid discovery_auth parameter.
/iscsi>

Only iSCSI initiators that can authenticate themselves with a user id of "mytargetuid" and a password of "mytargetsecret" can now discover this iSCSI target, and conversely, the iSCSI target can only discover iSCSI initiators that can authenticate themselves with a user id of "mymutualuid" and a password of "mymutualsecret".

Display the object hierarchy

The resulting iSCSI object hierarchy with CHAP Authentication (initiator or mutual) for one initiator looks as follows (displayed from the root object):

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [1 Target]
  | o- iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11 ......... [1 TPG]
  |   o- tpgt1 ....................................................... [enabled]
  |     o- acls ........................................................ [1 ACL]
  |     | o- iqn.1991-05.com.microsoft:ibm-t410s ................ [1 Mapped LUN]
  |     |   o- mapped_lun0 ......................................... [lun0 (rw)]
  |     o- luns ........................................................ [1 LUN]
  |     | o- lun0 .................................. [iblock/my_disk (/dev/sdb)]
  |     o- portals .................................................. [1 Portal]
  |       o- 192.168.1.139:3260 ........................................... [OK]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Persist the configuration

The target configuration can be persisted across OS reboots by using saveconfig from the root context:

/> saveconfig
WARNING: Saving rtsnode1 current configuration to disk will overwrite your boot settings.
The current target configuration will become the default boot config.
Are you sure? Type 'yes': yes
Making backup of srpt/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/srpt_start.sh
Making backup of qla2xxx/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/qla2xxx_start.sh
Making backup of loopback/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/loopback_start.sh
Making backup of LIO-Target/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/lio_backup-2012-02-27_23:19:37.660264.sh
Making backup of Target_Core_Mod/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/tcm_backup-2012-02-27_23:19:37.660264.sh
Generated Target_Core_Mod config: /etc/target/backup/tcm_backup-2012-02-27_23:19:37.660264.sh
Successfully updated default config /etc/target/lio_start.sh
Successfully updated default config /etc/target/tcm_start.sh
/>

Warning: Without saveconfig, the target configuration is ephemeral and will be lost upon rebooting or unloading the target service.

Spec file

RTS spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.

In particular, the iSCSI spec file /var/target/fabric/iscsi.spec is included via RTSlib.

# WARNING: This is a draft specfile supplied for demo purposes only.

# The iSCSI fabric module uses the default feature set.
features = acls, acls_auth, discovery_auth, nps, tpgts

# Obviously, this module uses IQN strings as WWNs
wwn_type = iqn

# The kernel module is default
# kernel_module = iscsi_target_mod

# The configfs group name is default
# configfs_group = iscsi

RFCs

• RFC 1994: PPP Challenge Handshake Authentication Protocol (CHAP)
• RFC 3720: Internet Small Computer Systems Interface (iSCSI)
• RFC 3721: Internet Small Computer Systems Interface (iSCSI) Naming and Discovery
• RFC 3722: String Profile for Internet Small Computer Systems Interface (iSCSI) Names
• RFC 3723: Securing Block Storage Protocols over IP (Scope: The use of IPsec and IKE to secure iSCSI, iFCP, FCIP, iSNS and SLPv2.)
• RFC 3724: The Rise of the Middle and the Future of End-to-End: Reflections on the Evolution of the Internet Architecture
• RFC 3747: Small Computer Systems Interface protocol over the Internet (iSCSI) Requirements and Design Considerations
• RFC 3783: Small Computer Systems Interface (SCSI) Command Ordering Considerations with iSCSI
• RFC 3980: T11 Network Address Authority (NAA) Naming Format for iSCSI Node Names
• RFC 4173: Bootstrapping Clients using the Internet Small Computer System Interface (iSCSI) Protocol
• RFC 4544: Definitions of Managed Objects for Internet Small Computer System Interface (iSCSI)
• RFC 4850: Declarative Public Extension Key for Internet Small Computer Systems Interface (iSCSI) Node Architecture
• RFC 5048: Internet Small Computer System Interface (iSCSI) Corrections and Clarifications
• RFC 5047: DA: Datamover Architecture for the Internet Small Computer System Interface (iSCSI)
• RFC 5046: Internet Small Computer System Interface (iSCSI) Extensions for Remote Direct Memory Access (RDMA)

Timeline

See also

• RTS OS, targetcli
• Target
• Fibre Channel, FCoE, InfiniBand, IBM vSCSI, tcm_loop
• ConfigFS: iSCSI/configFS (low level kernel API)

Notes

External links

• RTS OS Admin Manual
• RTSlib Reference Guide [HTML][PDF]
• iSCSI Wikipedia entry
• wetpaint.com Fun storage stuff (Ming Zhang)