LIO
The Linux SCSI Target Wiki
 | |
|---|---|
|
RisingTide Systems Multiprotocol Target Engine | |
| Original author(s) | Nicholas Bellinger |
| Developer(s) | RisingTide Systems LLC |
| Initial release | January 14, 2011 |
| Stable release | 4.1.0 / June 20, 2012 |
| Preview release | 4.2.0-rc5 / June 28, 2012 |
| Development status | Production |
| Written in | C |
| Operating system | Linux |
| Type | Target engine |
| License | GNU General Public License |
| Website | datera.io |
The Target (target_core_mod.ko) is the generic multiprotocol target engine of LIO. It is the standard Linux multiprotocol target engine with v2.6.38 (spring 2010 release).[1][2][3]
LIO Target supports a wide range of platforms (from PC architecture to mobile devices, STBs and game consoles), based on a wide range of CPU architectures (x86, ia64, Alpha, Cell, PPC, ARM, MIPS, etc.), basically all existing Linux block devices for backstores, and a growing number of fabric modules.
The LIO SCSI engine implements a significant SPC-3/SPC-4 feature set with support for high-end SCSI features in a fabric agnostic way, such as Persistent Reservations (PR), Asymmetric Logical Unit Assignment (ALUA), Multiple Connections per Session (MC/S) and ERL=2.
Contents |
Backstores
A backstore subsystem plugin is a physical storage object that provides the actual storage underlying an SCSI Endpoint. Backstore objects can be added via the Storage Hardware Abstraction Layer (SHAL) that brings storage hardware into the Target engine as raw block devices, on which the full Linux stack just works (including complex functionality such as software RAID, the LVM, snapshots, virtualization, etc.).
The Target supports the SCSI-3 standard for all backstore devices (block devices and/or VFS):
- FILEIO (Linux VFS devices): any file on a mounted filesystem. It may be backed by a file or an underlying real block device. FILEIO is using struct file to serve block I/O with various methods (synchronous or asynchronous) and (buffered or direct). The Linux kernel code for filesystems resides in linux/fs.
- IBLOCK (Linux BLOCK devices): any block device that appears in /sys/block. The Linux kernel code for block devices is in linux/block.
- PSCSI (Linux physical SCSI devices): any storage object that does direct pass-through of SCSI commands without SCSI emulation. This assumes an underlying SCSI device that appears with lsscsi in /proc/scsi/scsi, such as a SAS hard drive, such as any SCSI device. The Linux kernel code for device SCSI drivers resides in linux/drivers/scsi. SCSI-3 and higher is supported with this subsystem, but only for control CDBs capable by the device firmware.
- RAMDISK (Linux RAMDISK_MCP or RAMDISK_DR devices): any struct page device, such as a regular ramdisk. This type of device is typically used for bandwidth testing.
The SCSI functionality is implemented directly in the target engine in a fabric agnostic way, including a number of high-end features, such as Persistent Reservations (PRs) and Asymmetric Logical Unit Assignment (ALUA), which have been available with LIO 3, following the SPC-4 standard.
Fabric modules
The LIO Fabric Hardware Abstraction Layer (FHAL) accommodates novel fabric technologies and requirements, and allows all protocol-specific processing to be encapsulated in fabric modules:
- iSCSI (iscsi_target_mod.ko): provides SCSI access over IP networks. Status: released.
- FCoE (tcm_fc.ko): provides Fibre Channel access over Ethernet, originally based on open-fcoe.org. Status: released.
- Fibre Channel: drivers for FC Host Bus Adapters (HBAs).
- QLogic (tcm_qla2xxx.ko). Status: Beta.
- Emulex. Status: development.
- InfiniBand: drivers for Infiniband Host Bus Adapters (HBAs). Status: future release.
- tcm_loop (tcm_loop.ko): This high-speed SCSI emulation device can serve any type of raw hardware to local applications and virtual machines as a fully SCSI SPC-4 compliant block device, including emulation of advanced functionality, such as PRs, ALUA, etc. Status: released.
Setup
The LIO 3 CLI manual describes a simple userspace shell (lio-utils) for LIO 3 (Target+iSCSI). It also describes how to setup a number of different subsystem storage object backstores for LIO LUN endpoints:
- Target CLI: basic Target v3 CLI operations using tcm_node.py from lio-utils.
- iSCSI CLI: basic iSCSI v3 CLI operations using lio_node.py and lio_dump.py from lio-utils.
- HOWTO: starting the Target.
RisingTide Systems makes available upon request and for free RTSadmin, a comprehensive, powerful, yet easy storage management tool that can efficiently handle even complex LIO installations.
Lastly, the Target engine configuration is mapped into Target/configFS, where it can be configured manually from user space by using inter- and intra-module symbolic links.
Specifications
The following specifications are available as T10 Working Drafts:
- SCSI Architecture Model - 2 (SAM-2): SAM-2 describes the second generation of the SCSI Architecture Model. Status: Published, 9/11/2002
- SCSI Architecture Model - 3 (SAM-3): SAM-3 describes the third generation of the SCSI Architecture Model. Status: Published, 9/21/2004
- SCSI Architecture Model - 4 (SAM-4): SAM-4 describes the fourth generation of the SCSI Architecture Model. Status: Published, 5/8/2008
- SCSI Primary Commands - 2 (SPC-2): contains the second-generation definition of the basic commands for all SCSI devices. SPC-2 is used in conjuction with a standard for the specific device type. In 2006, the ANSI SPC-2 standard was replaced with ISO/IEC 14776-452:2005, which is available through ANSI. Status: Published, 7/18/2001
- SCSI Primary Commands - 3 (SPC-3): contains the third-generation definition of the basic commands for all SCSI devices. SPC-3 is used in conjuction with a standard for the specific device type. Status: Published, 5/4/2005
- SCSI Primary Commands - 4 (SPC-4): contains the fourth-generation definition of the basic commands for all SCSI devices. SPC-4 is used in conjuction with a standard for the specific device type. Status: Development, 11/16/2010
- Small Computer System Interface - 2 (SCSI-2): SCSI-2 defines the second generation of the Small Computer System Interface. Status: Published, 9/7/1993
Glossary
- Access Control List (ACL): Used to specify the access rights for Initiators to TPGs.
- Backstore: A physical storage object that provides the actual storage underlying an SCSI Endpoint.
- Command Descriptor Block (CDB): The standard format for SCSI commands. CDBs are commonly 6, 10, or 12 bytes long, though they can be 16 bytes or of variable length.
- Challenge Handshake Authentication Protocol (CHAP): An authentication technique for confirming the identity of one computer to another. Described in RFC 1994.
- Connection Identifier (CID): A 16-bit number, generated by the Initiator, that uniquely identifies a connection between two iSCSI devices. This number is presented during the login phase.
- Endpoint: The combination of an iSCSI Target Name with an iSCSI TPG (IQN + Tag).
- Extended Unique Identifier (EUI): A 64-bit number that uniquely identifies every device in the world. The format consists of 24 bits that are unique to a given company, and 40 bits assigned by the company to each device it builds.
- Initiator: The originating end of a SCSI session. Typically a controlling device such as a computer.
- Internet Protocol Storage (IPS): The class of protocols or devices that use the IP protocol to move data in a storage network. FCIP, iFCP, and iSCSI are all examples of IPS protocols.
- iSCSI Qualified Name (IQN): A name format for iSCSI that uniquely identifies every device in the world (e.g.
iqn.5886.com.acme.tapedrive.sn-a12345678). - Initiator Session Identifier (ISID): A 48-bit number, generated by the Initiator, that uniquely identifies a session between the Initiator and the Target. This value is created during the login process, and is sent to the target with a Login PDU.
- Multipath I/O (MPIO): A method by which data can take multiple redundant paths between a server and storage.
- Network Portal: The combination of an iSCSI Endpoint with an IP address plus a TCP port. The TCP port number for the iSCSI protocol defined by IANA is 3260.
- Organizationally Unique Identifier (OUI) is a 24-bit number that is purchased from the IEEE Registration Authority. This identifier uniquely identifies a vendor, manufacturer, or other organization (referred to by the IEEE as the “assignee”) globally or worldwide and effectively reserves a block of each possible type of derivative identifier (such as MAC addresses, group addresses, Subnetwork Access Protocol protocol identifiers, etc.) for the exclusive use of the assignee, see OUI Wikipedia entry. The OUI is subsequently used by the assignee to create particular instances of these identifiers for various purposes, such as the identification of a particular piece of equipment.
- SCSI Architectural Model (SAM): A document that describes the behavior of SCSI in general terms, allowing for different types of devices communicating over various media.
- Target: The receiving end of a SCSI session, typically a device such as a disk drive, solid state drive, tape drive, or scanner.
- Target Group: A collection of fabric module SCSI target endpoints containing SCSI target ports that provide access to individual storage objects.
- Target Port: The combination of an iSCSI Endpoint with one or more LUNs.
- Target Portal Group (TPG): A list of IP addresses and TCP port numbers that determines which interfaces a specific iSCSI target will listen to.
- Target Session Identifier (TSID): A 16-bit number, generated by the target, that uniquely identifies a session between the initiator and the target. This value is created during the login process, and is sent to the initiator with a Login Response PDU.
- World Wide Name (WWN) or World Wide Identifier (WWID) is a unique identifier which identifies a particular Fibre Channel, ATA or SAS target. Each WWN is an 8 byte number derived from an IEEE OUI and vendor-supplied information, see also WWN Wikipedia entry. There are two formats of WWN defined by the IEEE:
- Original format: addresses are assigned to manufacturers by the IEEE standards committee, and are built into the device at build time, similar to an Ethernet MAC address. The first 2 bytes are either hex 10:00 or 2x:xx (where the x's are vendor-specified) followed by the 3-byte vendor identifier and 3 bytes for a vendor-specified serial number
- New addressing schema: the first nibble is either hex 5 or 6 followed by a 3-byte vendor identifier and 36 bits for a vendor-specified serial number
RFCs
- RFC 1994: PPP Challenge Handshake Authentication Protocol (CHAP)
Timeline
| Timeline of the LinuxIO | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Release | Details | 2011 | 2012 | 2013 | 2014 | 2015 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | ||
| 4.x | Version | 4.0 | 4.1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Feature | LIO Core | Loop back | FCoE | iSCSI | Perf | SRP | CM WQ | FC USB 1394 | vHost | Perf | Misc | 16 GFC | iSER | Misc | VAAI | Misc | DIF Core NPIV | DIF iSER | DIF FC vhost | TCMU Xen | Misc | Misc | virtio 1.0 | Misc | NVMe OF | ||||||||||||||||||||||||||||||||||||
| Linux | 2.6.38 | 2.6.39 | 3.0 | 3.1 | 3.2 | 3.3 | 3.4 | 3.5 | 3.6 | 3.7 | 3.8 | 3.9 | 3.10 | 3.11 | 3.12 | 3.13 | 3.14 | 3.15 | 3.16 | 3.17 | 3.18 | 3.19 | 3.20 | 3.21 | 3.22 | ||||||||||||||||||||||||||||||||||||
See also
- Platforms: PC architecture (x86/ia64), PlayStation 2 and PlayStation 3 (Cell), etc.
- Fabric modules: iSCSI, FCoE, Fibre Channel and InfiniBand (future release)
- Distributions: RTS Director, RHEL 4.x/5.x, SLES11, CentOS, Debian, Ubuntu, etc.
- Configuration: RTSadmin and lio-utils
- ConfigFS low level kernel API: Target/configFS and iSCSI/configFS
- SCSI
Notes
- ↑ Linus Torvalds (1/14/2011). "Trivial merge". torvalds/2.6.git.
- ↑ James Bottomley (12/21/2010). "[SCSI] target: Add LIO target core v4.0.0-rc6". scsi-misc-2.6.git.
- ↑ Jonathan Corbet (12/22/2010). "Shooting at SCSI targets". lnw.net.
Wikipedia entries
- Fabrics: ATA over Ethernet (AoE), FibreChannel over Ethernet (FCoE), FibreChannel (FC), InfiniBand (IB), iSCSI, iSCSI Extensions for RDMA (iSER), Internet Wide Area RDMA Protocol (iWARP), and SCSI RDMA Protocol (SRP)
- Services: Service Location Protocol (SLP), Internet Storage Name Service (iSNS)
- Other: Storage Area Network (SAN), Serial attached SCSI (SAS), World Wide Name (WWN)
External links
- LIO Wikipedia entries: German
- Adapters by SCSI connector type
- Anatomy of the Linux SCSI subsystem
- Organizationally Unique Identifier (OUI) Wikipedia entry
- SCSI Wikipedia entry
- SCSI Tutorial
- SCSI/iSCSI/RAID/SAS Information Sheet
- T10 Technical Committee (SCSI standards)
- T10 Technical Committee - Protocols and Physical Layers
- T10 Primary Commands (SPC): using SPC-3/4
- T10 Block Commands (SBC) using SBC-3
- wetpaint.com Fun storage stuff (Ming Zhang)
