Error Recovery Level
The Linux SCSI Target Wiki
The Error Recovery Level (ERL) is negotiated during a leading iSCSI connection login in traditional iSCSI (RFC-3720) and iSCSI Extentions for RDMA (RFC-5046).
The following diagram represents the error recovery hierarchy:
+ / \ / 2 \ <-- Connection recovery +-----+ / 1 \ <-- Digest failure recovery +---------+ / 0 \ <-- Session failure recovery +-------------+
The following table provides an overview over the error recovery capabilities expected from each error recovery level implementation respectively.
|ERL||Associated Error recovery capabilities|
|0||Session recovery (Section 184.108.40.206 - Session Recovery)|
|1||Digest failure recovery (see note below) plus ERL=0|
|2||Connection recovery (Section 220.127.116.11 - Connection Recovery) plus ERL=1|
The following sections provide more detail over the error recovery capabilities expected from each error recovery level implementation.
ERL=0: Session Recovery
ERL=0 (Session Recovery) is triggered when failures within a command, within a connection, and/or within TCP occur. This causes all of the previous connections from the failed session to be restarted on a new session by sending a iSCSI Login Request with a zero TSIH.
Restart all iSCSI connections on any failure.
This is a special case for ERL=0 and recovering the existing I_T nexus.
ERL=1: Digest Failure Recovery
ERL=1 (Digest Failure Recovery) only applies to traditional iSCSI. For iSCSI/SCTP (which has its own CRC32C) and both types of iSER (so far), handling header and data checksum recovery can be disabled.
Within Connection Recovery
- CmdSN Retry Timer
- Logic to handle Recovery R2T
Within Command Recovery
- DataOut Timer
- Datain Timer
Logic to handle recovery / generate R2Ts
Logic to issue SNACK for missing StatSN or DataIN
- Support RDATA SNACK [this is still TODO]
ERL=2: Connection Recovery
ERL=2 (Connection Recovery), also known as "Internexus-MP", is an optional RFC-3720 feature that allows for both single and multiple communication path sessions within a iSCSI Nexus (and hence the SCSI Nexus) to actively perform realligence/retry on iSCSI ITTs from failed iSCSI connections. ERL=2 allows iSCSI fabrics to take advantage of recovery in all regards of transport level fabric failures, and in a completely OS independent fashion (i.e. below the host OS storage stack).
As it is a SCSI feature, it is generic to the underlying network protocol (fabric module), and has been implemented with iSCSI/TCP, iSCSI/SCTP, and is possible for iSER/DDP, iSER/IB. With the latter case (iSER) traditional iSCSI recovery logic that pertains to ERL=1 is disabled, as the underlying RCaP is handling integrity of payloads using CRC32C or greater checking.
Note that in ERL=0 sessions, all communication paths need to be shutdown/restarted after a recovery exception occurs.
- Handle Logout Request REMOVECONNFORRECOVERY (CSM-E)
- Handle generation of Recovery R2Ts for WRITE:
- Traditional iSCSI
- Handle recovery DATAIN for READ:
- Traditional iSCSI
- Handle changed MaxRecvDataSegmentLength across ERL=2
- Traditional iSCSI [TODO]
- iSER [TODO]
Handle new Login Request for existing iSCSI connection handle (CSM-I).