Fibre Channel over Ethernet

The Linux SCSI Target Wiki

(Difference between revisions)
Jump to: navigation, search
Line 77: Line 77:
   modprobe fcoe ; echo eth0 > /sys/module/fcoe/parameters/create
   modprobe fcoe ; echo eth0 > /sys/module/fcoe/parameters/create
-
  you should see something like the following in the target side kernel ring buffer:
+
you should see something like the following in the target side kernel ring buffer:
   [  469.432122] device eth0 entered promiscuous mode
   [  469.432122] device eth0 entered promiscuous mode
Line 83: Line 83:
   [  469.435336] host0: libfc: Link up on port (    0)
   [  469.435336] host0: libfc: Link up on port (    0)
-
  and the target will be configured!
+
and the target will be configured!
 +
 
 +
== Open-FCoE Initiator ==
 +
 
 +
Now we are ready to connect from an Open-FCOE Initiator.  Remember that in order to run the Initiator in Point to Point mode, the necessary patches
 +
to libfc are also required.  Just as on the target side, load the fcoe module and create the association to the network interface on the initiator capable of communication with the LPORT network interface configured above:
 +
 
 +
modprobe fcoe ; echo eth0 > /sys/module/fcoe/parameters/create
 +
 
 +
After a few seconds, the FCoE LUNs will be detected and registered with Linux/SCSI..  Here is what that output looks like on the Open-FCOE initiator side:
 +
 
 +
  [  853.183484] device eth0 entered promiscuous mode
 +
  [  853.184640] scsi0 : FCoE Driver
 +
  [  853.186064] host0: libfc: Link up on port (    0)
 +
  [  856.754764] host0: Assigned Port ID  10101
 +
  [  856.758376] scsi 0:0:0:0: Direct-Access    LIO-ORG  FILEIO          3.1  PQ: 0 ANSI: 5
 +
  [  856.818511] sd 0:0:0:0: [sda] READ CAPACITY(16) failed
 +
  [  856.819201] sd 0:0:0:0: [sda] Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
 +
  [  856.820127] sd 0:0:0:0: [sda] Sense not available.
 +
  [  856.820964] sd 0:0:0:0: [sda] 97656251 512-byte logical blocks: (50.0 GB/46.5 GiB)
 +
  [  856.822580] sd 0:0:0:0: [sda] Write Protect is off
 +
  [  856.823230] sd 0:0:0:0: [sda] Mode Sense: 2f 00 00 00
 +
  [  856.823640] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 +
  [  856.825018] sd 0:0:0:0: [sda] READ CAPACITY(16) failed
 +
  [  856.825688] sd 0:0:0:0: [sda] Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
 +
  [  856.826620] sd 0:0:0:0: [sda] Sense not available.
 +
  [  856.828304]  sda: unknown partition table
 +
  [  856.830403] sd 0:0:0:0: [sda] READ CAPACITY(16) failed
 +
  [  856.831109] sd 0:0:0:0: [sda] Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
 +
  [  856.832039] sd 0:0:0:0: [sda] Sense not available.
 +
  [  856.833689] sd 0:0:0:0: [sda] Attached SCSI disk
 +
  [  857.186393] ------------[ cut here ]------------
 +
  [  857.187054] WARNING: at drivers/scsi/libfc/fc_lport.c:1370 fc_lport_timeout+0x49/0x87 [libfc]()
 +
  [  857.188144] Hardware name:
 +
  [  857.191292] Modules linked in: sd_mod crc_t10dif fcoe libfcoe libfc scsi_transport_fc scsi_tgt ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi dm_multipath dm_mod scsi_dh loop parport_pc serio_raw parport tpm_tis virtio_balloon psmouse tpm tpm_bios pcspkr i2c_piix4 i2c_core processor button evdev ext3 jbd mbcache ide_cd_mod  ide_gd_mod cdrom ata_piix ata_generic libata scsi_mod piix ide_pci_generic floppy e1000e virtio_pci virtio_ring virtio 8139cp 8139too ide_core mii thermal fan thermal_sys
 +
  [  857.202922] Pid: 16, comm: events/1 Not tainted 2.6.34-rc1 #3
 +
  [  857.203563] Call Trace:
 +
  [  857.204064]  [<ffffffffa03594e1>] ? fc_lport_timeout+0x49/0x87 [libfc]
 +
  [  857.204747]  [<ffffffffa03594e1>] ? fc_lport_timeout+0x49/0x87 [libfc]
 +
  [  857.205434]  [<ffffffff8103892d>] ? warn_slowpath_common+0x77/0xa3
 +
  [  857.206102]  [<ffffffffa03594e1>] ? fc_lport_timeout+0x49/0x87 [libfc]
 +
  [  857.206878]  [<ffffffff8104cb06>] ? worker_thread+0x181/0x20f
 +
  [  857.207557]  [<ffffffffa0359498>] ? fc_lport_timeout+0x0/0x87 [libfc]
 +
  [  857.208285]  [<ffffffff81050126>] ? autoremove_wake_function+0x0/0x2e
 +
  [  857.209009]  [<ffffffff8104c985>] ? worker_thread+0x0/0x20f
 +
  [  857.209693]  [<ffffffff8104fcf9>] ? kthread+0x79/0x81
 +
  [  857.210424]  [<ffffffff81003694>] ? kernel_thread_helper+0x4/0x10
 +
  [  857.211141]  [<ffffffff8104fc80>] ? kthread+0x0/0x81
 +
  [  857.211791]  [<ffffffff81003690>] ? kernel_thread_helper+0x0/0x10
 +
  [  857.212497] ---[ end trace e194c9c11700625f ]---
 +
  [  857.265004] alua: device handler registered
 +
  [  857.290275] device-mapper: multipath round-robin: version 1.0.0 loaded
 +
  [  857.291599] sd 0:0:0:0: alua: supports implicit and explicit TPGS
 +
  [  857.292794] sd 0:0:0:0: alua: port group 00 rel port 02
 +
  [  857.293645] sd 0:0:0:0: alua: port group 00 state A supports TOUSNA
 +
  [  857.300249] sd 0:0:0:0: alua: port group 00 state A supports TOUSNA
 +
 
 +
and on the Open-FCoE TCM_FC target side:
 +
 
 +
  [  491.470139] host0: Assigned Port ID  10102
 +
  [  491.470789] host0: libfc: Port ( 10102) entered point-to-point mode
 +
  [  491.471679] TARGET_CORE[fc]: Registered fabric_sess_ptr: ffff88007d474340

Revision as of 00:43, 20 March 2010

As of March 19th, the TCM based Open-FCoE.org target fabric module in lio-core-2.6.git/tcm_fc has been committed into lio-core-2.6.git/master as LIO v3.4.0-rc1..!

100% of logic for v3.x Target_Core_Mod/ConfigFS Persistent_Reservations and ALUA is now running in multi-fabric mode between and across LIO-Target TCM_Loop and OpenFCoE SCSI Ports..!

This would not have been possible without the tremendous contributions of tcm_fc by Joe Eykholt and his ongoing work of making libfc target capable. Thanks Joe!

Contents

Description

The tcm_fc fabric module is based on the open-fcoe.org target code and has been modified to run using a modified upstream Linux libfc module.

This includes a patch series to reenable Point to Point module for libfc (which have been merged upstream), and a set of hooks into libfc to be able to hand off incoming FLOGI requests to FC based target modules.

Control interface

The tcm_fc fabric module is using the configfs interface in /sys/kernel/config/target/fc. Note that there is NO strict open-fcoe userspace code requirement in order to get up and running.

Source

http://git.kernel.org/?p=linux/kernel/git/nab/lio-core-2.6.git;a=tree;f=drivers/target/tcm_fc;hb=HEAD

What does a running example look like..?

 # Create the '20:00:' prefixed FCoE Lport from network inteface eth0
 mkdir -p /sys/kernel/config/target/fc/20:00:00:e0:81:c0:90:b3/tpgt_1/lun/lun_0
 # Chdir and create tcm_fc_port from TCM FILEIO object
 cd /sys/kernel/config/target/fc/20\:00\:00\:e0\:81\:c0\:90\:b3/tpgt_1/lun/lun_0/
 ln -s /sys/kernel/config/target/core/fileio_0/myfile/ tcm_fc_port
 # Chdir and created '20:00' prefixed Initiator WWPN from it's own network interface
 cd ../../ ; mkdir -p acl/20:00:00:e0:81:c0:90:b2/lun_0 ; cd acl/20\:00\:00\:e0\:81\:c0\:90\:b2/lun_0
 # Create explict MappedLUN=0 for NodeACl to TPG LUN 0
 ln -s /sys/kernel/config/target/fc/20\:00\:00\:e0\:81\:c0\:90\:b3/tpgt_1/lun/lun_0/ lun_0
target# tree /sys/kernel/config/target/fc/
 /sys/kernel/config/target/fc/
 |-- 20:00:00:e0:81:c0:90:b3
 |   `-- tpgt_1
 |       |-- acl
 |       |   `-- 20:00:00:e0:81:c0:90:b2
 |       |       |-- auth
 |       |       |   |-- node_name
 |       |       |   `-- port_name
 |       |       `-- lun_0
 |       |           `-- lun_0 -> ../../../../../../../target/fc/20:00:00:e0:81:c0:90:b3/tpgt_1/lun/lun_0
 |       `-- lun
 |           `-- lun_0
 |               |-- alua_tg_pt_gp
 |               |-- alua_tg_pt_offline
 |               |-- alua_tg_pt_status
 |               |-- alua_tg_pt_write_md
 |               `-- tcm_fc_port -> ../../../../../../target/core/fileio_0/myfile
 `-- version
 10 directories, 7 files
 [  176.370130] <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API >>>>>>>>>>>>>>>>>>>>>>
 [  176.371205] Initialized struct target_fabric_configfs: ffff88003787b800 for fc
 [  176.372739] <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>>
 [  180.529964] Target_Core_ConfigFS: REGISTER -> group: ffffffffa02b9480 name: fc
 [  180.531086] Target_Core_ConfigFS: REGISTER -> Located fabric: fc
 [  180.531803] Target_Core_ConfigFS: REGISTER -> ffffffffa0366f70
 [  180.532506] Target_Core_ConfigFS: REGISTER -> Allocated Fabric: fc
 [  180.532921] Target_Core_ConfigFS: REGISTER -> Set tf->tf_fabric for fc
 [  180.532921] TARGET_CORE[fc]: Allocated Normal se_portal_group_t for endpoint: 20:00:00:e0:81:c0:90:b3, Portal Tag: 1
 [  188.053789] fileio/fc: Adding to default ALUA Target Port Group: alua/default_tg_pt_gp
 [  188.054913] fc_TPG[1]_LUN[0] - Activated fc Logical Unit from CORE HBA: 1
 [  211.862437] fc_TPG[1] - Added ACL with TCQ Depth: 32 for fc Initiator Node: 20:00:00:e0:81:c0:90:b2
 [  219.960620] fc_TPG[1]_LUN[0->0] - Added RW ACL for  InitiatorNode: 20:00:00:e0:81:c0:90:b2
 modprobe fcoe ; echo eth0 > /sys/module/fcoe/parameters/create

you should see something like the following in the target side kernel ring buffer:

 [  469.432122] device eth0 entered promiscuous mode
 [  469.433179] scsi0 : FCoE Driver
 [  469.435336] host0: libfc: Link up on port (     0)

and the target will be configured!

Open-FCoE Initiator

Now we are ready to connect from an Open-FCOE Initiator. Remember that in order to run the Initiator in Point to Point mode, the necessary patches to libfc are also required. Just as on the target side, load the fcoe module and create the association to the network interface on the initiator capable of communication with the LPORT network interface configured above:

modprobe fcoe ; echo eth0 > /sys/module/fcoe/parameters/create

After a few seconds, the FCoE LUNs will be detected and registered with Linux/SCSI.. Here is what that output looks like on the Open-FCOE initiator side:

 [  853.183484] device eth0 entered promiscuous mode
 [  853.184640] scsi0 : FCoE Driver
 [  853.186064] host0: libfc: Link up on port (     0)
 [  856.754764] host0: Assigned Port ID  10101
 [  856.758376] scsi 0:0:0:0: Direct-Access     LIO-ORG  FILEIO           3.1  PQ: 0 ANSI: 5
 [  856.818511] sd 0:0:0:0: [sda] READ CAPACITY(16) failed
 [  856.819201] sd 0:0:0:0: [sda] Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
 [  856.820127] sd 0:0:0:0: [sda] Sense not available.
 [  856.820964] sd 0:0:0:0: [sda] 97656251 512-byte logical blocks: (50.0 GB/46.5 GiB)
 [  856.822580] sd 0:0:0:0: [sda] Write Protect is off
 [  856.823230] sd 0:0:0:0: [sda] Mode Sense: 2f 00 00 00
 [  856.823640] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 [  856.825018] sd 0:0:0:0: [sda] READ CAPACITY(16) failed
 [  856.825688] sd 0:0:0:0: [sda] Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
 [  856.826620] sd 0:0:0:0: [sda] Sense not available.
 [  856.828304]  sda: unknown partition table
 [  856.830403] sd 0:0:0:0: [sda] READ CAPACITY(16) failed
 [  856.831109] sd 0:0:0:0: [sda] Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
 [  856.832039] sd 0:0:0:0: [sda] Sense not available.
 [  856.833689] sd 0:0:0:0: [sda] Attached SCSI disk
 [  857.186393] ------------[ cut here ]------------
 [  857.187054] WARNING: at drivers/scsi/libfc/fc_lport.c:1370 fc_lport_timeout+0x49/0x87 [libfc]()
 [  857.188144] Hardware name: 
 [  857.191292] Modules linked in: sd_mod crc_t10dif fcoe libfcoe libfc scsi_transport_fc scsi_tgt ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi dm_multipath dm_mod scsi_dh loop parport_pc serio_raw parport tpm_tis virtio_balloon psmouse tpm tpm_bios pcspkr i2c_piix4 i2c_core processor button evdev ext3 jbd mbcache ide_cd_mod   ide_gd_mod cdrom ata_piix ata_generic libata scsi_mod piix ide_pci_generic floppy e1000e virtio_pci virtio_ring virtio 8139cp 8139too ide_core mii thermal fan thermal_sys
 [  857.202922] Pid: 16, comm: events/1 Not tainted 2.6.34-rc1 #3
 [  857.203563] Call Trace:
 [  857.204064]  [<ffffffffa03594e1>] ? fc_lport_timeout+0x49/0x87 [libfc]
 [  857.204747]  [<ffffffffa03594e1>] ? fc_lport_timeout+0x49/0x87 [libfc]
 [  857.205434]  [<ffffffff8103892d>] ? warn_slowpath_common+0x77/0xa3
 [  857.206102]  [<ffffffffa03594e1>] ? fc_lport_timeout+0x49/0x87 [libfc]
 [  857.206878]  [<ffffffff8104cb06>] ? worker_thread+0x181/0x20f
 [  857.207557]  [<ffffffffa0359498>] ? fc_lport_timeout+0x0/0x87 [libfc]
 [  857.208285]  [<ffffffff81050126>] ? autoremove_wake_function+0x0/0x2e
 [  857.209009]  [<ffffffff8104c985>] ? worker_thread+0x0/0x20f
 [  857.209693]  [<ffffffff8104fcf9>] ? kthread+0x79/0x81
 [  857.210424]  [<ffffffff81003694>] ? kernel_thread_helper+0x4/0x10
 [  857.211141]  [<ffffffff8104fc80>] ? kthread+0x0/0x81
 [  857.211791]  [<ffffffff81003690>] ? kernel_thread_helper+0x0/0x10
 [  857.212497] ---[ end trace e194c9c11700625f ]---
 [  857.265004] alua: device handler registered
 [  857.290275] device-mapper: multipath round-robin: version 1.0.0 loaded
 [  857.291599] sd 0:0:0:0: alua: supports implicit and explicit TPGS
 [  857.292794] sd 0:0:0:0: alua: port group 00 rel port 02
 [  857.293645] sd 0:0:0:0: alua: port group 00 state A supports TOUSNA
 [  857.300249] sd 0:0:0:0: alua: port group 00 state A supports TOUSNA

and on the Open-FCoE TCM_FC target side:

 [  491.470139] host0: Assigned Port ID  10102
 [  491.470789] host0: libfc: Port ( 10102) entered point-to-point mode
 [  491.471679] TARGET_CORE[fc]: Registered fabric_sess_ptr: ffff88007d474340
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Google AdSense