vHost
The Linux SCSI Target Wiki
Line 57: | Line 57: | ||
|- | |- | ||
! align="left" | 1x [[rd_mcp]] LUN | ! align="left" | 1x [[rd_mcp]] LUN | ||
- | | 8 || 155k IOPs || 145k IOPs | + | | 8 || ~155k IOPs || ~145k IOPs |
|- | |- | ||
! align="left" | 16x [[rd_mcp]] LUN | ! align="left" | 16x [[rd_mcp]] LUN | ||
- | | 16 || 315k IOPs || 305k IOPs | + | | 16 || ~315k IOPs || ~305k IOPs |
|- | |- | ||
! align="left" | 32x [[rd_mcp]] LUN | ! align="left" | 32x [[rd_mcp]] LUN | ||
- | | 16 || 425k IOPs || 410k IOPs | + | | 16 || ~425k IOPs || ~410k IOPs |
|} | |} | ||
<br/> | <br/> | ||
Line 76: | Line 76: | ||
|- | |- | ||
! align="left" | 1x [[rd_mcp]] LUN | ! align="left" | 1x [[rd_mcp]] LUN | ||
- | | 32 || 160K IOPs || 150K IOPs | + | | 32 || ~160K IOPs || ~150K IOPs |
|- | |- | ||
! align="left" | 16x [[rd_mcp]] LUN | ! align="left" | 16x [[rd_mcp]] LUN | ||
- | | 32 || 1125k IOPs || 1100k IOPs | + | | 32 || ~1125k IOPs || ~1100k IOPs |
|- | |- | ||
! align="left" | 32x [[rd_mcp]] LUN | ! align="left" | 32x [[rd_mcp]] LUN | ||
- | | 32 || 1185k IOPs || 1175k IOPs | + | | 32 || ~1185k IOPs || ~1175k IOPs |
|} | |} | ||
Revision as of 23:12, 22 August 2012
![]() | |
---|---|
![]() iSCSI fabric module | |
Original author(s) |
Nicholas Bellinger Stefan Hajnoczi |
Developer(s) | Datera, Inc. |
Preview release | 4.2.0-rc5 / July 30, 2012 |
Development status | Production |
Written in | C |
Operating system | Linux |
Type | Fabric module |
License | GNU General Public License |
Website | datera.io |
- See Target for a complete overview over all fabric modules.
vHost provides a very high performance local SCSI target for KVM guests.
Contents |
Overview
The RTS vHost fabric module implements I/O processing based on the Linux virtio mechanis. With this, Linux KVM guests can drive more than 3 times better performance with local Linux backstores than with any other SCSI target that is available under Linux.
Guests
RTS vHost allows effectively running very high-performance applications, such as database engines, in KVM guests on local storage.
Linux
The Linux performance numbers were measured on a dual Intel Xeon-E5-2687W 3.10 Ghz CPU Romley-EP system with 32x threads and 32 GB DDR3-1600 SDRAM.[1]
The results for one KVM VM and a varying number of LUNs on RTS OS with vHost on RAM are as follows.
IOPS:
KVM | |||
---|---|---|---|
Workload | Jobs | 25%/75% Read/Write | 75%/25% Read/Write |
1x rd_mcp LUN | 8 | ~155k IOPs | ~145k IOPs |
16x rd_mcp LUN | 16 | ~315k IOPs | ~305k IOPs |
32x rd_mcp LUN | 16 | ~425k IOPs | ~410k IOPs |
Native | |||
---|---|---|---|
Workload | Jobs | 25%/75% Read/Write | 75%/25% Read/Write |
1x rd_mcp LUN | 32 | ~160K IOPs | ~150K IOPs |
16x rd_mcp LUN | 32 | ~1125k IOPs | ~1100k IOPs |
32x rd_mcp LUN | 32 | ~1185k IOPs | ~1175k IOPs |
Throughput:
KVM | |||
---|---|---|---|
Workload | Jobs | 25%/75% Read/Write | 75%/25% Read/Write |
1x rd_mcp LUN | 8 | ~1800 MB/s | ~1800 MB/s |
16x rd_mcp LUN | 16 | ~8000 MB/s | ~8000 MB/s |
32x rd_mcp LUN | 16 | ~18500 MB/s | ~18500 MB/s |
Native | |||
---|---|---|---|
Workload | Jobs | 25%/75% Read/Write | 75%/25% Read/Write |
1x rd_mcp LUN | 32 | ~2048 MB/s | ~2048 MB/s |
16x rd_mcp LUN | 32 | ~17500 MB/s | ~17500 MB/s |
32x rd_mcp LUN | 32 | ~20480 MB/s | ~20480 MB/s |
The benchmarks were done with FIO with the following parameters:
[randrw] rw=rw rwmixwrite=25 rwmixread=75 ioengine=libaio direct=1 size=100G iodepth=64 iodepth_batch=4 iodepth_batch_complete=32 numjobs=32 blocksize=1M filename=/dev/sdb filename=/dev/sdX....
Windows
RTS will enable Windows guests to achieve the same level of performance with the virtio Virtual MegaRAID driver, and other future work on native Windows virtio drivers.
targetcli
targetcli from Datera, Inc. is used to configure vHost targets. targetcli aggregates service modules via a core library, and exports them through an API to the Unified Target, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).
I/O processing details
vHost processes I/Os from Linux guests to Unified Target backstores as follows:
- The KVM guest enqueues the SCSI I/O descriptor(s) to its virtio ring;
- The KVM guest kicks the Unified Target to wake up;
- The Unified Target wakes up, dequeues the I/O descriptor(s) off the virtio ring and processes them;
- The Unified Target dispatches the I/O to the backend storage device (HDDs, SSDs, flash, RAM, etc.);
- The Unified Target backend storage device completes the I/O;
- The Unified Target enqueues the resulting I/O descriptor(s) to the KVM guest virtio ring;
- The Unified Target kicks the KVM guest to wake up;
- The KVM guest wakes up and dequeues the I/O descriptor(s) off the virtio ring.
Spec file
RTS spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.
In particular, the iSCSI spec file /var/target/fabric/vhost.spec is included via RTSlib.
# WARNING: This is a draft specfile supplied for demo purposes only. # The vHost fabric module feature set features = nexus # Use naa WWNs. wwn_type = naa # Non-standard module naming scheme kernel_module = tcm_vhost # The configfs group name is default configfs_group = vhost
Timeline
Timeline of the LinuxIO | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Release | Details | 2011 | 2012 | 2013 | 2014 | 2015 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | ||
4.x | Version | 4.0 | 4.1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Feature | LIO Core | Loop back | FCoE | iSCSI | Perf | SRP | CM WQ | FC USB 1394 | vHost | Perf | Misc | 16 GFC | iSER | Misc | VAAI | Misc | DIF Core NPIV | DIF iSER | DIF FC vhost | TCMU Xen | Misc | Misc | virtio 1.0 | Misc | NVMe OF | ||||||||||||||||||||||||||||||||||||
Linux | 2.6.38 | 2.6.39 | 3.0 | 3.1 | 3.2 | 3.3 | 3.4 | 3.5 | 3.6 | 3.7 | 3.8 | 3.9 | 3.10 | 3.11 | 3.12 | 3.13 | 3.14 | 3.15 | 3.16 | 3.17 | 3.18 | 3.19 | 3.20 | 3.21 | 3.22 |
See also
Notes
- ↑ Nicholas A. Bellinger (8/7/2012). "SCSI small block random I/O performance on 3.6-rc0 using SCSI loopback ports". http://permalink.gmane.org.
External links
- RTS OS Admin Manual
- RTSlib Reference Guide [HTML][PDF]
- Multiqueue KVM
- QEMU roadmap
- QEMU/KVM vhost-scsi driver in qemu-kvm.git repository
- Hajnoczi, et al. Virtio SCSI an alternative virtualized storage stack for KVM. KVM Forum, Vancouver, Canada: YouTube.