User:Delimiter/FreeBSD/iSCSI
Introduction
This page is an attempt to explain the iSCSI options available to the FreeBSD community.
I will help document the quirks, performance, etc of iSCSI on the FreeBSD platform.
Option A - base kernel module & initiator
If all you need is initiator capability (mount remote storage), as of 7.0-RELEASE there is kernel module iscsi_initiator(4) and initiator command iscontrol(8). The latter is controlled by the /etc/iscsi.conf configuration file - see iscsi.conf(5)
Option B - userland components ported from NetBSD
iscsi-target is based on netbsd-iscsi written by Alistair Crooks for the NetBSD platform.
Recent versions also include initiator capability as these are utilized to for the test harness. However this is not provided by the port/package as of yet.
Known issues
(2008-Apr-10)
There is a compatibility problem due apparently to endianness disparity between 32-bit (x86) and 64-bit (sparc64) architectures. Mix and match at your OWN RISK.
Usage / Troubleshooting
Client-side (using iscontrol)
First setup /etc/iscsi.conf with a valid target0 declaration, e.g.
port = 3260
target0 {
targetaddress = 192.168.1.11 # your target IP/hostname
targetname = iqn.1994-04.org.netbsd.iscsi-target:target0
Now initiate the connection.
kldload iscsi_initiator iscontrol -c /etc/iscsi.conf -n target0
If all goes well you can see your block storage by issuing the dmesg command or looking in /var/log/messages
da0 at iscsi0 bus 0 target 0 lun 0 da0: <NetBSD NetBSD iSCSI 0> Fixed Direct Access SCSI-3 device
You can now manipulate da0 with fdisk or if already formatted with a recognizable fs, mount it!
Also, you can just do a simple discovery using this command...
iscontrol -d [v-]targetaddress=targethost initiatorname=initiatorhost
Client-side (using open-iscsi)
OK so open-iscsi is Linux but at some point it's all I had to test with.
# iscsiadm --mode discovery --type sendtargets --portal 192.168.1.12 192.168.1.12:3260,1 iqn.1994-04.org.netbsd.iscsi-target:target0 192.168.1.12:3260,1 iqn.1994-04.org.netbsd.iscsi-target:target1
# iscsiadm -m node -T iqn.1994-04.org.netbsd.iscsi-target:target0 -p 192.168.1.12:3260 node.name = iqn.1994-04.org.netbsd.iscsi-target:target0 node.tpgt = 1 node.startup = manual iface.hwaddress = default iface.iscsi_ifacename = default iface.net_ifacename = default iface.transport_name = tcp node.discovery_address = 192.168.1.12 node.discovery_port = 3260 node.discovery_type = send_targets node.session.initial_cmdsn = 0 node.session.initial_login_retry_max = 4 node.session.cmds_max = 128 node.session.queue_depth = 32 node.session.auth.authmethod = None node.session.auth.username = <empty> node.session.auth.password = <empty> node.session.auth.username_in = <empty> node.session.auth.password_in = <empty> node.session.timeo.replacement_timeout = 120 node.session.err_timeo.abort_timeout = 10 node.session.err_timeo.reset_timeout = 30 node.session.iscsi.InitialR2T = No node.session.iscsi.ImmediateData = Yes node.session.iscsi.FirstBurstLength = 262144 node.session.iscsi.MaxBurstLength = 16776192 node.session.iscsi.DefaultTime2Retain = 0 node.session.iscsi.DefaultTime2Wait = 0 node.session.iscsi.MaxConnections = 1 node.session.iscsi.MaxOutstandingR2T = 1 node.session.iscsi.ERL = 0 node.conn[0].address = 192.168.1.12 node.conn[0].port = 3260 node.conn[0].startup = manual node.conn[0].tcp.window_size = 524288 node.conn[0].tcp.type_of_service = 0 node.conn[0].timeo.logout_timeout = 15 node.conn[0].timeo.login_timeout = 15 node.conn[0].timeo.auth_timeout = 45 node.conn[0].timeo.active_timeout = 5 node.conn[0].timeo.idle_timeout = 60 node.conn[0].timeo.ping_timeout = 5 node.conn[0].timeo.noop_out_interval = 10 node.conn[0].timeo.noop_out_timeout = 15 node.conn[0].iscsi.MaxRecvDataSegmentLength = 131072 node.conn[0].iscsi.HeaderDigest = None,CRC32C node.conn[0].iscsi.DataDigest = None node.conn[0].iscsi.IFMarker = No node.conn[0].iscsi.OFMarker = No
Add -l to login to the target, then check dmesg for indication of what the device now attached is.
# iscsiadm -m node -T iqn.1994-04.org.netbsd.iscsi-target:target0 -p 192.168.1.12:3260 -l # dmesg ... [257646.128000] sdb: unknown partition table [257646.140000] sd 9:0:0:0: [sdb] Attached SCSI disk [257646.140000] sd 9:0:0:0: Attached scsi generic sg3 type 0
Server-side (using iscsi-target)
# iscsi-target -D -v all -f /usr/local/etc/iscsi/targets
Reading configuration from `/usr/local/etc/iscsi/targets'
target0:rw:192.168.0.0/16
extent0:/tmp/iscsi-target0:0:104857600
target1:rw:192.168.0.0/16
extent1:/tmp/iscsi-target1:0:209715200
pid 830:util.c:110: iscsi_malloc_atomic(64) = 0x8072040
DISK: 1 logical unit (204800 blocks, 512 bytes/block), type iscsi fs
DISK: LUN 0: 100 MB disk storage for "target0"
DISK: 1 logical unit (409600 blocks, 512 bytes/block), type iscsi fs
DISK: LUN 0: 200 MB disk storage for "target1"
TARGET: TargetName is iqn.1994-04.org.netbsd.iscsi-target
pid 830:target.c:1638: listener thread started
pid 830:target.c:1646: create, bind, listen OK
pid 830:util.c:652: connection 6 selected
pid 830:target.c:1668: waiting for IPv4 connection on port 3260
pid 830:target.c:1708: IPv4 connection accepted on port 3260 (local IP 192.168.1.12, remote IP 192.168.1.9)
pid 830:target.c:1710: TargetAddress = "192.168.1.12:3260,1"
pid 830:target.c:1192: session 0: started
...
Performance
Spew test, 50M in size.
# mount ... /dev/sdb1 on /tmp/sdb type ext2 (rw) /dev/sdc1 on /tmp/sdc type jfs (rw)
# spew -v --write 50M /tmp/sdb/spewtest WTR: 13263.21 KiB/s Transfer time: 00:00:03 IOPS: 26526.42 Total iterations: 1 Total runtime: 00:00:03 Total write transfer time (WTT): 00:00:03 Total write transfer rate (WTR): 13263.21 KiB/s Total write IOPS: 26526.42 IOPS # spew -v --write 50M /tmp/sdc/spewtest WTR: 13534.02 KiB/s Transfer time: 00:00:03 IOPS: 27068.03 Total iterations: 1 Total runtime: 00:00:04 Total write transfer time (WTT): 00:00:03 Total write transfer rate (WTR): 13534.02 KiB/s Total write IOPS: 27068.03 IOPS
Because these values are near what should be the throughput capacity for 100Mb network, these tests are inconclusive.