User:Delimiter/FreeBSD/iSCSI

From ConShell
Jump to: navigation, search

Introduction

This page is an attempt to explain the iSCSI options available to the FreeBSD community.

I will help document the quirks, performance, etc of iSCSI on the FreeBSD platform.

Option A - base kernel module & initiator

If all you need is initiator capability (mount remote storage), as of 7.0-RELEASE there is kernel module iscsi_initiator(4) and initiator command iscontrol(8). The latter is controlled by the /etc/iscsi.conf configuration file - see iscsi.conf(5)

Option B - userland components ported from NetBSD

iscsi-target is based on netbsd-iscsi written by Alistair Crooks for the NetBSD platform.

Recent versions also include initiator capability as these are utilized to for the test harness. However this is not provided by the port/package as of yet.

Known issues

(2008-Apr-10)

There is a compatibility problem due apparently to endianness disparity between 32-bit (x86) and 64-bit (sparc64) architectures. Mix and match at your OWN RISK.

Usage / Troubleshooting

Client-side (using iscontrol)

First setup /etc/iscsi.conf with a valid target0 declaration, e.g.

port = 3260
target0 {
targetaddress = 192.168.1.11 # your target IP/hostname
targetname = iqn.1994-04.org.netbsd.iscsi-target:target0

Now initiate the connection.

 kldload iscsi_initiator
 iscontrol -c /etc/iscsi.conf -n target0

If all goes well you can see your block storage by issuing the dmesg command or looking in /var/log/messages

da0 at iscsi0 bus 0 target 0 lun 0
da0: <NetBSD NetBSD iSCSI 0> Fixed Direct Access SCSI-3 device

You can now manipulate da0 with fdisk or if already formatted with a recognizable fs, mount it!

Also, you can just do a simple discovery using this command...

iscontrol -d [v-]targetaddress=targethost initiatorname=initiatorhost

Client-side (using open-iscsi)

OK so open-iscsi is Linux but at some point it's all I had to test with.

# iscsiadm --mode discovery --type sendtargets --portal 192.168.1.12
192.168.1.12:3260,1 iqn.1994-04.org.netbsd.iscsi-target:target0
192.168.1.12:3260,1 iqn.1994-04.org.netbsd.iscsi-target:target1
# iscsiadm -m node -T iqn.1994-04.org.netbsd.iscsi-target:target0 -p 192.168.1.12:3260 
node.name = iqn.1994-04.org.netbsd.iscsi-target:target0
node.tpgt = 1
node.startup = manual
iface.hwaddress = default
iface.iscsi_ifacename = default
iface.net_ifacename = default
iface.transport_name = tcp
node.discovery_address = 192.168.1.12
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 4
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.auth.authmethod = None
node.session.auth.username = <empty>
node.session.auth.password = <empty>
node.session.auth.username_in = <empty>
node.session.auth.password_in = <empty>
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 10
node.session.err_timeo.reset_timeout = 30
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 0
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 192.168.1.12
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.active_timeout = 5
node.conn[0].timeo.idle_timeout = 60
node.conn[0].timeo.ping_timeout = 5
node.conn[0].timeo.noop_out_interval = 10
node.conn[0].timeo.noop_out_timeout = 15
node.conn[0].iscsi.MaxRecvDataSegmentLength = 131072
node.conn[0].iscsi.HeaderDigest = None,CRC32C
node.conn[0].iscsi.DataDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No

Add -l to login to the target, then check dmesg for indication of what the device now attached is.

# iscsiadm -m node -T iqn.1994-04.org.netbsd.iscsi-target:target0 -p 192.168.1.12:3260 -l
# dmesg
...
[257646.128000]  sdb: unknown partition table
[257646.140000] sd 9:0:0:0: [sdb] Attached SCSI disk
[257646.140000] sd 9:0:0:0: Attached scsi generic sg3 type 0

Server-side (using iscsi-target)

# iscsi-target -D -v all -f /usr/local/etc/iscsi/targets 
Reading configuration from `/usr/local/etc/iscsi/targets'
target0:rw:192.168.0.0/16
        extent0:/tmp/iscsi-target0:0:104857600
target1:rw:192.168.0.0/16
        extent1:/tmp/iscsi-target1:0:209715200
pid 830:util.c:110: iscsi_malloc_atomic(64) = 0x8072040
DISK: 1 logical unit (204800 blocks, 512 bytes/block), type iscsi fs
DISK: LUN 0: 100 MB disk storage for "target0"
DISK: 1 logical unit (409600 blocks, 512 bytes/block), type iscsi fs
DISK: LUN 0: 200 MB disk storage for "target1"
TARGET: TargetName is iqn.1994-04.org.netbsd.iscsi-target
pid 830:target.c:1638: listener thread started
pid 830:target.c:1646: create, bind, listen OK

pid 830:util.c:652: connection 6 selected
pid 830:target.c:1668: waiting for IPv4 connection on port 3260
pid 830:target.c:1708: IPv4 connection accepted on port 3260 (local IP 192.168.1.12, remote IP 192.168.1.9)
pid 830:target.c:1710: TargetAddress = "192.168.1.12:3260,1"
pid 830:target.c:1192: session 0: started
...

Performance

Spew test, 50M in size.

# mount
...
/dev/sdb1 on /tmp/sdb type ext2 (rw)
/dev/sdc1 on /tmp/sdc type jfs (rw)
# spew -v --write 50M /tmp/sdb/spewtest
WTR:    13263.21 KiB/s   Transfer time: 00:00:03    IOPS:    26526.42

Total iterations:                                1
Total runtime:                            00:00:03
Total write transfer time (WTT):          00:00:03
Total write transfer rate (WTR):    13263.21 KiB/s
Total write IOPS:                   26526.42 IOPS

# spew -v --write 50M /tmp/sdc/spewtest
WTR:    13534.02 KiB/s   Transfer time: 00:00:03    IOPS:    27068.03

Total iterations:                                1
Total runtime:                            00:00:04
Total write transfer time (WTT):          00:00:03
Total write transfer rate (WTR):    13534.02 KiB/s
Total write IOPS:                   27068.03 IOPS

Because these values are near what should be the throughput capacity for 100Mb network, these tests are inconclusive.

Related