iSCSI

Brief introduction

The iSCSI protocol allows SCSI commands to be used over a TCP/IP network.
The main reason people want to use an iSCSI is reducing costs, since they don't need buying FC HBA and infrastructure is already setup.
The default port for iSCSI targets is 3260.

iSCSI versus NFS

Both are used for accessing storage device over network, so what’s the difference? 

1. NFS is used for accessing remote FILE SYSYEM data. 
   Many people can assess data so there is functionality of locking data when used by someone, so others have to wait. 

2. iSCSI is used for accessing BLOCKS on remote disk. 
   In this case many users cannot share this access, because there is not lock functionality for block level access. 

In this example I have SunFire T2000 directly connected to StorEdge 3510 (with FC cables). 
The server is running Solaris 10 update 7. 
This server is iSCSI target and is exporting block device (ZFS volume) in order to be accessed by another Solaris box over network (iSCSI initiator).  

Configuring iSCSI Target - no Authentication

You need next packages installed: 

system      SUNWiscsir                   Sun iSCSI Device Driver (root)
system      SUNWiscsitgtr                Sun iSCSI Target (Root)
system      SUNWiscsitgtu                Sun iSCSI Target (Usr)
system      SUNWiscsiu                   Sun iSCSI Management Utilities (usr)

Enable the service  svc:/system/iscsitgt:default

1. Create a base directory

The base directory is used to store the iSCSI target configuration data and needs to be defined prior to using the iSCSI target for the first time.


#  iscsitadm modify admin -d /var/iscsi 


2. Configure a backing store

The backing store contains the physical storage that is exported as iSCSI target.
With Solaris, next can be backing store: flat files, physical device, SVM, ZFS volume. 

Let's create ZFS volumes on StorEdge 3510, one on RAID1 and RAID5. 
> format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
       0. c1t40d0 
          /pci@7c0/pci@0/pci@1/pci@0,2/SUNW,qlc@1/fp@0,0/ssd@w216000c0ff89cacc,0
       1. c1t40d1 
          /pci@7c0/pci@0/pci@1/pci@0,2/SUNW,qlc@1/fp@0,0/ssd@w216000c0ff89cacc,1
       2. c2t0d0 
          /pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@0,0
       3. c2t1d0 
          /pci@7c0/pci@0/pci@1/pci@0,2/LSILogic,sas@2/sd@1,0

> zpool create -f drum-raid5 c1t40d0 

> zpool create -f drum-raid1 c1t40d1 

> zpool status
  pool: drum-raid1
 state: ONLINE
 scrub: none requested
config:
        NAME        STATE     READ WRITE CKSUM
        drum-raid1  ONLINE       0     0     0
          c1t40d1   ONLINE       0     0     0

  pool: drum-raid5
 state: ONLINE
 scrub: none requested
config:
        NAME        STATE     READ WRITE CKSUM
        drum-raid5  ONLINE       0     0     0
          c1t40d0   ONLINE       0     0     0

> zpool list
NAME         SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
drum-raid1    68G    94K  68.0G     0%  ONLINE  -
drum-raid5   136G   111K   136G     0%  ONLINE  -

> zfs create -V 30g drum-raid1/volume-no-CHAP 

> zfs list
NAME                        USED  AVAIL  REFER  MOUNTPOINT
drum-raid1                 30.0G  36.9G    18K  /drum-raid1
drum-raid1/volume-no-CHAP    30G  66.9G    30K  -
drum-raid5                  112K   134G    18K  /drum-raid5


The ZFS Volume (drum-raid1/volume-no-CHAP) is now created.

3. Create a target

Let's now create iSCSI target. 
> iscsitadm create target
iscsitadm: at least one option required
        iscsitadm create target  
        OPTIONS:
                -t, --type  
                -u, --lun  
                -z, --size  
                -a, --alias  
                -b, --backing-store  
For more information, please see iscsitadm(1M)

>iscsitadm create target -b /dev/zvol/dsk/drum-raid1/volume-no-CHAP testors-no-CHAP 


If you change mind and want to removing iSCSI targets, do:

> iscsitadm delete target --lun 0 testors-no-CHAP 


4. Verify the target configuration
> iscsitadm list target -v 
Target: testors-no-chap
    iSCSI Name: iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap
    Connections: 0
    ACL list:
    TPGT list:
    LUN information:
        LUN: 0
            GUID: 0
            VID: SUN
            PID: SOLARIS
            Type: disk
            Size:   30G
            Backing store: /dev/zvol/dsk/drum-raid1/volume-no-CHAP
            Status: online


The iSCSI configuration data file is created in base directory: 
# ls /etc/iscsi
total 12
drwxr-xr-x   2 root     sys          512 Mar 18 15:23 .
drwxr-xr-x  58 root     sys         4096 Mar 18 15:37 ..
-rw-------   1 root     root         548 Mar 18 15:23 iscsi_v1.dbc


Note about iSCSI name:

It can be in two formats: IQN or EUI.

1. IQN - has date, domain and node identification, like in my example. 
2. EUI - has 16 hexadecimal digits, resembles WWN of FC node. 

5. Once iSCSI initiator is configured (will be done in next steps), you can list it: 
> iscsitadm list initiator 
Initiator: cs2
    iSCSI Name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba69545
    CHAP Name: cs2

Configuring iSCSI Initiator - no Authentication

You need next packages installed: 

system      SUNWiscsir                   Sun iSCSI Device Driver (root)
system      SUNWiscsiu                   Sun iSCSI Management Utilities (usr)

Enable service svc:/network/iscsi_initiator:default

1. Configure a discovery method

Have only static discovery (use this for small number of targets or restrict targets that initiator can access)
# iscsiadm modify discovery 
iscsiadm: at least one option required
        iscsiadm modify discovery 
        OPTIONS:
                -s, --static  
                -t, --sendtargets  
                -i, --iSNS  
For more information, please see iscsiadm(1M)

# iscsiadm modify discovery -i disable
# iscsiadm modify discovery -t disable

# iscsiadm list discovery
Discovery:
        Static: enabled
        Send Targets: disabled
        iSNS: disabled

# iscsiadm add static-config iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap,192.168.24.35


2. Verify the targets
> iscsiadm list target -vS 
Target: iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap
        Alias: testors-no-chap
        TPGT: 1
        ISID: 4000002a0000
        Connections: 1
                CID: 0
                  IP address (Local): 192.168.20.222:32784
                  IP address (Peer): 192.168.24.35:3260
                  Discovery Method: Static
                  Login Parameters (Negotiated):
                        Data Sequence In Order: yes
                        Data PDU In Order: yes
                        Default Time To Retain: 20
                        Default Time To Wait: 2
                        Error Recovery Level: 0
                        First Burst Length: 65536
                        Immediate Data: yes
                        Initial Ready To Transfer (R2T): yes
                        Max Burst Length: 262144
                        Max Outstanding R2T: 1
                        Max Receive Data Segment Length: 8192
                        Max Connections: 1
                        Header Digest: NONE
                        Data Digest: NONE
        LUN: 0
             Vendor:  SUN
             Product: SOLARIS
             OS Device Name: /dev/rdsk/c4t2d0s2

> iscsiadm list target-param -v 
Target: iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap
        Alias: testors-no-chap
        Bi-directional Authentication: disabled
        Authentication Type: NONE
        Login Parameters (Default/Configured):
                Data Sequence In Order: yes/-
                Data PDU In Order: yes/-
                Default Time To Retain: 20/-
                Default Time To Wait: 2/-
                Error Recovery Level: 0/-
                First Burst Length: 65536/-
                Immediate Data: yes/-
                Initial Ready To Transfer (R2T): yes/-
                Max Burst Length: 262144/-
                Max Outstanding R2T: 1/-
                Max Receive Data Segment Length: 8192/-
                Max Connections: 1/-
                Header Digest: NONE/-
                Data Digest: NONE/-
        Configured Sessions: 1

> iscsiadm list initiator-node 
Initiator node name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba6c502
Initiator node alias: -
        Login Parameters (Default/Configured):
                Header Digest: NONE/-
                Data Digest: NONE/-
        Authentication Type: NONE
        RADIUS Server: NONE
        RADIUS access: unknown
        Configured Sessions: 1


3. Initialize and use the new targets

Target(s) is visible, device entries are needed to create with command:

> devfsadm -Cv -i iscsi 


> format
AVAILABLE DISK SELECTIONS:
       0. c1t0d0 
          /pci@1f,0/pci@1/scsi@8/sd@0,0
       1. c1t1d0 
          /pci@1f,0/pci@1/scsi@8/sd@1,0
       2. c4t2d0 
          /iscsi/disk@0000iqn.1986-03.com.sun%3A02%3A92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chapFFFF,0

4. Create file systems 

> zpool create iscsi-no-chap c4t2d0 

> zfs list
NAME            USED  AVAIL  REFER  MOUNTPOINT
iscsi-no-chap  89.5K  29.3G     1K  /iscsi-no-chap

A. In case you want to remove already configured iSCSI Target

Find your targets.
> iscsiadm list target
Target: iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap
        Alias: testors-no-chap
        TPGT: 1
        ISID: 4000002a0000
        Connections: 1


B. Remove the target

> iscsiadm remove static-config iqn.1986-03.com.sun:02:92b0edad-52cd-ca06-93bd-d39a31259a2b.testors-no-chap

The iSCSI authentication

The iSCSI authentication may be required since iSCSI target cannot determine if connection request is from valid host. 
For example, someone else can connect to already connected (in use) block device, and type 'newfs' and you are screwed. 

Target authenticates an initiator by using Challenge-Handshake Authentication Protocol (CHAP). 

Authentication can be:

1. Unidirectional: only target identifies initiator. 
2. Bidirectional: initiators also identifies target. 

Let's create new volume for testing iSCSI setup with Bidirectional authentication : 

> zfs create -V 40g drum-raid5/volume-yes-CHAP
> zfs list
NAME                         USED  AVAIL  REFER  MOUNTPOINT
drum-raid5                  40.0G  93.9G    18K  /drum-raid5
drum-raid5/volume-yes-CHAP    40G   134G    16K  -

An iSCSI initiator configuration for Bidirectional authentication

1. Set a secret key (between 12-16 characters)
> iscsiadm modify initiator-node --CHAP-secret
Enter secret:
Re-enter secret:


2. Set iSCSI initiator CHAP name (let's make it same as hostname of iSCSI initiator) 
> iscsiadm modify initiator-node --CHAP-name counterstrike2


3. Tell iSCSI initiator to use CHAP
> iscsiadm modify initiator-node --authentication CHAP

> iscsiadm list initiator-node
Initiator node name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba6c502
Initiator node alias: -
        Login Parameters (Default/Configured):
                Header Digest: NONE/-
                Data Digest: NONE/-
                Authentication Type: CHAP
                CHAP Name: counterstrike2 
        RADIUS Server: NONE
        RADIUS access: unknown
        Configured Sessions: 1


Just as reminder how to add iSCSI target in case it's not already added. 
> iscsiadm add static-config iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap,192.168.24.35


4. We want to enable bidirectional authentication on iSCSI target. 
> iscsiadm modify target-param --bi-directional-authentication enable iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap

> iscsiadm list target-param -v
Target: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
        Alias: testors-yes-chap
        Bi-directional Authentication: enabled 
        Authentication Type: NONE
        Login Parameters (Default/Configured):
                Data Sequence In Order: yes/-
                Data PDU In Order: yes/-
                Default Time To Retain: 20/-
                Default Time To Wait: 2/-
                Error Recovery Level: 0/-
                First Burst Length: 65536/-
                Immediate Data: yes/-
                Initial Ready To Transfer (R2T): yes/-
                Max Burst Length: 262144/-
                Max Outstanding R2T: 1/-
                Max Receive Data Segment Length: 8192/-
                Max Connections: 1/-
                Header Digest: NONE/-
                Data Digest: NONE/-
        Configured Sessions: 1


5. And setup authentication method which is CHAP.
> iscsiadm modify target-param --authentication CHAP iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap


6. The iSCSI target must also know secret-key, we already setup on iSCSI initiator in step 1. 
> iscsiadm modify target-param --CHAP-secret iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
Enter secret:
Re-enter secret:


7. Final bidirectional config is below. 
> iscsiadm list target-param -v                                                                                         
Target: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
        Alias: testors-yes-chap
        Bi-directional Authentication: enabled
        Authentication Type: CHAP  
                CHAP Name: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap < - - see this 
        Login Parameters (Default/Configured):
                Data Sequence In Order: yes/-
                Data PDU In Order: yes/-
                Default Time To Retain: 20/-
                Default Time To Wait: 2/-
                Error Recovery Level: 0/-
                First Burst Length: 65536/-
                Immediate Data: yes/-
                Initial Ready To Transfer (R2T): yes/-
                Max Burst Length: 262144/-
                Max Outstanding R2T: 1/-
                Max Receive Data Segment Length: 8192/-
                Max Connections: 1/-
                Header Digest: NONE/-
                Data Digest: NONE/-
        Configured Sessions: 1

The iSCSI target configuration for Bidirectional authentication

Quick reminder: how to create iSCSI target: 

> iscsitadm create target -b /dev/zvol/dsk/drum-raid5/volume-yes-CHAP testors-yes-CHAP

> iscsitadm list target
Target: testors-yes-chap
    iSCSI Name: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
    Connections: 0

> iscsitadm list target -v
Target: testors-yes-chap
    iSCSI Name: iqn.1986-03.com.sun:02:504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chap
    Connections: 0
    ACL list:
    TPGT list:
    LUN information:
        LUN: 0
            GUID: 0
            VID: SUN
            PID: SOLARIS
            Type: disk
            Size:   40G
            Backing store: /dev/zvol/dsk/drum-raid5/volume-yes-CHAP
            Status: online

1. Set iSCSI target CHAP name as its own hostname
> iscsitadm modify admin --chap-name testors


2. Set the secret (same secret as for iSCSI initiator)
> iscsitadm modify admin --chap-secret
Enter secret:
Re-enter secret:


3. Creat an initiator object (this will be associated to one/more targets)
> iscsitadm create initiator --iqn iqn.1986-03.com.sun:01:0003ba3559b8.4ba69545 cs2

> iscsitadm list initiator -v
Initiator: cs2
    iSCSI Name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba69545
    CHAP Name: Not set
    CHAP Secret: Not set


4. Create CHAP name
> iscsitadm modify initiator --chap-name cs2 cs2

> iscsitadm list initiator -v
Initiator: cs2
    iSCSI Name: iqn.1986-03.com.sun:01:0003ba3559b8.4ba69545
    CHAP Name: cs2		< -  see this
    CHAP Secret: Not set


5. Set CHAP secret  (same secret as for iSCSI initiator)
> iscsitadm modify initiator --chap-secret cs2
Enter secret:
Re-enter secret:


On iSCSI initiator: 

> format
Searching for disks...done
c4t4d0: configured with capacity of 23.91GB
AVAILABLE DISK SELECTIONS:
       0. c1t0d0 
          /pci@1f,0/pci@1/scsi@8/sd@0,0
       1. c1t1d0 
          /pci@1f,0/pci@1/scsi@8/sd@1,0
       2. c4t4d0 
          /iscsi/disk@0000iqn.1986-03.com.sun%3A02%3A504a7bec-02bb-603a-cb19-ea22f593a799.testors-yes-chapFFFF,0