Use FreeBSD10 to build ZFS-based iSCSI service

Overview

I have always strongly recommended ZFS because it is so easy to use. However, it can only run on Oracle's Solaris and FreeBSD systems until now. To share it with other systems, you can only use NAS or SAN.

The NAS method is very simple. I have been using Samba for implementation. Of course NFS is also acceptable. I haven't tried SAN before. Of course SAN is better to use, and the network speed is fast enough now. Although NAS and SAN are both external storage for storage, but for the client, it knows that NAS is a network device, and SAN is regarded as a local device, which is the main difference between them. This is because NAS uses the network layer protocol and SAN uses the underlying block device protocol.

Traditionally, SAN is relatively high, because they all use fiber channel (FC), and later the iSCSI rescue star-the SAN using iSCSI is also called IP-SAN. With iSCSI, the differences between NAS and SAN are not so obvious, because some NAS devices also provide iSCSI support.

Note: Because NAS uses the network protocol, it is a common Network File System Access Method for the client. You do not need to pay attention to the specific file system used by the server, this means that files in NAS can be shared in different systems. For example, I am storing files on the server in ZFS format and share them with Linux, Mac, and Windows through SAMBA. There is no problem in accessing shared files in four completely different systems. However, SAN uses the underlying block device protocol, so it is used exclusively by clients. The targets used in Linux cannot be shared with Windows (the formatted file system is different ), similarly, the specific content in the target cannot be directly seen on the server end. (You can use the local Initiator of the server end to connect and mount it to the specified file system for operations, but only the file system formats supported by the server itself ).

Although the previous version of FreeBSD supports iSCSI, It is a user-level application. I personally feel bad, so I have never tried it. In the latest FreeBSD 10, iSCSI has been integrated into the system. This is really good news. It feels good to try it recently.

The basic principles of iSCSI are as follows: iSCSI is a protocol and a virtual SCSI implementation on an IP network. The client can simulate a local block device (which can be understood as a virtual SCSI hard disk) through the iSCSI Initiator, and then the iSCSI Initiator transmits the received SCSI commands to the server through the IP network, the server then converts the corresponding commands into operations on the actual hard disk.

There are some physical or logical hard disks on the server (that is, the storage end), which are organized into so-called Luns (logical unit numbers) and can be understood as a logical volume, such as a hard disk and a partition, A group of RAID or ZFS. When the storage end uses iSCSI to provide external storage services, we call it iSCSI target. At the same time, the storage end can provide external services through multiple channels, such as using different IP addresses, different network adapters, and different identity authentication methods. Each channel is called a portal group. The portal group and target can be freely combined to meet various storage requirements of the client.

On the client side, it is implemented through the iSCSI Initiator mentioned earlier. It is represented as a virtual hard disk locally (there is a device name under/dev, but there is no actual physical device ), all its operations will be passed to the corresponding iSCSI target through iSCSI.

ISCSI target

First, create a ZFS on the server for target:

zfs create -s -V 4G -b 4k tank/testtarget

Note that the-V parameter is required to create a block device under/dev/zvol for iSCSI. -V indicates that a ZFS volume is created, and-s indicates that no space is allocated during creation. If this parameter is not added, a volume that actually occupies the specified capacity is created. -B specifies the block size (that is, the slice size in the traditional sense, usually 4096 or 512 ).

Add the following line to/etc/rc. conf to enable the ctld (iSCSI service ):

ctld_enable="YES"

Configure ctld and create the/etc/ctl. conf file with the following content:

portal-group san {        discovery-auth-group no-authentication        listen 192.168.x.x}target iqn.2014-05.com.example:target0 {        auth-group no-authentication        portal-group san        lun 0 {                path /dev/zvol/tank/testtarget                blocksize 4096                size 4G        }}

This is the simplest configuration, with only one target and one portal group. User authentication is not used. It is sufficient for testing.

Then you can start this target:

chmod 600 /etc/ctl.confservice ctld start

Note that the chmod step is required. Otherwise, the service cannot be started because a globally readable configuration file is insecure.

After the startup, you can check the log to confirm that there is no error.

tail /var/log/messages

ISCSI initiator

Generally, FreeBSD is not used as the client, but sometimes the target configuration needs to be tested on the server, so the Initiator under FreeBSD may still be used. Therefore, the following configuration method is recorded for reference. This section uses the actual desktop environment configuration as an example.

My desktop is Linux Mint 16, which is for reference below. However, different Linux releases should be similar. Mac seems to require commercial software support and cannot be introduced. Microsoft provides related Initiator software for Windows for free (later versions of Windows are built-in). The configuration method is attached.

Install software first:

sudo apt-get install open-iscsi open-iscsi-utils

Then start the service:

sudo service open-iscsi start

Search for target:

sudo iscsiadm -m discovery -t sendtargets -p 192.168.x.x

You can see the configured target in the result.

Login connection target:

sudo iscsiadm -m node -T iqn.2014-05.com.example:target0 -p 192.168.x.x -l

Now open the system preference-disk (or related tools on your release, or you are used to using command line) to see a new disk (on my computer, the device name is/dev/sdc), and the disk name is freebsd ctldisk, which is not formatted. Format it with EXT4 and then mount it to use it as a local disk.

So far, a basic ZFS-based iSCSI service is built.

More complex applications

As mentioned above, the target configuration is fully open, and all clients in the entire network segment can be connected freely. To ensure security, you must add user authentication.

The simplest way is to add the user name and password to the target Configuration:

target iqn.2014-05.com.example:target0 {        portal-group san        chap user password1234        lun 0 {                path /dev/zvol/tank/testtarget                blocksize 4096                size 4G        }}

Note that the password must be at least 12 characters by default. However, if multiple users need to use it, it is inconvenient. In this case, auth-group is required:

auth-group ag0 {        chap user1 password1234        chap user2 password1234}target iqn.2014-05.com.example:target0 {        auth-group ag0        portal-group san        lun 0 {                path /dev/zvol/tank/testtarget                blocksize 4096                size 4G        }}

Now you need to specify the user name and password to connect to the initiator. However, the user name and password are not entered in the command line, but must be configured for the open-iscsi service. Modify/etc/iscsi/iscsid. conf and add:

node.startup = automaticnode.session.auth.authmethod = CHAPnode.session.auth.username = usernode.session.auth.password = password1234

Node. startup is set to automatic to enable the initiator to automatically connect. This is not necessary. Its default value is manual, that is, you need to manually connect to the target, set it to automatic, and then automatically connect to the target after the system restarts, without the need to manually run iscsiadm to connect.

The username and password of node. session. auth are the username and password of the target configured earlier. Similarly, if you have configured user authentication for the portal group, you can also configure the discovery user name and password here.

Restart the open-iscsi service:

sudo service open-iscsi restart

Run the following command again:

sudo iscsiadm -m node -T iqn.2014-05.com.example:target0 -p 192.168.x.x -l

You can log on to the target. After the connection, you can mount it.

If you set node. startup to automatic as before, it will be automatically connected after startup. You only need to directly mount or configure automatic mount as follows:

Obtain UUID first:

sudo blkid /dev/sdc

Or use the path in the/dev/disk/py-path/mode and configure it to fstab (take the by-path as an example. For the UUID mode, see the default configuration in fstab ):

/dev/disk/by-path/ip-192.168.x.x:3260-iscsi-iqn.2014-05.com.example:target0-lun-0 /mnt/iscsi      ext4    _netdev,errors=remount-ro 0       1

Note: _ netdev must be included in the option; otherwise, the mount will fail after a long wait period at startup.

Try ZFS

It can be noted that the previous step is that the client needs to format the target. If I format the target in EXT4 format, is the server-side ZFS still valid? You can try it.

First, create some files in the mount path, and then take a snapshot on the server:

zfs snapshot tank/testtarget@test1

Then return to the client, delete or modify the relevant files, and then umount and disconnect (otherwise, server ZFS cannot be modified ):

sudo umount /dev/sdcsudo iscsiadm -m node -T iqn.2014-05.com.example:target0 -p 192.168.x.x -u

Then, the iSCSI service is stopped on the server:

service ctld stop

Now you can try to roll back the snapshot-of course, you can also perform a clone, and then point the target to the clone. Here we take a simple rollback operation as an example:

zfs rollbak tank/testtarget@test1

Restart the service:

service ctld start

Client reconnection:

sudo iscsiadm -m node -T iqn.2014-05.com.example:target0 -lsudo mount /dev/sdc /mnt/iscsi

Now let's look at the modified or deleted files and restore them to their original state. It can be seen that a ZFS volume can also implement the ZFS function and will not expire because it is formatted as EXT4.

Appendix: Initiator configuration under FreeBSD

Start iscsid service first. Generally, you can run the iscsid command to start the service, because you need to configure rc to start the service by running the service command. conf, and you can use the service command to stop it even if it is started directly.

Connect to target:

iscsictl -A -p 192.168.x.x -t iqn.2014-05.com.example:target0

Then, use iscsictl without parameters to check the connection status and device name. Note: If the status shows Waiting for iscsid (8), the iscsid service is not started.

If it's okay:

Mount-t fstype/dev/da0/mnt/iscsi

Fstype indicates the name of the file system and da0 indicates the name of the connected device.

Appendix: Initiator configuration for Windows

Because Mac is not available, I looked at the Windows configuration when I was depressed and found that it was quite simple, and I didn't even need to restart it.

First download Microsoft iSCSI initiator from Microsoft website

Then install it. Note: You must enable the ms dtc service.

After installation, click [Add] in the [Target Portals] column on the [Discovery] Page and enter the IP address or DNS name of the Portal, for example, 192.168.x.x.

Click [Targets] to view the targets configured On the server. Click [Log On] and the status changes from Inactive to Connected. Note that there is an option named "Automatically restore this connection when the system boots", which can be Automatically connected at system startup.

Now, open disk management in system management, and a prompt will pop up saying that there is another hard disk in the system. Select and format the file system format you want, such as FAT32 or NTFS, then you can use it like a general hard disk.