iSCSI Target IET Architecture

IET (iSCSI Enterprise Target) is a kernel-implemented iSCSI target, compared to the user-configured target (such as TGT), IET is relatively stable, and also a long history, IO directly through the kernel state, performance is better. This paper is mainly aimed at IET program architecture, from program module, command operation processing flow, initiator and target interaction, I/O process analysis.

1. Program Module

IET contains 3 parts: Ietadm,ietd,iscsi_trgt, where Ietadm is a command-line tool that provides Target,lun creation, deletion, parameter setting, and so on, while IETD is a user-state program, For the interaction with Ietadm and initiator, and the corresponding processing, iSCSI related discovery, login, authentication, logout and other operations are implemented in the user state; Iscsi_trgt is a kernel module, the resolution of the iSCSI protocol processing, i/ o processing is done within this kernel-state module.

1) Ietdadm and ietd interact through the IPC, and the IPC is initialized when the IETD program starts;
2) IETD will also create a listening socket when booting (the default port is 3260, and will generate a listener for each IP on the machine socket,initiator IP and port are specified when target is found), The initiator interacts with the IETD through the socket, a initiator and a target establish a long connection (the iSCSI protocol Standard says at least 1 connections), and subsequent I/O is the interaction through this connection;
3) ietd and ISCSI_TRGT use the IOCTL to interact, some operations (such as creating a delete target, creating a delete lun, etc.) are processed by IETD in the user state, send the command through the IOCTL to the ISCSI_TRGT, and then Iscsi_ The TRGT is processed in the kernel state;

2.initiator interaction with IET

Initiator to ietd request, ietd based on the type of request related processing, in the IETD process is discovery, login, auth, logout and other login authentication operations, if necessary ietd will send commands to Iscsi_ TRGT for processing. Here is an example of iSCSI login to simply analyze the processing flow.

, initiator initiates a login request to ietd,ietd to receive the login request, initialize the connection information (conn), and perform the corresponding processing of the login, then initialize the session, and after completion ietd use the IOCTL via Iscsi_ TRGT in the kernel state to create a session, the completion of the return to ietd, and then ietd return to the initiator response information, and then ietd notify ISCSI_TRGT takeover Fd,iscsi_trgt will be in the kernel to initialize the connection (conn), This connection is directly associated with the FD used by the user state ietd, directly using the FD corresponding kernel sock, the subsequent initiator end of the I/O is directly through the kernel sock and iscsi_trgt interaction.

3.I/O processing

Before you introduce the I/O process, you need to know the kernel thread that corresponds to each target.

When the target is created, the ISCSI_TRGT kernel module assigns a network thread nthread to the target (named ISTD, such as the target ID is 1000, then PS can see the "istd1000" process), And a number of worker threads Wthread (called istiod) are assigned, the default is 8, which can be configured by parameters. The network thread will parse the iSCSI protocol, put it into the work queue (work_queue) after processing, and wake up the worker thread, and multiple worker threads share the Work_queue, taking cmnd from Work_queue to do the corresponding processing.

, the initiator end of the iSCSI device IO Read and write through the file system, block device to the SCSI subsystem, and finally by the SCSI low layer of the ISCSI_TCP module sent I/O request to the Iscsi_trgt,iscsi_ The sock of the corresponding target in the TRGT receives the request, wakes the nthread corresponding to the target, then nthread the resolution of the Protocol, and then puts cmnd into the work queue work_queue after processing, and wakes up Wthread, Wthread takes cmnd from the queue and then handles it according to the request type (specifically disk_execute_cmnd in TARGET_DISK.C, including inquiry, Report_luns, Read_capacity, MODE _sense, Read_6/read_10/read_16, write_6/write_10/write_16, Synchronize_cache, etc.), such as read-write processing, will go to the backend storage type (Nullio, Blockio,fileio) The specific process flow: is through the make_request point to the corresponding request handler function, such as the blockio_make_request in Blockio, and then construct the bio, call Submit_ The bio is sent to the underlying disk device for processing, and if it is FileIO, it is called vfs_read/vfs_write for processing, and after I/O is complete, the iSCSI response is built into write_list (and each connection is a write_list, That is, different target is not the same), Wake Nthread to the iSCSI response related processing, and finally through the sock will be sent to the initiator side of the device.

iSCSI Target IET Architecture