iSCSI

iSCSI, short for Internet Small Computer Systems Interface, is a protocol that transports SCSI block storage commands over TCP/IP networks. In server storage, it lets a server use a disk volume located on a remote storage array as if that volume were a locally attached drive, using ordinary Ethernet switches and network adapters.

Standardized by the IETF in 2004 and later consolidated in a single updated specification, iSCSI became the most common way to build a storage area network (SAN) without Fibre Channel hardware. It is used for virtualization clusters, databases and backup targets in organizations of every size.

How iSCSI works

iSCSI wraps SCSI commands, the same read and write instructions a server sends to a local disk, into protocol data units carried over TCP, by default on port 3260. Because TCP provides ordering and retransmission, iSCSI can run on any routed IP network, although it performs best on a dedicated, low-latency segment.

Every iSCSI deployment involves a small set of roles and identifiers, summarized in the table.

The initiator first performs discovery, typically by asking a portal for its list of targets with the SendTargets method or by querying an iSNS server. It then logs in, negotiates parameters such as maximum data segment lengths, and the operating system presents each LUN as a raw block device that can be partitioned and formatted.

iSCSI as block storage

iSCSI delivers block storage, not files. The storage array knows only numbered blocks, and the file system lives on the initiator. This is the key difference from NAS protocols such as NFS and SMB, where the storage server manages the file system and clients request whole files.

One consequence is critical: a LUN formatted with an ordinary file system such as NTFS, ext4 or XFS must not be mounted by two servers at once, or the file system will be corrupted. Shared access requires a cluster-aware file system, such as VMFS in VMware environments or Cluster Shared Volumes in Windows failover clusters, which coordinate locking between hosts.

iSCSI compared with other storage protocols

iSCSI competes with both other block protocols and file protocols. The choice depends on existing infrastructure, required latency and the skills of the operations team.

Compared with Fibre Channel, iSCSI avoids separate switches, host bus adapters and specialist skills, at the cost of somewhat higher latency and CPU overhead for TCP processing. On all-flash arrays, NVMe over TCP is increasingly preferred where operating systems and arrays support it, because it was designed for the parallelism of solid-state storage.

Designing an iSCSI network

Most iSCSI performance and reliability problems are network design problems. Best practice treats storage traffic as infrastructure in its own right, separated from user traffic and built with redundancy at every layer.

Multipath I/O is preferred over link aggregation for iSCSI, because multipathing works at the storage layer and can balance and fail over sessions across separate subnets. Jumbo frames help only when every switch, adapter and target port uses the same MTU; a single mismatch causes hard-to-diagnose stalls.

Common iSCSI pitfalls

iSCSI is easy to start with, which is why misconfigurations are frequent. The following points cover the mistakes most often found in production environments.

• Sharing a LUN between servers without a cluster file system, which corrupts data quickly.
• Running storage traffic on the user network, where congestion causes latency spikes and timeouts.
• Leaving targets without authentication, so any host on the segment could connect to a volume.
• Mismatched MTU settings, which allow small packets to pass while large transfers fail.
• Relying on a single path, so one cable or switch failure takes all storage offline.

Monitoring should include latency per LUN, retransmissions and session state, because rising TCP retransmissions are often the first sign of a failing link or an overloaded switch long before applications report errors.

The simple explanation

Normally a computer’s hard drive sits inside the computer, connected by a short cable. iSCSI replaces that cable with the network. The disk can sit in a storage cabinet down the hall, yet the server treats it like its own internal drive.

This lets a company keep all its storage in one central place. It can give each server exactly the space it needs and add more space later without opening any machines. Ordinary network equipment does the job, so no special cabling is required.

How it is used in practice

A regional logistics company runs about 60 virtual machines on a cluster of three hypervisor hosts. Instead of placing disks in each host, it connects all three to a storage array over iSCSI, using two dedicated 25 GbE switches and a separate VLAN for storage traffic.

Each host has two storage adapters on different subnets with multipath I/O configured in round-robin mode, mutual CHAP authentication and jumbo frames verified end to end. The virtual machine volumes use a cluster file system so that all hosts can access them at once.

When one storage switch failed during a firmware update, traffic moved to the second path within seconds and no virtual machine lost access to its disks. The company can also move running virtual machines between hosts, because every host sees the same shared block storage.