Full Introduction to FCoE (Fibre Channel Over Ethernet)

Definition of FCoE

What is FCoE? Short for Fibre Channel over Ethernet, it is a computer network technology used to encapsulate Fibre Channel frames on an Ethernet network. This allows Fibre Channel to use 10 Gigabit Ethernet (or higher speeds) while retaining the Fibre Channel protocol. This specification was part of the International Committee for Information Technology Standards T11 FC-BB-5 standard released in 2009.

Functionality of FCoE

FCoE is independent of Ethernet forwarding schemes and directly transmits Fibre Channel over Ethernet. The FCoE protocol specification replaces the FC0 and FC1 layers of the Fibre Channel stack with Ethernet. By retaining the local Fibre Channel structure, FCoE could be integrated with existing Fibre Channel networks and management software.

Data centers utilized Ethernet for TCP/IP networks and Fibre Channel for storage area networks (SANs). With FCoE, Fibre Channel, together with traditional Internet Protocol (IP) traffic, becomes another network protocol that runs on Ethernet. Compared with iSCSI running on TCP and IP, FCoE runs directly on Ethernet in the network protocol stack. As a result, FCoE is not routable at the IP layer and cannot work on routed IP networks.

Unlike Fibre Channel, because traditional Ethernet didn’t have priority-based flow control, FCoE needed to enhance Ethernet standards to support priority-based flow control mechanisms (to reduce frame loss caused by congestion). The IEEE standards body added priorities to the Data Center Bridging (DCB) task group.

Fibre Channel requires three main extensions to provide the functionality of Fibre Channel over Ethernet networks:

• Encapsulate native Fibre Channel frames into Ethernet frames.
• The expansion of the Ethernet protocol itself prevents the Ethernet fabric from losing frames regularly during congestion.
• The mapping between Fibre Channel N_port ID (aka FCID) and Ethernet MAC address.

The computer can be connected to FCoE by using a converged network adapter (CNA), which includes both the Fibre Channel host bus adapter (HBA) and Ethernet network interface controller (NIC) functions on the same physical card.

CNA has one or more physical Ethernet ports. FCoE encapsulation can be done in software using conventional Ethernet network interface cards, but FCoE CNA offloads (from the CPU) the low-level frame processing and SCSI protocol functions traditionally performed by the Fibre Channel host bus adapter.

Application of FCoE

The main application of FCoE is in the data center storage area network (SAN). Because FCoE can reduce cabling, it has special applications in data centers. In server virtualization applications, FCoE usually requires many physical I/O connections for each server.

With FCoE, network (IP), and storage (SAN) data traffic can be combined using a single network. This combination can:

• Reduce the number of network interface cards required to connect to different storage and IP networks.
• Reduce the number of cables and switches.
• Reduce power and cooling costs.

Frame Format of FCoE

FCoE uses dedicated Ethertype 0x8906 to encapsulate over Ethernet. A single 4-bit field (version) meets the IEEE subtype requirements. The 802.1Q tag is optional but may be necessary for a given implementation. SOF (start of frame) and EOF (end of frame) are coded in accordance with RFC 3643.

The reserved bit is used to ensure that the FCoE frame meets the minimum length requirement of the Ethernet. Inside the encapsulated Fibre Channel frame, the frame header is reserved to connect to the storage network by passing on the Fibre Channel frame directly after de-encapsulation.

FIP (FCoE Initialization Protocol) is an integral part of FCoE. Its main goal is to discover and initialize FCoE-enabled entities connected to the Ethernet cloud. FIP uses the dedicated Ethertype of 0x8914.

Advantages of FCoE

Using Fibre Channel over Ethernet has many advantages, including:

• Reduce complexity. Because storage-related traffic is sent over a regular Ethernet network, no Fibre Channel switch is required. In turn, this reduces the complexity and cost of the architecture.
• Improve performance. Fibre Channel is a high-speed storage protocol, and some storage devices support speeds up to 128 Gbps. Even so, such equipment tends to be expensive. Therefore, many organizations are still using 8 Gbps storage devices and SANs. FCoE enables such organizations to achieve higher speeds, such as 10 Gbps or 40 Gbps, by using relatively inexpensive Ethernet networks.
• Simplify network management. There are countless tools for managing and monitoring Ethernet networks. Because FCoE routes storage traffic across standard Ethernet, organizations may be able to use their existing network tools to manage storage traffic.