I/O Consolidation in the Data Center

I/O Consolidation 1 Introduction 1 What Is I/O Consolidation 2 Merging the Requirements 3 Why I/O Consolidation Has Not Yet Been Successful 4 Fundamental Technologies 5 PCI-Express 5 10 Gigabit Ethernet 5 Additional Requirements 8 Buffering Requirements 8 Layer 2 Only 9 Switch Architecture 9 Low Latency 10 Native Support for Storage Traffi c 11 RDMA Support 11 Enabling Technologies 15 Introduction 15 Lossless Ethernet 15 PAUSE 15 Credits Versus PAUSE 17 PAUSE Propagation 18 Is Lossless Better? 19 Why PAUSE Is Not Widely Deployed 20 Priority-based Flow Control (PFC) 20 Additional Components 22 DCBX: Data Center Bridging eXchange 22 Bandwidth Management 23 Congestion Management 25 Delayed Drop 26 Going Beyond Spanning Tree 28 Active-Active Connectivity 32 Etherchannel 32 Virtual Switching System (VSS) 32 virtual Port Channel (vPC) 34 Ethernet Host Virtualizer 36 Layer 2 Multipath (L2MP) 38 Basic Mechanisms in L2MP 40 Cisco DBridges 47 IETF RBridges and the TRILL Project 51 VEB: Virtual Ethernet Bridging 52 Server Virtualization 53 SR-IOV 54 The IEEE Standard Effort 54 VEB in the Adapter 55 VEB in the Switch 56 VNTag 57 Fabric Extenders 59 VN-Link 60 Questions and Answers 63 Does FCoE Uses Credits? 63 High Availability of PAUSE and Credits 63 Queue Size 63 Long-Haul 63 FECN/BECN 64 Confi guration 64 Bandwidth Prioritization 64 Storage Bandwidth 64 Cisco DCB/FCoE Support 65 10GE NICs 65 IP Routing 65 Lossless Ethernet Versus Infi niband 66 Nomenclature 66 Fibre Channel over Ethernet 67 Introduction 67 Fibre Channel 69 Fibre Channel Architectural Models 71 FCoE Mapping 73 FCoE Architectural Models 74 FCoE Benefi ts 79 FCoE Data Plane 80 FCoE Topologies 82 FCoE Addressing 85 FCoE Forwarding 87 FPMAs and SPMAs 90 FIP: FCoE Initialization Protocol 92 FIP Messages 93 FIP VLAN Discovery 97 FIP Discovery 98 FIP Virtual Link Instantiation 103 FIP Virtual Link Maintenance 108 Converged Network Adapters 110 FCoE Open Software 112 Network Tools 113 FCoE and Virtualization 114

Silvano Gai, who grew up in a small village near Asti, Italy, has more than 27 years of experience in computer engineering and computer networks. He is the author of several books and technical publications on computer networking and multiple Internet Drafts and RFCs. He is responsible for 30 issued patents and 50 patent applications. His background includes 7 years as a full professor of computer engineering, tenure track, at Politecnico di Torino, Italy, and seven years as a researcher at the CNR (Italian National Council for Scientifi c Research). For the past 12 years, he has been in Silicon Valley where, in the position of Cisco Fellow, he was an architect of the Cisco Catalyst family of network switches, of the Cisco MDS family of storage networking switches, and of the Nexus family of data center switches. Silvano teaches a course on I/O Consolidation, Data Center Ethernet, and Fibre Channel over Ethernet at Stanford University (see http://scpd.stanford.edu/certifi cates/fcoe). Claudio DeSanti is a Distinguished Engineer in the Advanced Architectures & Research organization at Cisco. He represents Cisco in several National and International Standards Bodies, such as INCITS Technical Committee T11, IEEE 802.1, IETF, and in industry associations. He is vice chairman of the INCITS T11 Technical Committee, chairperson of various working groups, including FC-BB-5, where FCoE has been developed, and technical editor of different standards, including IEEE 802.1Qbb, where Priority-based Flow Control is defi ned. He is author of several patents and international publications, including ten RFCs in IETF and other standards in the American National Standard Institute. He received many honors and awards, including the INCITS 2008 Technical Excellence Award, the INCITS 2007 Team Award, and the INCITS 2006 Gene Milligan Award for Effective Committee Management. Claudio's research interests include network protocols, storage networking, routing, and security. He