The Policy Driven Data Center with ACI. Architecture, Concepts, and Methodology

Table of Contents
Foreword xx

Introduction xxi

Chapter 1 Data Center Architecture Considerations 1

    Application and Storage 1

        Virtualized Data Center 2

        Introduction 2

        Definition and Virtualization Concepts 3

        Network and Design Requirements 6

        Storage Requirements 7

        Big Data 7

        Definition 7

        Network Requirements 9

        Cluster Design with the Hadoop Building Blocks: the POD 10

        Storage Requirements 11

        Design Considerations 11

        High-Performance Compute 14

        Definition 14

        Network Requirements 14

        Storage Requirements 14

        Design Considerations 14

        Design Topologies 15

        Ultra-Low Latency 16

        Definition 16

        Network Requirements 17

        Storage Requirements 18

        Design Considerations 18

        Design Topologies 19

        Massively Scalable Data Center 21

        Definition 21

        Network Requirements 23

        Storage Requirements 24

        Design Considerations 24

        Design Topologies 25

        Design Topologies Examples 25

    The POD-based Designs 26

        The POD Model or the Data Model for Shared Infrastructure and Cloud Computing 26

        The FlexPod Design 28

    Data Center Designs 29

        End of Row 29

        Middle of Row 30

        Top of Rack: The Modern Data Center Approach 30

        Single-Homed Servers Design 32

        Logical Data Center Design with the Spine-Leaf ACI Foundation Architecture 33

    Summary 35

Chapter 2 Building Blocks for Cloud Architectures 37

    Introduction to Cloud Architectures 37

    Network Requirements of Clouds and the ACI Solution 39

    Amazon Web Services Model 41

    Automating Server Provisioning 43

        PXE Booting 43

        Deploying the OS with Chef, Puppet, CFengine, or Similar Tools 44

        Chef 45

        Puppet 46

    Orchestrators for Infrastructure as a Service 47

        vCloud Director 47

        OpenStack 48

        Project and Releases 48

        Multi-Hypervisor Support 49

        Installers 49

        Architecture Models 50

        Networking Considerations 51

        UCS Director 51

        Cisco Intelligent Automation for Cloud 52

        Conciliating Different Abstraction Models 55

    Summary 56

Chapter 3 The Policy Data Center 57

    Why the Need for the Policy-Based Model? 57

    The Policy Theory 59

    Cisco APIC Policy Object Model 61

        Endpoint Groups 63

        Cisco APIC Policy Enforcement 66

        Unicast Policy Enforcement 66

        Multicast Policy Enforcement 69

        Application Network Profiles 70

        Contracts 71

    Understanding Cisco APIC 79

        Cisco ACI Operating System (Cisco ACI Fabric OS) 79

        Architecture: Components and Functions of the Cisco APIC 80

        Policy Manager 81

        Topology Manager 81

        Observer 82

        Boot Director 82

        Appliance Director 83

        VMM Manager 83

        Event Manager 83

        Appliance Element 84

        Architecture: Data Management with Sharding 84

        Effect of Replication on Reliability 84

        Effect of Sharding on Reliability 85

        Sharding Technology 86

        User Interface: Graphical User Interface 87

        User Interface: Command-Line Interface 87

        User Interface: RESTful API 88

        System Access: Authentication, Authorization, and RBAC 88

    Summary 89

Chapter 4 Operational Model 91

    Introduction to Key Technologies and Tools for Modern Data Centers 92

        Network Management Options 92

        REST Protocol 93

        XML, JSON, and YAML 94

        Python 96

        Python Basics 96

        Where Is the main() Function? 97

        Functions Definition 97

        Useful Data Structures 98

        Parsing Files 99

        Verifying Python Scripts 101

        Where to Run Python 101

        Pip, EasyInstall, and Setup Tools 101

        Which Packages Do I Need? 101

        virtualenv 102

        Git and GitHub 103

        Basic Concepts of Version Control 103

        Centralized Versus Distributed 104

        Overview of Basic Operations with Git 104

        Installing/Setting Up Git 105

        Key Commands in Git 105

    Operations with the Cisco APIC 106

        Object Tree 108

        Classes, Objects, and Relations 109

        Naming Conventions 113

        Object Store 114

        Using REST to Program the Network 114

        Tools to Send REST Calls 115

        REST Syntax in Cisco ACI 117

        Modeling Tenants in XML 119

        Defining the Relationship Among EPGs (Providers and Consumers) 120

        A Simple Any-to-Any Policy 121

        ACI SDK 122

        ACI Python Egg 122

        How to Develop Python Scripts for ACI 123

        Where to Find Python Scripts for ACI 124

    For Additional Information 124

    Summary 125

Chapter 5 Data Center Design with Hypervisors 127

    Virtualized Server Networking 128

        Why Have a Software Switching Component on the Server? 129

        Overview of Networking Components 132

        Virtual Network Adapters 132

        Virtual Switching 133

        Endpoint Groups 133

        Distributed Switching 133

        Hot Migration of Virtual Machines 134

    Segmentation Options 134

        VLANs 134

        VXLANs 134

        VXLAN Packet Format 135

        VXLAN Packet Forwarding 136

        VXLANs Without Multicast 137

    Microsoft Hyper-V Networking 137

    Linux KVM and Networking 141

        Linux Bridging 142

        Open vSwitch 143

        OVS Architecture 143

        Example Topology 145

        Open vSwitch with OpenStack 146

        OpenFlow 147

    VMware ESX/ESXi Networking 149

        VMware vSwitch and Distributed Virtual Switch 150

        VMware ESXi Server Traffic Requirements 151

        VXLAN Tagging with vShield 151

        vCloud Director and vApps 152

        vCloud Networks 153

    Cisco Nexus 1000V 155

    Port Extension with VN-TAG 158

    Cisco ACI Modeling of Virtual Server Connectivity 160

        Overlay Normalization 160

        VMM Domain 161

        Endpoint Discovery 162

        Policy Resolution Immediacy 162

        Cisco ACI Integration with Hyper-V 162

        Cisco ACI Integration with KVM 163

        Cisco ACI Integration with VMware ESX 164

    Summary 165

Chapter 6 OpenStack 167

    What Is OpenStack? 167

        Nova 168

        Neutron 169

        Swift 173

        Cinder 173

        Horizon 174

        Heat 174

        Ironic 174

    OpenStack Deployments in the Enterprise 176

    Benefits of Cisco ACI and OpenStack 177

        Cisco ACI Policy Model 178

        Physical and Virtual Integration 179

        Fabric Tunnels 179

        Service Chaining 179

        Telemetry 179

    OpenStack APIC Driver Architecture and Operations 180

        How Integration Works 180

    Deployment Example 182

        Installation of Icehouse 183

        Configuration of the Cisco APIC Driver 185

        Neutron.conf File 186

        ML2_conf.ini File 186

        ML2_cisco_conf.ini File 186

        Configuration Parameters 187

        Host-Port Connectivity 188

        External Networks 188

        PortChannel Configuration 188

        Troubleshooting 188

    The Group Based Policy Project at OpenStack 190

    Summary 191

Chapter 7 ACI Fabric Design Methodology 193

    Summary of ACI Fabric Key Functionalities 194

        ACI Forwarding Behavior 194

        Prescriptive Topology 194

        Overlay Frame Format 196

        VXLAN Forwarding 197

        Pervasive Gateway 198

        Outside Versus Inside 199

        Packet Walk 201

        Segmentation with Endpoint Groups 202

        Management Model 204

    Hardware and Software 207

    Physical Topology 208

        Cisco APIC Design Considerations 210

        Spine Design Considerations 211

        Leaf Design Considerations 212

        Unknown Unicast and Broadcast 213

        Use of VLANs as a Segmentation Mechanism 214

        VLANs and VXLANs Namespaces 215

        Concept of Domain 216

        Concept of Attach Entity Profile 217

    Multi-tenancy Considerations 218

    Initial Configuration Steps 219

        Zero-Touch Provisioning 220

        Network Management 221

        Policy-based Configuration of Access Ports 223

        Configuring Switch Profiles for Each Leaf 228

        Configuring Interface Policies 228

        Interface Policy Groups and PortChannels 228

        Interface Policy Groups 229

        PortChannels 229

        Virtual PortChannels 231

        Virtual Machine Manager (VMM) Domains 233

        VMM Domain 233

        AEP for Virtualized Servers Connectivity 234

Configuring a Virtual Topology 235

        Bridge Domain 237

        Hardware Proxy 237

        Flooding Mode 238

        fvCtx 238

        Endpoint Connectivity 238

        Connecting a Physical Server 239

        Connecting a Virtual Server 239

        External Connectivity 240

    Summary 241

Chapter 8 Service Insertion with ACI 243

    Overview of ACI Design with Layer 4 Through Layer 7 Services 244

        Benefits 244

        Connecting Endpoint Groups with a Service Graph 244

        Extension to Virtualized Servers 245

        Management Model 245

        Service Graphs, Functions, and Rendering 246

    Hardware and Software Support 247

    Cisco ACI Modeling of Service Insertion 248

        Service Graph Definition 249

        Concrete Devices and Logical Devices 250

        Logical Device Selector (or Context) 251

        Splitting Bridge Domains 251

    Configuration Steps 252

        Definition of a Service Graph 253

        Defining the Boundaries of the Service Graph 253

        The Metadevice 254

        Defining an Abstract Node’s Functions 255

        Defining an Abstract Node’s Connectors 257

        Abstract Node Elements Summary 258

        Connecting Abstract Nodes to Create the Graph 258

        Definition of Concrete Devices and Cluster of Concrete Devices 260

        Configuration of the Logical Device and Concrete Device 261

        Configuration of the Logical Device Context (Cluster Device Selector) 264

        Naming Summary 265

    Summary 266

Chapter 9 Advanced Telemetry 267

    Atomic Counters 267

        The Principle 267

        Further Explanation and Example 268

        Atomic Counters and the APIC 270

    Latency Metrics 271

    ACI Health Monitoring 272

        Statistics 273

        Faults 274

        Events, Logs, Diagnostics 279

        Health Score 280

    The Centralized show tech-support ACI Approach 281

    Summary 282

Chapter 10 Data Center Switch Architecture 285

    Data, Control, and Management Planes 285

        Separation Between Data, Control, and Management Planes 286

        Interaction Between Control, Data, and Management Planes 287

        Protection of the Control Plane with CoPP 288

        Control Plane Packet Types 288

        CoPP Classification 290

        CoPP Rate-Controlling Mechanisms 290

    Data Center Switch Architecture 291

        Cut-through Switching: Performance for the Data Center 292

        Crossbar Switch Fabric Architecture 295

        Unicast Switching over Crossbar Fabrics 297

        Multicast Switching over Crossbar Fabrics 298

        Overspeed in Crossbar Fabrics 298

        Superframing in the Crossbar Fabric 299

        The Scheduler 301

        Crossbar Cut-through Architecture Summary 301

        Output Queuing (Classic Crossbar) 302

        Input Queuing (Ingress Crossbar) 303

        Understanding HOLB 304

        Overcoming HOLB with VoQ 304

        Multistage Crossbar 305

        Centralized Shared Memory (SoC) 306

        Multistage SoC 307

        Crossbar Fabric with SoC 307

        SoC Fabric 308

    QoS Fundamentals 309

        Data Center QoS Requirements 309

        Data Center Requirements 311

        Type of QoS Used in Different Data Center Use Cases 312

        Trust, Classification, and Marking Boundaries 313

        Data Center QoS Capabilities 315

        Understanding Buffer Utilization 315

        The Buffer Bloat 317

        Priority Flow Control 318

        Enhanced Transmission Selection 319

        Data Center Bridging Exchange 320

        ECN and DCTCP 320

        Priority Queue 321

        Flowlet Switching: Nexus 9000 Fabric Load Balancing 322

        Nexus QoS Implementation: The MQC Model 324

    Summary 326

Conclusion 329