BRKCOM-1005 UCS Architecture Overview

Architecture of the Cisco Unified Computing SystemBRKCOM-1005Dan HansonManager, Technical MarketingComputing Systems Product GroupAgenda•Introduction – Fundamental UCS Components•UCS Strategy: A Software Defined Server and Edge•UCS Implementation of Strategy•UCS Advantage Videos on YouTube •SummaryIntroduction – Fundamental UCS ComponentsCisco UCS is a Fundamental Unified Fabric LAN Any IEEE Compliant LAN SAN B Any ANSI T11 Compliant SAN Mgmt SAN AAny ANSI T11 Compliant SAN•Proliferation of NIC andHBA devices, cables, portsas servers are added•Linear costs per computenode added•State is built per server•Per server managementUnified fabric within the DCEVERY Rack-MountServer ProductionManagementBackupSAN BSAN A•Proliferation of NIC and HBAdevices, cables, ports, IO BayModules as bladed servers areadded•State is Built on a Per-ChassisBasis•Non-Trivial Intra-ChassisConnectivity•Large Unit Step Costs for addedChassisUnified fabric within the DC EVERY Blade Server Production Management Backup SAN B SAN A Blade Chassis•As Servers are added, thecabling remains constant•Management model remainsthe same•State is per-UCS domain andnot per rack or blade chassis•Small unit-step costs forchassis as satellites of UCSDomainNew Cabling Model EVERY UCS B Series Server Entire UCS Domain Production Management SAN BSAN A 1-20 Blade Chassis, 160 Rack UnitsFundamentals of Cisco UCS•We start with a data model that includes the existence, identity, and configuration of a server and its various sub-components–Deep model of very fundamental components within servers•We grow this data model to include upstream I/O needs to include the configuration of the upstream devices ports connecting to this server–Coupling of the “other end” of the cables that connect these servers to the DC•We include policies to define groupings of these servers, priorities, security segments, and many others–To offer differentiated services, for example different x86 processor types•We probe newly added hardware, to classify them into service level tiers by capability –To ease not only the customer consumption of services, but also provider capacity growth•We map these modeled servers and all surrounding component needs to these service level tiers when the actual services are requiredUCS Centralized Policy And Application Driven•EmbeddedManagement enables more configuration points to set up and provision HW, FW, and Advanced SettingsLANSANCisco UCS Service ProfileNIC Existence NIC MACsHBA Existence HBA WWPNs Server UUID BIOS Settings RAID SettingsVLAN Assignments VLAN TaggingFC Fabrics Assign. FC Boot Parameters Boot order PXE settings IPMI Settings QoSTemplate Association Org & Sub Org Assoc. Server Pool Assoc. Statistic Thresholds BIOS scrub actions Disk scrub actions BIOS firmware Adapter firmware BMC firmwareAdvanced NIC settings Serial over LAN settingsUCS Server Physical ArchitectureUCS Cluster of Servers Presented to Data Center•Pre-”pinned” forwarding paths within UCS System including ingress/egress – not a forwarding based on {MAC address | VLAN}•No Spanning Tree Protocol–Reduces CPU load on upstream switches–All links carrying traffic–Doubles effective bandwidth•Maintains server MAC table for UCS servers only•Prevents Loops by pinning a MAC Address to Only One Port or Port-Channel –No traffic can pass between upstream segments•Upstream VSS/vPC optional for all bandwidth usage•Completely transparent to Next Hop Switch (need not be multi-chassis aware)•Traffic on same L2 subnet switched locally inside FIUCS System ComponentsUCS Server Physical Architecture•Typical Infrastructure: Assign abusiness use to a chassis–Some chassis filled and requiresnew purchases of chassis, LANI/O modules, SAN I/O modules,Management modules, Intra-DCcabling all required–Open capacity in other chassis arestranded–Leads to slower business turn-around•With UCS, grouping of businessfunctions can be virtual withinarray – or on Rack-Mountedarrays near-term – SingleManagement Entity for a UCSarrayUCS Fabric Interconnects Different Ports for Dedicated FunctionsBladed Deployments (Can mix with Rack Mounts) •Wire once forbandwidth, notconnectivity•Policy-drivenbandwidth allocation•Integrates as a singlesystem into your datacenter•In mixed blade andrack environments,160 servers is the limitRack Mount Deployments (Can mix with Blades) •Wire once forbandwidth, notconnectivity•Policy-drivenbandwidth allocation•To get higherbandwidth on racks,we cable fewer serversto the 2232’s•In mixed blade andrack environments,160 servers is the limitConnecting UCS to Multiple Security SegmentsVLAN ListUplink GroupServer Interface List10,11,12… E1/1,E1/2Blade1 NIC0 & NIC1, Blade 2 NIC0 20,21,22… E1/10,E1/11 None30,31,32…E1/20,E1/21Blade 2 NIC1, Blade 3 NIC0 & NIC1501 FC0,FC1Blade 1 vHBA0, Blade 2 vHBA0UCSForwarding BoundaryUCS Server Models for Blade and Rack MountInside the x86 Server – Straightforward Components and OS ViewCisco UCS: Many Server Form Factors, One SystemUCS C220 M3Versatile, General Purpose EnterpriseInfrastructure, and Application ServerEnterprise PerformanceUCS C240 M3Ideal Platform for Big Data, ERP,and Database ApplicationsUCS B200 M3Optimal Choice for VDI, PrivateCloud, or Dense Virtualization/Consolidation WorkloadsIntensive/Mission CriticalUCS B420 M3Enterprise Class, 4-SocketBlade for Large, Memory-Intensive Bare Metaland Virtualized ApplicationsUCS C420 M3Enterprise Class, 4-Socket Server forLarge, Memory-Intensive Bare Metal, andVirtualized ApplicationsUCS C24 M3Entry, Expandable Rack Server for StorageIntensive WorkloadsUCS C22 M3Entry Rack Server for Distributed and WebInfrastructure ApplicationsUCS B22 M3Entry Blade Server for IT Infrastructure and WebApplicationsScale OutRackBladeUCS C260 M2Mission-Critical, 2-Socket ExtendedMemory Server for Large, Memory-Intensive ApplicationsUCS C460 M2Mission-Critical, 4-SocketServer for Large, CPU-IntensiveApplicationsUCS B440 M2Mission-Critical, 4-Socket Blade for Large, CPU-Intensive Bare MetalUCS B230 M2Density-optimized CPU andMemory-Intensive 2-Socket Blade for BareMetal and Virtualized ApplicationsUCS C460 M4Mission-Critical, 4-SocketServer for Large, CPU-IntensiveApplicationsUCS B260/B460 M4Mission-Critical, 2/4-SocketBlades for Large, CPU-Intensive Bare Metaland Virtualized ApplicationsUCS Strategy: A Software Defined Server and EdgeUCS Logical ArchitectureA Legacy Server – Physical Builds of the DevicesInfrastructure Deployment and Operational Models •Structured DC Design forCompute, LAN, Storage,Security, and Facilities•Services Capacity andGeographical Capabilities•Business Need to ServiceDelivery Process andTiming CapabilitiesBusiness Need timing Impacts – Whiteboard to Service RealizationTRADITIONALINDUSTRYAPPROACH -SIMPLIFIEDWhiteboardDesignProduce cutsheets toteamsIdentifyNeedsProvisionStorageIdentifyServerClassIdentifyNetworkNeedsMaskLUNsIdentifyServerInstanceShareWWN’sConfigureSAN EdgeDetermineDCPlacementShareLocationZoneFabricConfigureNetworkEdgeFacilitiesStageServerFirmwareUpdatesBIOSPolicySettingsRAIDSettingsImage OSCoordinateEnds ofCablesCoordinateEnds ofCablesJoinSystemsMgmtDomainUpdateSecurityPolicyInstallApplicationRethinking the Process•Abstracting, Converging, and Virtualizing more of the DC Infrastructure for Deployment and Operational Benefits•Easing the Restrictions of Structured Design•Embracing Service Delivery Independent of Location – Including Campus Needs •Tighter Coupling of Provisioning and Delivery for Accelerated Deployments•Tight Coupling of Needs to Provisioning –Self Service Typically Mandates VM’s •Bare Metal Model Unchanged – Used for Virtualization Provider Rollouts •Management and Troubleshooting of Overlays not shown here•Single Admin with Full Control on InfrastructureVirtualizing the Server and Some Infrastructure is Partial SolutionAFTERINFRASTRUCTUREPRE-PROVISIONINGON HYPERVISORWhiteboardDesign Producecut sheets – butfewer teamsinvolved at timeof needIdentifyNeedsIdentifyVirtual DCDeploy VMfrom OVFUpdateSecurityPolicyUpdateVirtualizedServicesNeedsImage OS JoinSystemsMgmtDomainInstallApplicationConfigureNetworkEdgeIdentifySharedData StoreMaking These Programmable Goals Pervasive – Not just in VM Pods •Begins with a Separation of Control Plane and Data Plane•Easier Programmatic Control•Don’t Deconstruct App Components to Fit in DC–Make the DC Infrastructure Morph to the App Policy•API with the Centralized Controller Architecture Common–Abstract extensions for optimized services to API methods–Move to centralized policies and pools of how resources are consumed •Industry offerings vary in terms of where some middleware would live –Middleware to control advanced ASIC functionality, and other HW capabilities –Support of these API extensions–Expanding these into Open Source living on the PlatformsWhat Really Does a Controller Do? Other Examples Yesterday and Today•These are not new–Wireless Controllers to centrally manage Access Points–Controlling Bridge in all FEX architectures–VM Managers – things like Auto-Deploy•More than just Control Plane/Data Plane Separation –All modern modular devices have that separation in a sense•Expanding the scope–Span entire DC segments (or inter-DC) at scale–Host the Running Images and Components (FEX, Auto Deploy, etc.)–Host the Device Configurations (FEX, Auto Deploy, WiFi, etc.)–Model Driven Imposition of detailed configuration lines no longer in user space –Administrators configure the models end to end now–API’s to allow the end using processes, portals, etc. to configure the models –API’s for the reporting and subscription to monitoring and event subscriptionControllers Do Not Equate to Merchant Hardware •One Direction is to EliminateControl Plane completely onElement•White Box Servers•Merchant Silicon on networking•“White Box” Network and Storagegear carry little management•Goal is to effectively open sourcecode of devices to users•Different views by MarketSegmentTightly Coupling the Controller to Hardware Capability •Moving to Direct Control ofPolicies and Pooling of theUnderlying Infrastructure•Feature Rich Integration•Controller to Element Closed in itsNature•Northbound Methods Open•UCS Model Fits This Model TodayTradeoffs with Software Only Options•Cisco Assertion: Underlying Hardware will Still Matters – Cannot Rely on Software Libraries/Overlays Only•White Box Servers with little management–Do we have precedents here?–One example: Are their advantages in processor families for workloads (VT-x, VT-d, TXT for VM boot, ) or is cheapest OK? Should I Virtualize Servers on ARM? •“White Box” Network and Storage gear with little management–Same example: Will all networking ASICs lose any value, will cheapest device be OK?Industry Directions inside the Data Center•Goal is to effectively open source code of devices to users–Do we want the ability to openly access internals for some customizations, or do we want to fully write our complete control plane?–Do we want to invent and maintain our infrastructure? Dev, QA, Docs, Support, etc.?–Key item here – What Optimizations are Needed and how to put in API?•Different views by Market Segment–Web Providers will be more capable of developing this IP and its lifecycle – will others want this vs.Off the Shelf?–Example: Do we want to expose the core OS for loading of custom code, or offer API to control? Tradeoffs with Software Only OptionsUCS Implementation of StrategyWhen we Add in a Unified Fabric Architecture and Policy Driven ConfigurationsWhen we Add in Templates and PoolsUCS Manager Data Management Engine Review of the UCS Management ArchitectureOrganizational Groupings*•Optional to Deploy•First fundamental co-tenancy unit is an organization–Maximum is only based on resources•Organizations are logical divisions of resources and policy •Can be tiered with sub-organizations–5 levels deep maximumPooling Servers does NOT equate to Chassis Boundaries* •Method of pulling physicalblades into server poolshown•Legacy boundaries are perchassis•Can be any blade on anychassis, or rack-mountpooled together•Segmentation can bevirtual within UCSServer Pool Qualification Policies – SLA Tiering for Software Defined Servers* •As hardware added into UCS, it can be automatically added to pools to offer SLA’s–By chassis/slot/organization for ownership of hardware–Qualified by this policy, and made available to users as unconfigured device in the appropriate pool•Method to select minimums for hardware pool candidates•Server pool policy to map pools to minimal qualifiers* Unique to Cisco UCSFirmware Versioning Included within the Software Defined Server •Complete control of firmware that isqualified for use per application that will berunning*•Various Components without Admin UEFI*–Blade Management Controller–BIOS–Option ROM–RAID Controller–NIC’s–HBA’s–Main Board•Pulled into a profile or a profile template*•Allows for assigning the firmware state onserver–Easier automation possibleBIOS and RAID Controller Configuration within the Software Defined Server•Complete control of BIOS settings*–All pertinent fields are configurable in a BIOSpolicy–When profile is associated, the BIOS settingsare also matched to the preference of theapplication•Complete control of RAID settings*–RAID configuration can be setup on a profilebasis–RAID can be protected – if a profileapplication to an existing RAID server, canraise exception rather than applying•Allows for assigning the low-levelconfiguration state on serverGrouping Server IO needs and Network Edge in Software•Coupling of the configuration of LAN/SAN access port with the NIC/HBA using it in the server–Goal of reducing the manual touches on the infrastructure, and adhere to established policies –Entire “I/O Carriages” can be pre-built for rapid addition to a new software defined server within UCS •Single XML object in UCS, multiple coordinated hardware items on legacy servers/LAN/SAN•A-priori configuration–No manual NX-OS or IOS configuration of physical port–No manual SAN-OS / NX-OS configuration of physical port–Abstract into objects that are using them (adapters)–No manual adapter configuration – abstract to XML object–Create templates of these adapters by useFundamental I/O Linkages on the Unified Fabric•Converging the Infrastructure Silos for Deployment and Operations Advantages •Makes Programmatic I/O Possible without Server Virtualization Need •The key is how this is accomplished??Moving to Programmable I/OIndividualEthernets (vNICs)DCB EthernetBlade ManagementChannels (BMC Connections)x86 Server NIC NIC NICMgmt Mgmt HBA HBAPCIe Treex86 ServerPCIe TreeServer NIC Creation and Configuration within the Software Defined Server •Define X(*) NICs and their attributes–VLAN(s)–MAC Addresses–QoS Attributes (Speeds, MTU’s, Markings, etc.)–Security Policy (Native allowed, MAC forge, etc.)–Order on PCIe bus*–Which adapter and fabric to use*–Which L2 Network to Connect to above UCS(Different DMZs)*–Fabric Failover to allow HA in UCS without OSteaming/bonding configuration*•Define a Whole Group of NICs*–Security Policy to an IO Assembly•Later Slides will talk to VM-FEX*Pre-Creating Templates of Server NICs•Centralized per Organization/Root•Adapter Templates Can be used as Initial or Updating •Tightly Controlled Centralized Policy–Easier Audit•Policy Managed by Security and Network Administrator •Applicable to Physical and Virtual Servers–Tied into Service Profile Templates–Tied into vNIC Templates–Tied to VM-FEX vSwitching Port Profiles•Highly Automated Process to Add Connectivity to Server with PolicyExtending UCS FEX Endpoints to the Virtual Machines (VM-FEX)*•Each Virtual Machine vNIC now connected to the UCS network edge–1:1 mapping between a VM virtual adapter and the UCS network–Services can remain on network – similar to pizza box deployments today•Live Migration is Supported with this “bypass” mode – adapters not on ESXi Shell, nor on Windows 2012 hosts•Multiple Security levels can exist within a given vCenter/SCVMM/KVM environment on UCS with hardware isolation•As Virtual Machines move around infrastructure, the network edge port stays consistent with the virtual adapter that is moving•RHEL with KVM support in 6.3, Microsoft Hyper-V in Windows 2012, vSphere since 4.0u1Server HBA Creation and Configuration within the Software Defined Server •Define Y HBAs and their attributes*–VSAN–WWPN–QoS Attributes (Speeds, etc.)–Order on PCIe bus–Which adapter and fabric to use–Which Upstream FC port or channel topresent the HBA on•Define a Whole Group of HBAs*–Security Policy to an IO AssemblyPre-Creating Templates of Server HBAs •Centralized per Organization/Root •Adapter Templates Can be used as Initial or Updating•Tightly Controlled Centralized Policy–Easier Audit•Highly Automated Process to Add SAN Connectivity to Server with Policy•Policy Managed by Security and SAN Administrator–Tied into Service Profile Templates–Tied into vHBA TemplatesPre-Creating Templates of All Server IO •Administratively defined by Security, Storage, and Network teams before use•Template of full I/O rules (numbers of NICs and where they connect, numbers of HBAs and what they can see, etc.) valid for an organization•Organization server administrator can attach the I/O sub-assembly to a server in a server definition–CANNOT however add a random adapter withother network/storage behind it, nor modifyassemblyEnsuring Service Levels for Server Adapters within Unified IOClass NameFCGoldEthernet BECOS Value31Drop/No-Drop No-Drop Drop Drop Strict Priority No No No Bandwidth/Weight1 (20%)3 (60%)1 (20%)UCS Server A Global System Class DefinitionsUCS Server BFC TrafficHigh Priority Ethernet Best Effort Ethernet。