Verified Scalability Guide for Cisco APIC, Release 3.2(1l)

22 Pages·2017·1.01 MB·English

Verified Scalability Guide for Cisco APIC, Release 3.2(1l), 3.2(1m), 3.2(2l),
and3.2(3h),Multi-Site,Release1.2(1g),1.2(2b),and1.2(3a),andCiscoNexus
9000 Series ACI-Mode Switches, Release 13.2(1l), 13.2(2l), and 13.2(3h)
Overview 2
GeneralScalabilityLimits 2
MultipleFabricOptionsScalabilityLimits 5
CiscoACIMulti-SiteScalabilityLimits 5
FabricTopology,SPAN,Tenants,Contexts(VRFs),ExternalEPGs,BridgeDomains,Endpoints,andContracts
ScalabilityLimits 7
VMMScalabilityLimits 21
Layer4-Layer7ScalabilityLimits 24
AD,TACACS,RBACScalabilityLimits 24
QoSScalabilityLimits 25
Revised:July12,2022,
Overview
ThisguidecontainsthemaximumverifiedscalabilitylimitsforACIparametersfortheCiscoAPICRelease3.2.(1l),CiscoMulti-Site
Release1.2(1g),andCiscoNexus9000SeriesACI-ModeSwitches,Release13.2.(1l).Thesevaluesarebasedonaprofilewhere
eachfeaturewasscaledtothenumbersspecifiedinthetables.ThesenumbersdonotrepresentthetheoreticallypossibleACIfabric
scale.
General Scalability Limits
•L2Fabric:InLegacymodethereisnorouting,L3context,norcontractenabledintheL2fabricprofile. Atenantinthisprofile
doesnotneedtobemappedtoonededicatedACItenant. AtenantcanberepresentedbyasetofEPGsinstead.Toimprovethe
loadsharingamongAPICcontrollernodes,youmustdistributeEPGsandBDsacrossanACItenant.
•L3Fabric:TheACIL3fabricsolutionprovidesafeature-richhighlyscalablesolutionforpubliccloudandlargeenterprise.
Withthisdesign,almostallsupportedfeaturesaredeployedatthesametimeandaretestedasasolution.Thescalabilitynumbers
listedinthissectionaremulti-dimensionalscalabilitynumbers.Thefabricscalabilitynumbersrepresenttheoverallnumberof
objectscreatedonthefabric.Theper-leafscalenumbersaretheobjectscreatedandpresentedonanindividualleafswitch.The
fabriclevelscalabilitynumbersrepresentAPICclusterscalabilityandthetestedupperlimits.Someoftheper-leafscalability
numbersaresubjecttohardwarerestrictions.Theper-leafscalabilitynumbersarethemaximumlimitstestedandsupportedby
leafswitchhardware.Thisdoesnotnecessarilymeanthateveryleafswitchinthefabricwastestedwithmaximumscalenumbers.
•StretchedFabric:Stretchedfabricallowsmultiplefabrics(upto3)distributedinmultiplelocationstobeconnectedasasingle
fabricwithasinglemanagementdomain.Thescalefortheentirestretchedfabricremainsthesameasforasinglesitefabric.
ForexampleaL3stretchedfabricwillsupportupto200leafswitchestotalwhichisthemaximumnumberofleafswitches
supportedonasinglesitefabric.Parametersonlyrelevanttostretchedfabricarementionedinthetablesbelow.
•Multi-Pod:Multipodenablesprovisioningamorefault-tolerantfabriccomprisedofmultiplepodswithisolatedcontrolplane
protocols.Also,multipodprovidesmoreflexibilitywithregardtothefullmeshcablingbetweenleafandspineswitches.For
example,ifleafswitchesarespreadacrossdifferentfloorsordifferentbuildings,multipodenablesprovisioningmultiplepods
perfloororbuildingandprovidingconnectivitybetweenpodsthroughspineswitches.
MultipodusesasingleAPICclusterforallthepods;allthepodsactasasinglefabric.IndividualAPICcontrollersareplaced
acrossthepodsbuttheyareallpartofasingleAPICcluster.
•Multi-Site:Multi-SiteisthearchitectureinterconnectingandextendingthepolicydomainacrossmultipleAPICclusterdomains.
Assuch,Multi-SitecouldalsobenamedasMulti-Fabric,sinceinterconnectsseparateAvailabilityZones(Fabrics)eachdeployed
asasinglePodinthisreleaseandmanagedbyanindependentAPICcontrollercluster.AnACIMulti-Sitepolicymanageris
partofthearchitectureandisusedtocommunicatewiththedifferentAPICdomainstosimplifythemanagementofthearchitecture
andthedefinitionofinter-sitepolicies.
NOTE:Themaximumnumberofleafswitchesoverallis400perfabricscale.Themaximumnumberofphysicalportsis19,200per
fabric.Themaximumnumberofremoteleafswitchesis40perfabric
2
Feature L2Fabric L3Fabric LargeL3Fabric
NumberofAPICcontrollers 3*or4nodeAPICcluster 3*or4nodeAPICcluster 5*,6,or7nodeAPICcluster
Note *denotespreferredcluster
size.
Whilethehighernumberof
controllersissupported,the
preferredsizeisbasedonthe
numberofleafswitchesin
theenvironment.
Numberofleafswitches 80 80 300for5-or6-nodecluster
400for7-nodecluster
Numberofspineswitches Maximumspinesperpod:6. Maximumspinesperpod:6. Maximumspinesperpod:6.
Totalspines24. Totalspines24. Totalspines24.
NumberofFEXs N/A 20perleafswitch,576 N/A
ports/leafswitch,650per
fabric
Numberoftenants N/A 1000 3000
NumberofLayer3(L3)contexts N/A 1000 3000
(VRFs)
Numberofcontracts/filters N/A •10,000contracts •10,000contracts
•10,000filters •10,000filters
Numberofendpointgroups(EPGs) 21,000(500maximumper 15,000(500maximumper 15,000(500maximumper
tenant,or4,000ifthereisa tenant,or4,000ifthereisa tenant,or4,000ifthereisa
singletenantinafabric) singletenantinafabric) singletenantinafabric)
NumberofIsolationenabledEPGs 400 400 400
Numberofbridgedomains(BDs) 21,000 15,000 15,000
NumberofBGP+numberofOSPF N/A 3,000 3,000
sessions+EIGRP(forexternal
connection)
NumberofMulticastgroups N/A 8000 8000
NumberofMulticastgroupsperVRF N/A 8000 8000
Numberofstaticroutestoasingle N/A 5,000 10,000
SVI/VRF
Numberofstaticroutesonasingleleaf N/A 5,000 10,000
switch
3
Feature L2Fabric L3Fabric LargeL3Fabric
NumberofvCenters N/A •200VDS •200VDS
•50AVS •50AVS
•50CiscoACIVirtual •50CiscoACIVirtual
Edge Edge
NumberofServiceChains N/A 1000 1000
NumberofL4-L7devices N/A 30managedor50unmanaged 30managedor50unmanaged
physicalHApairs,1,200 physicalHApairs,1,200
virtualHApairs(1,200 virtualHApairs(1,200
maximumperfabric) maximumperfabric)
NumberofESXihosts-VDS N/A 3200 3200
NumberofESXihosts-AVS N/A 3200(Only1AVSinstance 3200(Only1AVSinstance
perhost) perhost)
NumberofESXihosts-AVE N/A 3200(Only1AVEinstance 3200(Only1AVEinstance
perhost) perhost)
NumberofVMs N/A Dependsuponserverscale Dependsuponserverscale
Numberofconfigurationzonesper 30 30 30
fabric
NumberofBFDsessionsperleaf 256 256 256
switch
MinimumBFDtimer MinimumBFDtimer MinimumBFDtimerrequired
requiredtosupportthisscale: requiredtosupportthisscale: tosupportthisscale:
•minTx:50 •minTx:50 •minTx:50
•minRx:50 •minRx:50 •minRx:50
•multiplier:3 •multiplier:3 •multiplier:3
Multi-Pod 3*or4nodeAPICcluster,6 3*or4nodeAPICcluster,6 •5*or6nodeAPIC
pods,80leafswitchesoverall pods,80leafswitchesoverall cluster,6pods,200leaf
NOTE:*ispreferredclustersize
switchesmaxperpod,
300leafswitchesmax
overall
•7nodeAPICcluster,12
pods,200leafswitches
maxperpod,400leaf
switchesmaxoverall
L3EVPNServicesoverFabricWAN N/A 1000VRFs,60,000routesin 1000VRFs,60,000routesin
-GOLF(withandwithoutOpFlex) afabric afabric
Layer3Multicastroutes N/A 8,000 8,000
4
Feature L2Fabric L3Fabric LargeL3Fabric
NumberofRoutesinOverlay-1VRF 1,000 1,000 1,000
Multiple Fabric Options Scalability Limits
ConfigurableOptions PerLeafScale PerFabricScale
StretchedFabric
Maximumnumberoffabricsthatcanbeastretched N/A 3
fabric
MaximumnumberofRouteReflectors N/A 6
Multi-Pod
MaximumnumberofPODs N/A 12
MaximumnumberofleafswitchesperPOD N/A 200
Maximumnumberofleafswitchesoverall N/A 400
Cisco ACI Multi-Site Scalability Limits
Stretchedvs.nonStretched—Stretchedinmulti-sitemeanstheMulti-SitefabrichasstretchedobjectssuchasEPG,BD,VRFor
SubnetacrossmultiplesitesorhascrosssitecontractsbetweenEPGs.Thescaleparametersforthisscenarioisdescribedinthe
"Stretched(Multi-Site)"column.Non-StretchedinMulti-SitemeansallobjectssuchasEPG,contract,andBDarelocaltoasiteonly
anddonotcrossthelocal-siteboundary.Thescaleparametersforthisscenarioisdescribedinthe"Non-Stretched(APIC)"column
inthetablebelow:
Note FormaximumscaleMulti-Siteconfigurationswithmanyfeaturesenabledsimultaneously,itisrecommendedthatthose
configurationsbetestedinalabpriortodeployment.
Multi-SiteGeneralScalabilityLimits
ConfigurableOptions Scale
Sites 10
Podspersite 12
Leafswitchespersite 200
5
Multi-SiteObjectScale
ConfigurableOptions Scale
PolicyObjectsperSchema 500
TemplatesperSchema 5
ApplicationProfilesperSchema 200
NumberofSchemas 60
NumberofTemplates 300
Multi-SiteOrchestratorUsers(nonparallel*) 50
*Multi-SiteOrchestratorprocessesrequestssequentiallyfrom
multipleuserseveniftheyaredeployingdifferentschemas.
Multi-SiteScalabilityLimitsforStretchedObjects
ScalingItem Stretched(Multi-Site) Non-Stretched(APIC)
Tenants 300 2500
VRFs 800 3000
BDs 3000 10,000
Contracts 3000 2000
EndPoints 100,000 150,000including:
•100,000-stretchedfromothersites
•50,000-locallylearnedinsite-local
EPGs 3000 10,000
IsolatedEPGs 400 400
MicroSegmentEPGs 400 400
IGMPSnooping 8000 8000
L3OutexternalEPGs 500 2400
Subnets 8000 10,000
NumberofL4-L7Logical 400 RefertoL4-L7SiteLocalScalabilityNumbers
devices
Numberofgraphinstances 250 RefertoL4-L7SiteLocalScalabilityNumbers
Numberofdeviceclustersper 10 RefertoL4-L7SiteLocalScalabilityNumbers
tenant
6
ScalingItem Stretched(Multi-Site) Non-Stretched(APIC)
Numberofinterfacesperdevice Any RefertoL4-L7SiteLocalScalabilityNumbers
cluster
Numberofgraphinstancesper 125 RefertoL4-L7SiteLocalScalabilityNumbers
devicecluster
Fabric Topology, SPAN, Tenants, Contexts (VRFs), External EPGs, Bridge
Domains, Endpoints, and Contracts Scalability Limits
Thetablebelowshowsthemappingofthe"ALE/LSEType"tothecorrespondingTORswitches.Thisinformationishelpfulto
determinewhichTORswitchisaffectedwhenweusethetermsALEv1,ALEv2,LSEorLSE2inremainingsections.
Note Inthefollowingtable,theN9K-C9336C-FX2switchislistedasLSEforscalabilitylimitspurposeonly.Theswitchiscapable
ofsupportingLSE2platformfeatures,consultspecificfeaturedocumentationforfulllistofsupporteddevices.
ALE/LSEType ACI-SupportedTORs
ALEv1 •N9K-C9396PX+N9K-M12PQ
•N9K-C93128TX+N9K-M12PQ
•N9K-C9396TX+N9K-M12PQ
ALEv2 •N9K-C9396TX+N9K-M6PQ
•N9K-C93128TX+N9K-M6PQ
•N9K-C9396PX+N9K-M6PQ
•N9K-C9372TX64K
•N9K-C9332PQ
•N9K-C9372PX
LSE N9K-C93108TC-EX+N9K-C93180YC-EX+N9K-C9336C-FX2
LSE2 N9K-C93108TC-FX+N9K-C93180YC-FX+
N9K-C9348GC-FXP
Note Unlessexplicitlycalledout,LSErepresentsbothLSEandLSE2andALErepresentsbothALEv1andALEv2intherestof
thisdocument.
7
ConfigurableOptions PerLeafScale PerFabricScale
FabricTopology
NumberofPCs,vPCs 320(withFEXHIF) N/A
Numberofencapsperaccessport,PC,vPC(non 3,000 N/A
FEXHIF)
NumberofencapsperFEXHIF,PC,vPC 20 N/A
NumberofmemberlinksperPC,vPC* 16 N/A
*vPCtotalports=32,16perleaf
NumberofportsxVLANS(globalscopeandno 64,000 N/A
FEXHIF)
168,000(whenusinglegacyBDmode)
NumberofportsxVLANS(FEXHIFsand/orlocal ForALEv1andv2:9,000 N/A
scope)
ForLSEandLSE2:
10,000
Numberofstaticportbindings ForALEv1andv2:30,000 400,000
ForLSEandLSE2:
60,000
STP AllVLANs N/A
Mis-CablingProtocol(MCP) 256VLANsperinterface N/A
2000logicalports(portxVLAN)per
leaf
8
ConfigurableOptions PerLeafScale PerFabricScale
Maximumnumberofendpoints(EPs) Defaultprofile(Dualstack)— 16-slotand8-slotmodularspine
switches:
•ForALEv1andv2:
Max.450,000ProxyDatabaseEntries
•MAC:12,000
inthefabric,whichcanbetranslated
•IPv4:12,000or intoanyoneofthefollowing:
•IPv6:6,000or •450,000MAC-onlyEPs(eachEP
withoneMAConly)
•IPv4:4,000
•225,000IPv4EPs(eachEPwith
IPv6:4,000 oneMACandoneIPv4)
•150,000dual-stackEPs(eachEP
DefaultProfileorHighLPMProfile—
withoneMAC,oneIPv4,and
•ForLSEorLSE2: oneIPv6)
•MAC:24,000
Theformulatocalculateinmixed
•IPv4:24,000 modeisasfollows:
#MAC+#IPv4+#IPv6<=450,000
•IPv6:12,000
NOTE:Fourfabricmodulesare
requiredonallspinesinthefabricto
IPv4Scaleprofile—
supportabovescale.
•ForLSEandLSE2:
4-slotmodularspineswitches:
•MAC:48,000
Max.360,000ProxyDatabaseEntries
•IPv4:48,000
inthefabric,whichcanbetranslated
intoanyoneofthefollowing:
•IPv6:Notsupported
•360,000MAC-onlyEPs(eachEP
•ForALEv1andv2:Notsupported withoneMAConly)
•180,000IPv4EPs(eachEPwith
HighDualStackScaleprofile—
oneMACandoneIPv4)
•ForLSE:
•120,000dual-stackEPs(eachEP
•MAC:64,000
withoneMAC,oneIPv4,and
oneIPv6)
•IPv4:64,000
•IPv6:24,000 Theformulatocalculateinmixed
modeisasfollows:
•ForLSE2:
#MAC+#IPv4+#IPv6<=360,000
•MAC:64,000
NOTE:Fourfabricmodulesare
•IPv4:64,000 requiredonallspinesinthefabricto
supportabovescale.
•IPv6:48,000
•ForALEv1andv2:Notsupported
9
ConfigurableOptions PerLeafScale PerFabricScale
Fixedspineswitches:
Max.180,000ProxyDatabaseEntries
inthefabric,whichcanbetranslated
intoanyoneofthefollowing:
•180,000MAC-onlyEPs(eachEP
withoneMAConly)
•90,000IPv4EPs(eachEPwith
oneMACandoneIPv4)
•60,000dual-stackEPs(eachEP
withoneMAC,oneIPv4,and
oneIPv6)
Theformulatocalculateinmixed
modeisasfollows:
#MAC+#IPv4+#IPv6<=180,000
NumberofMACEPGs N/A 125
NumberofMulticastGroups DefaultorHighLPMprofile(Dual 8000
stack):8,000
IPv4Scaleprofile:8,000
HighDualstackScaleprofile:512
NumberofMulticastGroupsperVRF 8000 8000
NumberofIPsperMAC 4096 4096
SPAN ALEbasedToRs: N/A
•4uni-directionalor2bi-directional
access/tenantsessions
•4uni-directionalor2bi-directional
fabricsessions
LSEbasedToRs:
•8uni-directionalor4bi-directional
sessions(fabric,access,ortenant)
10