PKI Planes in Cisco APIC-EM (1.3.x)

14 Pages·2016·1.2 MB·English

PKI Planes in Cisco APIC-EM (1.3.x) Tech Note
PKIPlanesinCiscoAPIC-EM1.3.x 2
OverviewofSecureConnectionsinCiscoAPIC-EMv.1.3.x 3
PKIPlanesinCiscoAPIC-EM1.3.x 5
Summary:PKIPlanesinCiscoAPIC-EMv.1.3.x 10
Revised: October 21, 2016,
PKI Planes in Cisco APIC-EM 1.3.x
EffectivemanagementoftheCiscoAPIC-EMPKIrequiresanunderstandingofthemechanismsthatsecurevarioustypesofnetwork
connection.ThistopicconcernsitselfprimarilywithPKI-basedcontrolleranddeviceconnections,describingotherkindsof
connectionsonlyforpurposesofcomparisonandcontrastwithPKI-securedconnections.Detaileddescriptionsofnon-PKIconnections
areoutsidethescopeofthisdiscussion.
TheCiscoAPIC-EMprovidesPKI-basedconnectionsinseveraldistinctPKIplanes.
•ControllerPKIPlane:HTTPSconnectionsinwhichthecontrolleristheserverintheclient-servermodel,andthecontroller's
servercertificatesecurestheconnection.Thecontroller'sservercertificatecanbeself-signed(default)orissuedbyanexternal
CA(recommended.)
•DevicePKIPlane: DMVPNconnectionsbetweendevicesinthecontrolplaneofthenetwork,bilaterallyauthenticatedand
securedbythedeviceIDcertificatesofbothdevicesthatparticipateintheconnection.TheseDMVPNtunnelssecuredata-plane
trafficasittravelsbetweennetworkdevices.AprivateCAprovidedbytheAPIC-EMcontroller(theDevicePKICA)manages
thesecertificatesandkeys.
•GrapevineServicePKIPlane:ThegrapevinerootmanagesthisinternalPKIplanethatsecurescommunicationsbetween
Grapevineservicesinamulti-hostcluster;theGrapevineServicePKIPlaneisnotexternallyaccessible,soitisnotdiscussed
furtherhere.
Table 1: PKI Planes in Cisco APIC-EM
Authentication Encryption Use Case
ControllerPKIPlane:externalcallerinitiatesconnectiontocontroller
HTTPS Callerpresentsusername+passwordor Yes RESTclient,includingCiscoNetwork
callerpresentsserviceticket;Controller PlugNPlay(PnP)mobileapporCisco
presentsservercertificate PrimeInfrastucture
HTTPS One-way:controllerpresentsitsserver Yes CiscoNetworkPlugNPlay(PnP)
certificate provisioningworkflows
DevicePKIPlane:device-to-deviceconnections
DMVPN BilateralauthenticationviaIKEv2using Yes DMVPNconnectionsbetweendevices
deviceIDcertificates/keysissuedbythe forthesecureexchangeofdata-plane
privateCAthattheAPIC-EMcontroller traffic
provides(theDevicePKICA).DeviceID
certificatesandkeyssecure
device-to-deviceconnectionsbetween
IWAN-manageddevices.
GrapevineServicePKIPlane:connectionsbetweengrapevineservices
HTTPS Connectionsbetweengrapevineservices Yes Systemuseonly.Notaccessibleto
externalcallers.
2
InthedefaultconfigurationofCiscoAPIC-EM,theDevicePKICAisarootCA;thereisnoparentCAaboveit.Thisconfiguration
oftheDevicePKICAisknownasrootCAmode.NotethatrootCAmodeappliesONLYtotheDevicePKICA;theDevicePKI
CAhasnothingtodowiththeControllerPKIplane.
Optionally,version1.3.xofCiscoAPIC-EMprovidestheabilityfortheDevicePKICAtouseaCAcertificatethathasbeenissued
byanexternalCA.ThisconfigurationoftheDevicePKICAisknownassubCAmode.Regardlessofmode,theDevicePKICA
neverinteractsdirectlywiththeexternalCAandnoautomatedmanagementoftheDevicePKICA'sCAcertificateeveroccurs.
Again,regardlessofmode,onlytheDevicePKICAmanagesthecertificatesandkeysthatsecuredevice-to-deviceconnectionsin
theDevicePKIplane;anexternalCA,ifused,neverhasaccesstothesecertificatesandkeys.
IntherareeventthatanexternalCArevokestheCAcertificateoftheDevicePKICA,auserwhohasROLE_ADMINinscopeALL
mustreplacethiscertificatemanually.Notethatdoingsorequiresre-configurationoftheclusterandmanualdeprovisioningof
devicesthatusecertificatesandkeysissuedundertheoldCAcertificate;thereisnootherworkflowforreplacingtheDevicePKI
CA'sCAcertificate.
TounderstandsubCAmode,andAPIC-EMPKIimplementationingeneral,simplyremembertwopointsthattheremainderofthis
topicexploresfurther:
•subCAmodeaffectsONLYtheCAcertificateoftheDevicePKICA.ItdoesnotaffectANYothercertificatesorkeys.
•TheAPIC-EMprovidesNOautomatedinteractionswithanyexternalCA.
Overview of Secure Connections in Cisco APIC-EM v. 1.3.x
TwoindependentPKIplanes(ControllerPKIPlaneandDevicePKIPlane)securetwomaincategoriesofPKI-basedconnection.
TheAPIC-EMcontrolleralsosupportsothertypesofsecureconnectionthatdonotusePKI.
Note ThegrapevinerootmanagesaninternalPKIplanethatsecurescommunicationsbetweenGrapevine
servicesinamulti-hostcluster;theGrapevineServicePKIPlaneisnotexternallyaccessible,soitisnot
discussedfurtherhere.
PKI-Based Connections
AllHTTPSconnectionstotheAPIC-EMusetheControllerPKIPlane.Device-to-deviceconnectionsusetheDevicePKIPlane,
whichiscompletelyseparatefromtheControllerPKIPlane.
Controller PKI Plane: Externally Initiated HTTPS Connections to the Controller
WhenthecontrollerrespondstoarequestforanHTTPSsession,itistheserverinaclient-servermodelthatusesPKItosecurethe
connection.InresponsetotherequestforanHTTPSsession,thecontrollerpresentsitsservercertificate.Therefore,externallyinitiated
HTTPSconnectionstothecontrollertakeplaceintheControllerPKIPlane.
HTTPSrequestscancomefromdevicesinthecontrolplaneofthenetworkortheycancomefromNBRESTAPIcallers.The
controllerneverinitiatesHTTPSconnectionstodevices.
3
Device PKI Plane: DMVPN Connections Between IWAN-Managed Devices
AseparatePKIplanesecurestheDynamicMultipointVPN(DMVPN)connectionsthatIWAN-manageddevicesformamongst
themselvesforthesecureexchangeofdata-planetraffic.ThisDevicePKIPlaneismanagedbyaprivateCAthattheAPIC-EM
controllerprovides(theDevicePKICA.)
Bydefault,theDevicePKICArunsasarootCA;inthismode(knownasrootCAmode),theCAcertificateoftheDevicePKICA
istheapexofthecertificatechainfordevicecertificates.Optionally,theDevicePKICAcanbeconfiguredtouseanexternallyissued
CAcertificate(so-calledsubCAmode),whichsubordinatestheDevicePKICAtotheexternalCA.
•Inthedefaultconfiguration(rootCAmode),anexternalCAcannotmanagethecertificatesandkeysthatsecuretheDevicePKI
Plane.
•InsubCAmode,auserwhohasROLE_ADMINinscopeALLmustmanuallyuploadtotheprivateCAaCAcertificateissuedby
anexternalCA.InsubCAmode,theprivateCAdoesnotinteractdirectlywiththeexternalCA,noautomatedmanagementof
theprivateCA'sCAcertificateoccurs,andtheexternalCAstillcannotmanageanyofthecertificatesorkeysthattheprivate
CAissuestoIWAN-manageddevices.
WhileinsubCAmode,ifyouusethesameCAtomanagetheDevicePKICA'sCAcertificateandthecontroller'sserver
certificate,therespectivecertificatechainsoftheDevicePKIplaneandtheControllerPKIplanehaveacommonancestor,but
nousecasetakesadvantageofthisancestry.Formoreinformation,seeDevice-to-DeviceDMVPNConnections, onpage8.
Grapevine Service PKI Plane: Connections Between Grapevine Services
ThegrapevinerootmanagesthisinternalPKIplanethatsecurescommunicationsbetweenGrapevineservicesinamulti-hostcluster;
theGrapevineServicePKIPlaneisnotexternallyaccessible,soitisnotdiscussedfurtherhere.
Non-PKI Secure Connections
ThecontrolleralsosupportsthefollowingsecureconnectionsthatdonotusePKI.
Controller-Initiated Non-PKI Secure Connections to Devices
Controller-initiatedsecureconnectionstodevicescanuseSSHorAuthenticatedSNMPv3.Theseconnectionsareauthenticated,but
theydonotuseaCA;therefore,theseconnectionsdoNOTtakeplaceintheControllerPKIPlane.
•SSHfromthecontroller:WhenthecontrollerinitiatesanSSHconnectiontoadevice,thecontrollerpresentsasharedsecret
(username/passwordpair)andthedevicepresentsitspublickeytocreateasecureconnection.
•AuthenticatedSNMPv3:Authentication-enabledSNMPv3usesasharedsecrettoestablishtrustbetweenthecontrollerand
thedevice.Whenthecontrollerusesauthentication-enabledSNMPv3toinitiateaconnectiontoadevice,itpresentsausername
andpasswordthatatrustedadministratorsuppliedout-of-bandtoboththecontrollerandthedevice:
-Theadminsuppliedcredentialstothedevicebycreatingonthedevicealoginaccountthatthecontrollercanusefor
discoverypurposes.
-Theadminsuppliedcredentialstothecontrollerbycreatingdiscoverycredentialsonthecontroller.Thesecredentials
enablethecontrollertosupplyavalidusername/passwordpairtologintothedevicefordiscoverypurposes.
Ifthedeviceacceptstheusernameandpasswordthatthecontrollerpresents,thenthecontrollertruststhedevice.(Notethat
SNMPv3canbeconfigurednottoauthenticatetheconnection;ifso,theconnectionisnotsecuredanditisoutsidethescope
ofthisdiscussionofsecureconnections.Optionally,authenticatedSNMPv3connectionscanalsobeencrypted.)
4
Externally Initiated Non-PKI Secure Connections to the Controller
SSHtothecontroller:Whenanexternalcaller(suchasanadministrativeremoteterminalsession)initiatesanSSHconnectionto
thecontroller,thecontrollerpresentsitshostpublicRSAorECDSAkey.RequestsforSSHsessionscomefromadministrators
openingremoteconsolesessionswiththegrapevineroot.NetworkdevicesneverinitiateSSHconnectionstothecontroller.
Controller-to-Controller Secure Tunneling
APIC-EMcontrollersinamulti-hostclustercanuseasecure,encryptedchannelforcommunicatingamongstthemselves.This
communicationsinfrastructureisnotaccessiblebymeansofanyAPIoruserinterface.ThissecuritymechanismisnotPKI-based;
itusesIPSectunnelssecuredbyprivatekeysthatthegrapevinerootmanages.Formoreinformation,see"ConfiguringIPSecTunneling
forMulti-HostCommunications"intheCiscoApplicationPolicyInfrastructureControllerEnterpriseModuleDeploymentGuide.
PKI Planes in Cisco APIC-EM 1.3.x
TheAPIC-EMmaintainsmultipleseparatePKIplanes.EachPKIplanesecuresaparticularsetofconnections:
•ControllerPKIPlane:Client-initiatedHTTPSconnectionstothecontroller
WhenanexternalcallerinitiatesanHTTPSconnectiontothecontroller,thecontrollerpresentsitsservercertificate.Such
connectionsincludethefollowing:
◦LoginstotheAPIC-EMGUIviaHTTPS
◦GrapevineAPIcalls(HTTPSonport14141,redirectedtoport443)
◦InvocationsoftheNBRESTAPIviaHTTPS
WhenaNBRESTAPIcallerinitiatesanHTTPSconnectiontothecontrollertoinvokeaNBRESTAPIortodownload
afile(suchasadeviceimage,aconfiguration,andsoon)thecontroller(server)presentsitsservercertificatetothecaller
(client)thatrequestedtheconnection.
Notethatcontroller-initiatedconnectionstodevicesdoNOTtakeplacewithintheControllerPKIPlane.Eveniftheconnections
useSSHorSNMPv3,noCAmanagesthekeysinvolved,sotheconnectionisnotconsideredtobePKI-based.Thecontroller
mayinitiateconnectionstodevicesforpurposesthatincludediscovery,managingtags,pushingpolicytodevices,orinteracting
withdevicesononbehalfofaRESTcaller.Forcompatibilitywitholderdevices,discoverycanoptionallyusetheTELNET
protocol,whichisinsecureandthereforeoutsidethescopeofthisPKIdiscussion.
•DevicePKIPlane:Device-to-deviceDMVPNconnections
IWAN-managedcontrol-planedevicesformDynamicMultipointVPN(DMVPN)connectionsamongthemselvesforthesecure
exchangeofdata-planetraffic.AprivateCertificateAuthority(CA)providedbytheCiscoAPIC-EM(theDevicePKICA)
provisionsthecertificatesandkeysthatsecuretheseDMVPNconnections.ThePKIbrokerservicemanagesthesecertificates
andkeysasdirectedbyanadminintheIWANGUIorasdirectedbyaRESTcallerthatusesthe/certificate-authority
and/trust-pointNBRESTAPIs.
TheprivateCAcanruninrootCAmode(default)orsubCAmode:
◦InrootCAmode(default),theDevicePKICAistherootCAandthereisnoparentCAaboveit.ItsCAcertificatecannot
bereplaced.Itcannotbeasub-CAorintermediateCAtoanyexternalCA.
◦InsubCAmode,theDevicePKICAusesaCAcertificatethatwasissuedbyanexternalCA.However,thisrelationship
doesnotenabletheexternalCAtoperformanysortofautomatedmanagementofanyPKIitemsonthecontrolleroron
thedevicenetwork.
•GrapevineServicePKIPlane:Connectionsbetweengrapevineservices
5
ThegrapevinerootmanagesaninternalPKIplanethatsecurescommunicationsbetweenGrapevineservicesinamulti-host
cluster;theGrapevineServicePKIPlaneisnotexternallyaccessible,soisnotdiscussedfurtherhere.
RegardlessofwhethertheDevicePKICAoperatesasarootCAorasasubCA,thefollowingrulesalwaysapply:
•TheDevicePKICAneverinteractsdirectlywiththeexternalCA.
•NoautomatedmanagementoftheDevicePKICACAcertificateeveroccurs.IftheexternalCArevokestheCAcertificateof
theDevicePKICA,auserwhohasROLE_ADMINinscopeALLmustlearnofthisout-of-bandandmustreplacethesubCA
certificatemanually.
•TheexternalCAcannotmanagethecertificatesandkeysthattheDevicePKICAissuestosecuredevice-to-deviceconnections
betweenIWAN-manageddevices.ThesecertificatesandkeysarealwaysmanagedONLYbytheDevicePKICA.
•TheuseofsubCAmodedoesnotalterthebehaviorofthe/certificate-authorityand/trust-pointNBRESTAPIs.For
example,ifyouusetheNBRESTAPItorevoketheCAcertificateoftheDevicePKICA,thecontrollerdoesNOTcalloutto
theexternalCA.
ItisalsoimportanttounderstandthattheDevicePKICAnevermanagesthecontroller'sservercertificate.
Formoreinformation,seeDevice-to-DeviceDMVPNConnections, onpage8.
HTTPS Connections to the Controller
WhenanexternalcallerinitiatesanHTTPSconnectiontotheCiscoAPIC-EMcontroller,thatconnectiontakesplaceinthePKIplane
thatthecontroller'sservercertificatesecures(theControllerPKIPlane.)Inthisclient-servermodel,thecontrolleristheserverthat
presentsitscertificatetotheclient(RESTclientorWebbrowser)toestablishatrustedconnection.(Ifthecontrollerisbehinda
firewall,itsgateway-proxycertificateparticipatesinthetrustchain.)
Thecertificatethatthecontroller(orproxy)presentscanbeself-signedorCA-issued.IfthecertificateisCA-issued,anativeCisco
callermayrefertothetrustpoolbundletoestablishtrustwiththeCA.APnP-managedCiscodevicedownloadedthetrustpoolbundle
fromthecontrolleraspartoftheCiscoNetworkPlugNPlay(PnP)workflowthatprovisionedthedevice.Anon-PnPCiscodevice
canalsobeconfiguredtousethetrustpoolbundle.Non-Ciscodevicesorcallerscannotusethetrustpoolbundle"as-is"buttheycan
establishtrustwithtrustpoolCAsbymeansoftheirowncertificatechain.
UponestablishingtrustwiththecontrollerandanyrequiredCAs,thecallercanuseHTTPStoinvokeNBRESTAPIsonthecontroller,
suchasthosewhichprovideone-timedownloadsofconfigurationfiles,certificates,keys,andsoon.Forexample,thePnPmobile
applicationmayinitiateHTTPSwiththecontrollerforthepurposeofprovisioningnetworkdevices,butthedeploymentofconfiguration
filestothedevicesbythecontrollertakesplaceoveracontroller-initatedSSHconnectionthatisNOTwithintheControllerPKI
Plane.
Security-consciouscallerstypicallywouldnotconnecttoacontrollerthatpresentsanexpiredorrevokedservercertificate,although
itispossibleforthemtodosoattheirownrisk.
Expiration of the Controller's Server Certificate
TheAPIC-EMcontrolleroranetworkdevicedoesnotneedtheassistanceofaCAtodeterminewhetheracertificatehasexpired.
Theexpirationdateofthecertificateiscontainedinthecertificateitself;thedeviceorcontrollersimplycomparesthecertificate's
expirationdatewithcurrentsystemtime.
TheAPIC-EMcontrollerwarnsadministratorsoftheimpendingexpirationofitsservercertificate.Thiswarningappearsinthe
controllerGUIonly.Thecontrollerprovidesnoautomatedmanagementofthiscertificate;auserwhohasROLE_ADMINinscopeALL
musttakeexplicitactiontoreplacetheservercertificatebeforeitexpires.ThisadministratorcanusetheControllerSettingspanel
intheGUIorthe/certificateNBRESTAPItoreplacetheservercertificate.Similarly,thisadmincanusetheGUIorthe
/proxy-certificateNBRESTAPItoreplacetheproxycertificate.
6
Revocation of the Controller's Server Certificate
Thecontroller'sservercertificatecanbeself-signed(default)orCA-issued(recommended.)
•Aself-signedcertificatecan'tberevokedinthetruesenseoftheword:withouttheuseofanexternalCertificateAuthority,
thereisnomechanismforcommunicatingtheinvalidstatusofthecertificatetothosewhohaveestablishedatrustrelationship
withthatcertificate.However,theself-signedcertificatecanbedeletedorreplaced,breakingthechainoftrustthathadbeen
establishedwiththeoldcertificate.
•TheCA-basedrevocationworkflowappliestoCA-issuedcertificatesonly.Inthisworkflow,atrustedCertificateAuthority
mayrevokeacertificate,communicatetherevokedstatusofthatcertificatetoothermembersofitstrustdomain,andperhaps
supplyavalidreplacementcertificate.
TheworkflowthatresultsfromtheCA-basedrevocationofthecontroller'sservercertificatevariesaccordingtothecontextinwhich
theCAinteracts.Usecasestoconsiderincludenon-PnPdevices,NBRESTAPIcallers,PnP-manageddevices,theAPIC-EMcontroller
itselfandinvalidationofanintermediaryCAcertificatethatispartofthetrustpoolbundle.
Non-PnP Devices and NB REST Callers
Ifconfiguredtodoso,anon-PnPnetworkdevicecancontacttheappropriateCAtolearnoftherevocationofaCA-issuedserver
certificatebymeansoftheCertificateRevocationList(CRL)orOnlineCertificateStatusProtocol(OSCP).However,ifthedevice
isnotconfiguredtoperformarevocationcross-checkwithanexternalCA,thedevicecannotdeterminewhethertheexternalCAhas
revokedthatcertificate.Asaresult,thedevicemaytrustacertificatethatanexternalCAhasrevoked.
Note Thisexampledescribesnon-PnPdevicesthatcommunicatewithanexternalCA,notPnP-manageddevices
thatinteractwiththeDevicePKICA.Non-PnPdevicescannotcommunicatewiththeDevicePKICA,
andtheDevicePKICAdoesnotacceptOSCPrequests.
Adeviceperformsarevocationcross-checkwiththeexternalCAonlywhenitsCRLdistributionpoint(CDP)pointstotheexternal
CA.IfthedeviceisnotconfiguredtocheckaCRLortoissueanOSCPrequesttotheexternalCA,thedevicesimplychecksitsown
internaltruststoreofvalidand/orprivateCArootcertsalongwiththeexpirationdateoftheservercertificatethatthecontroller
presentstoit.Ifthetruststorecontainsstalerevocationdataandthecertificateisnotexpired,itispossibleforthedevicetotrusta
revokedcertificate.
Themostlikelycircumstanceunderwhichacontroller'sservercertificatemightberevokedwouldbeanexplicitrequestbythe
controlleradmintotheexternalCAtorevoketheservercertificate.Theadministratormightissuethisrequestifthecontrollerwas
stolenoriffailedhardwarewasreturnedtoCiscowithouthavingbeenprocessedproperlyforreturn.Inthissituation,itisreasonable
toassumethattheadminissuingtherevocationrequestwouldknowoftherevocationandwouldtakestepstogenerateandinstalla
new,valid,CA-issuedreplacementservercertificateonthecontrolleroronareplacementcontroller.
DevicesconfiguredtointeractwiththetrustedCAthatmanagestheservercertificateshouldcontinuetoworkcorrectlyuponinstallation
ofanothervalid,CA-issuedservercertificateonthecontroller.DevicesthatdonotinteractwiththetrustedCAmightneedtobe
updatedmanuallyasnecessarytotrustthenewservercertificate;untilthisupdatetakesplace,thesedevicesmayrefusetoconnect
tothecontroller,andRESTAPIrequeststhatinvolvethesedevicesmayfail.
PnP-Managed Devices and IWAN-Managed Devices
PnP-manageddevicesandIWAN-manageddevicesneverinteractwithanyCAotherthantheAPIC-EMprivateCA,evenwhenthe
privateCArunsinsubCAmode.Therefore,PnP-manageddevicescannotlearnoftherevocationofthecontroller'sservercertificate
directlyfromanexternalCA.Hence,thestatusoftheAPIC-EMservercertificateisofnodirectconsequencetosuchdevices.These
devicesmightrespondtoachangeinstatusoftheprivateCAservercertificate,however,asdescribedinDevice-to-DeviceDMVPN
Connections, onpage8.
7
Important TheprivateCAthatsecuresDistributedMultipointVPNconnectionsdoesnotmanagethecontroller's
servercertificate.Therefore,itcannotproviderevocationstatusofthecontroller'sservercertificate.For
moreinformation,seeDevice-to-DeviceDMVPNConnections, onpage8.
APIC-EM Controller
AlthoughaCA-issuedservercertificatecanbeinstalledontheAPIC-EMcontroller,theAPIC-EMcontrolleritselfdoesNOTinteract
directlywithanyexternalCA;therefore,ithasnowaytolearnoftherevocationofitsservercertificatebyanexternalCA.Note,
also,thatthecontrollerdoesnotupdateitsservercertificateautomaticallyunderanycircumstances.Replacementofanexpiredor
revokedservercertificaterequiresexplicitactiononthepartofauserwhohasROLE_ADMINinscopeALL.
Intermediary CA Certificate in the Trustpool Bundle
InvalidationofanintermediaryCAcertificateinthetrustpoolbundleisaspecialcase.Whenthetrustpoolbundlechanges,the
controllerGUIdoesdisplayanotifcationtouserswhohaveROLE_ADMINinscopeALL,anditprovidesabuttonthatthistypeofadmin
canclicktodownloadandinstallanewtrustpoolbundleonthecontroller.However,themeansbywhichnetworkelementsgetthe
newtrustpoolbundlevaryaccordingtohowthebundlewasinstalledonthosedevices.DevicesnotmanagedbyPnPcannotgetthe
trustpoolbundlefromthecontroller,buttheymaybeconfiguredtodownloadanewtrustpoolbundlefromtheCiscocloudautomatically.
PnP-manageddevicesthatgotthetrustpoolbundlefromthecontrollerwillcontinuetotrustthecontroller'snewintermediarycertificate
ifithasavalidRootCAcertificate.ThesameistrueofdevicesnotmanagedbyPnP.Therefore,althoughbestpracticerecommends
manualupdateofdeviceswiththenewtrustpoolbundleintimelyfashion,achangetoanintermediaryCAisnotlikelytocausean
immediateproblem.
Device-to-Device DMVPN Connections
IWAN-manageddevicescanformDynamicMultipointVPN(DMVPN)connectionsamongthemselvesforthesecureexchangeof
data-planetraffic.TheDevicePKICAandthepki-brokerserviceworktogethertoprovisionthedeviceIDcertificatesandkeysthat
securetheseDMVPNconnections.Thepki-brokerservicealsoexposesaNBRESTAPIthatcanbeusedtomanagethesedeviceID
certificatesandkeysmanually.
WhentheDevicePKICArunsinrootCAmode(default),theDevicePKICAisnotrecognizedbyanyexternalCAasanintermediate
CA.Therefore,thisinternalCAisnotamemberofthetrustpool(ios.p7b)bundlethattheAPIC-EMprovidestodevicesinthe
NetworkPlugnPlayprovisioningworkflow.ExternalCAsinthetrustpoolbundlehavenoknowledgeofthecertificatesthatthe
controller'sinternalCAdolesouttoIWANdevicesprivately.CertificatesissuedbytheDevicePKICAcanberevokedmanuallyby
usingthe/trust-pointNBRESTAPIthattheCiscoAPIC-EMcontrollerexposes.
RunningtheDevicePKICAinsubCAmoderequirestheDevicePKICAtouseaCAcertificatethatissignedbyanexternalCA.
However,subCAmodedoesnotenabletheDevicePKICAtointeractwiththeexternalCA,anditdoesnotprovideautomated
managementoftheDevicePKICA'sCAcert.IfanexternalCArevokestheCAcertificateoftheDevicePKICA,theDevicePKI
CAcannotlearnofthisrevocationbecauseitneverinteractsdirectlywiththeexternalCA.AlthoughtherevocationoftheCA
certificateoftheprivateCAinvalidatesthePKIbroker'sservercertificate,which,inturn,invalidatesalldeviceIDcertificatesthat
thePKIbrokerissued,theAPIC-EMprovidesnoautomatedmanagementoftheDevicePKICA'sCAcert;therefore,itispossible
fordevicestocontinuetrustingthepki-brokerservercertificateevenwhenthesubCACAcertificatehasbeenrevokedbytheexternal
CA.
Asaresult,theuseofsubCAmodedoesnotchangetheend-user-visiblebehavioroftheprivateCAitself.TheuserwhohasROLE_ADMIN
inscopeALLmustlearnoftherevocationoftheCAcertificateout-of-bandandinstallanewCAcertificateintheprivateCA.Note
thatrevocationoftheCAcertificateisanextremelyrareoccurrence,andinstallationofanewCAcertificateintheprivateCAisa
non-trivialtask.ThecontrollerdoesnotprovideaGUIoranAPIforreplacingthesubCAcertificate.OncesubCAmodeisenabled,
theonlywaytoreplacetheCAcertificateoftheDevicePKICAistodoacompleteresetthatbringsthecontrollerbacktothedefault
8
rootCAmode,andthensubsequentlyredotheconversiontoSubCAmodeusingthenewsubCAcertificate.Beforeconvertingthe
controllerbacktosubCAmode,youmustremovealldeviceIDcertificatesandkeysissuedtonetworkdevicesundertheprevious
configurationoftheDevicePKICA.ThedevicesmustbetakenofflinebeforeconvertingthecontrollertosubCAmodewiththe
newsubCAcertificate,andthenalldeviceswillneedtobereprovisionedbythePKIbrokerserviceusingthenewconfigurationof
theDevicePKICA.
IWAN-participatingdeviceslearnoftherevocationofinternalCA-issueddeviceIDcertificatesbymeansoftheCRLdistribution
pointthatistheprivateCAitself.WhentwodevicesattempttocreateaDMVPNtunnel,theypresentdeviceIDcertificatestoeach
other.Todeterminewhetherthecertificatepresentedtoithasbeenrevoked,eachdevicepollsitsCRL.WheneveradeviceID
certificateisrevoked,theprivateCAgeneratesanewCRL.
AprivateCA-issuedcertificate(whichisusedtosecureDMVPNconnections)isvalidforoneyearfromthedateofissue(default)
oruntilanadministratively-setexpirationdate.IWAN-participatingdevicescanattemptautomatedrenewalofaprivateCA-issued
certificatebeforethecertificateactuallyexpires.ExpirationorrenewalofaprivateCA-issuedcertificategeneratesPKIeventsthat
appearintheauditlogs.
Intersection of the Device and Controller PKI Planes
TherequestforacertificatefromtheDevicePKICAcouldbeviewedasthepointatwhichthetwoPKIplanesintersect,thoughin
differentcontexts.Thisrequestfromadevicetothecontrollerrequiresatrustrelationshipthatthecontroller’sservercertificate
guarantees;thecontroller'sservercertificateisNOTissuedbytheDevicePKICA.However,thepayloadoftheresponseconcerns
thedeviceIDcertificatethattheDevicePKICAissuestothedeviceaspartofthedevice-provisioningworkflow.ThisdeviceID
certificateprotectstheDMVPNconnectionsthatIWAN-participatingdevicesformamongthemselvesintheDevicePKIPlane.
Tounderstandthesestatementsmoreclearly,let'sexamineoneexampleofaworkflowthatintersectsbothPKIplanes.ThePOST
/trust-pointrequestgeneratesthecertificatesandkeysnecessarytoenrollaspecificdeviceinthetrustdomainoftheDevicePKI
CA.ThistrustrelationshipenablesthedevicetoparticipateinDMVPNconnectionswithotherdevicesthattrusttheDevicePKICA.
AnycallercanissuethisrequesttoenrolladeviceinthetrustdomainthattheDevicePKICAmanages.Forexample,youmight
createacustomRESTapplicationthatenrollsdevices,oranadministratormightusethisAPItoenrolldevicesmanually,ifnecessary.
Requiredargumentstothisrequestidentifyaspecificdevice(forexample,bydeviceserialnumberaswellasitsFQDNorIPaddress).
Theresponsetothisrequestcannotbeusedtoenrollanyotherdevice,anditisvalidforalimitedamountoftimeonly.
Forillustrativepurposes,assumethatanexternalcallerhasissuedthePOST/trust-pointrequest,supplyingaserialnumber,FQDN
andIPaddressthatidentifyaspecificdevicetoenroll.ThecontrollerreceivesthisHTTPSrequestintheControllerPKIplaneand
forwardsittothepki-brokerservice,whichinvokestheDevicePKICAintheDevicePKIplanetogeneratecertificatesandkeys
intendedforuseintheDevicePKIplane.TheresponsetotherequestthentravelsbacktothecallerintheControllerPKIplane.
Becausegenerationofthecertificatesbundlecompletesasynchronouslytotheoriginalrequest,thePOST/trust-pointresponse
bodyindicatesonlywhethertherequestwasaccepted;iftheHTTPresponseis202,thentheresponsebodycontainsataskIDthat
thecallercanusetodeterminewhenthetaskhascompleted.WhentheGET/task/{taskID}responsebodyindicatesthatthetask
hascompletedsuccessfully,itcontainsanidelementthatcanbepassedasthetrustPointIdargumenttothetheGET
/trust-point/{trustPointId}/configrequest.ThisrequestreturnstheinformationnecessarytoretrieveandinstallthePKCS12
bundlethatthecontrollergeneratedforthisspecificdevice.ThisrequestoriginatesintheControllerPKIplaneandreturnsapayload
thatenablesaccesstothedeviceIDcertificate/keybundlethatwasgeneratedintheDevicePKIplane;uponinstallationofthisbundle
inthedevice,thedevice(anditscertificate/keybundle)resideintheDevicePKIplane.
Note The/trust-pointAPIprovidesanumberofwaystoretrievetrustpointIDs.Formoreinformation,see
theRESTAPIdocumentation.
Thus,the/trust-pointNBRESTAPIisanexampleofadevicelifecycle-managementoperationthatoccursintheControllerPKI
Planethatthecontroller'sservercertificateprotects.AlthoughthePOST/trust-pointandGET
/trust-point/{trustPointId}/configrequestsareprotectedbytheservercertificate,thePKIpayloadsoftheirresponsesconcern
9
deviceIDcertificates,keysandCRLsthatprotecttransactionsintheDevicePKIPlane,suchasDMVPNtunnelingamongIWAN
devicesthemselves.
AsimilarcontextapplieswhenadeviceintheDevicePKIplaneretrievesanewCRLfromtheDevicePKICA.Thedevicesendsa
requesttothecontroller,whichrequestsaCRLfromtheDevicePKICA.Becausethedevicehasanexistingtrustrelationshipwith
theDevicePKICA,itcangettheCRLfromtheDevicePKICAdirectly.Oneimportantdifference,however,isthatthedevice's
requestforthenewCRLgoestothecontrolleroverHTTPratherthanHTTPS.Becausenocertificatesecurestheconnection,this
requestisnotconsideredtooccurinaPKItrustdomain,buttheworkflowissimilarinthesensethatarequestfromtheDevicePKI
planetothecontrollerresultsinthegenerationofaPKIconstructthatisusedintheDevicePKIplane.
AnotherexampleinwhichthecontrolleractsasanintermediarybetweenanIWAN-manageddeviceandtheDevicePKICAisthe
workflowinwhichthedevice'sdeviceIDcertificateisduetoexpireandthedevicerequestsanewdeviceIDcertificatefromthe
DevicePKICA.Again,thedeviceissuesanHTTPrequesttothecontroller,whichinstructstheDevicePKICAtogenerateanew
deviceIDcertificateforthedevice.Inthiscase,however,thecurrentdeviceIDcertificateonthedeviceisstillvalid,sotheDevice
PKICAcaninstallthereplacementdeviceIDcertificatedirectlyonthedevice.Oncethenewcertificateisinstalled,theDevicePKI
CAgeneratesanewCRLthatrevokestheolddeviceIDcertificate.Thus,thisparticularworkflowincludestworequeststhatoriginate
intheDevicePKIplane,traveltothecontroller,andresultintheinstallationofadeviceIDcertificateandCRLusedintheDevice
PKIplane.
Summary: PKI Planes in Cisco APIC-EM v. 1.3.x
ThefollowingfactorsgovernsecureconnectionstotheAPIC-EMcontrollerandsecuredevice-to-deviceconnections:
•PKIfornetworkcontrolplaneiscompletelyseparatefromPKIforconnectionstoAPIC-EMcontroller
TwocompletelyindependentPKIplanesseparatethecontroller'sinteractionwithexternalcallersfromdevice-to-device
interactionsinthenetworkcontrolplane.(AnotherPKIplane,theGrapevineServicePKIPlane,isnotexternallyaccessible.)
◦DevicePKIPlane:DMVPNconnectionsbetweennetworkdevices
DevicesmanagedbyCiscoNetworkPlugNPlay(PnP-manageddevices)makedevice-to-deviceDMVPNconnections
autonomouslyamongthemselvesforthesecureexchangeofdata-planetraffic.AprivateCertificateAuthority(CA)
providedbytheAPIC-EMcontroller(theDevicePKICA)managesthecertificatesandkeysthatsecuretheseconnections.
TheDevicePKICAcanoperateasarootCA(default)orasasubordinatetoanexternalCA(subCAmode.)
-InrootCAmode,theCAcertificateoftheDevicePKICAcannotbemanagedbyanyexternalCA,andtheDevice
PKICAcannotbeasub-CAorintermediateCAtoanyotherCA.
-InsubCAmode,theDevicePKICAusesaCAcertificatethatisissuedbyanexternalCA.However,theDevice
PKICAdoesnotinteractdirectlywiththeexternalCA,andrevocationoftheCAcertificatebytheexternalCAdoes
notresultintheautomatedreplacementofanycertificatesorkeys.IfauserhavingROLE_ADMINinscopeALLreplaces
therevokedCAcertificatewithanewone,thecontrollergeneratesanewservercertificateforthePKIbroker,device
IDcertificatessignedbytheoldservercertificatearerevoked,andPnP-manageddevicesretrieveanewCRL.
Regardlessofmode,neithertheprivateCAnorthecontrolleritselfcaninteractdirectlywithanyexternalCA.PnP-managed
devicesneverinteractwithanyCAotherthantheprivateCA.Devicescannotbe"hybridprovisioned"tointeractwith
boththeDevicePKICAandanotherCA.
◦ControllerPKIPlane:HTTPSconnectionstothecontrollersecuredbycontroller'scertificate
ThecontrollerpresentsitsservercertificateinresponsetoHTTPSconnectionrequests.Thiscertificatecanbeself-signed
(default)orCA-issued(recommended),butthecontrolleritselfdoesnotinteractwithanyexternalCA.
10