Table Of Content

Cloudera Administration ImportantNotice ©2010-2021Cloudera,Inc.Allrightsreserved. Cloudera,theClouderalogo,andanyotherproductor servicenamesorsloganscontainedinthisdocumentaretrademarksofClouderaand itssuppliersorlicensors,andmaynotbecopied,imitatedorused,inwholeorinpart, withoutthepriorwrittenpermissionofClouderaortheapplicabletrademarkholder.If thisdocumentationincludescode,includingbutnotlimitedto,codeexamples,Cloudera makesthisavailabletoyouunderthetermsoftheApacheLicense,Version2.0,including anyrequirednotices.AcopyoftheApacheLicenseVersion2.0,includinganynotices, isincludedherein.AcopyoftheApacheLicenseVersion2.0canalsobefoundhere: https://opensource.org/licenses/Apache-2.0 HadoopandtheHadoopelephantlogoaretrademarksoftheApacheSoftware Foundation.Allothertrademarks,registeredtrademarks,productnamesandcompany namesorlogosmentionedinthisdocumentarethepropertyoftheirrespectiveowners. Referencetoanyproducts,services,processesorotherinformation,bytradename, trademark,manufacturer,supplierorotherwisedoesnotconstituteorimply endorsement,sponsorshiporrecommendationthereofbyus. Complyingwithallapplicablecopyrightlawsistheresponsibilityoftheuser.Without limitingtherightsundercopyright,nopartofthisdocumentmaybereproduced,stored inorintroducedintoaretrievalsystem,ortransmittedinanyformorbyanymeans (electronic,mechanical,photocopying,recording,orotherwise),orforanypurpose, withouttheexpresswrittenpermissionofCloudera. Clouderamayhavepatents,patentapplications,trademarks,copyrights,orother intellectualpropertyrightscoveringsubjectmatterinthisdocument.Exceptasexpressly providedinanywrittenlicenseagreementfromCloudera,thefurnishingofthisdocument doesnotgiveyouanylicensetothesepatents,trademarkscopyrights,orother intellectualproperty.ForinformationaboutpatentscoveringClouderaproducts,see http://tiny.cloudera.com/patents. Theinformationinthisdocumentissubjecttochangewithoutnotice.Clouderashall notbeliableforanydamagesresultingfromtechnicalerrorsoromissionswhichmay bepresentinthisdocument,orfromuseofthisdocument. Cloudera,Inc. 395PageMillRoad PaloAlto,CA94306 info@cloudera.com US:1-888-789-1488 Intl:1-650-362-0488 www.cloudera.com ReleaseInformation Version:ClouderaEnterprise5.5.x Date:February3,2021 Table of Contents About Cloudera Administration................................................................................7 Managing CDH and Managed Services.....................................................................8 ManagingCDHandManagedServicesUsingClouderaManager........................................................................8 Configuration Overview.........................................................................................................................................................8 Managing Clusters...............................................................................................................................................................32 ManagingServices...............................................................................................................................................................36 Managing Roles...................................................................................................................................................................45 ManagingHosts...................................................................................................................................................................49 MaintenanceMode..............................................................................................................................................................62 ManagingCDHUsingtheCommandLine..........................................................................................................65 StartingCDHServicesUsingtheCommandLine..................................................................................................................66 StoppingCDHServicesUsingtheCommandLine................................................................................................................71 MigratingDatabetweenClustersUsingdistcp....................................................................................................................73 ManagingIndividualServices.............................................................................................................................78 ManagingFlume..................................................................................................................................................................78 ManagingtheHBaseService...............................................................................................................................................80 Managing HDFS.................................................................................................................................................................134 ManagingHive...................................................................................................................................................................165 Managing Hue...................................................................................................................................................................173 Managing Impala..............................................................................................................................................................197 ManagingKey-ValueStoreIndexer....................................................................................................................................208 ManagingMapReduceandYARN......................................................................................................................................209 ManagingOozie.................................................................................................................................................................217 Managing Solr...................................................................................................................................................................225 ManagingSpark.................................................................................................................................................................228 ManagingtheSqoop1Client.............................................................................................................................................231 ManagingSqoop2.............................................................................................................................................................232 ManagingZooKeeper.........................................................................................................................................................232 ConfiguringServicestoUsetheGPLExtrasParcel.............................................................................................................233 Resource Management........................................................................................235 Schedulers........................................................................................................................................................235 Cloudera Manager Resource Management.....................................................................................................235 LinuxControlGroups(cgroups)........................................................................................................................237 ResourceManagementwithControlGroups.....................................................................................................................239 ConfiguringResourceParameters......................................................................................................................................240 StaticServicePools...........................................................................................................................................241 Dynamic Resource Pools..................................................................................................................................242 ManagingDynamicResourcePools...................................................................................................................................243 YARNPoolStatusandConfigurationOptions....................................................................................................................245 AssigningApplicationsandQueriestoResourcePools......................................................................................................246 Configuration Sets.............................................................................................................................................................248 Scheduling Rules................................................................................................................................................................249 Managing Impala Admission Control...............................................................................................................250 Managingthe Impala Llama ApplicationMaster..............................................................................................252 EnablingIntegratedResourceManagementUsingClouderaManager............................................................................253 DisablingIntegratedResourceManagementUsingClouderaManager...........................................................................254 ConfiguringLlamaUsingClouderaManager.....................................................................................................................254 Impala Resource Management........................................................................................................................254 AdmissionControlandQueryQueuing..............................................................................................................................254 IntegratedResourceManagementwithYARN...................................................................................................................262 Performance Management...................................................................................264 OptimizingPerformanceinCDH.......................................................................................................................264 ChoosingaDataCompressionFormat.............................................................................................................267 Tuningthe Solr Server......................................................................................................................................268 TuningtoCompleteDuringSetup......................................................................................................................................268 GeneralTuning...................................................................................................................................................................268 Other Resources.................................................................................................................................................................275 Tuning Spark Applications................................................................................................................................275 Tuning YARN.....................................................................................................................................................281 Overview............................................................................................................................................................................282 Cluster Configuration.........................................................................................................................................................285 YARN Configuration...........................................................................................................................................................286 MapReduce Configuration.................................................................................................................................................288 Step7:MapReduceConfiguration.....................................................................................................................................288 Step7A:MapReduceSanityChecking................................................................................................................................288 ConfiguringYourClusterInClouderaManager.................................................................................................................288 High Availability...................................................................................................290 HDFS High Availability......................................................................................................................................290 IntroductiontoHDFSHighAvailability...............................................................................................................................290 ConfiguringHardwareforHDFSHA...................................................................................................................................291 EnablingHDFSHA..............................................................................................................................................................292 DisablingandRedeployingHDFSHA..................................................................................................................................304 ConfiguringOtherCDHComponentstoUseHDFSHA.......................................................................................................305 AdministeringanHDFSHighAvailabilityCluster...............................................................................................................308 ChangingaNameserviceNameforHighlyAvailableHDFSUsingClouderaManager......................................................312 MapReduce(MRv1)andYARN(MRv2)HighAvailability..................................................................................312 YARN(MRv2)ResourceManagerHighAvailability.............................................................................................................313 WorkPreservingRecoveryforYARNComponents.............................................................................................................320 MapReduce(MRv1)JobTrackerHighAvailability..............................................................................................................322 ClouderaNavigatorKeyTrusteeServerHighAvailability.................................................................................334 ConfiguringKeyTrusteeServerHighAvailabilityUsingClouderaManager......................................................................334 ConfiguringKeyTrusteeServerHighAvailabilityUsingtheCommandLine......................................................................335 RecoveringaKeyTrusteeServer........................................................................................................................................337 KeyTrusteeKMSHighAvailability....................................................................................................................337 HighAvailabilityforOtherCDHComponents...................................................................................................338 HBaseHighAvailability......................................................................................................................................................338 Hive Metastore High Availability.......................................................................................................................................343 HueHighAvailability.........................................................................................................................................................345 Llama High Availability......................................................................................................................................................348 ConfiguringOozieforHighAvailability..............................................................................................................................349 Search High Availability.....................................................................................................................................................350 ConfiguringClouderaManagerforHighAvailabilityWithaLoadBalancer.....................................................352 IntroductiontoClouderaManagerDeploymentArchitecture...........................................................................................352 PrerequisitesforSettingupClouderaManagerHighAvailability......................................................................................353 High-LevelStepstoConfigureClouderaManagerHighAvailability.................................................................................354 DatabaseHighAvailabilityConfiguration..........................................................................................................................380 TLSandKerberosConfigurationforClouderaManagerHighAvailability.........................................................................381 Backup and Disaster Recovery..............................................................................383 PortRequirementsforBackupandDisasterRecovery.....................................................................................383 Data Replication...............................................................................................................................................384 Designating a Replication Source......................................................................................................................................386 HDFS Replication................................................................................................................................................................387 Hive Replication.................................................................................................................................................................396 Impala Metadata Replication............................................................................................................................................404 UsingSnapshotswithReplication......................................................................................................................................404 EnablingReplicationBetweenClustersinDifferentKerberosRealms................................................................................404 Replication of Encrypted Data...........................................................................................................................................406 HBase Replication..............................................................................................................................................................407 Snapshots.........................................................................................................................................................414 Cloudera Manager Snapshot Policies................................................................................................................................414 Managing HBase Snapshots..............................................................................................................................................417 ManagingHDFSSnapshots................................................................................................................................................428 Cloudera Manager Administration........................................................................433 Starting,Stopping,andRestartingtheClouderaManagerServer...................................................................433 ConfiguringClouderaManagerServerPorts....................................................................................................433 MovingtheClouderaManagerServertoaNewHost.....................................................................................433 ManagingtheClouderaManagerServerLog...................................................................................................434 Viewing the Log.................................................................................................................................................................434 SettingtheClouderaManagerServerLogLocation..........................................................................................................435 ClouderaManagerAgents................................................................................................................................435 Starting,Stopping,andRestartingClouderaManagerAgents..........................................................................................436 ConfiguringClouderaManagerAgents.............................................................................................................................437 ManagingClouderaManagerAgentLogs.........................................................................................................................440 ChangingHostnames........................................................................................................................................441 Configuring Network Settings..........................................................................................................................443 Alerts................................................................................................................................................................443 Managing Alerts................................................................................................................................................................444 Managing Licenses...........................................................................................................................................451 SendingUsageandDiagnosticDatatoCloudera.............................................................................................456 ConfiguringaProxyServer.................................................................................................................................................456 ManagingAnonymousUsageDataCollection..................................................................................................................456 ManagingHueAnalyticsDataCollection..........................................................................................................................456 DiagnosticDataCollection.................................................................................................................................................457 ExportingandImportingClouderaManagerConfiguration.............................................................................459 Backing up Cloudera Manager.........................................................................................................................459 BackingupDatabases........................................................................................................................................................461 OtherClouderaManagerTasksandSettings...................................................................................................461 Settings..............................................................................................................................................................................461 Alerts..................................................................................................................................................................................462 Users..................................................................................................................................................................................462 Kerberos.............................................................................................................................................................................462 License...............................................................................................................................................................................462 UserInterfaceLanguage....................................................................................................................................................462 Peers..................................................................................................................................................................................462 Cloudera Management Service........................................................................................................................462 Cloudera Navigator Data Management Component Administration......................468 ClouderaNavigatorAuditServer......................................................................................................................468 PublishingAuditEvents......................................................................................................................................................471 ClouderaNavigatorMetadataServer...............................................................................................................472 Managing Metadata Extraction........................................................................................................................................478 ManagingMetadataPolicies.............................................................................................................................................479 Appendix: Apache License, Version 2.0.................................................................480 AboutClouderaAdministration About Cloudera Administration ThisguidedescribeshowtoconfigureandadministeraClouderadeployment.Administratorsmanageresources, availability,andbackupandrecoveryconfigurations.Inaddition,thisguideshowshowtoimplementhighavailability, anddiscussesintegration. ClouderaAdministration|7 ManagingCDHandManagedServices Managing CDH and Managed Services IfyouuseClouderaManagertomanageyourcluster,configuringandmanagingyourcluster,aswellasindividual servicesandhosts,usesadifferentparadigmthanifyouuseCDHwithoutClouderaManager.Forthisreason,many oftheseconfigurationtasksoffertwodifferentsubtasks,oneeachforclustersmanagedbyClouderaManagerand oneforclusterswhichdonotuseClouderaManager.Often,thetasksarenotinterchangeable.Forinstance,ifyouuse ClouderaManageryoucannotusestandardHadoopcommand-lineutilitiestostartandstopservices.Instead,youuse ClouderaManagertoperformthesetasks. Managing CDH and Managed Services Using Cloudera Manager YoumanageCDHandmanagedservicesusingtheClouderaManagerAdminConsoleandClouderaManagerAPI. ThefollowingsectionsfocusontheClouderaManagerAdminConsole. ConfigurationOverview WhenClouderaManagerconfiguresaservice,itallocatesrolesthatarerequiredforthatservicetothehostsinyour cluster.Theroledetermineswhichservicedaemonsrunonahost. Forexample,foranHDFSserviceinstance,ClouderaManagerconfigures: • OnehosttoruntheNameNoderole. • OnehosttorunasthesecondaryNameNoderole. • OnehosttoruntheBalancerrole. • RemaininghostsastorunDataNoderoles. Arolegroupisasetofconfigurationpropertiesforaroletype,aswellasalistofroleinstancesassociatedwiththat group.ClouderaManagerautomaticallycreatesadefaultrolegroupnamedRoleTypeDefaultGroupforeachrole type. Whenyouruntheinstallationorupgradewizard,ClouderaManagerconfiguresthedefaultrolegroupsitadds,and addsanyotherrequiredrolegroupsforagivenroletype.Forexample,aDataNoderoleonthesamehostasthe NameNodemightrequireadifferentconfigurationthanDataNoderolesrunningonotherhosts.ClouderaManager createsaseparaterolegroupfortheDataNoderolerunningontheNameNodehostandusesthedefaultconfiguration forDataNoderolesrunningonotherhosts. ClouderaManagerwizardsautoconfigurerolegrouppropertiesbasedontheresourcesavailableonthehosts.For propertiesthatarenotdependentonhostresources,ClouderaManagerdefaultvaluestypicallyalignwithCDHdefault valuesforthatconfiguration.ClouderaManagerdeviateswhentheCDHdefaultisnotarecommendedconfiguration orwhenthedefaultvaluesareillegal. ClouderaManagerConfigurationLayout AfterrunningtheInstallationwizard,useClouderaManagertoreconfiguretheexistingservicesandaddandconfigure additionalhostsandservices. ClouderaManagerconfigurationscreensoffertwolayoutoptions:new(thedefault)andclassic.Youcanswitchbetween layoutsusingtheSwitchtoXXXlayoutlinkatthetoprightofthepage.Keepthefollowinginmindwhenyouselecta layout: • Ifyouswitchtotheclassiclayout,ClouderaManagerpreservesthatsettingwhenyouupgradetoanewversion. • Selectionsmadeinonelayoutarenotpreservedwhenyouswitch. • Certainfeatures,includingcontrolsforconfiguringNavigatorauditeventsandHDFSlogredaction,aresupported onlyinthenewlayout. 8|ClouderaAdministration ManagingCDHandManagedServices Newlayoutpagescontaincontrolsthatallowyoufilterconfigurationpropertiesbasedonconfigurationstatus,category, andgroup.Forexample,todisplaytheJournalNodemaximumlogsizeproperty(JournalNodeMaxLogSize),clickthe CATEGORY>JournalNodeandGROUP>Logsfilters: Whenaconfigurationpropertyhasbeensettoavaluedifferentfromthedefault,aresettodefaultvalueicon displays. Classiclayoutpagesareorganizedbyrolegroupandcategorieswithintherolegroup.Forexample,todisplaythe JournalNodemaximumlogsizeproperty(JournalNodeMaxLogSize),selectJournalNodeDefaultGroup>Logs. ClouderaAdministration|9 ManagingCDHandManagedServices Whenaconfigurationpropertyhasbeensettoavaluedifferentfromthedefault,aResettothedefaultvaluelink displays. Thereisnomechanismforresettingtoanautoconfiguredvalue.However,youcanusetheconfigurationhistoryand rollbackfeaturetorevertanyconfigurationchanges. ModifyingConfigurationPropertiesUsingClouderaManager Note: ThistopicdiscusseshowtoconfigurepropertiesusingtheClouderaManager"newlayout."Theolder layout,calledthe"classiclayout"isstillavailable.Forinstructionsonusingtheclassiclayout,see ModifyingConfigurationProperties(ClassicLayout)onpage15. Toswitchbetweenthelayouts,clickeithertheSwitchtothenewlayoutorSwitchtotheclassic layoutlinksintheupper-rightportionofallconfigurationpages. MinimumRequiredRole:Configurator(alsoprovidedbyClusterAdministrator,FullAdministrator) WhenaserviceisaddedtoClouderaManager,eitherthroughtheinstallationorupgradewizardorwiththeAdd Servicesworkflow,ClouderaManagerautomaticallysetstheconfigurationproperties,basedontheneedsoftheservice andcharacteristicsoftheclusterinwhichitwillrun.Theseconfigurationpropertiesincludebothservice-wide configurationproperties,aswellasspecificpropertiesforeachroletypeassociatedwiththeservice,managedthrough rolegroups.Arolegroupisasetofconfigurationpropertiesforaroletype,aswellasalistofroleinstancesassociated withthatgroup.ClouderaManagerautomaticallycreatesadefaultrolegroupnamedRoleTypeDefaultGroupfor eachroletype.SeeRoleGroupsonpage48. ChangingtheConfigurationofaServiceorRoleInstance 1. Gototheservicestatuspage.(Cluster>servicename) 2. ClicktheConfigurationtab. 3. Locatethepropertyyouwanttoedit.Youcantypeallorpartofthepropertynameinthesearchbox,orusethe filtersontheleftsideofthescreen: • TheStatussectionlimitsthedisplayedpropertiesbytheirstatus.Possiblestatusesinclude: 10|ClouderaAdministration

Description:
Hadoop and the Hadoop elephant logo are trademarks of the Apache Software. Foundation. All other trademarks, registered trademarks, product
ebook img

Cloudera Administration PDF

481 Pages·2017·8.82 MB·English
by  
#cloudera #administration
Download
Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Download Cloudera Administration PDF Free - Full Version

by Unknow| 2017| 481 pages| 8.82| English

Download Cloudera Administration by in PDF format completely FREE. No registration required, no payment needed. Get instant access to this valuable resource on PDFdrive.to!

Free Download PDF

About Cloudera Administration

Hadoop and the Hadoop elephant logo are trademarks of the Apache Software. Foundation. All other trademarks, registered trademarks, product

Detailed Information

Author:Unknown
Publication Year:2017
Pages:481
Language:English
File Size:8.82
Format:PDF
Price:FREE
Download Free PDF

Safe & Secure Download - No registration required

Why Choose PDFdrive for Your Free Cloudera Administration Download?

  • 100% Free: No hidden fees or subscriptions required for one book every day.
  • No Registration: Immediate access is available without creating accounts for one book every day.
  • Safe and Secure: Clean downloads without malware or viruses
  • Multiple Formats: PDF, MOBI, Mpub,... optimized for all devices
  • Educational Resource: Supporting knowledge sharing and learning

Free Books Similar to Cloudera Administration

Frequently Asked Questions

Is it really free to download Cloudera Administration PDF?

Yes, on https://PDFdrive.to you can download Cloudera Administration by completely free. We don't require any payment, subscription, or registration to access this PDF file. For 3 books every day.

How can I read Cloudera Administration on my mobile device?

After downloading Cloudera Administration PDF, you can open it with any PDF reader app on your phone or tablet. We recommend using Adobe Acrobat Reader, Apple Books, or Google Play Books for the best reading experience.

Is this the full version of Cloudera Administration?

Yes, this is the complete PDF version of Cloudera Administration by Unknow. You will be able to read the entire content as in the printed version without missing any pages.

Is it legal to download Cloudera Administration PDF for free?

https://PDFdrive.to provides links to free educational resources available online. We do not store any files on our servers. Please be aware of copyright laws in your country before downloading.

The materials shared are intended for research, educational, and personal use in accordance with fair use principles.

Upgrade Premium
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.
We use cookies

We use cookies to ensure you get the best browsing experience on our website. By clicking "Accept Cookies", you agree that we can store cookies on your device in accordance with our Terms and Privacy.

DMCA & Copyright: Dear all, most of the website is community built, users are uploading hundred of books everyday, which makes really hard for us to identify copyrighted material, please contact us if you want any material removed.
Copyright @ PDFdrive.to, 2022 | We love our users