ebook img

Big Data: Related Technologies, Challenges and Future Prospects PDF

100 Pages·2014·1.491 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Big Data: Related Technologies, Challenges and Future Prospects

SPRINGER BRIEFS IN COMPUTER SCIENCE Min Chen Shiwen Mao Yin Zhang Victor C.M. Leung Big Data Related Technologies, Challenges and Future Prospects 123 SpringerBriefs in Computer Science SeriesEditors StanZdonik PengNing ShashiShekhar JonathanKatz XindongWu LakhmiC.Jain DavidPadua Xuemin(Sherman)Shen BorkoFurht V.S.Subrahmanian MartialHebert KatsushiIkeuchi BrunoSiciliano SushilJajodia Forfurthervolumes: http://www.springer.com/series/10028 Min Chen • Shiwen Mao (cid:129) Yin Zhang Victor C.M. Leung Big Data Related Technologies, Challenges and Future Prospects 123 MinChen ShiwenMao SchoolofComputerScience AuburnUniversity andTechnology Auburn,AL,USA HuazhongUniversityofScience andTechnology VictorC.M.Leung Wuhan,China ElectricalandComputerEngineering TheUniversityofBritishColumbia YinZhang Vancouver,BC SchoolofComputerScience Canada andTechnology HuazhongUniversityofScience andTechnology Wuhan,China ISSN2191-5768 ISSN2191-5776(electronic) ISBN978-3-319-06244-0 ISBN978-3-319-06245-7(eBook) DOI10.1007/978-3-319-06245-7 SpringerChamHeidelbergNewYorkDordrechtLondon LibraryofCongressControlNumber:2014937319 ©TheAuthor(s)2014 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer. PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations areliabletoprosecutionundertherespectiveCopyrightLaw. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. While the advice and information in this book are believed to be true and accurate at the date of publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with respecttothematerialcontainedherein. Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) Preface “Howbigisbig?”SciencewriterStephenStraussasksinhisfunbookforkidstitled HowBigisBigandexplainsthat“bignessissomethingnoonecanconsume.” In this book, we aim to answer this interesting question, but in the context of computer data. In the big data era, we are dealing with the explosive increase of global data and enormous datasets. Unlike seemingly similar terms such as “massivedata”or“verybigdata,”bigdatareferstothe datasetsthatcouldnotbe perceived,acquired,managed,andprocessedbytraditionalInformationTechnology (IT)andsoftware/hardwaretoolswithinatolerabletime.Itcanbecharacterizedby four Vs, i.e., Volume (greatvolume), Variety (variousmodalities), Velocity (rapid generation),andValue(hugevaluebutverylowdensity). Inthisbook,weprovideacomprehensiveoverviewofthebackgroundandrelated technologies, challenges and future prospects of big data. We first introduce the general background of big data and review related technologies, such as cloud computing, Internet of Things (IoT), data centers, and Hadoop. We then focus on the four phases of the value chain of big data, i.e., data generation, data acquisition,datastorage,anddataanalysis.Foreachphase,weintroducethegeneral background,discuss the technical challenges, and review the latest advances. We nextexaminetheseveralrepresentativeapplicationsofbigdata,includingenterprise management, IoT, online social networks, healthcare and medical applications, collectiveintelligence,andsmartgrid.Thisbookisconcludedwithadiscussionof openproblemsandfuturedirections.Weaimtoprovidethereadersacomprehensive overviewandbig-pictureofthisexcitingarea.Wehopethismonographcouldbea usefulreferenceforgraduatestudentsandprofessionalsinrelatedfields,andgeneral readerswhowillbenefitfromanunderstandingofthebigdatafield. WearegratefultoDr.Xuemin(Sherman)Shen,theSpringerBriefsSeriesEditor on Wireless Communications. This book would not be possible without his kind support during the process. Thanks also to the Springer Editors and Staff, all of whomdidtheirusualexcellentjobingettingthismonographpublished. This work was supported by China National Natural Science Foundation (No. 61300224), the Ministry of Science and Technology (MOST), China, the International Science and Technology Collaboration Program (Project No.: v vi Preface 2014DFT10070),andtheHubeiProvincialKeyProject(No.2013CFA051).Shiwen Mao’sresearchissupportedinpartbytheUSNationalScienceFoundation(NSF) under Grants CNS-1320664, CNS-1247955, CNS-0953513, and DUE-1044021, andthroughtheNSFBroadbandWirelessAccess&ApplicationsCenter(BWAC) SiteatAuburnUniversity(NSFGrantIIP-1266036).TheresearchofVictorLeung issupportedbytheCanadianNaturalSciencesandEngineeringResearchCouncil, BC InnovationCouncil, Qatar Research Foundation,TELUS, and other industrial partners.Anyopinions,findings,andconclusionsorrecommendationsexpressedin thismaterialarethoseoftheauthorsanddonotnecessarilyreflecttheviewsofthe foundation. Wuhan,China MinChen Auburn,AL ShiwenMao Wuhan,China YinZhang Vancouver,BC,Canada VictorC.M.Leung January2014 Contents 1 Introduction ................................................................... 1 1.1 DawnoftheBigDataEra ............................................... 1 1.2 DefinitionandFeaturesofBigData..................................... 2 1.3 BigDataValue ........................................................... 5 1.4 TheDevelopmentofBigData........................................... 6 1.5 ChallengesofBigData .................................................. 7 References...................................................................... 9 2 RelatedTechnologies ......................................................... 11 2.1 CloudComputing........................................................ 11 2.1.1 CloudComputingPreliminaries ................................ 11 2.1.2 RelationshipBetweenCloudComputingandBigData ....... 12 2.2 IoT........................................................................ 13 2.2.1 IoTPreliminaries ................................................ 13 2.2.2 RelationshipBetweenIoTandBigData ....................... 14 2.3 DataCenter............................................................... 15 2.4 Hadoop ................................................................... 16 2.4.1 HadoopPreliminaries............................................ 16 2.4.2 RelationshipbetweenHadoopandBigData................... 17 References...................................................................... 18 3 BigDataGenerationandAcquisition ...................................... 19 3.1 BigDataGeneration..................................................... 19 3.1.1 EnterpriseData................................................... 19 3.1.2 IoTData.......................................................... 20 3.1.3 InternetData ..................................................... 21 3.1.4 Bio-medicalData ................................................ 21 3.1.5 DataGenerationfromOtherFields............................. 22 3.2 BigDataAcquisition..................................................... 23 3.2.1 DataCollection .................................................. 23 vii viii Contents 3.2.2 DataTransportation.............................................. 26 3.2.3 DataPre-processing ............................................. 27 References...................................................................... 30 4 BigDataStorage.............................................................. 33 4.1 StorageSystemforMassiveData....................................... 33 4.2 DistributedStorageSystem.............................................. 35 4.3 StorageMechanismforBigData........................................ 37 4.3.1 DatabaseTechnology............................................ 38 4.3.2 DesignFactors................................................... 44 4.3.3 DatabaseProgrammingModel.................................. 45 References...................................................................... 48 5 BigDataAnalysis ............................................................. 51 5.1 TraditionalDataAnalysis................................................ 51 5.2 BigDataAnalyticMethods.............................................. 53 5.3 ArchitectureforBigDataAnalysis ..................................... 55 5.3.1 Real-Timevs.OfflineAnalysis ................................. 55 5.3.2 AnalysisatDifferentLevels..................................... 56 5.3.3 AnalysiswithDifferentComplexity............................ 57 5.4 ToolsforBigDataMiningandAnalysis................................ 57 References...................................................................... 58 6 BigDataApplications ........................................................ 59 6.1 ApplicationEvolution ................................................... 59 6.2 BigDataAnalysisFields ................................................ 61 6.2.1 StructuredDataAnalysis........................................ 61 6.2.2 TextDataAnalysis............................................... 61 6.2.3 WebDataAnalysis .............................................. 63 6.2.4 MultimediaDataAnalysis....................................... 64 6.2.5 NetworkDataAnalysis.......................................... 65 6.2.6 MobileTrafficAnalysis ......................................... 67 6.3 KeyApplications......................................................... 69 6.3.1 ApplicationofBigDatainEnterprises......................... 69 6.3.2 ApplicationofIoTBasedBigData............................. 70 6.3.3 ApplicationofOnlineSocialNetwork-Oriented BigData.......................................................... 70 6.3.4 ApplicationsofHealthcareandMedicalBigData............. 73 6.3.5 CollectiveIntelligence........................................... 74 6.3.6 SmartGrid ....................................................... 75 References...................................................................... 76 Contents ix 7 OpenIssuesandOutlook..................................................... 81 7.1 OpenIssues............................................................... 81 7.1.1 TheoreticalResearch ............................................ 81 7.1.2 TechnologyDevelopment....................................... 83 7.1.3 PracticalImplications............................................ 84 7.1.4 DataSecurity..................................................... 84 7.2 Outlook................................................................... 86 References...................................................................... 89

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.