Business Intelligence : The Savvy Manager’s Guide

372 Pages·05.669 MB·Russian

by  Абашева Л. П.

Business Intelligence
The Savvy Manager’s Guide
Second Edition
David Loshin
AMSTERDAM(cid:1)BOSTON(cid:1)HEIDELBERG(cid:1)LONDON
NEWYORK(cid:1)OXFORD(cid:1)PARIS(cid:1)SANDIEGO
SANFRANCISCO(cid:1)SINGAPORE(cid:1)SYDNEY(cid:1)TOKYO
MorganKaufmannisanimprintofElsevier
AcquiringEditor:AndreaDierna
DevelopmentEditor:RobynDay
ProjectManager:DanielleS.Miller
Designer:GregHarris
MorganKaufmannisanimprintofElsevier
225WymanStreet,Waltham,MA02451,USA
(cid:1)2013ElsevierInc.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,
includingphotocopying,recording,oranyinformationstorageandretrievalsystem,withoutpermissioninwriting
fromthepublisher.Detailsonhowtoseekpermission,furtherinformationaboutthePublisher’spermissionspolicies
andourarrangementswithorganizationssuchastheCopyrightClearanceCenterandtheCopyrightLicensing
Agency,canbefoundatourwebsite:www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(otherthanas
maybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchand
experiencebroadenourunderstanding,changesinresearchmethodsorprofessionalpractices,
maybecomenecessary.Practitionersandresearchersmustalwaysrelyontheirownexperi-
enceandknowledgeinevaluatingandusinganyinformationormethodsdescribedherein.
Inusingsuchinformationormethodstheyshouldbemindfuloftheirownsafetyandthesafety
ofothers,includingpartiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,
assumeanyliabilityforanyinjuryand/ordamagetopersonsorpropertyasamatterof
productsliability,negligenceorotherwise,orfromanyuseoroperationofanymethods,
products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
Loshin,David,1963-
Businessintelligence:thesavvymanager’sguide/DavidLoshin.–2nded.
p.cm.
ISBN978-0-12-385889-4
1. Businessintelligence.2. Informationtechnology–Management.3. Managementinformationsystems. I.Title.
HD38.7.L672012
658.4’72–dc23
2012032793
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary.
ISBN:978-0-12-385889-4
PrintedintheUnitedStatesofAmerica
1213141510987654321
ForinformationonallMKpublications
visitourwebsiteatwww.mkp.com
Preface
Introduction
IthasbeennearlytenyearssinceIbegantheprocessofwritingthefirsteditionofthis
book;insomewaysthatspanofyearsseemslikeaneternity,whileinothers,ablink
of the eye. At that time, business intelligence (BI), business analytics, OLAP, and
dataminingwerestillmaturingtechniques.Butoneinterestingobservation,looking
back at the text of the first edition: our objective in developing the book (titled
BusinessIntelligence–TheSavvyManager’sGuide)wastoprovideanoverviewof
a collection of techniques that were gradually being adopted to help business
understand ways towork better, and this objectiveanticipated the eventualdemoc-
ratizationofthecapabilitiesforrepurposinginformationfrommanysourcesinways
thatcouldleadto businessvalue. At thattime, Isuggested that:
The boundary that divides business and technology is a fuzzy one, and this border
erodes more and more as organizational managers recognize how integral knowl-
edge and information management are to the bottom line. A natural development
ofthisistheconceptofbusinessintelligence(BI),which(looselydefined)incorpo-
rates the tools, methods, and processes needed to transform data into actionable
knowledge.
WhatIfindcuriousaboutBIisthatitisnotjusttechnology,norisitjustpractices
andmethods.Itismoreacombinationofthebestofboththebusinessworldandthe
technical worlddusing advanced algorithms and data management techniques to
better implement the way a business works. But what prevents BI programs from
beingsuccessfulispreciselywhatformsthedividinglinebetweenbusinessandtech-
nology.Ifwearemovingtowardabusinessenvironmentwhereprofitsaredrivenby
theexploitationofinformation,thenitiscriticalforthosewhorun,or,moreproperly,
improve,thebusinesstounderstandwhatkindsofvalueliewithinacompany’sinfor-
mation and how tounlockthat valueand transformit into profits.
AsIwascompletingmypreviousbook,ThePractitioner’sGuidetoDataQuality
Improvement,IwasapproachedbyRickAdamsatMorganKaufmann(MK)torevise
thematerialinthefirsteditionintoasecondedition,whichIagreedtodobecauseI
thought that many things had changed in the ways that we guide our customers in
implementingaBIprogram.Iembarkedonthesetoftasksofupdatingthematerial
in the book as well as adding material about what has changed in the industry and xix
xx Preface
expandingsectionsandchaptersaboutthingsandideasthathavechangedovertime.
ButwhatIfoundinterestingisthatasIreviewedthematerial,Iwasstruckbyhow
muchofthefirsteditionhasremainedrelevant,whichIbelievehascontributedtothe
long shelf-life ofthe originalversion.
And in turn, it seems that Morgan Kaufmann’s interest in business intelligence
hasmaturedaswell;Iwasinvitedtobetheserieseditorforthebusinessintelligence
series, and (after some fits and starts) found a strong MK editor partner, Andrea
Dierna.AtthesametimethatIhavebeenrevisingthisbook,AndreaandIhavebeen
diligently working at laying the foundation for expanding the catalog of books to
guidedatamanagementandbusinessintelligenceprofessionals.Alotofthematerial
that is introduced in this book is, or will be treated in much finer detail in the
upcomingbooksoftheseries.Thisbookisintendedtobeacornerstonepieceofthat
strategy. My overall goal is to paint a broad-brush overview of the objectives and
practices for designing and deploying a BI program, and other books in the series
willprovidegreater depth about manyofthe topics we discuss here.
What This Book Is
ThereisalogicalsequencetounderstandingthebasicsofaBIprogram.Thisbook
will progress through that sequence, starting with the value of information, the
mechanicsofplanningforsuccess,datamodelinfrastructure,anddatapreparation,
followed by data analysis, integration, knowledge discovery, and finally the actual
useofdiscovered knowledge.My goals for this book include:
n Providingaknowledgebaseforthedecisionmakertodeterminethevalueofinte-
grating business intelligence, querying, and reporting technologies into the
company;
n Providinga high-leveldescription (i.e., not deep with technical jargon)oftech-
nicalconceptstargetedattheknowledgeablereader,followed bymorein-depth
descriptions;
n Providingsummaryinformationabouttechnologyconceptsandtheiradvantages
anddisadvantages;
n Providingleadershipconceptsassociatedwithimplementingorintegratingthese
technical components;
n Actingasamultiple-usetextthatdoublesasadetailedexplanatoryguideandas
aquickreference;
n Providingaclearexplanationoftheutilityoftechnologywithouttryingtoexplain
howtoimplement thattechnology.
As an example, there are some very good resources on developing a business
intelligenceroadmap,‘‘bigdataanalytics,’’ordataminingandknowledgediscovery
Preface xxi
that may cover the same topics covered here at a high level, but whose authors
provide much greater depth in the techniques or algorithms. Just as someone
watching a television doesn’t need to know how to build one, a data analyst, IT
manager, or business user does not need to know how to build an association rule
discovery system totake advantageof the results ofusing one.
Eachchapterinthisbookismeanttostandonitsown,andalthoughthesequence
of the chapters relates to a typical implementation sequence, you should not feel
constrainedbythatsequence.Myintentionistoprovideabookthatcanbereferred
toforguidance during the entire processof buildingand improvinga BI program.
Why You Should Be Reading This Book
Youhaveprobablypickedupthisbookforoneormoreoftheseverygoodreasons:
n Youareaseniormanagerseekingtotakeadvantageofyourorganization’sinfor-
mation to create or add to corporate value by increasing revenue, decreasing
costs, improving productivity, mitigating risks, or improving the customer
experience.
n You are the Chief Information Officer or Chief Data Officer of an organization
who desires to make the best useof the enterpriseinformation asset.
n You are amanager who has been asked todevelopa newBI program.
n You are amanager who has been asked totake overa flounderingBI program.
n You are amanager who has been asked totake overa successfulBI program.
n You are a senior business executive who wants to explore the value that a BI
program can addto yourorganization.
n Youareabusinessstaffmemberwhodesiresmoreinsightintothewaythatyour
organization does business.
n You are a database or software engineer who has been appointed a technical
manager for a BI program.
n You are asoftware engineer who aspires to bethe manager ofa BI program.
n YouareadatabaseorsoftwareengineerworkingonaBIprogramwhoaspiresto
replaceyour current manager.
n You are abusinessanalyst who hasbeen asked to joina BI team.
n Youareaseniormanagerandyourdirectlyreportingmanagershavestartedtalk-
ingabout BI using terminology youthink theyexpectyouto understand.
n Youareamiddle-levelmanagerorengineerandyourmanagerhasstartedtalking
about BIusing terminologyyou think theyexpectyou tounderstand.
n You are justinterested inBI.
How do I know so much about you? Because at many times in my life, I was
youdeither working on or managing a project for which I had some knowledge
xxii Preface
gaps.Andatthetime,Iwouldlovetohavehadastraightforwardbooktoconsultfor
aquicklookuporamorein-depth read,withouthavingtospend ahugeamount of
moneyonatechnicalbookthatonlybrieflyaddressedatopicofinterest.Instead,I
haveawallfulloftechnicalbooksthatcostafortune,andbecausetheyneverreally
addressedmyrealneeds,theyjustsittheregatheringdust.SoIhavedecidedtotake
a stab at writing the book I wish I had twenty years ago when I first embarked on
acareer ininformationutilization.
Organization of the Book
The book is organized into 21 chapters that cover the business intelligence and
analyticsprogramfromdevelopingavalueproposition,andplanning,throughdesign
and development, then focusing on preparation, integration, and then use of the
businessintelligenceresults.Thechapterdescriptionsareprovidedingreaterdetail
inTable P.1.
TABLEP.1 LayoutoftheBook
Chapter Title Description
1 Business Thischapterprovidesadefinitionofbusinessintelligence,
Intelligence–An providesanoverviewofthedriversforbusinessintelligence
Introduction andanalyticsactivities,andsuggeststhecapabilitiesthat
wouldbeincludedaspartofabusinessintelligenceand
analyticsprogram.
2 ValueDrivers Thischapterprovidesadescriptionofhowbusiness
intelligencecanimprovethewaysthatbusinessisdone.This
includesboth‘‘horizontal’’applicationsthatcanbeapplied
acrossmanydifferenttypesoforganizationsand‘‘vertical’’
applicationsassociatedwithspecificindustries.
3 Planningfor Thischapterwasreengineeredtoconcentrateonassessing
Success organizationalpreparednessforexploitingtheresultsof
abusinessintelligenceprogramdestablishingperformance
measureswhileinstitutingchangestotheorganizationthat
allowactiontobetakenasaresultofanalytics.Thischapter
alsoprovidesanoverview
ofthesuccessfactorsforthebusinessintelligence
program.
4 Developing Thisshortchapterprovidesahigh-leveloverviewand
aBusiness exampleorderingofthetaskstobeperformedasbusiness
Intelligence intelligenceisdeployedacrosstheorganization.
Roadmap
(Continued)
Preface xxiii
TABLEP.1 LayoutoftheBook(Continued)
Chapter Title Description
5 TheBusiness Thischapterprovidesahigh-leveloverviewofcapabilities
Intelligence withinabusinessintelligenceenvironment.Herewereview
Environment thefundamentalsofbusinessintelligence:architecture,
design,datamodeling,datapreparation,integration,data
quality,alongwiththedifferentmethodsandstylesof
deliveringactionableknowledge.
6 BusinessModels Thischapterdescribesapproachesformodelingtheflowof
andInformation informationacrosscross-functionalbusinessprocesses,the
Flow desireforperformancemeasurementsalongthoseprocesses,
andtheirneedforreportingandanalytics.
7 Data Thischapterprovidessuggestionsandguidelinesfor
Requirements engagingbusinessdataconsumers,understandingthetypes
Analysis ofbusinessproblemstheyarelookingtosolve,andsoliciting
datarequirementsforreportingandanalytics.
8 Data Thischapterfocusesontheplatformsforanalysis,largely
Warehousesand focusingondatawarehouses,operationaldatastores,and
theTechnical associateddatamodelssupportingthebusinessintelligence
Business activity.
Intelligence
Architecture
9 Business Thetopicofmetadataisimportantenoughtowarrantitsown
Metadata chapterinwhichwelookmorecloselyattheconceptsof
businessmetadata,businesstermglossaries,definitions,and
semantics.
10 DataProfiling Herewelookatthebeginningofthedatapreparationphase
anddatadiscovery,wherecandidatedatasourcesare
subjectedtostatisticalanalysistoidentifypotential
anomaliesthatwouldcompromisetheleveloftrustinthe
finalinformationproducts.
11 BusinessRules Werefertotheseparationofbusinesslogicfromlogic
implementationasthebusinessrulesapproach.Thesimplest
waytodescribeabusinessrulessystemisasawell-
describedsetofenvironmentstates,acollectionof
environmentvariables,asetofformallydefinedrulesthat
reflectbusinesspolicies,preferences,guidelines,andsuch,
indicatinghowtheenvironmentisaffected,andas
amechanismforoperationalizingthoserules.
12 DataQuality Thischaptercentersontheimportanceofdefiningdata
qualityexpectationsandmeasuringdataqualityagainst
thoseexpectations.Wewillalsolookatthegeneral
perceptionofdataqualityandwhatthesavvymanager
needstoknowtodistinguishbetweendatacleansingand
dataquality.
(Continued)
xxiv Preface
TABLEP.1 LayoutoftheBook(Continued)
Chapter Title Description
13 DataIntegration Thenextstepistounderstandhowinformationcanbe
sharedacrosstheenterprise,bothflowingintoandoutofthe
analyticalenvironments.Thischapterdiscussesthebroad
spectrumofdetailssurrounddataextraction,transformation,
loading,aswellasdatavirtualization.
14 High- Thischapterlooksattheneedforhigh-performance
Performance analyticalplatforms,anddiscussesparallelismandother
Business approachesforhardware-basedorcommodity-basedBI
Intelligence appliances,aswellasalternatessuchasHadoop.
15 Alternate Thischapterconsiderstheaspectsofincorporatingexternal
Information datasetsandhowdatasetsareusedforenrichmentand
Contexts enhancement.
16 Location Thischapterlooksattheemergingbroaduseofgeographic
Intelligenceand informationinavarietyofcontextstoprovideactionable
SpatialAnalysis intelligencebasedonlocationintelligenceandspatial
analytics.
17 Knowledge Thischapterprovidesdetailsontheuseofdatamining
Discovery,Data techniquesandpredictiveanalyticstodeveloppredictive
Mining,and modelstohelpautomatedecision-makinginmany
Analytics operationalapplications.
18 Repurposing Inthischapterwelookatwaystoevaluateadditionaldata
Publicly sets,particularlyonesthatarepubliclyavailable(suchas
AvailableData government-provideddatasets)asawaytoenhanceyour
corporatedataasset.
19 Knowledge Thischapterlooksatmethodsfordeliveringbusiness
Delivery intelligence,rangingfromreports,adhocqueries,to
scorecards,dashboards,andmash-ups.
20 Newand Hereweprovidesomereviewandsomedeeperdivesinto
Emerging emergingtechnologiessuchastextanalytics,sentiment
Techniques analysis,ESPandcomplexeventprocessing,andsocial
mediaandnetworkanalytics.
21 QuickReference Thisfinalchapterofthebookisdedicatedasaquick
Guide referenceguideforselectedBIterminology.Eachimportant
topicmentionedinthebookisgivenanentryinthissection
insummarizedformforaquickrefresherofthenamedtopic.
Our Approach to Knowledge Transfer
As I have mentioned in the prefaces to both Master Data Management and The
Practitioner’s Guide to Data Quality Improvement, I remain devoted to helping
organizations strategically improve their capabilities in gaining the best advantage
Preface xxv
fromwhatmightbecalled‘‘informationutility.’’Mypriorexperiencesinfaileddata
managementactivitiesdrovemetoquitmylast‘‘realjob’’(asIliketosay)andstart
myownconsultingpracticetoprovethattherearebetterwaystoorganizeandplan
information-oriented program.
My company, Knowledge Integrity, Inc. (www.knowledge-integrity.com), was
developed to help organizations form successful business intelligence, analytics,
information quality, data governance, and master data management programs. As
awayofdistinguishingmyeffortfromotherconsultingcompanies,Ialsoinstituted
afewimportant corporate rules about theway wewould do business:
1. Ourmissionwastodevelopandpopularizemethodsforenterprisedatamanage-
ment.Asopposedtothecrazeforpatentingtechnology,methods,andprocesses,
wewould openly publish our ideas so as to benefit anyonewilling to investthe
time andenergytointernalize the ideas wewerepromoting.
2. Wewouldencourageclientstoadoptourmethodswithintheirsuccesspatterns.It
isachallenge(andperhapsinaway,insulting)towalkintoanorganizationand
tellpeoplewhohavedonetheirjobssuccessfullythattheyneedtodropwhatthey
aredoing andchange every aspect oftheway theywork. Webelievethat every
organization has its own methods for success, and our job is to craft a way to
integrate performance-based information quality management into the existing
organizationalsuccess structure.
3. Wewouldnotestablishourselvesaspermanentfixtures.Webelievethatinforma-
tionmanagementisacorecompetencythatshouldbemanagedwithintheorga-
nization,andourgoalforeachengagementistoestablishthefundamentalaspects
oftheprogram,transfertechnologytointernalresources,andthenbeonourway.
Ioftensaythatifwe doourjobright, weworkourselvesoutofacontract.
4. Wearenot‘‘sellingaproduct,’’weareengagedtosolvecustomerproblems.We
arelessconcernedaboutrigidcompliancetoatrademarkedmethodologythanwe
are about making sure that the customer’s core issues are resolved, and if that
means adapting our methods to the organization’s, that is the most appropriate
waytoget thingsdone.Ialsoliketosaythat weare successfulwhen theclient
comes up with ourideas.
5. Effective communication is the key to change management. Articulating how
good information management techniques enhance organizational effectiveness
andperformanceisthefirststepinengaging businessclientsandensuringtheir
supportandsponsorship.Wewouldinvestpartofeveryengagementinestablish-
ing a strong business case accompanied by collateral information that can be
socialized within and across the enterprise.
With these rules in mind, our first effort was to consolidate our ideas for
semantic, rule-oriented data quality management in a book, Enterprise Knowledge
Management–TheDataQualityApproach,whichwaspublishedin2001byMorgan
xxvi Preface
Kaufmann.Ihavebeentoldbyanumberofreadersthatthebookiscriticalintheir
development of a data quality management program, and the new technical ideas
proposedforrule-baseddataqualitymonitoringhave,intheinterveningyears,been
integratedinto all the major data quality vendorproductsuites.
The first edition of this book, Business Intelligence – The Savvy Manager’s
Guide,wasreleasedin2003,andhasbeennotedbymanyindustryexpertsasagreat
introduction to business intelligence. That was followed by Master Data Manage-
ment in 2009 and The Practitioner’s Guide to Data Quality Improvement in 2011.
Overthespanoftime,wehavedevelopedagraduate-levelcourseondataqualityfor
NewYorkUniversityandmultipleday-coursesforTheData WarehousingInstitute
(www.tdwi.org); presented numerous sessions at conferences and chapter meetings
for DAMA (the Data Management Association), and course and online content for
DATAVERSITY(www.dataversity.net);providedcolumnsforRobertSeiner’sData
Administration Newsletter (www.tdan.com) and monthly columns for DM Review
(www.dmreview.com);providedadownloadablecourseondataqualityfromBetter
Management (www.bettermanagement.com); and host an expert channel and
monthlynewsletterattheBusinessIntelligenceNetwork(www.b-eye-network.com)
andTechTarget(www.TechTarget.com).
Wearefrequentlyaskedbyvendorsacrossthespectrumtoprovideanalysisand
thought leadership in many areas of data management. We have consulted in the
publicsectorforbothfederal,state,andotherglobalgovernmentagencies.Wehave
guideddatamanagement projects inanumberofindustries,includinggovernment,
financialservices,healthcare,manufacturing,energyservices,insurance,andsocial
services, among others.
Since we started the company, the awareness of the value of information
management has been revealed to be one of the most important topics that senior
management faces. In practices that have emerged involving the exploitation of
enterprise data, such as Enterprise Resource Planning (ERP), Supply Chain
Management (SCM), Customer Relationship Management (CRM), among others,
there is a need for a consolidated view of high-quality data representing critical
views of business information. Increased regulatory oversight, increased need
for information exchange, business performance management, and the value of
service-oriented architecture are driving a greater focus on performance-oriented
management of enterprise data with respect to utility: accessibility, consistency,
currency,freshness, and usabilityof acommon informationasset.
Contact Me
While my intention is that this book will provide an overview of a business intel-
ligence and business analytics framework and program, there are situations where