The DAMA Guide to Data Management Body of Knowledge (DAMA DMBOK)

430 Pages·2016·4.91 MB·English

The DAMA Guide to
The Data Management
Body of Knowledge
(DAMA-DMBOK Guide)

First Edition
Mark Mosley, Editor – Development
Michael Brackett, Editor – Production
Susan Earley, Assistant Editor
Deborah Henderson, Project Sponsor
Published by:
Technics Publications, LLC
Post Office Box 161
Bradley Beach, NJ 07720 U.S.A.
www.technicspub.com/books.htm
All rights reserved. No part of this CD may be reproduced or transmitted in any form or
by any means, electronic or mechanical, including photocopying and recording, or by any
information storage and retrieval system, without written permission from DAMA
International, except for appropriately attributed quotations. No part of this CD may be
placed on a server, web page, or other electronic means for distribution to multiple
audiences.
The author and publisher have taken care in the preparation of this book, but make no
expressed or implied warranty of any kind and assume no responsibility for errors or
omissions. No liability is assumed for incidental or consequential damages in connection
with or arising out of the use of the information or programs contained herein.
Copyright © 2009 by DAMA International
ISBN, print ed. 978-0-9771400-8-4
First Printing 2009
Printed in the United States of America
Library of Congress Control Number: 2008912034
Dedicated to all the professionals contributing to development
of a data management profession.
Contents
Figures ............................................................................................................................. xiii
Tables ................................................................................................................................. xv
Foreword ......................................................................................................................... xvii
Preface............................................................................................................................... xix
Acknowledgements ........................................................................................................... xxi
1 Introduction ..................................................................................................................... 1
1.1 Data: An Enterprise Asset ........................................................................................ 1
1.2 Data, Information, Knowledge ................................................................................. 2
1.3 The Data Lifecycle .................................................................................................... 3
1.4 The Data Management Function.............................................................................. 4
1.5 A Shared Responsibility ........................................................................................... 5
1.6 A Broad Scope ........................................................................................................... 6
1.7 An Emerging Profession ........................................................................................... 7
1.8 A Growing Body of Knowledge ................................................................................. 8
1.9 DAMA–The Data Management Association ............................................................ 8
1.10 Purpose of the DAMA-DMBOK Guide ................................................................... 9
1.11 Goals of the DAMA-DMBOK Guide ....................................................................... 9
1.12 Audiences of the DAMA-DMBOK Guide ...............................................................10
1.13 Using the DAMA-DMBOK Guide ..........................................................................10
1.14 Other BOK Guides .................................................................................................11
1.15 The DAMA Dictionary of Data Management ........................................................11
1.16 The DAMA-DMBOK Functional Framework ........................................................11
1.17 Structure of the DAMA-DMBOK Guide ................................................................15
1.18 Recurring Themes ..................................................................................................16
2 Data Management Overview ..........................................................................................17
2.1 Introduction .............................................................................................................17
2.2 Mission and Goals ....................................................................................................18
2.3 Guiding Principles ...................................................................................................19
2.4 Functions and Activities ..........................................................................................19
2.4.1 Data Management Activities ............................................................................21
2.4.2 Activity Groups..................................................................................................24
2.5 Context Diagram Overview .....................................................................................27
2.5.1 Suppliers ............................................................................................................27
2.5.2 Inputs ................................................................................................................28
2.5.3 Participants .......................................................................................................28
2.5.4 Tools ...................................................................................................................28
2.5.5 Primary Deliverables ........................................................................................28
2.5.6 Consumers .........................................................................................................28
2.5.7 Metrics ...............................................................................................................28
2.6 Roles .........................................................................................................................29
2.6.1 Types of Organizations ......................................................................................29
© 2009 DAMA International v
DAMA-DMBOK Guide
2.6.2 Types of Individual Roles .................................................................................. 30
2.7 Technology ............................................................................................................... 30
2.7.1 Software Product Classes ................................................................................. 31
2.7.2 Specialized Hardware ....................................................................................... 34
2.8 Recommended Reading ............................................................................................ 35
3 Data Governance ............................................................................................................ 37
3.1 Introduction ............................................................................................................. 37
3.2 Concepts and Activities ........................................................................................... 37
3.2.1 Data Governance ............................................................................................... 38
3.2.2 Data Stewardship ............................................................................................. 39
3.2.3 Data Governance and Stewardship Organizations .......................................... 41
3.2.4 Data Management Services Organizations ...................................................... 43
3.2.5 The Data Management Executive .................................................................... 44
3.2.6 The Data Governance Office ............................................................................. 44
3.3 Data Governance Activities ..................................................................................... 45
3.3.1 Data Strategy .................................................................................................... 45
3.3.2 Data Policies ...................................................................................................... 47
3.3.3 Data Architecture ............................................................................................. 48
3.3.4 Data Standards and Procedures ....................................................................... 48
3.3.5 Regulatory Compliance ..................................................................................... 49
3.3.6 Issue Management ............................................................................................ 50
3.3.7 Data Management Projects .............................................................................. 52
3.3.8 Data Management Services .............................................................................. 53
3.3.9 Data Asset Valuation ........................................................................................ 53
3.3.10 Communication and Promotion ...................................................................... 54
3.3.11 Related Governance Frameworks ................................................................... 55
3.4 Summary .................................................................................................................. 56
3.4.1 Guiding Principles ............................................................................................ 56
3.4.2 Process Summary .............................................................................................. 57
3.4.3 Organizational and Cultural Issues ................................................................. 59
3.5 Recommended Reading ............................................................................................ 61
3.5.1 Websites ............................................................................................................ 61
3.5.2 Prominent Books ............................................................................................... 61
3.5.3 Regulatory and Compliance Books ................................................................... 62
3.5.4 General Books ................................................................................................... 62
4 Data Architecture Management..................................................................................... 63
4.1 Introduction ............................................................................................................. 63
4.2 Concepts and Activities ........................................................................................... 64
4.2.1 Architecture Overview ...................................................................................... 65
4.2.2 Activities............................................................................................................ 72
4.3 Summary .................................................................................................................. 80
4.3.1 Guiding Principles ............................................................................................ 81
4.3.2 Process Summary .............................................................................................. 82
4.3.3 Organizational and Cultural Issues ................................................................. 84
4.4 Recommended Reading ............................................................................................ 85
4.4.1 Books ................................................................................................................. 85
4.4.2 Articles and Websites ........................................................................................ 86
vi © 2009 DAMA International
Contents
5 Data Development ..........................................................................................................87
5.1 Introduction .............................................................................................................87
5.2 Concepts and Activities ...........................................................................................87
5.2.1 System Development Lifecycle (SDLC) ............................................................87
5.2.2 Styles of Data Modeling ....................................................................................89
5.2.3 Data Modeling, Analysis, and Solution Design ................................................90
5.2.4 Detailed Data Design ...................................................................................... 101
5.2.5 Data Model and Design Quality Management ............................................... 113
5.2.6 Data Implementation ...................................................................................... 116
5.3 Summary ................................................................................................................ 120
5.3.1 Guiding Principles ........................................................................................... 120
5.3.2 Data Development Process Summary ............................................................ 121
5.3.3 Organizational and Cultural Issues ............................................................... 123
5.4 Recommended Reading .......................................................................................... 124
5.4.1 Data Modeling and Database Design ............................................................. 124
5.4.2 Business Rules ................................................................................................ 126
5.4.3 Information Engineering ................................................................................. 126
5.4.4 Agile Development .......................................................................................... 127
5.4.5 Object Orientation and Object-Oriented Design ............................................ 127
5.4.6 Service-oriented architecture (SOA) ............................................................... 127
5.4.7 SQL .................................................................................................................. 128
5.4.8 Software Process Improvement ...................................................................... 128
5.4.9 XML ................................................................................................................. 128
6 Data Operations Management ..................................................................................... 129
6.1 Introduction ........................................................................................................... 129
6.2 Concepts and Activities ......................................................................................... 129
6.2.1 Database Support ............................................................................................ 129
6.2.2 Data Technology Management ....................................................................... 138
6.3 Summary ................................................................................................................ 144
6.3.1 Guiding Principles ........................................................................................... 144
6.3.2 Process Summary ............................................................................................ 144
6.3.3 Organizational and Cultural Issues ............................................................... 146
6.4 Recommended Reading .......................................................................................... 150
7 Data Security Management .......................................................................................... 151
7.1 Introduction ........................................................................................................... 151
7.2 Concepts and Activities ......................................................................................... 151
7.2.1 Understand Data Security Needs and Regulatory Requirements ................. 152
7.2.2 Define Data Security Policy ............................................................................ 153
7.2.3 Define Data Security Standards ..................................................................... 154
7.2.4 Define Data Security Controls and Procedures .............................................. 155
7.2.5 Manage Users, Passwords, and Group Membership ...................................... 155
7.2.6 Manage Data Access Views and Permissions ................................................. 157
7.2.7 Monitor User Authentication and Access Behavior ....................................... 158
7.2.8 Classify Information Confidentially ............................................................... 159
7.2.9 Audit Data Security ........................................................................................ 159
7.3 Data Security in an Outsourced World ................................................................. 160
7.4 Summary ................................................................................................................ 161
© 2009 DAMA International vii
DAMA-DMBOK Guide
7.4.1 Guiding Principles .......................................................................................... 162
7.4.2 Process Summary ............................................................................................ 163
7.4.3 Organizational and Cultural Issues ............................................................... 164
7.5 Recommended Reading .......................................................................................... 166
7.5.1 Texts and Articles ........................................................................................... 166
7.5.2 Major Privacy and Security Regulations ........................................................ 167
8 Reference and Master Data Management ................................................................... 171
8.1 Introduction ........................................................................................................... 171
8.2 Concepts and Activities ......................................................................................... 173
8.2.1 Reference Data ................................................................................................ 174
8.2.2 Master Data .................................................................................................... 176
8.2.3 Understand Reference and Master Data Integration Needs ......................... 180
8.2.4 Identify Reference and Master Data Sources and Contributors .................... 180
8.2.5 Define and Maintain the Data integration Architecture ............................... 181
8.2.6 Implement Reference and Master Data Management Solutions ................... 185
8.2.7 Define and Maintain Match Rules.................................................................. 185
8.2.8 Establish Golden Records ............................................................................... 187
8.2.9 Define and Maintain Hierarchies and Affiliations ......................................... 189
8.2.10 Plan and Implement Integration of New Data Sources ............................... 190
8.2.11 Replicate and Distribute Reference and Master Data ................................. 190
8.2.12 Manage Changes to Reference and Master Data ......................................... 190
8.3 Summary ................................................................................................................ 192
8.3.1 Guiding Principles .......................................................................................... 192
8.3.2 Process Summary ............................................................................................ 193
8.3.3 Organizational and Cultural Considerations ................................................. 195
8.4 Recommended Reading .......................................................................................... 195
9 Data Warehousing and Business Intelligence Management....................................... 197
9.1 Introduction ........................................................................................................... 197
9.2 Concepts and Activities ......................................................................................... 199
9.2.1 Data Warehousing—A Brief Retrospective and Historical Tour ................... 199
9.2.2 DW / BI Architecture and Components .......................................................... 201
9.2.3 Tactical, Strategic and Operational BI ........................................................... 208
9.2.4 Types of Data Warehousing ............................................................................ 209
9.2.5 Dimensional Data Modeling Concepts and Terminology ............................... 210
9.3 DW-BIM Activities ................................................................................................ 218
9.3.1 Understand Business Intelligence Information Needs .................................. 218
9.3.2 Define and Maintain the DW-BI Architecture ............................................... 220
9.3.3 Implement Data Warehouses and Data Marts .............................................. 222
9.3.4 Implement Business Intelligence Tools and User Interfaces ........................ 223
9.3.5 Process Data for Business Intelligence .......................................................... 230
9.3.6 Monitor and Tune Data Warehousing Processes ........................................... 231
9.3.7 Monitor and Tune BI Activity and Performance ............................................ 232
9.4 Summary ................................................................................................................ 232
9.4.1 Guiding Principles .......................................................................................... 232
9.4.2 Process Summary ............................................................................................ 233
9.4.3 Organizational and Cultural Issues ............................................................... 235
9.5 Recommended Reading .......................................................................................... 235
viii © 2009 DAMA International
Contents
9.5.1 Data Warehousing ........................................................................................... 235
9.5.2 Business Intelligence ...................................................................................... 237
9.5.3 Data Mining .................................................................................................... 237
9.5.4 OLAP ............................................................................................................... 238
10 Document and Content Management ........................................................................ 239
10.1 Introduction ......................................................................................................... 239
10.2 Concepts and Activities........................................................................................ 240
10.2.1 Unstructured Data ........................................................................................ 241
10.2.2 Document / Record Management .................................................................. 241
10.2.3 Content Management .................................................................................... 248
10.3 Summary .............................................................................................................. 253
10.3.1 Guiding Principles ......................................................................................... 253
10.3.2 Process Summary .......................................................................................... 253
10.3.3 Organizational and Cultural Issues ............................................................. 255
10.4 Recommended Reading ........................................................................................ 255
10.4.1 Document / Content Management ................................................................ 255
10.4.2 Records Management .................................................................................... 256
10.4.3 Enterprise Information Portals .................................................................... 256
10.4.4 Meta-data in Library Science ........................................................................ 256
10.4.5 Semantics in XML Documents ...................................................................... 257
10.4.6 Unstructured Data and Business Intelligence ............................................. 257
10.4.7 Standards ...................................................................................................... 257
10.4.8 E-Discovery .................................................................................................... 257
11 Meta-data Management ............................................................................................. 259
11.1 Introduction ......................................................................................................... 259
11.2 Concepts and Activities........................................................................................ 260
11.2.1 Meta-data Definition ..................................................................................... 261
11.2.2 Meta-data History 1990 - 2008 ..................................................................... 266
11.2.3 Meta-data Strategy ....................................................................................... 267
11.2.4 Meta-data Management Activities ............................................................... 268
11.3 Summary .............................................................................................................. 284
11.3.1 Guiding Principles ......................................................................................... 284
11.3.2 Process Summary .......................................................................................... 285
11.3.3 Organizational and Cultural Issues ............................................................. 287
11.4 Recommended Reading ........................................................................................ 288
11.4.1 General Reading ............................................................................................ 288
11.4.2 Meta-data in Library Science ........................................................................ 289
11.4.3 Geospatial Meta-data Standards .................................................................. 289
11.4.4 ISO Meta-data Standards ............................................................................. 289
12 Data Quality Management ......................................................................................... 291
12.1 Introduction ......................................................................................................... 291
12.2 Concepts and Activities........................................................................................ 292
12.2.1 Data Quality Management Approach ........................................................... 292
12.2.2 Develop and Promote Data Quality Awareness ........................................... 294
12.2.3 Define Data Quality Requirements .............................................................. 295
12.2.4 Profile, Analyze and Assess Data Quality .................................................... 297
© 2009 DAMA International ix
DAMA-DMBOK Guide
12.2.5 Define Data Quality Metrics ......................................................................... 298
12.2.6 Define Data Quality Business Rules ............................................................ 300
12.2.7 Test and Validate Data Quality Requirements ............................................ 301
12.2.8 Set and Evaluate Data Quality Service Levels ............................................ 302
12.2.9 Continuously Measure and Monitor Data Quality ....................................... 303
12.2.10 Manage Data Quality Issues ...................................................................... 303
12.2.11 Clean and Correct Data Quality Defects .................................................... 305
12.2.12 Design and Implement Operational DQM Procedures .............................. 306
12.2.13 Monitor Operational DQM Procedures and Performance .......................... 307
12.3 Data Quality Tools ............................................................................................... 307
12.3.1 Data Profiling ................................................................................................ 308
12.3.2 Parsing and Standardization ........................................................................ 308
12.3.3 Data Transformation .................................................................................... 309
12.3.4 Identity Resolution and Matching ................................................................ 309
12.3.5 Enhancement ................................................................................................ 311
12.3.6 Reporting ....................................................................................................... 312
12.4 Summary .............................................................................................................. 312
12.4.1 Setting Data Quality Guiding Principles ..................................................... 312
12.4.2 Process Summary .......................................................................................... 313
12.4.3 Organizational and Cultural Issues ............................................................. 315
12.5 Recommended Reading ........................................................................................ 316
13 Professional Development .......................................................................................... 319
13.1 Characteristics of a Profession ............................................................................ 319
13.2 DAMA Membership ............................................................................................. 320
13.3 Continuing Education and Training ................................................................... 321
13.4 Certification ......................................................................................................... 321
13.4.1 How Do You Obtain a CDMP? ...................................................................... 321
13.4.2 CDMP Examination Criteria ........................................................................ 322
13.4.4 CDMP Qualifying Examinations .................................................................. 323
13.4.5 Accepted Vendor Training Certifications ..................................................... 323
13.4.6 Preparation for Taking Exams ..................................................................... 325
13.4.7 Taking CDMP Exams ................................................................................... 325
13.4.8 Professional Development / Recertification .................................................. 325
13.5 Professional Ethics .............................................................................................. 328
13.6 Notable Data Management Professionals ........................................................... 332
13.6.1 Lifetime Achievement Award ....................................................................... 332
13.6.2 Professional Achievement Award ................................................................. 332
13.6.3 Government Achievement Award ................................................................. 333
13.6.4 Academic Achievement Award ..................................................................... 333
13.6.5 DAMA Community Award ............................................................................ 333
Afterword ......................................................................................................................... 335
A1 Data Management Suppliers ..................................................................................... 337
A2 Data Management Inputs .......................................................................................... 339
A3 Data Management Participants ................................................................................ 341
x © 2009 DAMA International