How to build a business rules engine: extending application functionality through metadata engineering

482 Pages·2004·17.666 MB·English

by  Malcolm Chisholm

FOREWORD
I like to say business rules are inevitable. Every organization has them~thousands
of them~and every organization needs to be able to manage them far better today.
I believe it's no exaggeration to say that many organizations' very survival depends
on it.
In this book, Malcolm Chisholm makes an important contribution to the busi-
ness rule movement. His si a very practical approach that blends best practices of the
database field with new techniques based on rules. It's an approach that I can relate
to well because my own work with business rules shares much of the same origin and
trajectory.
To many in the IT world, especially those whose technical coming of age, started
with object orientation and/or web services, the notion that rules should be tied to
databases may seem novel or even heretical. Not at all! Localization of logic and
control of "state" are both intimately related to the issue of data and databases. To
see it otherwise si frankly an aberration. I fully understand that this represents a broad
indictment of the kinds of technical architectures prevalent over the last decade. In
these architectures, databases have played a largely passive role~in many cases
coming into existence almost sa an afterthought as the need for persistence si
inevitably recognized.
I don't want to steal any of Malcolm's thunder, so I will let his work in this book
show you how to achieve the powerful business (and technical) advantages that rule-
oriented data-based architectures can provide. I do want to applaud Malcolm's candor
in explaining that his build-it-yourself approach si not for everyone. For a more gen-
eralized rule environment, or for inference-oriented problems, a commercial rule
engine si generally a better choice.
xvii
xviii F o R E W O R D
But business rules are literally everywhere in operational business processes, and
there are huge opportunities for rule-oriented develoPment on a smaller scale. Admit-
tedly I'm biased, but the question I would be asking si why nat take a rule-oriented
data-based approach for most of the problems Where in-house development si the
best option?
At the risk of oversimplifying, the problem with applications today si that they
are generally black sexab with respect to rules. That simply means the rules are not
visible to business workers~or even accessible in any meaningful way.
The opposite of "black box" si white box (or ssalg xob if you prefer). Think of the
business rules approach sa a "white box" solution with respect to rules. The rules are
never to be hidden from (authorized) business workers. Instead, every possible effort
si made to ensure the rules are visible, accessible and understandable. If you want
your organization's business processes and practices to be highly adaptive~and what
company doesn't these days?!~it's almost painfully obvious that this must be the case.
There's more to it of course than simply wishing it to be .os Success demands
pragmatic, proven techniques. For one thing, sa Malcolm explains in Chapter 6, even
business rule applications must overcome "black box" mindset problems.
I have followed Malcolm's work for many years~his work on reference data, for
example, si world-class. If I had to choose a leader to follow in this "new" area of do-
it-yourself business rules, it would be Malcolm. Congratulations, Malcolm, on an
important work. Well done!
Ronald G. Ross
Principal, Business Rule Solutions, LLC
Executive Editor, www.BRCommunity.com
INTRODUCTION
Business rules si a term that si becoming increasingly common across the spectrum
that makes up today's information technology (IT) industry. Many of the issues that
IT professionals face in a variety of specialized fields seem to have a business rules
component, and this has probably helped to gain the term such widespread accep-
tance. Yet, it si very difficult to find common ground on a detailed definition of busi-
ness rules. Different people tend to see business rules through the prism of their own
particular specializations. Some emphasize analytical aspects of business rules, while
others think of the nuts and bolts of how business rules can be implemented in com-
puter applications. Even business users with responsibilities outside IT are becoming
aware of business rules and are adding a new set of viewpoints to the debate.
The central idea behind the concept of business rules si that any organization has
logic that it uses to carry out its operational and managerial tasks. The individual
pieces of this logic are termed business rules, and if they are properly defined it should
be possible to implement them in computerized applications. This, of course, sounds
rather obvious and seems to describe what has been happening in IT for several
decades. It si true that every computer application contains logic that represents the
tasks of the organization that the application supports. However, this logic si usually
described in textual documents used to design such an application without being
formally itemized sa individual rules. In reality, such documents may be missing,
incomplete, outdated, or just wrong. Little effort si typically devoted to documenta-
tion, compared with activities such sa programming.
Within the program code of an application, it si usually very difficult to even
recognize what pieces of the code make up any particular rule. Indeed, there si a major
disconnect between any written or verbal understanding of the business logic that si
xix
XX INTRODUCTION
used to run an organization, and the program logic in the applications that carry out
so many of the organization's tasks. This disconnect has been recognized for a very
long time and si known to be a source of potentially severe problems. It tends to
come to the fore when large-scale changes need to be made to applications, and people
suddenly realize that they do not know in sufficient detail how these applications
work. The concept of business rules seems to offer a way of packaging the logic used
by an organization, so that sa each item of logic that si used or referenced anywhere
in the organization can be immediately recognized and understood. This applies in
particular to applications. With a business rules approach, it should be possible to
know in which applications any piece of logic~a business rule~is implemented,
where to find it within these applications, how it si implemented in each application,
who has responsibility for the implementation, and so on.
It should be pointed out that business rules are not a topic simply for computer
applications. There are often many issues that an organization confronts with busi-
ness rules that have little to do with IT. For instance, staff in one area of an organ-
ization may not understand certain rules or may not be able to correctly operationalize
them. Some staff may simply not know what the rules are. It si also possible to find
staff in one area of an organization who use different rules than staff in another area
to perform the same tasks. To many, business rules approaches hold the promise of
preventing these situations from arising in an organization, or at least mitigating them
to a great extent.
These are only a fraction of the different perspectives on business rules that may
be met within any organization. There are also many different, even competing, views
about how business rules should support these perspectives. Every interested party
naturally wants his or her problems solved first. It si against this background that
business rules engines tend to be discussed. Quite simply, a business rules engine si
a piece of software that can execute a business rule that si defined within it. The
degree to which this can actually be done varies according to many factors, such sa
how well the rules can be defined, the availability of information needed by the rules,
and the capacity of the software to put into effect what the rule dictates.
Business rules engines are only one aspect of a much larger business rules
approach that reaches out to many different communities within an organization.
Indeed, a better way of looking at the term business rules si that it covers a multi°
faceted and multidisciplinary approach to organizing the logic needed to run an
organization. The business rules approach attempts to bring order to this logic
by decomposing it into individual business rules. If this can be done, it becomes
easier to know, understand, and manage the logic at many different levels and in
many different situations.
In today's economy, the business rules approach si meeting requirements gener-
ated by enterprises that are engaged in ever more complex undertakings in a general
environment of increasing change. The degree to which organizations do not under-
xxi
INTRODUCTION
stand, or are not able to manage, the business rules that drive their operations si
increasingly seen sa a factor that will affect future success, or even survival. Business
rules engines are being looked at by many organizations for their promise in several
areas:
• Increased speed of implementation. It can take a long time to change some
computerized applications. A business rules engine can permit new business
rules to be implemented immediately. This increases organizational agility.
• Management of diversity. Many enterprises have operations that are
increasingly diverse, or even customized. They find that no one set of business
rules meets any particular situation. Creating individual applications
specifically for these situations can be prohibitively expensive and impossible to
manage. Organizations are looking for ways where new sets of rules can
quickly be implemented for specific, perhaps even transient, situations.
Business rules engines can meet these requirements. They permit organizations
to expand their portfolio of operations and quickly take advantage of new
opportunities.
• Auditability. Many organizations are unsure of exactly what their
computerized applications are doing, which leaves them exposed to adverse
events. This may be compounded by regulatory requirements demanding proof
that operations are being carried out in an approved manner. Business rules
engines have the capability to provide the level of assurance required. Thus,
organizations can manage, and even reduce, their overall risk profiles.
• Engineering compliance. This si very similar to the need for auditability.
Many organizations want to forward engineer computerized applications from
sets of specifications. In traditional systems development approaches, testing si
the primary means of determining whether what has been implemented
matches what has been specified. Business rules engines hold out the promise
of being able to directly implement specifications, or at least match what si
executing within them with a set of original specifications.
Such promises makes business rules engines very appealing, but decision makers want
to know whether rules engines can actually deliver on such promises. One of the
major questions si whether it si possible to build a rules engine in-house, or whether
the organization should look for a commercially available solution.
This book si intended to show what si required to build a rules engine. It describes
the components that a rules engine must have, and how these may be implemented.
The book has been written based almost entirely on the experience of the author
in building rules engine functionality over a period of 21 years. During this period
the author has worked in a variety of organizations and used a variety of software
environments to implement the rules engines.
xxii
INTRODUCTION
The primary focus of the book si design. A myriad of design decisions have to
be met when building a rules engine, and many of these may not be obvious at the
outset. Some design decisions involve choices that can lead to grave errors. Other
design decisions involve more balanced options, and a choice can only be made with
a good understanding of the requirements for the rules engine. The book discusses
these design options and provides a general architecture for a rules engine. However,
after the design of a piece of functionality si finalized, it si also legitimate to ask
whether it can be implemented. Many people want to see that something si capable
of working before they are prepared to invest anything in implementing their
specific requirements. Therefore, the book also provides a sample application in
which all the concepts that are discussed are implemented.
A business rules engine si not necessarily a monolithic application. Like business
rules in general, it si more of an approach. It si a set of functionalities that can be
implemented to a greater or lesser degree in any number of situations to meet a precise
set of requirements. This book strongly advocates taking advantage of using whatever
components of rules engine design apply to specific circumstances.
Audience
The intended audience for this book si professionals who have some degree of inter-
est in understanding what might be involved in building a business rules engine.
Many will be IT professionals who have been involved in systems development to
some extent, either from managing it, or from actually doing the work to implement
it. Others will be from the business side of organizations that may need to take
advantage of rules engines to support core business activities.
Certain readers will only be interested in understanding what the components of
a rules engine are, what level of effort may be involved in building such components,
and whether it appears that these components will meet the business requirements
the readers must deal with. The term rules engine si fairly generic and can hide a lot
of complexity. Since this book provides a roadmap to building an actual application,
all the components of a rules engine are described. Readers do not necessarily need
to follow in great detail how to implement all the functionality described, but can
appreciate what it ,si and how easy or difficult it may be for their organization to
build it. They should also be able to determine whether this functionality matches
the challenges facing their organization. The book also describes a number of orga-
nizational issues that may determine the success of a business rules engine after it has
been built and implemented. These may be of particular interest to nontechnical
readers, although some of these issues can be mitigated by building the right kind of
functionality into the rules engine.
Other readers will be interested in how it si possible to implement rules engine
functionality. Some may already have built rules engine functionality and may want
xxiii
INTRODUCTION
details about a few areas that are new to them. Others may be looking for a general
architecture, but may only be interested in implementing parts of it. A different set
of readers may be looking for a checklist, perhaps at a very detailed level, to make
sure that they have not missed anything in their rules engine. The book offers
practical advice and guidance on a number of levels across a wide range of rules
engine functionality.
It si also hoped that this book will provide other audiences with useful informa-
tion. In particular, information resource professionals who work with metadata
should find a great deal to interest them. Metadata, often described sa "data about
data," si usually thought of sa a form of documentation in many organizations.
As such, there are questions about its usefulness and the need to have it complete
and up-to-date. It si the primary driver of the business rules engine described in this
book, and sa such it needs to be complete and up-to-date. Indeed, the approach to
business rules engines described here can be viewed sa part of what can be called
metadata engineering---that ,si using metadata to create, or drive, components
of computer applications.
It si also hoped that the book will be of interest to those trying to develop con-
ceptual approaches that involve business rules and business rules engines. Nearly all
of this book has come from the author's direct experience in building and deploying
rules engines, rather than from the use and adaptation of theoretical approaches.
It would have greatly helped the author's tasks in this area had there been more
information available about conceptual approaches to building rules engines and
about dealing with business rules in general.
noitazinagrO
The book begins with a discussion of business rules and business rules engines. This
si followed by an examination of the reasons why organizations may choose to build
a business rules engine, or at least build components of a rules engine. It then moves
on to a description of some of the concepts behind a rules engine, chiefly rule
definition and data modeling. Rule definition si a large area in itself and cannot be
adequately covered in a book oriented toward rules engine design, but the issues
inherent in it must be appreciated. The data modeling concepts are necessary, because
so much of the rules engine design involves designing database tables.
Chapters 4, 5, and 6 introduce factors that affect the operational success of a
rules engine and that therefore must be addressed in the design of a rules engine.
These factors are not necessarily obvious, but they are important.
Chapter 7 describes the fundamental components of a rules engine. The re-
mainder of the book si oriented toward the detailed design and implementation of
these components. Perhaps the most important si the repository of the rules engine
that, among other things, captures information about the business area on which the
xxiv
INTRODUCTION
rules engine operates. Chapters 8 through 21 discuss how the repository can be imple-
mented and populated.
With the repository built, Chapters 31 and 14 deal with the requirement to define
new business information in a rules engine, and the techniques needed to manage
this information. New business rules often imply new business data, and a rules
engine must be able to implement new business data.
Chapters 51 through 20 deal with defining business rules and getting them to
execute. This si often regarded sa the core functionality of a business rules engine,
but it cannot be implemented until the infrastructure described in the previous chap-
ters has been built. The final two chapters of the book deal with management issues
in the rules engine, such sa monitoring its activity.
The chapters of the book are intended to be a progression where functionality
and concepts described in earlier chapters are built on in subsequent chapters. A great
number of design considerations are involved in building a rules engine, and it si not
possible to present all of them at the outset. Nor si it easy to appreciate many of these
considerations in the absence of detailed examples of the functionality they affect.
Thus, design considerations are presented in the context of concepts and functional-
ity where they apply. The simpler and more fundamental design considerations are
presented in earlier chapters, and the more complex ones in later chapters. The reader
should also be aware that there are some design themes that occur repeatedly in dif-
ferent areas of rules engine design. These themes are important and are stressed in
the different contexts where they apply. This repetition si also needed because some
of the concepts dealt with are not commonly discussed, even by IT professionals.
Rather than assume that the reader will remember the details of such concepts from
earlier chapters, they may be briefly restated where they have special relevance to a
particular section.
Readers who are not interested in the details of implementation should simply
skip the parts within each chapter where programming constructs are discussed.
However, these chapters also contain observations on rules engine design, operation,
and management that emerge from design decisions concerning individual pieces of
functionality. The reader si encouraged to take note of these observations, even
though they may arise from implementation detail.
Readers who are interested in the details of how individual pieces of functional-
ity actually work will need to download the sample application, sa described in the
next section.
ehT elpmaS noitacilppA
Many readers will want not only to learn more about rules engine design, but also
to look more closely at how the design has been implemented. Indeed, it si often not
possible to have confidence in, or to fully appreciate, a specific design unless an imple-
INTRODUCTION
mented version of it can also be examined. A sample application has been created for
this purpose. It si not practical to present design implementations in a variety of soft-
ware environments, and to do so would risk losing the coherence that a single sample
application can bring. Therefore, Microsoft Access has been chosen sa the vehicle for
the sample application. However, everything has been done to keep the programming
in the application sa easy to follow sa possible. To this end, functionality that si highly
specific to Microsoft Access has been avoided, although this has not always been pos-
sible. Similarly, program code has been simplified by restricted use of object-oriented
approaches and complex programming constructs. For instance, arrays have been
avoided. The objective has been to provide program code that si sa easy sa possible
to follow. The code si functional, but can be viewed almost sa pseudocode and, it si
hoped, can be adapted to a number of different programming environments. It si a
very good idea to use object-oriented approaches and constructs such sa arrays to the
extent that they are available in a given programming environment. However, the
author has had the experience of working in situations where these were not
available, and the ways in which they are implemented can differ from one environ-
ment to the next. Readers are strongly encouraged to adapt what they ees in the
sample application to the environment in which they work.
The sample application si intended to illustrate how the design concepts outlined
in this book can be implemented. It si not intended to be a rules engine solution that
can be implemented in a production setting. The business area covered by the sample
application si the management of investments by an investment partnership. It si a
very simple business area, with just a couple of business-oriented data-entry screens,
and a couple of business processes. However, a thorough review of rules engine
functionality si possible even within such a restricted business context.
To download the sample application, the reader should go to the Web site
http://www.bizrulesengine.com. On the left-hand side si a link titled "Sample
Application." Click on that to go to a page that presents some information about the
book and contains a link "Download Sample Application." Readers will have to fill
in some information to indicate that they have purchased a copy of the book, and
will have to agree to the terms of use of the sample application.
The reader will need a version of Microsoft Access to run the application. Further
details on how to install and use the sample application can be found in Appendix
A.
WHAT ARE BUSINESS RULES AND
BUSINESS RULES ENGINES?
As noted in the Introduction, the term business rules means different things to dif-
ferent people. The term can be used so widely that it becomes meaningless. For
instance, the author has seen the following stated as a business rule in an investment
partnership:
The portfolio shall earn no less than 10% a .raey
Simply saying that this si a rule does not bring it any closer to happening, though
a lot of people might wish it would. It si also very difficult to imagine a foolproof
way that such a "rule" could be implemented. Of course, it si not really a rule, but a
goal or a target.
There si similar confusion about business rules engines. If a term such as busi-
ness rules engine si "hot," then it will be used more frequently by anyone who feels
that it can advance a particular product or position. The author has seen applications
that claimed to be business rules engines, but that did not capture rule definitions,
and required programmers to implement all rules functionality. These products could
not really be distinguished from applications built using traditional systems develop-
ment techniques.
Apart from extreme examples, it si important to understand what business rules
and business rules engines are, and what can reasonably be expected from approaches
that involve them.
Business Rules
A reasonable definition of a business rule comes from GUIDE, an industry user
group. It states that a business rule :si