BSTJ 60: 2. February 1981: The Quality Measurement Plan (QMP). (Hoadley, Bruce)

24 MB·English

con Ao aa i aug ee

The Quality Measurement Plan {QMP)

By SRUCE HOADLEY
(ohare raatved August 8, 1980)

This paper describes the Quality Measurement Plan (93r)
recently implemented system far reporting the quality asturance audit
renutt to Dell System masnngeruent. ue replaces the Tate aystem,
lehich evoled from the piomenring statistical cork of Shewhart and
Dodge ioing the 1800's and 1830% at Bell Laboratories. Hox and
lwhisher plats are used for graphically daplaying confidence intereals
for Due quality ofthe current production. The confidence interval ix
‘computed from both current and past data and is derived fram a new
Bayesian proach {0 te empirical Bayes problem for Poisson
‘observations, Here wsedlineuss the rationale, mathematical deriva
tions, dynamicn, operating sharacteristies, and many comparative
‘examples, We show thal @ue reduce statistical ervara relative to the
earlier T rate systom

|. wrmooueTION
11 Quality teurance

"The remnonsbiity ofthe Hell Lalirnories Quality Assurance Center
(eae) ito ensure tha he rommunieations reducta designed by Hell
Laboratories and bought by Hell Syatem operating compan from
Western Elecerie Company, Incoryorated wil mee! quali andana
‘and will parform as the designers ileewed." This abvites the need
for aach cperting sunjony to carry outite onm acceprance inspection,

"To meet this ressnnailiity, the qac works witha Western Hlectric
(WE) agents, the Quality Assurance Dizectorato(QAD], and Purchased
Procuets Tnapeecion ret) organizations, However, as elated in Het 1,
"The prewary responsibility fr quality lcs withthe lin organizations
Bel Laboratories for the quality of design and Western Plectre far
‘he quality of manufacture, insinlaion, and repair” The quality
assurance organizations conde independent activities to amare qual:
ity to the operating companies

ais
1.2 uatty eseurance sult

“Tho quality nesuranve organlzations have two major activites. "The
fina into conduel quality dite where preducts change ends, cither
within WE oF hetteen WE and the operating companies, Examples
remanufacturing, installation, and repair uudits. The second concerns
‘eullection of Bold quality monioring activities, Feamples are the
Product Performance Surveys Thee are designed sample surveys of
repnited fel troubles

"Aa sud ia a highly atrcmared system af inepoctions done on 8
sampling basis ‘The ingevlients of an and are: (0) aarapling method,
(Gi) seope uf inwpertion, Gi) quality standards, (ie) nonconformance
rocedres, (e) defect assessment practices, (1) quality rating method,
2nd (oi) report formats.

‘The sampling method slong wich the scope of inapecton determines
‘what tata will be porformed on what unita of product or atrbates of
product, The staliior and economics of sampling, the engineering
Fequltement, and the eld effect of defects play the central roles in
Aolermining the sazpling ad the scope of inepection,

‘The qualizy standards are numerical values exprosscd in defects
Aefoctives or demmerta por unt, They eset by Une qac in coal ion
trith che quo, The standard are Larget values, vellecing & UradeofT
‘tween fr! vost and maintenance caste

‘The nonconformance procedures are rules for detocting and dispos-
iny of audited lots thac are excessively dafective with respect to a
artivular eel of engineering requirement,

"The defect astessment practices are a0 of transformations tha
imap defect found into defects uasesel for quality eating puree. A
terminal rrip may have all ven connections off By one postion, but,
the consequences of thee ten defects found are much less than ven,
independent accurrance of this defect. Therafore, ese than ton defocts
ste assented.

"The quality rting method and report formuts determine how the
resulta of the audit ae presented to Bell System management, For
fexample, a product i reported w "Below Normal," when It fll a
‘Ratiteal tet of the hypothe tha Une quality landed ix elng met

1.2 The quatty measurement plan (MP)

were developed
bay Showhar, Podge, and other, searting in the 1990's and continuing
through to the middle 1980’. ‘This wark seas dacumented in the
eerature in Rafe, 3 6

Tn recent yours, rearch hat boon carved out wo evaluate the
pplication of modern staniel eories vo the sud ingredients, An
innportane idee it eummaarized in an arile by Eftoa and Moris which

216 THE BELL SYSTEM TECHNICAL JOURNAL, FEBRUARY 1961
explains a paradox discovered by Stein! When you have samples from
‘imsilar populations, dhe individual sample charactaisti are not the
Tet estimates of the individual population characteristics. Tocal error
{i redaced by sinking the individual sumple charncterisica part way
towards the grand meen over all sunples, Efron and Morris used
‘baseball hating averages to lustrato the point. Dut the problem of
estimating percent defective in quality assurance is the same problem.
‘And you ave always concerned with similar populations—for example,
‘the population of design line telephones produced for cach of several
month,

"This idea was orginlly explored in Rf. 9. Thelen has ow evolved
into the Quality Mearurement Plan (22) qu is Use recently imple
‘mental system for conducting ehree ofthe audit ingredients: defect
fzeceument, quality rating, and quality reporting

‘Asn quick introduction to qu, consider Fig. Thisis a comparison
cof the aur reporting format (Fig, 1a) with the old T-xate reporting
forroat (Pig. Th. Hach yesr is divided into sight periods. On the
Vollom, the Tirate ie plotted for each period and it meamuree the
liffrence between the observed and standard defect rates in unis of
‘sampling standard deviation (given standard quality). Th ia is hat
if the Prato las than negncive Ewe, then the hypothe oF
standard quality i rected. Bection TT considers the exact res for
‘exception reporting under the Toran syste,

‘Under qu, a box tnd whisker chert is plttad exch period, ‘The box
chart is a graphical representation of the posterior distribution of
‘eurent population quality on an index scale. The index value one is
‘the wandont on the Index scale and the value ewo means tice at
many deferie sf expected under the quality standard. ‘The posterior
‘probability thatthe population index Inger than the top whisker is
0.99, The top of the box, thy holiom of the bux, and the bottom
‘whiskor correspond to Une probabilities 0.95, 005, and 0.0, respee-
tively,

“The heavy “dot” isa Bayesestimate ofthe lang run process avert
the “cron dhe observed value in the current aap; wd ne “dash
hear the middle of the box i the posterior mean of the current
‘ovulation index and is ealled the Leet Measure of current quality.
‘The process averages, “dot,” nee joined ta show trond.

‘Alenough the T-rala chart and Uhe Quer chart often convey soar
rereanen (hore aredilferencen ‘The QUE chart providen « measure of
‘ality; che T-rate chart dose not. For example, in 7806 (Period 6 of
1978) both chares imply chat the quality is substandard, but the axe
‘har ale implies tet the population indox is somewhere hetween one
land evo.

'aMP and the Pato use the past data in very different ways. ar
Natt eg Feel fon oe fib ae esaw ir the

‘ona the past sample indices, bt males an inference about current
‘duality, ‘The T:rale system wxen tune criteria based on arcibuces of
the Pirate, such ns "Tew hon sere" and ean make an inference about
past quality. In Fix, 2, 7707, the Tate signals an exception, because
Se Terace in a ow ae les than 20r0 indicating that quality has not

‘218 THE BELL SYSTEM TECHNICAL JOURNAL, FEGRUARY 1861
ben standard forall six periods, But for qui, the standard is well
wwchin the box, indicating normal current quality. To different treat-
ene of past data ie also instead In Fig. 1. Comparing 7708 with
ADA reveala very similar T-rtes, bt quar box chares with different
meats.

"The T-rate systema based on the msurnption that the ial number
‘of dlfeca ina rating period bas a normal disribucion. qu is bated on
‘the Poisson dietibution, This difference ie imporcant for srall audi,
asshown in Section VIL

‘que mas on tral for pwo yeaes and was applied to 2,000 sets of
auc data. ‘The relalively simple qr algorithm published in Ret 9

cca bs ae sa finials vine irae Fer gur trough:
Piedocion ealy, own Oegh pat tats eds en

QUALITY MEASUREMENT PLAN 210
‘was used originlly. This simple algorithm worked for most data sta,
‘but not all (eg, zero defects in every period), The relatively complex
algorithm dissed in Section 1V is the result of a length fne-coning
roots, designed co make che algorithm work for every cso, This is
‘why the ful power of Bayes theorem with empirically based prior
Gistributions had to be used.

144 Retatonship tothe empirical Bayes sporazch

Note that in the GMP box chart, the Best Messure always les
between the elimaled process average and the curren: sample index
‘The Best Memare ie a shrinkage of tho sample index cowards the
csimated process average, ln 706 of Fig. I, he shrinkage is ose
‘ona standard; bu, in 7504, iin toward Ue starr

"The Dest Moasure is related o the clas of estimators deceibed by
‘stron ancl Morrie Tn the eited reerence, they provide a foundation
for Stein's parades with an empirical Hayes approach, In Hef. 7, they
‘aud Buse data Lo lucrate Sean's paredox, There isa clear analogy
between percene defective in a quality nesurance epplication and a
tbasetallbatcing average, ‘The doea in Ref. 7 was for many players at
2 given poine in time. The qu algorithm works with the data for one
product over Lime. so a beter baseball analogy would be one player

‘Tah T concains bacting average dats for Thurman Munson from
1970 chron 1998. This daca was eollected and analyzed by S. G.
CCenwford nd is displayed graphically in Mg. 3. The "erosser™ are
“Munsen's batting averages Yeported on the ln Sunviny of Apri for
fich year. The "boxes" are Munson’s baling averaye athe end of
the seavon, The dashed line i the average ofthe “erase,

‘he early season vragen are analogous to the audit data. The
verges are the rule from small ample of che populations ‘The
populations are the fnice populations of “a bats” foreach eoason. In

‘Table Batting average data for Thurman Munson
Teo Lat at otoeamn ue

Seay in ho Pan
how Bag Ealing

220 THE BELL SYSTEM TECHNICAL JOURNAL, FEBRUARY 1981
it RESINS nae

aj nee Ta uo rn
ales ahs tana desea Ae aie
EASES CG ata meaner

the audi, we are interested in making a statistical inference exch
peried about the curvent pepalation. So our problem in Fig. 9 is to
Inake e slatstienl inferenes each yoar about the season balling average
‘sing ony the early sewn nverages observed to dace.

‘As an estimate, one would be tempted co use the maximum like
lihood estimate, Use early 2eazen average, But, in every year, the
movement of the batting average from the wily senso I Une season
‘end it in the dizection of the agerozal eal season average overtime.
50, paradoxically, the early soon averages from other yeats seem to
be relevanc to the current seman average. Ici clear fom the date,
that a beter estimate of the venwon average ia some kindof hrinkage
of the early season nverage towards the aggregate early season average
over time And the ameunt of shrinkage ean depend only on the
fallble dnta-the early seacon averages

"What we teally have here is muliverinte problem. We observe a
rine-dimensional vector of obsorvations whose mean it a vector of
population characteristics, onc of which we are particularly incerested
fn. Seein® showed (or Une normal distribution) that che maximum
UHlelihood estimate of the vec ie inadmiable. Why’ this ia true
smanifeetsitelf in boseball lore. A player that stats the season rol

‘QUALITY MEASUREMENT PLAN 221
tively hot, tually cools of and a player thet starts in a relative stump
‘usually improves, Th s due to the nature of sampling error. The hot
is usually partially lcky and the slumping player ia uavally

ny in Fig a thatthe Bast Measure ofthe popolation index ia between
the curzent observed index and the eetimatd long-run process average.

"The approgch used for gu is actually Bayesian empizial Bayes,
‘The shrinkage factor usod i « Bayes estimate ofan optimal shrinkage
factor. So the Best Mesure ha the form

[estimated feurzeni
W process | + (1-7) |eampte
Laverage index
‘where Wis @ Baves estimate of
[sampling variance]
[Bammpling variance] + [process varanoeT

“The bigger the sumpling varianee is relative 10 the procs variance,
put on the eetimated procece average

Bayes upproach
cover the approach in Tf. 10. One ix tha the weight, Wis ulvays
riety becmoon aero and one. This is hecause Wie a Bayes eatimate
fof an unknown optimal wright, which has «nondegenerate posterior
titebution on the inverval [,{], The approach taken in Ref. 108 co
tate maximum Likelihood ertimstos of o, which ean be one; io, total
shrinkage to che procea average

"The second advantage is that an interval estimace of the current
population index ean be constructed from its posterior datruton.
‘Moat of the literarare (e.g, Ref. 10) (rats the estimation problem
thoroughly, but it provides litle guidance forthe interval estimation
problem,

"The ur algorthm la applied to the Munson data and the aw’
eatimates of the season averages are given in ‘Table ‘The sues of tho
fbeolute errors forthe maximum liked estimates (Aprll averages)
tnd the gus satimates are 0.603 and 0.81, respectively—a forty-five
percent improvement. Notie thatthe qu estimate for 1970 and 1971
tre close to the April averages, This Because there was no history on
Munson. The reduction in (otal absolute erzor for the years 1073
rough 1978 wns sisty-fve percent, because ofthe benefit of history

18 Objectives

‘This paper is intended to docurnent qu, Tt contains the rationale
for changing the rang mytem, a aynopais of qu features, mathemat-

222 THE BELL SYSTEM TECHNICAL JOURNAL, FEBRUARY 1981
fea! derivations of the riting formas, che dynamics of que, the
operating characteristics of aM, any example, and the QM report
ing fora.

"Readers who are interested only in the mathematin of qe and how
it relate to empirical Bayes, roy ship Section IL Readers, who are
pot interested in the mathematical derivation of qtr, my skip Section
W.

1 PRATE gvSTEM

“To understand the rationale bchind qu, one must firs uerstand
the Tate syste. Fron thia wo chall see where things have changed
land whore things have remained Use same.

2.1 Finding detect

‘The sampling methods slong with the seopo of inspection provide
for a sample of units of cont fer each sot of inspections. A‘unit of
Counts ether anit of produce ara unit of a produets wltribate sich
fn solderlees wrapped connections.

‘The remule of conlucing ast of inspostions is lst of defects found
and their dessriprions. Frequents, unierlying a defect ix a variable
mmesnizement” that falls outside w cunge- GMP dows not affect. the
process of Finding detects.

2.2 Ascoesing detects

‘The defertn found sometimes occur in cluster for which the effect
of the chuster a nonaltives i, che effects leas than the sum of the
‘fects ofthe individual defecn aoewering by themselves. In chis cas
the mumber of defete assed for rating purpose is less than the
‘number found. The defo! assessment practices forthe Terave xylem
volved over u 50 svar peviod, 20 thoee practves were based en a
‘arity of eter and engineering jwements The defect assessment
Drecices uner gu amount lo » redesign of the practices using &
nls principle, which i deveribed in Seetion 31.

23 Datect weighting and demerts

‘The defects ansemed are transforamel into demerit or defectives oF
ray remain a simple unweighted defects. Im an audit based on
Aomori each defect assenved bs assigned a number of demerit: 100,
50,10, or for 2, or D weit defect, eepectively. Cuidelines for
asigning demerit weighix wre contained in oumerous general and

(QUALITY MEASUREMENT PLAN 223,
special purpose demerit ists, ‘The principles underlying these demerit
tite are dascribed by Dodge in Ret. 5 nnd In an audit based on
defectives, all defects found in w unit of product are anulyatdl to
determine ifthe unit a considered defective. The arzextment ix either
tone or ero defectives. These transformations to demerits, defectives,
for defects ure not affected hy avr

24 Quay standede

For any set of ingpecton, the quality engineers in the aac have
stale quality standards do Ui, hy eonsored aut seope
Shop capability, eld performance, ecoemicy compen, ct The
Pilon of standards is described in Rt 3, For aude based on
{eects or defectives che standards nr exresed in deft dee
tives per nit Por aud bused on demerit Ue standards re deived
from fundamental defect pr unit uf count uandars fr A, By C, D>
{ape defor tn addon we wo Poin ws the standard dtibution
athe number of eye A defeets (or example)

To mirth lar, scold 4 spe example. Suppose in a
sails ete ae av de Note A H.C D det, The
Alinton of canderd quali i that Xa, nye independent an
dave Boimon dntbutons wilh mow Na, wn, =2~ The amber
demi in he sume

Dm 100s + HM + 1K + Xp
“The monn an vrianes of , sven andar quay, are
FADS) ~ 00a +a +
= AOD + ha + 106+ od
=n,
Vib |S) = 400 tA) + Onda) +
= n.orhs + (0)Ra + 0)Re + Ao}

anc,

‘The notation “P| reads “D conditional on 82"

[Nove that Us the demerit per unit standard and Cis a variance
per unit standard ‘Theae are the numbers chat would be publidbed in
the offical lst of standards celled the Master Heference fs. ‘The
‘quality standards are not affected by @sr

2.5 Rating classes and paris

For the purpose of reporting quality results to management, the
products are grouped into rating clases. An example is! rsa No. 1

224 THE BELL SYSTEM TECHNICAL JOURNAL, FEBRUARY 1981