ERIC EJ918449: Development of a Comprehensive Heart Disease Knowledge Questionnaire

2011·0.24 MB·English

by  ERIC

Research Article
Development of a Comprehensive
Heart Disease Knowledge Questionnaire
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
ABSTRACT
Background: Heart disease is the number one killer of both men and women in the United States, yet a comprehen-
sive and evidence-based heart disease knowledge assessment is currently not available. Purpose: This paper describes
the two-phase development of a novel heart disease knowledge questionnaire. Methods: After review and critique of
the existing literature, a questionnaire addressing five central domains of heart disease knowledge was constructed.
In Phase I, 606 undergraduates completed an 82-item questionnaire. In Phase II, 248 undergraduates completed a
revised 74-item questionnaire. In both phases, item clarity and difficulty were evaluated, along with the overall fac-
tor structure of the scale. Results: Exploratory and confirmatory factor analyses were used to reduce the scale to 30
items with fit statistics, CFI = .82, TLI = .88, and RMSEA = .03. Scores were correlated moderately positively with
an existing scale and weakly positively with a measure of health literacy, thereby establishing both convergent and
divergent validity. Discussion: The finalized 30-item questionnaire is a concise, yet discriminating instrument that
reliably measures participants’ heart disease knowledge levels. Translation to Health Education Practice: Health
professionals can use this scale to assess their patients’ heart disease knowledge so that they can create a tailored
program to help their patients reduce their heart disease risk.
Bergman HE, Reeve BB, Moser RP, Scholl S, Klein WMP. Development of a comprehensive heart disease knowledge questionnaire. Am J Health Educ.
2011;42(2):74-87. This paper was submitted to the Journal on August 5, 2010, revised and accepted for publication on November 19, 2010.
BACKGROUND estimated that every 30 seconds an American
For over 80 years, heart disease has been will suffer from a coronary event and every Hannah E. Bergman is a cancer research
the leading cause of mortality for both men 60 seconds the event is fatal.3 training award fellow at the National Cancer
and women in the United States.1,2 In 2005, Not only is heart disease the number Institute, Rockville, Maryland 20852; E-mail:
heart disease claimed the lives of more than one killer of Americans, but some of the [email protected]. Bryce B. Reeve is an
860 000 Americans.3 Coronary heart disease disease’s modifiable risk factors comprise the associate professor at The University of North
(CHD), the most common type of heart leading three direct contributors to death in Carolina at Chapel Hill, CB 741, Chapel Hill,
disease in the U.S.,4 is responsible for about the U.S.: tobacco use, poor diet and physical NC 27599-7411. Richard P. Moser is a research
500 000 deaths per year.3 Results from the inactivity, and alcohol consumption.5 Other psychologist, National Cancer Institute, Room
National Heart, Lung, and blood Institute’s modifiable risk factors for developing heart 4052, Rockville, MD 20852. Sarah Scholl is
Framingham Heart Study found that the disease include obesity, high blood pres- a project manager, University of Pittsburgh,
lifetime risk for developing CHD at age 40 is sure, high blood cholesterol, and stress.6 Pittsburgh, PA 15213. William M. P. Klein is
a 1-in-2 chance (48.6%) for men and 1-in-3 Thus, much of an individual’s risk of heart an associate director, National Cancer Institute,
chance (31.7%) for women.4 Moreover, it is disease—and likewise, of death—can be Room 4060, Rockville, MD 20852.
74 American Journal of Health Education — March/April 2011, Volume 42, No. 2
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
mitigated by addressing one or more of these atory; we developed and tested a paper and response scales (e.g., Likert-type scales).
modifiable factors through individual means pencil 84-item questionnaire with a univer- Our review process revealed many issues
such as by quitting smoking, exercising, im- sity student sample. The primary goals were (addressed in the critique below) with the
proving diet, or obtaining medical treatment to validate the questionnaire’s five-domain existing scales and led us to believe that our
for conditions such as diabetes, high blood structure and to reduce the number of items. current scale would not only address these
pressure, or unhealthy blood cholesterol. We also included additional items that evalu- issues, but add to the field of cardiovascular
Heart disease will continue to be a ated participants’ perceptions of their heart disease patient education by providing a
modern-day health threat unless the Ameri- disease knowledge. In Phase II, we validated more comprehensive and updated scale. See
can public has a clear and comprehensive a refined version of the scale with a second Table 1 for summary of findings.
understanding of what, and how, risk fac- university student sample. Many scales only addressed a subset of
tors contribute to the development of the We identified existing measures of heart the five domains.15-19 Others were tailored
disease. Further, individuals need not only disease knowledge through a search of specifically either to participants in an
be able to identify their risk factors, but also literature published as of February 2010 intervention program or to patient sub-
to understand them in the context of overall using PubMed and PsycINFO databases populations.16,17,20-23 In other cases, item
heart health knowledge. Ensuring that indi- and Google Scholar. Articles reviewed were wording was outdated or the use of termi-
viduals possess a basic understanding of the restricted to those written in English, with nology was too technical. For example, scales
disease itself—including its symptomatol- no limits on year of publication. We used that contain terms such as “angina pectoris,”
ogy, as well as related medical and dietary keyword search combinations such as: “atherosclerosis”24,25 and “myocardial infarc-
knowledge—will enable them to identify “heart health” and/or “heart disease” with tion”18,23 may not be practical for those with
specific actions they can take to reduce their “knowledge,” “development of,” “test,” low levels of education.
risks and to build a foundation upon which “scale,” ”questionnaire,” and “assessment” to Some scales may also lead to ceiling ef-
they can effectively take control of their find related articles. We did not review scales fects in performance. For example, among
health. Hence, having the ability to assess that included a majority of open-ended or the 25 items used in the Heart Disease Fact
peoples’ level of knowledge about heart Likert-scale type questions where there was questionnaire,22 18 items were found to
disease through a standardized tool would not an obvious correct answer7-11 or scales have a mean difficulty score of .80 or above,
be valuable in aiding the design of effective that focused specifically on detection, as meaning that the majority of the items were
health interventions, as well as in measur- opposed to more general knowledge of answered correctly 80% of the time by par-
ing the efficacy of such interventions. In heart disease.12 We also excluded from the ticipants. Without inclusion of items having
addition, such a tool could effectively serve review any studies that included participants a broader range of difficulty, such scales do
as a means by which to tailor educational aged 18 and younger,10,13,14 since the current not allow for the detection of meaningful
materials to individuals directly, to address study’s samples comprised of participants differences in knowledge. Further, many
their specific gaps in knowledge. 18 and older. Sixteen scales ultimately met scales contain too few or too many items. A
our criteria for review. scale with too few items may not adequately
PURPOSE Each of the 16 scales was examined using measure individuals’ knowledge levels with
The goal of the current study was to a set of criteria developed by the authors that sufficient sensitivity, and may also raise
develop an easily-administered and psycho- were based on scale construction, adminis- content validity concerns.24,26,27 Conversely, a
metrically sound heart disease knowledge tration and scoring. Within scale construc- scale with too many items increases respon-
questionnaire that could be used in a tion, degree of comprehensiveness (evalu- dent burden and may render it impractical
variety of adult populations. To achieve ation of how many knowledge domains for use in settings in which time may be
our purpose, we identified and extracted were addressed), degree of generalizability, limited, such as during visits with health
selected items from existing scales, merging understandability and suitability of wording, practitioners.25,28,29
them with newly developed items based degree of difficulty of the questions, and The modes by which some scales were
on current cardiovascular research. The total number of items were evaluated. Scale administered may have generated invalid
current scale encompasses five relevant administration review involved assessing results, such as those conducted via tele-
knowledge domains: dietary knowledge, mode of administration, as well as sample phone27,29 or otherwise in the form of an oral
epidemiology, medical information, risk group and sample size (i.e., whether the interview.18,27 Administering a scale verbally
factors, and heart attack symptoms. Sources scale could be used on multiple population introduces the possibility of interviewer bias
for the existing and new items are reported samples and was tested on a large enough or inconsistency. Further, respondents are
in the Methods section. sample size). Finally, our assessment of scale also more likely to answer in socially desir-
The questionnaire was developed in two scoring took into account incorporation of able ways when surveyed orally.31 Another
phases. The first phase (Phase I) was explor- open-ended items and use of continuous problem is utilization of a small sample.29
American Journal of Health Education — March/April 2011, Volume 42, No. 2 75
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
Table 1. Criteria Met by the Previous Scales Table 1. Criteria Met by the Previous Scales
Scale Construction Administration Scoring
Authors Year
Comprehensive Generalizability Wording Item Difficulty # of Items Mode Sample Open-ended/ Continuous Responsec “I don’t know”d
Collins et al. 2004 xa x NRb x x x x x
DeForge et al. 1998 x NR x x x x
Farquhar et al. 1990 x x NR x x x
Folsom et al. 1988 x x x x x
John et al. 2009 x x x x x
Momtahan et al. 2004 x x x x x
Mosca et al. 2000 x x x NR x x
Mosca et al. 2004 x x x NR x x
Oliver-McNeil & Artinian 2002 x x x NR x x
Smith et al. 1991 x NR x x x
Suminski et al. 1999 x x x x x x
Tate & Cade 1990 x x x x x x
Thanavaro et al. 2006 x x NR x x x x x
Thanavaro et al. 2010 x x x x x x x
Wagner et al. 2005a x x NR x x x x x
Wagner et al. 2005b x x x x x x x
Notes:
aAn “x” indicates criterion met by a particular scale.
bNot reported “NR” indicates that item difficulty criterion was not reported or that it was not reported for all items.
cAn “x” indicates articles that did not include open-ended or continuous response items.
dAn “x” indicates articles that did include the answer option “I don’t know.”
Scales which utilize open-ended or con- demiology, and symptoms that would add self-regulatory model of illness behavior lit-
tinuous response items are challenging for to the field of cardiovascular disease patient erature,34,35 and the findings from an ad hoc
the administrator to score.18,27-30 Open-ended education. We did so using two samples expert panel consisting of a board certified
questions elicit a wide range of responses and cross-validated the resulting scale by internist, a health psychologist, and psy-
and as such, are difficult to score objec- correlating it with a previously used scale,26 chometrician. based on our findings, these
tively.32 The use of items with continuous as well as a common measure of health five domains are of equal importance in the
response scales renders determining “cor- literacy,33 to establish both convergent and realm of heart disease knowledge.
rect” answers problematic, because no clear divergent validity. The self-regulatory model of illness
correct response exists. Further, several behavior stipulates that individuals make
METHODS
scales were also found to contain true/false health decisions over three stages: cognitive
items, but did not include the “I don’t know” Instrument Development representations stage, action plan stage, and
option as a choice.16,17,19,20,23,24,26,27,30 Including The current scale encompasses five rel- appraisal stage.36 The cognitive representa-
this option helps to reduce guessing.32 evant knowledge domains: dietary knowl- tion stage is defined by five dimensions
Our aim was to address these issues via edge, epidemiology, medical information, of the health risk: identity/characteristic
construction of a novel scale—one based risk factors, and heart attack symptoms. symptoms, cause, consequences, duration,
upon the most current medical knowledge These current domains were derived from and control.36-39 We believe that our do-
about heart disease and its risk factors, epi- our literature review of previous scales, the mains, and their associated items, conform
76 American Journal of Health Education — March/April 2011, Volume 42, No. 2
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
Table 1. Criteria Met by the Previous Scales
Scale Construction Administration Scoring
Authors Year
Comprehensive Generalizability Wording Item Difficulty # of Items Mode Sample Open-ended/ Continuous Responsec “I don’t know”d
Collins et al. 2004 xa x NRb x x x x x
DeForge et al. 1998 x NR x x x x
Farquhar et al. 1990 x x NR x x x
Folsom et al. 1988 x x x x x
John et al. 2009 x x x x x
Momtahan et al. 2004 x x x x x
Mosca et al. 2000 x x x NR x x
Mosca et al. 2004 x x x NR x x
Oliver-McNeil & Artinian 2002 x x x NR x x
Smith et al. 1991 x NR x x x
Suminski et al. 1999 x x x x x x
Tate & Cade 1990 x x x x x x
Thanavaro et al. 2006 x x NR x x x x x
Thanavaro et al. 2010 x x x x x x x
Wagner et al. 2005a x x NR x x x x x
Wagner et al. 2005b x x x x x x x

to this stage of the self-regulation model and blood Institute, National Center for An ad hoc panel was created to address
of illness. Health Statistics, Department of Health and and review the content validity and face
The selection of existing items and de- Human Services, United States Department of validity of the domains and items selected
velopment of new ones were based on how Agriculture, Centers for Disease Control and for the finalized questionnaire (as seen in
the items fell within the five pre-established Prevention, Mayo Clinic, University of Mary- Phase II methods and results). The group
knowledge domains. The process was un- land Medical Center, Harvard Medical School, was composed of three experts in cardio-
dertaken by the authors, including a health and Coalition of Labor Union Women. vascular disease, health psychology and
psychologist, a research psychologist and a The true-false format was chosen over psychometrics. The three experts were asked
psychometrician. Some items were extracted the multiple-choice item format for the to independently check to see if the items
from existing, yet dated scales, such as the current scale because we wanted to reduce had the correct response, fit under the pre-
Cardiovascular Disease (CVD) Knowledge participant burden, given that there were established domain, and were up-to-date.
Test,26 while others were developed using over 80 items in the original questionnaire Phase I Questionnaire
findings and information gleaned from the (see Phase I instrument description). The Instrument description. The initial ques-
Pittsburgh Healthy Heart Study, American “I don’t know” option was included to help tionnaire consisted of 82 items that measured
Heart Association, Harvard Center for Can- improve the scale’s reliability to reduce dietary knowledge (N = 20), epidemiology
cer Prevention, Women’s Heart Foundation, guessing that is often associated with the knowledge (N = 8), medical knowledge (N
Heart Healthy Women, National Heart, Lung true-false format.32 = 24), risk factors knowledge (N = 20), and
American Journal of Health Education — March/April 2011, Volume 42, No. 2 77
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
heart attack symptoms knowledge (N = 10). “I don’t know” and 1 = correct answer for After comparison of the alternative
Additionally, participants completed three each item on the scale. one- and five-factor models, item reduction
items that measured their self-perceived The data from the 594 participants who began by considering the difficulty of the
degree of knowledge about heart disease: “I answered at least 68 of the 71 items (95%) items (assessed by looking at mean scores),
know more than the average person about as well as all 3 items about self-perceived and the discrimination of the items (assessed
heart disease,” “I believe that I answered 75% knowledge were analyzed. The overall scale by looking at factor loadings and item-total
or more of the above questions correctly,” score was calculated as the sum of the correct score correlations), inter-item correlations
and “I believe that I answered more of the answers, with percentage score derived by (not reported in text; please contact corre-
above questions correctly than the average dividing each score by 71. We used list-wise sponding author for them), and contribu-
person will.” Participants also reported age deletion for missing values. tion of each item’s content towards overall
and gender. Analysis plan. The main purposes of knowledge were taken into consideration
Participants. Participants were 606 Phase I analyses were to evaluate the factor when selecting items for the questionnaire.
University of Pittsburgh undergraduates structure of the questionnaire and item For difficulty, we wanted to ensure that we
enrolled in an introductory psychology reduction. Our goal was to produce a ques- had items that tapped into different levels
course. Participants were 56.6% female with tionnaire that contained a reduced number of heart disease knowledge; this guarantees
a mean age of 18.67 (SD = 1.66, range = 17 of items, but retained the breadth of do- that we could discriminate amongst indi-
to 44). Three participants did not provide mains regarding heart disease knowledge. viduals who may have limited knowledge
demographic information. Participation SPSS Version 17 was used to obtain de- from those who have advanced knowledge.
fulfilled a course requirement. scriptive statistics on the overall score of the For discrimination, we wanted items that
Procedure. Participants completed the questionnaire and the three items that mea- were related to the overall construct of heart
scale as part of a mass testing session. sured self-perceived degree of knowledge. disease knowledge to maximize our ability to
The instructions for the heart disease Upon item reduction (explained below), differentiate among individuals at different
questionnaire asked the participants to the correlation between the participants’ skill/knowledge levels.
answer “true,” “false,” or “I don’t know” to overall scores and scores on each of the three In some cases, an item with a low factor
a number of questions addressing their be- perception items was calculated. loading was retained upon having been
liefs and knowledge about various aspects The statistical software data analysis pro- deemed to measure a key concept not re-
of heart disease. gram MPlus Version 5.21 ran the confirma- flected in other questions. We also took into
Data preparation. From the original 82 tory and exploratory factor analyses, given consideration mean scores reflected in our
items, we excluded 9 items—8 of which its ability to model items with dichotomous data to ensure the finalized questionnaire
were no longer accurate based on new find- response format. Two confirmatory factor had a broad enough range of difficulty to
ings in the medical literature at the time analyses (CFA) were conducted; a five-factor assess different knowledge levels.
of analysis (for example, “Lack of physical solution reflecting the theoretical structure Phase II Questionnaire
activity is the principal cause of obesity,” is of the questionnaire and a one-factor solu- Instrument description. This instru-
incorrect since it is now understood that tion representing all items as measuring a ment was a revised version of the Phase I
no 1 principle cause of obesity exists)40 and general knowledge of heart disease. To assess questionnaire. The updated scale excluded
1 item due to ambiguity (“Men and women model fit Comparative Fit Index (CFI),42 8 items from Phase I that were no longer
experience the same symptoms of a heart Tucker-Lewis Index (TLI),43 and Root Mean accurate based on new findings in the medi-
attack.” Women often experience different Square Error of Approximation (RMSEA) cal literature. The item that was discarded
sets of symptoms, but many are symptoms were reported.44 We attempted to identify for analysis in Phase I due to ambiguity
common to both men and women).41 We sources for misfit if the CFA model did not was rewritten and included in the Phase II
also excluded from analysis two items that fit one or more of the model fit criteria (i.e., questionnaire, as were the 2 items which
had a mean difficulty score (% participants CFI > .95, TLI >. 95, RMSEA <. 05). For had been excluded from analysis due to
answering correctly) of .06 or below or .94 or example, we examined the model modifi- having had poor item-difficulty scores. The
above. The remaining 71 items were divided cation indices to identify items that loaded resulting scale consisted of 74 items. Par-
among each knowledge domain as follows: on more than one domain. An exploratory ticipants also completed 11 items from the
dietary knowledge (N = 16), epidemiology factor analysis (EFA) was also conducted to CVD Knowledge Test26 (3 items from this
knowledge (N = 8), medical knowledge (N = look for the number of factors and patterns 14-item scale were already included in the
22), risk factors (N = 17), and symptoms (N represented in the observed data by examin- scale), 3 items that measured self-perceived
= 8). Within any one domain, less than 2% ing both the magnitude of the eigenvalues degree of knowledge about heart disease (the
of data was missing. Participants’ answers and interpretability of the data associated same items from Phase I), and 7 items that
were coded either 0 = incorrect answer or with each extracted factor. measured health behaviors to see if there
78 American Journal of Health Education — March/April 2011, Volume 42, No. 2
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
was a relationship between health behavior score ranging from 0 to 5. We expect that an exploratory factor analysis (EFA) of the
status and heart disease knowledge level. A the results should be the same from these 71 items revealed at least five factors. The
final item measured participants’ perceived two different calculation methods. first factor accounted for 15% of the total
health literacy using a conventional 1 item Analysis plan. The goal of Phase II was variance. Factors 1 through 5 accounted
scale, “How confident are you filling out to validate the reduced item questionnaire for 32% of the total variance. because the
medical forms by yourself?”33 Participants resulting from Phase I and to validate the sixth and seventh factors only accounted
also reported age and gender. finalized questionnaire with other heart for a small amount of variance, 3% each,
Participants. Participants were 248 disease measures. and did not provide additional interpre-
University of Pittsburgh undergraduates SPSS Version 17 was used to obtain de- tive solutions, we retained the five-factor
enrolled in an introductory psychology scriptive statistics on the reduced heart dis- model. The five-factor EFA model had good
course. Participants were 50.4 % male with ease knowledge questionnaire, the 10 items fit statistics with CFI = .96, TLI = .97, and
a mean age of 18.68 (SD = 1.08, range = about health behaviors and perceptions, RMSEA = .02.
17 to 27). Participation fulfilled a course and the 14-item CVD Knowledge Test.26 We based on the factor loadings of the
requirement. obtained internal reliability estimates (coef- 71-item five-domain model CFA, 38 items
Procedure. Participants completed the ficient alpha) of the finalized scale (obtained loaded well onto one of the domains (fac-
scale as part of a larger questionnaire in through item reduction), the five knowledge tor loadings > .40). From these 38 items, we
an unrelated research study. Participants domains as well as correlations among the discarded 17 items due to their difficulty
scheduled individual appointments with scale scores. The Kuder-Richardson formula mean (too easy or too difficult) and content
an undergraduate research assistant and 20 for the internal reliability estimates was (items’ topics were addressed in other items).
completed the questionnaire privately on reported because our heart disease knowl- For the 33 items that did not load well, we
a computer in the laboratory. The session edge scale is dichotomous (true/false).46 The reviewed their content and mean difficulty
lasted about one hour. The instructions for correlations between the overall scale score level, and ultimately opted to retain 8 of
the Phase II questionnaire were identical to on the heart disease knowledge question- these items, as their mean difficulties fell
those of Phase I. naire and participants’ responses to the within the desired range and their content
Data preparation. From the original health literacy item were also calculated. was deemed to be worthy of inclusion in the
74 items, three items had mean difficulty The statistical software analysis program questionnaire. The item reduction of Phase
scores higher than .94 and were discarded MPlus Version 5.21 was used to conduct I’s questionnaire produced 29 items. In sum,
from analyses. Participants’ item answers the confirmatory factor analysis on the criteria used to get from 71 to 29 items were
were coded either 0 = incorrect answer or same five-domain reduced questionnaire. factor loadings, item content area, and dif-
“I don’t know” and 1 = correct answer. The To assess model fit we reported the same fit ficulty mean.
statistics as in Phase I.
scores of 242 participants were used for Descriptive statistics: 29-item scale. The
analyses after excluding participants who average score for the 29-item scale was 15.77
RESULTS
did not answer all the items in the question- with a standard deviation of 4.81 and a range
naire, answer the 10 items about health be- Phase I Questionnaire of 0 to 28. Sixty-one percent of participants
haviors and perceptions, or those from the Descriptive statistics. For the 71-item answered 15 or more of the items correctly.
14-item CVD Knowledge Test.26 The overall scale, the average score was 35.80 with a stan- The item difficulty mean ranged from .18
heart disease scale score was calculated by dard deviation of 8.87 and a range of 0 to 58. to .79. Correlations between the domains
two different methods. We did these two Fifty-four percent of participants answered for the 29 items were low to moderate
analyses to show that the five domains are 36 or more of the items correctly. The item (Table 2).
equally important to the total knowledge mean difficulty ranged from .13 to .93. The 29-item questionnaire showed statis-
score. In Method I, the number of correct Factor analyses. The one-factor confirma- tically significant associations with 3 items
answers was counted and divided by the tory factor analysis (CFA) solution had fit about self-perceived knowledge regarding
final number of items that resulted from statistics of CFI = .76, TLI = .79, RMSEA heart disease: “I know more than the aver-
the item reduction process (see below). In = .04. The five-factor CFA solution had the age person about heart disease,” r(592) =
Method II, we calculated the overall score same results of CFI = .76, TLI = .79, and .24, P < 0.01; “I believe that I answered 75%
by taking the sum of the correct answers in RMSEA = .04. Given that neither model fit, or more of the above questions correctly,”
each domain, dividing this number (correct we explored the modification indices. We (r(592) = .25, P < 0.01); and “I believe that
items) by the total number of items in the found no modification in item loadings that I answered more of the above questions cor-
domain. The result of this process gave us would significantly improve model fit (all rectly than the average person will,”(r(592)
five percents (one for each domain), which changes in χ2 were less than 4). = .37, P < 0.01). In other words, people
were added together to give a total scale An examination of the eigenvalues from who performed better on the scale tended
American Journal of Health Education — March/April 2011, Volume 42, No. 2 79
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
Table 2. Correlations Among the 5 Knowledge Domains (29 items)
Variables Dietary Epidemiology Medical Risk Factor Symptoms
Dietary - .25 .37 .38 .25
Epidemiology - .31 .47 .27
Medical - .42 .31
Risk Factor - .33
Symptoms -
Note: All correlations significant at P < .001.
to rate their performance more highly, The mean scale score for the 14-item CVD and the mean difficulty.
although the moderate magnitude of these Knowledge Test26 was 7.70 with a standard Ad hoc panel. An ad hoc panel reviewed
correlations suggests a fair degree of error deviation of 1.59 and a range of 2 to 12. our final 30-item scale and revisions were
in self-perceived knowledge. Eighty percent of participants answered 7 done based on their feedback (Appendix 1).
Phase II Questionnaire or more items correctly. The panel suggested rewording three items,
The 30-item questionnaire showed sta- “Taking an aspirin each day is thought to
Descriptive analyses. by Method I calcu-
tistically significant associations with the 3 decrease the risk of getting heart disease” to
lations (see Methods-Phase II section), the
mean scale score for the 30-item scale (29 items about self-perceived knowledge about “Taking an aspirin each day decreases the risk
items from Phase I, plus 1 rewritten item heart disease (out of the 10 items about of getting heart disease,” “The best kind of ex-
with the addition of “many of,” “Men and health behaviors and perceptions): “I know ercise involves rapid breathing for a sustained
women experience many of the same symp- more than the average person about heart period of time” to “The healthiest kind of ex-
toms of a heart attack,” to improve clarity disease,” r(240) = .33, P < 0.01; “I believe ercise for the heart involves rapid breathing for
and included in the Phase II questionnaire) that I answered 75% or more of the above a sustained period of time,” and “Turning pale
was 15.68 with a standard deviation of 4.71 questions correctly,” r(240) = .18, P < 0.01; or gray is a common symptom of having heart
and a range of 2 to 25. Sixty-three percent and “I believe that I answered more of the attack” to “Turning pale or gray is a symptom
of participants answered 15 or more of the above questions correctly than the average of having heart attack.” Rewording the items
items correctly. by Method II calculations, person will,” r(240) = .39, P < 0.01). Scores did not change the correct response answer,
the mean scale score for the 30-item scale were also positively correlated with the item, nor do we think it influenced the results
was 2.61 with a standard deviation of .80 and “I know my blood pressure level,” r(240) = before the change.
a range of .39 to 4.33. because the 30-item .15, P < 0.05. The correlation between the
scale was divisible by the scale score range 30 items and the 14-item CVD Knowledge DISCUSSION
of 5, these percentage scores were multiplied Test26 was moderately positive, r(240) = .59, This paper describes the development of
by 6, so that Method I and Method II final P < 0.01, suggesting both convergent and a novel heart disease knowledge question-
scale scores could be compared. Method divergent validity. The correlation between naire through two phases. The purpose of
II’s converted mean scale score was 15.67 the 30 items and the health literacy item Phase I was to validate the five-factor model
with a standard deviation of 4.79 and a was weakly positive, r(240) = .32, P < 0.01, and perform item reduction. The result was a
range of 2.36 to 25.95. The item mean dif- suggesting divergent validity. 29-item questionnaire. The purpose of Phase
ficulty ranged from .18 to .75. The Kuder- Factor analyses. The five-factor confirma- II was to validate the final 30-item question-
Richardson 20 formula for the 30-item scale tory factor analysis (CFA) solution produced naire (29 items from Phase I along with an
was acceptable (.73), but was not for the model fit statistics of CFI = .82, TLI = .88, item that was rewritten and reintroduced).
five individual knowledge domains (range and RMSEA = .03. Table 3 summarizes both We also evaluated the extent to which par-
= .29 to .47). the mean (difficulty) and factor loading ticipants accurately discerned the scope of
The mean scale score for the 10 items (discrimination ability) of each item in each their heart disease knowledge in both phases,
about health behaviors and perceptions was knowledge domain for the five-factor CFA. and validated the new scale by correlating it
3.88 with a standard deviation of 1.57 and a Appendix 1 presents the final 30-item heart with an existing but dated measure of heart
range of 0 to 9. Approximately 59% of par- disease knowledge questionnaire and in- disease knowledge,26 as well as with a con-
ticipants answered 4 or more items correctly. cludes the item content, the correct answer, ventional measure of health literacy.33
80 American Journal of Health Education — March/April 2011, Volume 42, No. 2
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
Table 3. Standardized Model Results—Factor Loadings of the 5 Knowledge Domains (30 items)
Risk
Dietary Epidemiology Medical Symptoms Difficulty
Items Factors
(N = 6) (N = 4) (N = 7) (N = 4) Mean
(N = 9)
Polyunsaturated fats are healthier for
28 63
the heart than saturated fats.
Trans-fats are healthier for the heart
.49 .66
than most other kinds of fats.
Most of the cholesterol in an egg is in
.56 .65
the white part of the egg.
Dietary fiber lowers blood cholesterol. .44 .48
Margarine with liquid safflower oil is
healthier than margarine with hydro- .29 .38
genated soy oil.
Many vegetables are high in choles-
.29 .71
terol.
Women are less likely to get heart dis-
.54 .34
ease after menopause than before.
Heart disease is the leading cause of
.44 .60
death in the United States.
Most women are more likely to die
.61 .38
from breast cancer than heart disease.
Heart disease is better defined as a
short-term illness than a chronic, long- .56 .70
term illness.
Most people can tell whether or not
.51 .72
they have high blood pressure.
The best kind of exercise involves rapid
breathing for a sustained period of .29 .38
time.
A healthy person’s pulse should return
to normal within 15 minutes after .41 .73
exercise.
Cardiopulmonary resuscitation (CPR)
.42 .65
helps to clear clogged blood vessels.
HDL refers to “good” cholesterol, and
.12 .31
LDL refers to “bad” cholesterol.
Atrial defibrillation is a procedure
where hardened arteries are opened .48 .19
to increase blood flow.
“High” blood pressure is defined as
.37 .26
110/80 (systolic/diastolic) or higher.
Having had chicken pox increases the
.36 .52
risk of getting heart disease.
Eating a lot of red meat increases heart
.20 .65
disease risk.
The most important cause of heart at-
.45 .34
tacks is stress.
Walking and gardening are considered
types of exercise that can lower heart .54 .75
disease risk.
Continues on next page
American Journal of Health Education — March/April 2011, Volume 42, No. 2 81
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
Table 3. Standardized Model Results—Factor Loadings of the 5 Knowledge Domains (30 items) (Con’t)
Risk
Dietary Epidemiology Medical Symptoms Difficulty
Items Factors
(N = 6) (N = 4) (N = 7) (N = 4) Mean
(N = 9)
Smokers are more likely to die of lung
.44 .23
cancer than heart disease.
Taking an aspirin each day is thought
to decrease the risk of getting heart .44 .70
disease.
Taller people are more at risk for get-
.36 .38
ting heart disease.
People who have diabetes are at
.43 .64
higher risk of getting heart disease.
Eating a high fiber diet increases the
.41 .60
risk of getting heart disease.
Turning pale or gray is a common
.44 .51
symptom of having heart attack.
Sudden trouble seeing in one eye is a
common symptom of having a heart .47 .18
attack.
Feeling weak, lightheaded, or faint is
a common symptom of having a heart .58 .71
attack.
Men and women experience many of
.26 .70
the same symptoms of a heart attack.
Note: Standardized model results; STDYX Standardization
Phase I Questionnaire decision that the five-factor model was a majority of participants answered at least 4
The majority of the participants an- good choice for our 71-item scale. The cor- items on the 10-item scale about perceptions
swered at least half of the items correctly for relations among the 5 knowledge domains and health behaviors and at least 7 items cor-
both the 71 and 29-item scales. Only about for the 29 items were low to moderate, all rectly on the 14-item CVD Knowledge Test.26
a quarter of the participants answered 2 or positive and significant. These correlations The level of difficulty was appropriate.
more of the 4 items about self-perceived indicate that the content information in The low alpha levels of the five knowl-
knowledge in such a way as to reflect con- each domain did not overlap, but that all edge domains suggest that while looking at
fidence in their knowledge of heart disease, domains were indeed measuring heart dis- how participants perform on each subscale
both absolutely and in comparison to the ease knowledge. of heart disease knowledge is informative,
average person. The item difficulty means of The correlation between the 29-item it cannot be considered a reliable scale for
the 29 items fell within an appropriate range, heart disease knowledge questionnaire and assessing group differences or changes over
with 11 items having a difficulty mean of .40 the 3 items assessing self-perceived knowl- time. However, the overall 30-item scale
to .60. The average difficulty mean for the 29 edge was statistically significant, though had an acceptable internal reliability47 of .73
items was .54. Scales with difficulty means moderate. Thus, people who know more are for examining group level scores on heart
that are close to .50 give a more accurate also generally more aware that they know disease knowledge.
assessment of an individual’s knowledge more, but there is still a fair degree of error The correlation between the 30-item
level because they provide “maximum dis- in these self-assessments. scale and 14-item CVD Knowledge Test26
crimination”46 in individuals’ degree of heart Phase II Questionnaire was moderately positive (.59) which sup-
disease knowledge. both methods for calculating the overall ports our assumptions that our current scale
The CFA and EFA conducted on the 71 score for the 30-item scale produced similar and the previous scale showed both conver-
items produced a reduced 29-item scale. results. As with Phase I, the majority of par- gent and divergent validity. It is evidence
Although the CFA produced mixed fit sta- ticipants answered at least half of the items of convergent validity because it is
tistics for our model, the EFA validated the correctly for the 30-item questionnaire. A associated with another scale tapping into
82 American Journal of Health Education — March/April 2011, Volume 42, No. 2
Hannah E. Bergman, Bryce B. Reeve, Richard P. Moser, Sarah Scholl, and William M. P. Klein
Appendix 1. Heart Disease Knowledge Questionnaire
Instructions:
On the following page, you will be asked to respond to a number of True/False questions addressing your beliefs and knowledge
about various aspects of heart disease.
Please answer each by circling “T” for True and “F” for False.
Very few people answer all these questions correctly—just do the best you can. Feel free to circle ‘Don’t know’ if you are unsure of
an answer.
EXAMPLE:
High blood pressure increases the risk of getting heart disease……………. T F Don’t know
Correct Difficulty
Item Item Content
Response Mean
HD1 Polyunsaturated fats are healthier for the heart than saturated fats. True .63
HD2 Women are less likely to get heart disease after menopause than before. False .34
HD3 Having had chicken pox increases the risk of getting heart disease. False .52
HD4 Eating a lot of red meat increases heart disease risk. True .65
HD5 Most people can tell whether or not they have high blood pressure. False .72
HD6 Trans-fats are healthier for the heart than most other kinds of fats. False .66
HD7 The most important cause of heart attacks is stress. False .34
HD8 Walking and gardening are considered types of exercise that can lower heart disease risk. True .75
HD9 Most of the cholesterol in an egg is in the white part of the egg. False .65
HD10 Smokers are more likely to die of lung cancer than heart disease. False .23
HD11 Taking an aspirin each day decreases the risk of getting heart disease. True .70
HD12 Dietary fiber lowers blood cholesterol. True .48
HD13 Heart disease is the leading cause of death in the United States. True .60
HD14 The healthiest exercise for the heart involves rapid breathing for a sustained period of time. True .38
HD15 Turning pale or gray is a symptom of having a heart attack. True .51
HD16 A healthy person’s pulse should return to normal within 15 minutes after exercise. True .73
HD17 Sudden trouble seeing in one eye is a common symptom of having a heart attack. False .18
HD18 Cardiopulmonary resuscitation (CPR) helps to clear clogged blood vessels. False .65
HD19 HDL refers to “good” cholesterol, and LDL refers to “bad” cholesterol. True .31
Atrial defibrillation is a procedure where hardened arteries are opened to increase
HD20 False .19
blood flow.
HD21 Feeling weak, lightheaded, or faint is a common symptom of having a heart attack. True .71
HD22 Taller people are more at risk for getting heart disease. False .38
HD23 “High” blood pressure is defined as 110/80 (systolic/diastolic) or higher. False .26
HD24 Most women are more likely to die from breast cancer than heart disease. False .38
HD25 Margarine with liquid safflower oil is healthier than margarine with hydrogenated soy oil. True .38
HD26 People who have diabetes are at higher risk of getting heart disease. True .64
HD27 Men and women experience many of the same symptoms of a heart attack True .70
HD28 Eating a high fiber diet increases the risk of getting heart disease. False .60
HD29 Heart disease is better defined as a short-term illness than a chronic, long-term illness. False .70
HD30 Many vegetables are high in cholesterol. False .71
Note: Items HD4 and HD28 overlap with both dietary and risk factor domains. However, we kept them in the risk factor domain because they refer to a par-
ticular dietary risk factor of heart disease, as opposed to general dietary aspects pertaining to overall heart health, which are in the dietary domain.
American Journal of Health Education — March/April 2011, Volume 42, No. 2 83