Individual Differences and Personality

392 Pages·2013·2.77 MB·English

by  Michael C. Ashton (Auth.)

CHAPTER1
Basic Concepts in Psychological
Measurement
Contents
1.1. Some Simple Statistical Ideas 2
1.1.1 Levels of measurement 2
1.1.2 Standard scores 3
1.1.3 Correlation coefficients 5
1.2. Assessing Quality of Measurement: Reliability and Validity 9
1.2.1 Reliability 9
1.2.1.1 Internal-consistency reliability 10
1.2.1.2 Interrater (interobserver) reliability 13
1.2.1.3 Test-retest reliability 14
1.2.2 Validity 16
1.2.2.1 Content validity 16
1.2.2.2 Construct validity: convergent and discriminant 17
1.2.2.3 Criterion validity 18
1.3. Methods of Measurement: Self- and Observer Reports, Direct Observations, Biodata 19
1.3.1 Self-reports 19
1.3.2 Observer reports 20
1.3.3 Direct observations 21
1.3.4 Biodata (life outcome data) 22
1.3.5 Comparing the methods of measurement 22
1.4. Summary and Conclusions 24
Before we can really begin to understand personality, we need to figure out how to
measure it. And before we can measure personality, it would be useful to have some
common terms for describing our measurements.
In this chapter, we will introduce some basic concepts that allow us to describe
psychological measurements. By using these concepts, we will have some quick and
simple ways of understanding the results of personality research. For example, if
a researcher reports that women have a higher level than do men of some personality
characteristic, you would probably want to know how much higher. Or, if a researcher
reports that a given personality characteristic is related to enjoyment of a particular kind
of music, you would probably want to know how much they are related.
Also in this chapter, we will consider the basic ways of evaluating whether or
not our measurements are accurate. Whenever we try to measure a psychological
Individual Differences and Personality Ó 2013 Elsevier Inc.
ISBN 978-0-12-416009-5, http://dx.doi.org/10.1016/B978-0-12-416009-5.00001-3 All rights reserved. 1
2 Individual Differences and Personality
characteristic, we need to make sure (a) that we really have measured some meaningful
characteristic, and (b) that this characteristic really is the same one that we are trying
to measure. Measuring psychological characteristics accurately can be tricky, so it is
important to have some ways of expressing how well we have measured those
characteristics.
And finally, we will also explore in this chapter some of the methods that psychol-
ogists use when measuring personality and related characteristics. As you will see later in
this chapter, there exists a variety of methods, each of which has its advantages and
disadvantages.
1.1. SOME SIMPLE STATISTICAL IDEAS
1.1.1 Levels of measurement
One difficulty in psychological measurement, as opposed to measurement in other areas
of science, is that there is usually not a meaningful “zero” level of a psychological trait.
When astronomers describe a variable such as distance, it is easy to imagine what zero
distance is. But when psychologists describe a characteristic such as intelligence or
rebelliousness or irritability, it is difficult to imagine what a zero level would be. Even if
someone has a score of zero on an intelligence test, it does not seem meaningful to say
that the person has zero intelligencedpresumably, he or she would get a score higher
than zero if the test were easier. Because there is no clear zero point, we cannot really say
that one person is “three times” more rebellious or “50%” less irritable than another, in
the way that an astronomer might say that one planet has twice as much mass as another.
That is, in psychology we are usually not able to describe ratios between people’s levels of
a variable or a person’s absolute amount of a variable.
But the lack of a true zero level of psychological characteristics does not mean that we
cannot measure those characteristics. In fact, there are several different ways by which we
could compare people’s levels of any given trait. One of these is simply to rank people:
For example, we could measure people’s levels of ambition, and then record their
positions relative to each other, such as 1st, 2nd, 3rd, ., 654th, .. This is a sensible
approach, but it has some shortcomings. One difficulty is that the differences between the
ranks are not always meaningful. For example, the person with the highest level of a trait
in a given sample might be just slightly higher than the person with the 2nd highest level,
but the person who is 2nd highest might be far, far ahead of the person in 3rd. This fact
means that ranks are less than ideal for calculating statistics based on our measurements.
For example, when we want to compute the average level of a trait, our computation is
much more meaningful if the differences (or “intervals”) between the numbers always
mean the same thing. So, the numbers provided by ranks are not as useful as we might
like them to be.
Basic Concepts in Psychological Measurement 3
In measuring people’s characteristics, therefore, psychologists would like to obtain
scores that have meaningful differences between them (even though the ratios need not be
meaningful). For example, if we are trying to measure the trait of “assertiveness,” we
would like to be confident that a score of 60 really does mean a level of the trait that is
halfway between the levels indicated by a 50 and a 70. Note that assertiveness is not really
being measured in any particular “units,” and that it does not matter if the average score is
60, or 360, or 60, or whatever. The important thing, for the purpose of making
meaningful comparisons among people and of being able to calculate statistics, is simply
that equal differences, or intervals, between scores represent roughly equal differences in
the level of the trait. For example, when psychologists measure intelligence using an
“IQ” test, they would hope that the difference between an IQ of 110 and an IQ of 120
really does have the same meaning as the difference between an IQ of 130 and an IQ of
140. (Note again, by the way, that an IQ of 0 does not indicate zero intelligence; note
also that the average IQ level has been set arbitrarily at 100, even though any other value
could have been chosen as the average instead.)
How do psychologists know if their measurements meet the requirement of having
meaningful differences? The methods for testing this are beyond the scope of this
textbook, but we can say here that most well-designed psychological measurements are
close enough to this ideal to be useful for statistical analysis.
1.1.2 Standard scores
It was mentioned before that psychological characteristics are not measured in any
particular “units,” and that it does not matter how high or low the “scores” on a char-
acteristic tend to be, as long as the differences between scores are meaningful. But
differences in the numbers used for measuring variables might cause difficulties when we
want to compare someone’s scores across two or more traits. For example, suppose that
Bob has an IQ of 90 (where the average person’s IQ is 100) and that Bob also has a score
of 60 on a “sociability” scale (which, let us say, has an average score of 50). At first glance,
it seems that Bob’s IQ score (90) is higher than his sociability score (60), but in fact Bob is
below the average on IQ and above the average on sociability. Therefore, we need some
way to relate scores on one scale to scores on another scale, so that we can compare levels
of one characteristic with levels of another, or to compare scores on the same charac-
teristic as measured by different scales.
Psychologists are able to make meaningful comparisons across different kinds of
measurement scales by converting scores into standard scores. The first step in calculating
a standard score is to take an individual’s score on a given scale, and then subtract the mean
score (i.e., the average score) for the persons who have been measured. This difference
between the individual’s score and the mean score tells us whether the person is above the
average (if the difference is positive) or below the average (if the difference is negative).
4 Individual Differences and Personality
But this is not the only step. If we merely subtract the mean score from the indi-
vidual’s score, we still might not have a meaningful idea of how far above or below the
average that person is. This is because different scales of measurement differ in terms of
how “spread out” people’s scores are. For example, on a typical IQ test, about two-thirds
of people are within 15 points of the average (i.e., between 85 and 115), and about 95%
of people are within 30 points of the average (i.e., between 70 and 130). So, a person
who has an IQ of 110 is above average, but not very far above average. But imagine that
we have a different test, on which people’s scores are much more tightly bunched (say,
two-thirds of people between 95 and 105, and 95% of people between 90 and 110). On
this scale, a score of 110 would be very high. So, we need some way to compare scales
that have different amounts of variability in people’s scores, as well as different average
scores.
In order to do this, psychologists use a second step, after having first subtracted the
average score on a scale from the individual’s score on that scale. They then divide this
BOX 1-1 The Normal Distribution
The examples in the text are based on what is called a normal distribution of scores; when
drawn as a graph, this produces the well-known bell-shaped curve (see Figure 1-1). For many
physical and psychological characteristics, the distribution of scores is roughly normal: Most
people have scores close to the average value, with relatively few people being far above or far
below that average. Notice that a person whose score is equal to the mean will have a score
that is higher than that of 50% of people. If a person’s score is one standard deviation above
z-score –3.5 –3 –2.5 –2 –1.5 –1 –0.5 0 0.5 1 1.5 2 2.5 3 3.5
% of scores below <1 2 7 16 31 50 69 84 93 98 >99
Figure 1-1 The normal distribution of scores.
the mean, then his or her score is higher than that of about 84% of people; if it is two standard
deviations above, then it is higher than that of about 98% of people. Conversely, a score that is
one standard deviation below the mean is higher than that of about 16% of people, and
a score that is two standard deviations below is higher than that of about 2% of people.
Basic Concepts in Psychological Measurement 5
difference by the standard deviation, a number that indicates how much variability there
1
is among people on a variable. For many psychological characteristics, about two-
thirds of people are within one standard deviation above or below the mean, and about
95% of people are within two standard deviations above or below the mean. (For
example, in the situation mentioned before for the typical IQ test, the standard devi-
ation is 15.)
The result of the preceding two stepsdfinding the difference between the individ-
ual’s score and the average score, and then dividing this difference by the variability
(standard deviation) of the scoresdis to give a universal or standard way of expressing
people’s scores on a given characteristic, regardless of the original distribution of scores
on that characteristic. These scores, known as standard scores, have two special prop-
erties: First, the average score on a standard-score scale is exactly zero, and second, the
standard deviation of a standard-score scale is exactly one. So, after we have calculated
standard scores for our variables, we can meaningfully compare a person’s scores across
different variables. This applies not only to different scales measuring the same variable
(e.g., two different IQ test scales), but also to scales measuring different variables (e.g., an
2
IQ test scale and a sociability scale, or an “orderliness” scale and an “originality” scale).
1.1.3 Correlation coefficients
The correlation coefficient, known by the symbol r, tells us how strongly two variables
“go together”, or covary with each other. The values of the correlation coefficient can
range from a maximum of +1 (indicating a perfect positive correlation between two
variables) to a minimum of 1 (indicating a perfect negative correlation between two
variables). A correlation of 0 means that the two variables are unrelated to each other.
1
The standard deviation is calculated by, first, finding the difference between each person’s score and the average score
across all persons, then squaring each of these values, then adding up all of the squared values, then dividing this sum
by the number of people who were measured, and then finding the square root of this result. Suppose that there are
N individuals, and each of the individuals, i, has a score of Xi on variable X, with a mean score of m for those
individuals. The formula for the standard deviation, s, will then be as follows:
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u
u
u"PN ðXi mÞ2#
u
t i¼l
s ¼
N
If we do not take the square root, then we have the square of the standard deviation of scores, which is called the
2
variance of scores, s .
2
Sometimes, standard scores are converted from their basic form, in which the mean is zero and the standard
deviation is one, into some other form. As mentioned above, IQ scores are nowadays calculated as standard scores,
but with the mean set to 100 and the standard deviation set to 15. Many other tests use a form of standard scores
called T-scores, whereby the mean is set to 50 and the standard deviation to 10. Scores on the well-known SAT
examination, used in admissions to higher education in the United States, were originally standard scores for which
the mean was set to 500 and the standard deviation to 100.
6 Individual Differences and Personality
The correlation coefficient tells us an important fact about how people’s score on the
two variables are related: A difference of 1 standard deviation unit on one variable is
associated with a difference of r standard deviation units on the other variable. Let us
consider some examples that show what this means.
Suppose that variables X and Y have a perfect positive correlation with each other
(i.e., r ¼ 1): the higher variable X is, the higher variable Y must be, and vice versa. In this
case, we know that a person’s level of variable X (expressed in z-score units) will be equal
to his or her level of variable Y (expressed in z-score units). For example, a person who is
1 standard deviation above the mean on variable X must also be 1 standard deviation
above the mean on variable Y. Likewise, a person who is 2 standard deviations below the
mean on variable Y must also be 2 standard deviations below the mean on variable X.
Figure 1-2 (panel a) shows a graph that depicts what a correlation of +1 looks like;
notice that the dots (each of which represents a person’s score on the two variables) make
a straight line from the lower left of the graph (indicating low levels of both variable X and
variable Y) to the upper right of the graph (indicating high levels of both variable X and
variable Y).
Now suppose instead that variables X and Y have a perfect negative correlation with
each other (i.e., r ¼ 1): the higher variable X is, the lower variable Y must be, and vice
versa. In this case, we know that a person’s level of variable X (expressed in z-score units)
will be equal in size but opposite in sign to his or her level of variable Y (expressed in
z-score units). A person who is 1.5 standard deviations above the mean on variable X
must be 1.5 standard deviations below the mean on variable Y. Likewise, a person who is
0.5 standard deviations above the mean on variable Y must be 0.5 standard deviations
below the mean on variable X.
Figure 1-2 (panel b) shows a graph that depicts what a correlation of 1 looks like;
notice that the dots (each of which represents a person’s score on the two variables) make
a straight line from the upper left of the graph (indicating high levels of Yand low levels
of X) to the lower right of the graph (indicating low levels of Y and high levels of X).
It is unusual for two variables to correlate perfectly with each other, either positively
or negatively. In fact, even when we measure the same characteristic in two different
ways, we usually find a correlation smaller than +1, due to various sources of error in
measurement (as we will discuss later in this chapter, in the section on Reliability). When
the correlation between two variables is not +1 or 1, then we do not know exactly
what a person’s level of one variable will be, simply by knowing his or her level of the
other variable. However, for a large group of people who have a given level of one
variable, the correlation does let us know roughly what the average level of the other
variable will be for those particular people.
For example, if the correlation between variable X and variable Y is .50, and if we
have some people who are 2 standard deviations above the mean on variable X, then
their average level on variable Y will be about 1 standard deviation above the mean
Basic Concepts in Psychological Measurement 7
(a) r = +1.00 (b) r = −1.00
(c) r = +.50 (d) r = −.50 (e) r = .00
Figure 1-2 Scatterplots showing scores on variable Y (vertical axis) and variable X (horizontal axis)
having correlations of (a) +1.00, (b) 1.00, (c) +.50, (d) .50, and (e) .00.
(because 2 .50 ¼ 1). Likewise, if the correlation between variable X and variable
Y is .50, and if we have some people who are 2 standard deviations above the mean on
variable X, then we know that their average level on variable Y will be about 1 standard
deviations below the mean (because 2 .50 ¼ 1).
To get a sense of what a correlation of a given size “means”, consider the remaining
panels of Figure 1-2. Notice that in these panels, the association between X and Y is
not perfect, but you can still notice a tendency for X and Y to go together positively
(panel c) or negatively (panel d).
Figure 1-2 (c) shows a correlation of about +.50, which is a moderately large positive
correlation. As an example, this might be close to the correlation that you would find, for
a group of adults, between body weight and weightlifting ability. (On average, heavier
people can lift more weight than lighter people can, but there are still some light people
who can lift a lot and some heavy people who cannot lift very much.)
Figure 1-2 (d) shows a correlation of about .50, which is a moderately large
negative correlation. This might be close to the correlation that you would find, for
a group of adults, between body weight and distance running ability. (On average,
8 Individual Differences and Personality
heavier people cannot sustain as fast a running pace as lighter people can, but there are
still some heavy people who can run at a fast pace and some light people who cannot run
at a fast pace.)
Figure 1-2 (e) shows a correlation of .00, which means that the two variables are
completely unrelated to each other. A zero correlation (r ¼ .00) means that the two
variables are completely unrelated to each other. In this case, people’s levels of variable X
do not go along in either direction with their levels of variable Y. No matter what is the
level of variable X shown by a given person, your best guess for that person’s level of
variable Y is simply the average level for the entire sample of persons (i.e., 0 standard
deviations from the mean). In this kind of situation, we say that the two variables are
perfectly uncorrelated. Such a result might happen if we were to examine, in a group of
adults, the correlation between height and intelligence. On average, taller people are
probably no more and no less intelligent than shorter people are.
There is no strict rule as to what value of a correlation makes it a “small” (or “low”,
“weak”) as opposed to a “large” (or “high”, “strong”) correlation. But as a rough
guideline, correlations of between about .20 and +.20 are often considered small,
correlations between about .20 and .40 and between about .20 and .40 are
considered moderate in size, and correlations beyond .40 or beyond +.40 are
considered large. (A correlation of, say, +.80 is very large, and usually would be found
only when the same variable is being measured by two similar methods.)
Sometimes people tend to downplay the importance of correlations that are not
very large. But even a rather modest correlation can provide useful information. For
example, suppose that a personality variable correlates .25 with some outcome variable
(such as job performance or marital satisfaction). What would this mean? People with
very high levels of this personality variable (say, 2 standard deviation units above the
mean) would, on average, be about 0.5 standard deviation units above the mean on the
outcome variable (because 2 .25 ¼ .50). Likewise, people with very low levels of this
personality variable (say, 2 standard deviation units below the mean) would, on average,
be about 0.5 standard deviation units below the mean on the outcome variable
(because 2 .25 ¼ .50). Therefore, there would be about a 1 standard deviation
unit difference in the outcome variable between people who are very high and people
who are very low in the personality variable (because .50 e ( .50) ¼ 1.0). A difference
of 1 standard deviation unit is fairly large, so the information provided by this rather
modest correlation coefficient is important.
Here is the formula for calculating the correlation, r, between two variables, x and y,
where Zxi and Zyi are standard scores on those variables for each of N individuals, i:
N
P
ZxiZyi
i¼1
rxy ¼
N
Basic Concepts in Psychological Measurement 9
The idea is that for each of the N individuals we have measured, we find the product
of his or her standard scores on the two variables. Then we add together the products
obtained from each individual, and we divide this total by N, the number of individuals
we have measured. Notice that, if most people tend to have positive z-scores on both
variables or negative z-scores on both variablesdrather than a positive z-score on one
variable and a negative z-score on the otherdthen the products will be positive, and
their sum will be a positive number, thereby producing a positive correlation. If instead
many people have a positive z-score on one variable and a negative z-score on the other,
then the products will be negative, and their sum will be a negative number, thereby
3
producing a negative correlation.
1.2. ASSESSING QUALITY OF MEASUREMENT: RELIABILITY AND
VALIDITY
The preceding sections have described some of the basic statistical concepts that are used
by psychologists who measure people’s levels of various characteristics. But now we need
to consider the question of how to assess the quality of those measurements: When we try
to measure a characteristic in a sample of people, how do we know whether or not we
have been successful? In other words, how can we know how accurately we have measured
that characteristic? There are several aspects of measurement quality to be considered, but
these are generally classified into two broad properties known as reliability and validity.
1.2.1 Reliability
The reliability of a measurement is the extent to which it agrees with other measurements
of the same characteristic. When there is good agreement between measurements, this
tells us that they are assessing some real characteristic, rather than just being meaningless
random numbers. It is important to evaluate the reliability of our measurements, because
whenever we try to measure a characteristic, there is likely to be some random error in
those measurements.
There are several different ways in which reliability can be assessed, depending on the
kind of random error that we consider. For example, if we measure a characteristic by
3
Differences between two groups (such as men and women) on a variable can be expressed in terms of standard
deviation units. For example, if the standard deviation of height for men and for women is 8 centimetres, and if the
average man is 12 centimeters taller than the average woman, then we can say that the difference in height between
men and women equals 1.5 standard deviation units. This method of expressing the differences between groups is
related to the correlation coefficient. If, for example, men and women differ by 1.5 standard deviation units on
a variable, then this is equivalent to a correlation of .60 between that variable and a person’s sex; likewise, a difference
of 0.5 standard deviation units is equivalent to a correlation of about .25 and a difference of 1.0 standard deviation
units is equivalent to a correlation of about .45. (The formula for this conversion is beyond the scope of this
textbook.)
10 Individual Differences and Personality
using a test or scale that consists of several questions or “items”, then there is error in the
sense that each of those questions is imperfect as an indicator of the characteristic. If we
measure a characteristic by asking several raters to assess people’s levels of that character-
istic, then there is error in the sense that each of those raters is imperfect in making their
assessments. And on any given occasion when we measure a characteristic, there is error in
the sense that the characteristic may fluctuate across occasions. Let us consider the different
kinds of reliability, which deal with these different kinds of error in measurement.
1.2.1.1 Internal-consistency reliability
When evaluating the quality of a psychological measurement, we need to consider the
error that results from differences among the “items” or parts of the measurement, such
as the various questions on a test or a scale. Whenever we try to measure a characteristic,
we use a limited number of possible questions or statements, which we call “items.”
These items are combined to produce an overall score for the test or scale. You are
already familiar with this idea: For example, a golfer adds up his or her scores for each of
the 18 holes to get his or her overall score for the round of golf.
The process of averaging across the items is crucial for making a reliable measure-
ment. Any particular item is not a pure measure of the characteristic that we are trying to
assess. Instead, each item will assess that characteristic only partially, and will also assess
some other variable that is specific to that item. To the extent that an item measures some
specific variable of its own, rather than the characteristic that we are trying to assess, we
say that the item has “error” variance. But if we can average a person’s score across many
items, then the error associated with any single item will tend to be cancelled out. The
overall, average score will therefore have less error; it will be a more reliable indication of
whatever the items have in common.
To appreciate this point, consider the golf analogy again. A golfer’s score on any given
holewill depend partlyon howgood a golfer he or she is, but it will also depend on random
chance, and on specific features of that hole. Regardless of a person’s golf ability, he or she
might be very lucky or very unlucky on any particular hole. And even if two golfers were
equal in ability, one of them might find a particular hole to be easy, and the other might
find that hole to be difficult. Therefore, wewould not want to assess someone’s golf ability
based on his or her score on a single hole of golf. But if we were to find the golfer’s overall
score for an entire round of golf, then good luck or bad luck on any particular hole would
tend to be cancelled out. The overall score would give us a much better idea of how good
a golfer he or she really is. In other words, we get a more reliable indication of a golfer’s
ability by averaging out (or adding up) across the holes of a golf course.
But even when we average a person’s score across items, the resulting overall score
will not be perfectly reliabledit will still have some error. The reliability of a score that is
found by averaging responses to several items (assuming that the items have roughly equal
standard deviations) basically depends on two things: the number of items, and the