Pisa 2009 results.

398 Pages·2010·5.648 MB·English

by  OECD

PISA 2009 Results:
Students On Line
DIGITAL TECHNOLOGIES AND PERFORMANCE PISA 2009 Results:
VOLUME VI
Students On Line
Are students well prepared to meet the challenges of the future? Can they analyse, reason and communicate
their ideas effectively? Have they found the kinds of interests they can pursue throughout their lives as productive
members of the economy and society? The OECD Programme for International Student Assessment (PISA) seeks
to answer these questions through the most comprehensive and rigorous international assessment of student
knowledge and skills. Together, the group of countries and economies participating in PISA represents nearly 90% DIGITAL TECHNOLOGIES AND PERFORMANCE
of the world economy.
PISA 2009 Results presents the fi ndings from the most recent PISA survey, which focused on reading and also VOLUME VI
assessed mathematics and science performance. The report comprises six volumes:
• Volume I, What Students Know and Can Do: Student Performance in Reading, Mathematics and Science,
compares the knowledge and skills of students across countries. P
IS
• Volume II, Overcoming Social Background: Equity in Learning Opportunities and Outcomes, looks at how
A
successful education systems moderate the impact of social background and immigrant status on student and 2
school performance. 0
0
• Volume III, Learning to Learn: Student Engagement, Strategies and Practices, examines 15-year-olds’ motivation, 9
R
their engagement with reading and their use of effective learning strategies.
e
s
• Volume IV, What Makes a School Successful? Resources, Policies and Practices, examines how human, u
fi nancial and material resources, and education policies and practices shape learning outcomes. lt
s
• Volume V, Learning Trends: Changes in Student Performance Since 2000, looks at the progress countries have : S
made in raising student performance and improving equity in the distribution of learning opportunities. tu
d
• Volume VI, Students On Line: Digital Technologies and Performance, explores students’ use of information e
technologies to learn. n
t
s
PISA 2009 marks the beginning of the second cycle of surveys, with an assessment in mathematics scheduled O
n
for 2012 and one in science for 2015. L
in
e
THE OECD PROGRAMME FOR INTERNATIONAL STUDENT ASSESSMENT (PISA) D
PISA focuses on young people’s ability to use their knowledge and skills to meet real-life challenges. This orientation refl ects a change IG
in the goals and objectives of curricula themselves, which are increasingly concerned with what students can do with what they learn ITA
at school and not merely with whether they have mastered specifi c curricular content. PISA’s unique features include its: L T
– Policy orientation, which highlights differences in performance patterns and identifi es features common to high-performing students, EC
schools and education systems by linking data on learning outcomes with data on student characteristics and other key factors that H
N
shape learning in and outside of school. O
L
– Innovative concept of “literacy”, which refers both to students’ capacity to apply knowledge and skills in key subject areas and to their O
G
ability to analyse, reason and communicate effectively as they pose, interpret and solve problems in a variety of situations. IE
S
– Relevance to lifelong learning, which goes beyond assessing students’ competencies in school subjects by asking them to report on A
N
their motivation to learn, their beliefs about themselves and their learning strategies. D
– Regularity, which enables countries to monitor their progress in meeting key learning objectives. PE
R
– Breadth of geographical coverage and collaborative nature, which, in PISA 2009, encompasses the 34 OECD member countries and F
O
41 partner countries and economies. R
M
A
N
C
E
–
Please cite this publication as: VO
L
OECD (2011), PISA 2009 Results: Students On Line: Digital Technologies and Performance (Volume VI), U
M
PISA, OECD Publishing. E
V
http://dx.doi.org/10.1787/9789264112995-en I
This work is published on the OECD iLibrary, which gathers all OECD books, periodicals and statistical
databases. Visit www.oecd-ilibrary.org, and do not hesitate to contact us for more information.
Programme for International Student Assessment
ISBN 978-92-64-11291-9 -:HSTCQE=VVW^V^:
98 2011 03 1 P
www.oecd.org/publishing

PISA 2009 Results:
Students On Line
Digital technologies
anD Performance
(Volume Vi)
this work is published on the responsibility of the secretary-general of the oecD. the opinions
expressed and arguments employed herein do not necessarily reflect the official views of the
organisation or of the governments of its member countries.
Please cite this publication as:
oecD (2011), PISA 2009 Results: Students on Line: Digital Technologies and Performance (Volume VI)
http://dx.doi.org/10.1787/9789264112995-en
isBn 978-92-64-11291-9 (print)
isBn 978-92-64-11299-5 (PDf)
the statistical data for israel are supplied by and under the responsibility of the relevant israeli authorities. the use of such data
by the oecD is without prejudice to the status of the golan heights, east Jerusalem and israeli settlements in the West Bank
under the terms of international law.
Photo credits:
getty images © ariel skelley
getty images © geostock
getty images © Jack hollingsworth
stocklib image Bank © Yuri arcurs
corrigenda to oecD publications may be found on line at: www.oecd.org/publishing/corrigenda.
Pisatm, oecD/Pisatm and the Pisa logo are trademaks of the organisation for economic co-operation and Development (oecD).
all use of oecD trademarks is prohibited without written permission from the oecD.
© oecD 2011
You can copy, download or print oecD content for your own use, and you can include excerpts from oecD publications, databases and multimedia
products in your own documents, presentations, blogs, websites and teaching materials, provided that suitable acknowledgment of oecD as source
and copyright owner is given. all requests for public or commercial use and translation rights should be submitted to [email protected]. requests for
permission to photocopy portions of this material for public or commercial use shall be addressed directly to the copyright clearance center (ccc)
at [email protected] or the centre français d’exploitation du droit de copie (cfc) at [email protected].
Foreword
one of the ultimate goals of policy makers is to enable citizens to take advantage of a globalised world economy.
this is leading them to focus on the improvement of education policies, ensuring the quality and sustainability
of service provision, a more equitable distribution of learning opportunities and stronger incentives for greater
efficiency in schooling.
Such policies all hinge on reliable information on how well education systems prepare students for life. most
countries monitor students’ learning and the performance of schools. But in a global economy, the yardstick for
success is no longer improvement by national standards alone, but how education systems perform internationally.
the oeCd has taken that challenge up by developing PISA, the Programme for International Student Assessment,
which evaluates the quality, equity and efficiency of school systems in some 70 countries that, together, make up
nine-tenths of the world economy. PISA represents a commitment by governments to monitor the outcomes of
education systems regularly within an internationally agreed framework and it provides a basis for international
collaboration in defining and implementing educational policies.
the results from the PISA 2009 assessment reveal wide differences in education outcomes, both within and across
countries. the education systems that have been able to secure strong and equitable learning outcomes, and to
mobilise rapid improvements, show others what is possible to achieve. naturally, GdP per capita influences
educational success, but this only explains 6% of the differences between average student performance. the other
94% reflect the potential for public policy to make a difference. the stunning success of Shanghai-China, which
tops every league table in this assessment by a clear margin, show what can be achieved with moderate economic
resources and in a diverse social context. In mathematics, more than a quarter of Shanghai’s 15-year-olds can
conceptualise, generalise, and creatively use information based on their own investigations and modelling of
complex problem situations. they can apply insight and understanding and develop new approaches and strategies
for addressing novel situations. In the oeCd area, just 3% of students reach that level of performance.
While better educational outcomes are a strong predictor of economic growth, wealth and spending on education
alone are no guarantee for better educational outcomes. overall, PISA shows that an image of a world divided
neatly into rich and well-educated countries and poor and badly-educated countries is out of date.
this finding represents both a warning and an opportunity. It is a warning to advanced economies that they cannot
take for granted that they will forever have “human capital” superior to that in other parts of the world. At a time of
intensified global competition, they will need to work hard to maintain a knowledge and skill base that keeps up
with changing demands.
PISA underlines, in particular, the need for many advanced countries to tackle educational underperformance so
that as many members of their future workforces as possible are equipped with at least the baseline competencies
and skills that enable them to participate in social and economic development. the high social and economic cost
of poor educational performance in advanced economies risks otherwise to become a significant drag on economic
development. At the same time, the findings show that poor skills are not an inevitable consequence of low national
income – an important outcome for countries that need to achieve more with less.
But PISA also shows that there is no reason for despair. Countries from a variety of starting points have shown the
potential to raise the quality of educational outcomes substantially. Korea’s average performance was already high in
2000, but Korean policy makers were concerned that only a narrow elite achieved levels of excellence in PISA. Within
less than a decade, Korea was able to double the share of students demonstrating excellence in reading literacy.
3
PISA 2009 ReSultS: StudentS on lIne – Volume VI © OECD 2011
Foreword
A major overhaul of Poland’s school system helped to dramatically reduce performance variability among schools,
reduce the share of poorly performing students and raise overall performance by the equivalent of more than half a
school year. Germany was jolted into action when PISA 2000 revealed below-average performance and large social
disparities in results, and has been able to make progress on both fronts. Israel, Italy and Portugal have moved closer
to the oeCd average and Brazil, Chile, mexico and turkey are among the countries with impressive gains from very
low levels of performance.
But the greatest value of PISA lies in inspiring national efforts to help students to learn better, teachers to teach better,
and school systems to become more effective.
A closer look at high-performing and rapidly improving education systems shows that these have much in common
that transcends differences in their history, culture and economic evolution.
First, while most nations declare their commitment to education, the test comes when these commitments are
weighed against others. How do they reward teachers compared to the way they pay other highly-skilled workers?
How are education credentials weighed against other qualifications when people are being considered for jobs?
Would you want your child to be a teacher? How much attention do the media pay to schools and schooling? Which
matters more, a community’s standing in the sports leagues or its standing in the student academic achievement
league tables? Are parents more likely to encourage their children to study longer and harder or to want them to
spend more time with their friends or playing sports?
In the most successful education systems, the political and social leaders have persuaded their citizens to make the
choices needed to show that they value education more than other things. But placing a high value on education
will get a country only so far if the teachers, parents and citizens of that country believe that only some subset of
the nation’s children can or need to achieve world class standards. this report shows clearly that education systems
built around the belief that students have different pre-ordained professional destinies to be met with different
expectations in different school types tend to be fraught with large social disparities. In contrast, the best-performing
education systems embrace the diversity in students’ capacities, interests and social background with individualised
approaches to learning.
Second, high-performing education systems stand out with clear and ambitious standards that are shared across the
system, focus on the acquisition of complex, higher order thinking skills, and are aligned with high stakes gateways
and instructional systems. In these education systems, everyone knows what is required to get a given qualification,
in terms both of the content studied and the level of performance that has to be demonstrated to earn it. Students
cannot go on to the next stage of their life – be it work or further education – unless they show that they are qualified
to do so. they know what they have to do to realise their dream and they put in the work that is needed to achieve it.
third, the quality of an education system cannot exceed the quality of its teachers and principals, since student
learning is ultimately the result of what goes on in classrooms. Corporations, professional partnerships and national
governments all know that they have to pay attention to how the pool is established from which they recruit; how
they recruit; the kind of initial training their recruits get before they present themselves for employment; how they
mentor new recruits and induct them into their service; what kind of continuing education they get; how their
compensation is structured; how they reward their best-performers and how they improve the performance of
those who are struggling; and how they provide opportunities for the best-performers to acquire more status and
responsibility. many of the world’s best-performing education systems have moved from bureaucratic “command
and control” environments towards school systems in which the people at the frontline have much more control
of the way resources are used, people are deployed, the work is organised and the way in which the work gets
done. they provide considerable discretion to school heads and school faculties in determining how resources
are allocated, a factor which the report shows to be closely related to school performance when combined with
effective accountability systems. And they provide an environment in which teachers work together to frame what
they believe to be good practice, conduct field-based research to confirm or disprove the approaches they develop,
and then assess their colleagues by the degree to which they use practices proven effective in their classrooms.
last but not least, the most impressive outcome of world class education systems is perhaps that they deliver high-
quality learning consistently across the entire education system such that every student benefits from excellent
learning opportunity. to achieve this, they invest educational resources where they can make the greatest difference,
they attract the most talented teachers into the most challenging classrooms, and they establish effective spending
choices that prioritise the quality of teachers.
4
© OECD 2011 PISA 2009 ReSultS: StudentS on lIne – Volume VI
Foreword
these are, of course, not independently conceived and executed policies. they need to be aligned across all aspects
of the system, they need to be coherent over sustained periods of time, and they need to be consistently implemented.
the path of reform can be fraught with political and practical obstacles. moving away from administrative and
bureaucratic control toward professional norms of control can be counterproductive if a nation does not yet have
teachers and schools with the capacity to implement these policies and practices. Pushing authority down to lower
levels can be as problematic if there is not agreement on what the students need to know and should be able to do.
Recruiting high-quality teachers is not of much use if those who are recruited are so frustrated by what they perceive
to be a mindless system of initial teacher education that they will not participate in it and turn to another profession.
thus a county’s success in making these transitions depends greatly on the degree to which it is successful in
creating and executing plans that, at any given time, produce the maximum coherence in the system.
these are daunting challenges and devising effective education policies will become ever more difficult as schools
needs to prepare students to deal with more rapid change than ever before, for jobs that have not yet been created,
to use technologies that have not yet been invented and to solve economic and social challenges that we do not yet
know will arise. But those school systems that do well today, as well as those that have shown rapid improvement,
demonstrate that it can be done. the world is indifferent to tradition and past reputations, unforgiving of frailty and
complacency and ignorant of custom or practice. Success will go to those individuals and countries that are swift
to adapt, slow to complain and open to change. the task of governments will be to ensure that countries rise to this
challenge. the oeCd will continue to support their efforts.
***
the report is the product of a collaborative effort between the countries participating in PISA, the experts and
institutions working within the framework of the PISA Consortium, and the oeCd Secretariat. this volume of
the report was drafted by a team led by Juliette mendelovits with guidance from the PISA Reading expert Group
and the oeCd PISA team, led by Andreas Schleicher. Contributing authors were Alla Berezner, John Cresswell,
miyako Ikeda, Irwin Kirsch, dominique lafontaine, tom lumley, Christian monseur, Johannes naumann,
Soojin Park and Jean-François Rouet. editorial and analytical support were provided by Francesca Borgonovi,
michael davidson, maciej Jakubowski, Guillermo montt, oscar Valiente, Sophie Vayssettes, elisabeth Villoutreix
and Pablo Zoido of the oeCd PISA team. Further advice was provided by marilyn Achiron, Simone Bloem,
marika Boiron, Simon Breakspear, Henry Braun, nihad Bunar, Jude Cosgrove, Aletta Grisay, tim Heemsoth,
donald Hirsch, david Kaplan, Henry levin, Barry mcCrae, dara Ramalingam, Wolfgang Schnotz, eduardo Vidal-
Abarca and Allan Wigfield. Administrative support was provided by Juliet evans and diana tramontano.
the PISA assessment instruments and the data underlying the report were prepared by the PISA Consortium, under
the direction of Raymond Adams at the Australian Council for educational Research (ACeR) and Henk moelands
from the dutch national Institute for educational measurement (CIto). the expert group that guided the preparation
of the reading assessment framework and instruments was chaired by Irwin Kirsch.
the development of the report was steered by the PISA Governing Board, which is chaired by lorna Bertrand
(united Kingdom), with Beno Csapo (Hungary), daniel mcGrath (united States) and Ryo Watanabe (Japan) as vice
chairs. Annex C of the volumes lists the members of the various PISA bodies, as well as the individual experts and
consultants who have contributed to this report and to PISA in general.
Intel Corporation provided a generous financial contribution towards publishing this volume.
Angel Gurría
OECD Secretary-General
5
PISA 2009 ReSultS: StudentS on lIne – Volume VI © OECD 2011

Table of Contents
ExEcutivE summary ...........................................................................................................................................................................................................19
introduction to Pisa ...................................................................................................................................................................................................23
rEadEr’s GuidE ........................................................................................................................................................................................................................29
ChapTEr 1 contExt of thE Pisa diGital rEadinG assEssmEnt............................................................................................31
new technologies for text, new ways of reading ...................................................................................................................................................32
• differences in the readability and usability of text .....................................................................................................................................33
• new features of digital texts .......................................................................................................................................................................................34
impact of digital texts on reading literacy ..................................................................................................................................................................36
• Which aspects of reading are affected by digital text? .............................................................................................................................36
some issues for assessing digital reading .....................................................................................................................................................................37
conclusions ......................................................................................................................................................................................................................................38
ChapTEr 2 studEnt PErformancE in diGital and Print rEadinG .................................................................................39
digital reading................................................................................................................................................................................................................................40
• texts ............................................................................................................................................................................................................................................40
• Cognitive processes .........................................................................................................................................................................................................42
• Situation ...................................................................................................................................................................................................................................44
how the Pisa 2009 reading results are reported ..................................................................................................................................................44
• How the PISA 2009 digital reading tests were designed, analysed and scaled ......................................................................44
What students can do in digital reading ......................................................................................................................................................................49
• Students reading the different levels of proficiency on the digital reading scale ...................................................................49
• Average level of proficiency .......................................................................................................................................................................................51
• Gender differences in performance on the digital reading scale ......................................................................................................52
Examples of digital reading items from the Pisa 2009 assessment ..........................................................................................................54
• IWANTTOHELP ..................................................................................................................................................................................................................54
• SmELL........................................................................................................................................................................................................................................60
• JOb SEArcH ........................................................................................................................................................................................................................66
similarities and differences between digital and print reading assessment ........................................................................................71
• Framework characteristics and test construct .................................................................................................................................................71
• test design and operational characteristics ......................................................................................................................................................73
a comparison of performance in digital and print reading ............................................................................................................................74
• Students reaching the different levels of proficiency .................................................................................................................................74
• Average level of proficiency .......................................................................................................................................................................................76
• Gender differences in performance on the digital and print reading scales .............................................................................78
a composite scale for digital and print reading .....................................................................................................................................................80
• Students reaching the different levels of proficiency on the composite reading scale .......................................................82
• Average level of proficiency .......................................................................................................................................................................................83
• Gender differences in performance on the composite reading scale ............................................................................................85
conclusions ......................................................................................................................................................................................................................................86
7
PISA 2009 ReSultS: StudentS on lIne – Volume VI © OECD 2011
Table of ConTenTs
ChapTEr 3 naviGation in thE Pisa 2009 diGital rEadinG assEssmEnt .....................................................................89
General patterns in the relationship between navigation and performance in digital and print reading .....................90
• Relevance of pages...........................................................................................................................................................................................................91
• Indicators used to describe navigation ...............................................................................................................................................................91
• distribution of navigation indices at the country level ............................................................................................................................93
• Relationships among navigation, print and digital reading ..................................................................................................................97
• Correlations between navigation and performance ...................................................................................................................................97
• Regression of digital reading performance on print reading and navigation ............................................................................98
• non-linear effects of navigation on digital reading performance ..................................................................................................100
case studies: navigation behaviour of students in selected digital reading tasks .......................................................................102
• tasks analysed in the case studies .......................................................................................................................................................................103
• IWANTTOHELP ...............................................................................................................................................................................................................105
• SmELL.....................................................................................................................................................................................................................................113
• JOb SEArcH .....................................................................................................................................................................................................................117
conclusions ...................................................................................................................................................................................................................................120
ChapTEr 4 rElationshiPs BEtWEEn diGital rEadinG PErformancE and studEnt BackGround,
EnGaGEmEnt and rEadinG stratEGiEs ......................................................................................................................................................123
family background ...................................................................................................................................................................................................................124
• Socio-economic background .................................................................................................................................................................................124
• Immigrant status ..............................................................................................................................................................................................................127
• languages spoken at home .....................................................................................................................................................................................127
• Performance differences within and between schools ..........................................................................................................................128
student engagement and attitudes................................................................................................................................................................................128
• engagement in reading and digital reading proficiency .......................................................................................................................129
• do students who enjoy reading read better on line? .............................................................................................................................131
• the association between the diversity of print material students read and digital reading proficiency ...............132
• online reading practices ...........................................................................................................................................................................................133
• Gender differences in online reading practices .........................................................................................................................................134
• online reading practices and digital reading proficiency ..................................................................................................................135
reading strategies .....................................................................................................................................................................................................................138
• Awareness of strategies to understand and remember information ..............................................................................................138
• Awareness of effective strategies to summarise information .............................................................................................................138
model for the relationship between reading performance and student background characteristics ............................139
• Parents’ occupation .......................................................................................................................................................................................................139
• Parents’ education ..........................................................................................................................................................................................................139
• number of books in the home ..............................................................................................................................................................................139
• Cultural possessions .....................................................................................................................................................................................................139
• Home educational resources ..................................................................................................................................................................................139
conclusions ...................................................................................................................................................................................................................................140
ChapTEr 5 studEnts’ familiarity With information and communication tEchnoloGiEs .......143
students’ access to ict .........................................................................................................................................................................................................144
• the number of students who have never used a computer ................................................................................................................144
• Students’ access to a computer and the Internet at home ..................................................................................................................146
• Students’ access to computers and the Internet at school ..................................................................................................................150
how students use technology at school and at home .....................................................................................................................................157
• Students’ use of ICt at home ..................................................................................................................................................................................157
• Students’ use of ICt at school ................................................................................................................................................................................162
students’ attitudes towards and self-confidence in using computers ..................................................................................................167
• Students’ attitudes towards using computers ...............................................................................................................................................167
• Students’ confidence in computer use and technical proficiency ................................................................................................170
conclusions ...................................................................................................................................................................................................................................175
8
© OECD 2011 PISA 2009 ReSultS: StudentS on lIne – Volume VI