ebook img

Use of an Aptitude Test in University Entrance – A Validity - ERIC PDF

130 Pages·2007·0.59 MB·English
by  
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Use of an Aptitude Test in University Entrance – A Validity - ERIC

R ESEARCH Use of an Aptitude Test in University Entrance – A Validity Study Relationships between SAT® scores, attainment measures and background variables Catherine Kirkup, Ian Schagen, Rebecca Wheater Jo Morrison and Chris Whetton National Foundation for Educational Research Research Report RR846 Research Report No 846 Use of an Aptitude Test in University Entrance – A Validity Study Relationships between SAT® scores, attainment measures and background variables Catherine Kirkup, Ian Schagen, Rebecca Wheater Jo Morrison and Chris Whetton National Foundation for Educational Research The views expressed in this report are the authors’ and do not necessarily reflect those of the Department for Education and Skills. © NFER Trading Ltd 2007 ISBN 978 1 84478 934 4 Acknowledgements The NFER project team for this work consisted of: Chris Whetton Project Director Catherine Kirkup Researchers Rebecca Wheater Stuart Gordon Design Margaret Parfitt Project Administration Assistant Anne Milne Research Data Services Dave Hereward Ed Wallis Ian Schagen Statistics, Research and Analysis Group Jo Morrison Ben Styles Tilaye Yeshanew The NFER also gratefully acknowledges the advice given by members of the project steering group and the assistance of the College Board and Educational Testing Services for providing and scoring the SAT Reasoning TestTM. This project is co-funded by the Department for Education and Skills, the Sutton Trust, the National Foundation for Educational Research and the College Board. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Department for Education and Skills, the Sutton Trust or the College Board. © NFER Trading Ltd 2007 Table of contents 1 Executive summary.......................................................................1 1.1 Introduction 1 1.2 Key findings 1 1.3 Structure of the report 2 2 Introduction....................................................................................3 2.1 Background 3 2.2 Objectives of the study 5 3 Methodology..................................................................................6 3.1 The SAT Reasoning TestTM 6 3.1.1 Contacting schools .....................................................................6 3.1.2 Materials.....................................................................................7 3.1.3 Response rates...........................................................................8 3.2 Data matching 9 3.3 Student survey – spring 2006 9 3.4 Student survey – autumn 2006 9 4 Student samples..........................................................................11 4.1 Representation of the main sample 11 4.2 Student survey sub-samples 14 4.2.1 Additional background characteristics from the spring survey..17 4.2.2 Additional information from the autumn survey.........................20 5 Relationships between SAT® scores and attainment............23 5.1 Descriptive statistics 23 5.1.1 Attainment data.........................................................................23 5.1.2 SAT® data................................................................................25 5.3 Exploring the relationships between the main study variables 34 6 Disadvantaged students: Can the SAT® identify students with the potential to benefit from higher education whose ability is not adequately reflected in their A level results because of their (economically or educationally) disadvantaged circumstances? ........................................................ 37 6.1 Conclusions 42 7 High-achieving students: Can the SAT® distinguish helpfully between the most able applicants who get straight As at A level?.........................................43 7.1 Conclusions 51 8 Questionnaire findings...............................................................52 8.1 Spring 2006 questionnaire 52 8.1.1 Future Plans.............................................................................52 8.1.2 Views on university...................................................................53 8.1.3 Views on school........................................................................53 8.1.4 Careers advice..........................................................................54 8.1.5 Students not intending to go to university.................................54 8.1.6 Opinions of the SAT Reasoning Test™....................................55 8.2 Autumn 2006 questionnaire 55 8.2.1 HE entrants in 2006..................................................................55 8.2.2 Gap year students....................................................................56 8.2.3 No immediate plans for HE.......................................................56 9 Future phases of the research...................................................57 10 References...................................................................................59 Appendix 1: Spring 2006 Student Questionnaire Appendix 2: Autumn 2006 Student Questionnaire Appendix 3: SAT® Descriptive Statistics Appendix 4: Functioning of the SAT® 1 Executive summary 1.1 Introduction In September 2005 the National Foundation for Educational Research (NFER) in association with the Department for Education and Skills (DfES), the Sutton Trust and the College Board, began a five-year research study to examine the validity of an aptitude test in higher education admissions. This report describes and explores the relationships between scores on the SAT Reasoning TestTM (also known in brief as the SAT®), attainment measures (A levels and GCSEs) and background characteristics of the student sample. The SAT® has three measures: Critical Reading, Mathematics and Writing. 1.2 Key findings The number of students who took the SAT® in autumn 2005 and agreed that their data could be used within the research was 9022. Of these SAT® participants, 8041 students were matched to both GCSE and A level attainment data and form the basis of the analysis for this report (hereafter referred to as the main sample). The key findings to date were as follows: • The mean SAT® scores of the English sample were very similar to US mean scores and individual items functioned in a similar way in comparable samples of students. • The correlation between A level scores and the SAT® was 0.64 (i.e. high A level scores were generally associated with high scores on the SAT®). This correlation was higher than correlations of between 0.33 and 0.50 in a previous pilot study (McDonald et al., 2001a), most likely due to restructuring of both measures in the intervening years. • SAT® scores and total A level (or equivalent) score points were generally related in a similar way to a number of educational and socio-economic factors, including the type of institution attended, ethnicity, eligibility for free school meals (FSM), etc. • Female students had higher total GCSE and A level points scores and achieved significantly higher scores on the SAT® Writing component than male students. Male students performed significantly better on the SAT® Mathematics component and on the SAT® as a whole. • Two regression analyses, one controlling for A level total scores and one controlling for both A level scores and average prior attainment at GCSE, showed that female students, some ethnic minorities, students with special educational needs (SEN) and students learning English as an additional 1 language (EAL) appeared to perform less well on the SAT® than expected compared to default categories (i.e. boys, white students, etc). • Students in independent schools did better than expected on the SAT®, and students in FE colleges did worse, when A level results alone were taken into account. In both cases, they neither did better nor worse than expected when average prior attainment at GCSE was also taken into account. • Students who had gained three or more grade As at A level achieved significantly higher scores than the rest of the sample on all components of the SAT®. However there was considerable variation in the SAT® scores of this group offering the possibility of differentiation between high ability HE candidates should the SAT® prove to be a valid predictor of degree outcomes. • Students who achieved very high SAT® scores were not always the same students who achieved three or more A grades at A level and vice versa. Students who achieved very high SAT® scores but did not form part of the ‘three A grades’ group tended to be male students. At this stage, the analysis was focussed on overall attainment and broad background variables. Further analyses will be carried out during the next twelve months, looking at differences between students according to their A level subjects and examining the social, educational and economic background variables using more complex statistical modelling of the data. This analysis will incorporate data on the attainment of the institutions attended, census data relating to the home postcodes of students in the sample and, where available, information supplied by the students themselves (e.g. parental education). Without higher educational outcomes it will not be possible, as yet, to answer the main research questions. However, this further analysis may reveal more about the performance of students across the SAT®, A levels and GCSEs according to more sensitive measures of disadvantage. 1.3 Structure of the report Section 2 describes the background to the research and the aims and objectives of the study. Section 3 outlines the methodologies employed and section 4 details the representation and background characteristics of the main SAT® sample and the subsequent sub-samples who took part in the student surveys in spring and autumn 2006. In section 5 of this report we report the findings of the analysis of the quantitative data examining the relationships between the SAT® and both A level and GCSE attainment. These results are then explored in relation to two key research questions in sections 6 and 7. Section 8 presents the findings from the questionnaire surveys and section 9 outlines future potential phases of this research study and suggested milestones when further results will be disseminated. 2 2 Introduction In September 2005 the National Foundation for Educational Research (NFER) in association with the Department for Education and Skills (DfES), the Sutton Trust and the College Board, began a five-year research study to examine the validity of an aptitude test in higher education admissions. This report describes and explores the relationships between scores on the SAT Reasoning TestTM (also known in brief as the SAT®), attainment measures (A levels and GCSEs) and background characteristics of the student sample. It incorporates information given in the unpublished interim report (Kirkup et al., 2006) on the administration and technical functioning of the SAT®, the characteristics of the student sample and a preliminary analysis of the first student survey. 2.1 Background Higher education brings considerable benefits to graduates in terms of salary, job security, employment opportunity, and so on. Although the number of students entering higher education (HE) has grown enormously in recent years, some groups are still under-represented. In a report into the participation in higher education over the period 1994-2000 (HEFCE, 2005) it was noted that young people living in the most advantaged 20 per cent of areas were five or six times more likely to go into higher education than those from the least advantaged 20 per cent of areas. The benefits of higher education vary according to the course studied and the institution attended. The demand for university places generally exceeds the supply available, particularly for popular courses or popular institutions. Where there are competing applicants, universities and other higher education institutions have to assess the merit and potential of each student in order to decide who to admit. In its report, the Admissions to Higher Education Steering Group, chaired by Professor Steven Schwartz (DfES, 2004), identified several issues to be addressed to bring about improvements to the admissions system in England and to ensure a fair and transparent system for all students.1 As the system stands at present, most of the offers of university places to prospective students are made on the basis of predicted grades rather than examination results. Although the process is moving towards a post-qualification application system, a difficulty that will remain for admissions staff is that they may have to choose from an increasingly large number of highly-qualified candidates who achieve a string of A grades in their A level examinations. A further issue identified by the Schwartz report was that the information used to assess university applicants may not be equally reliable. Although ‘prior educational attainment remains the best single indicator of success at undergraduate level’ (page 1 One recommendation of the report was the creation of a central source of expertise and advice on admissions issues, resulting in the Supporting Professionalism in Admissions (SPA) Programme. 3

Description:
Admissions Test for Law (LNAT). Oxford University has already introduced a. History Aptitude Test and is planning to introduce something similar for English.
See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.