Evaluation and Application of Instruments Measuring Spatial Ability and Attitude for College ...

193 Pages·2015·3.16 MB·English

University of South Florida
Scholar Commons
Graduate Theses and Dissertations Graduate School
11-20-2014
Evaluation and Application of Instruments
Measuring Spatial Ability and Attitude for College
Chemistry Students
Xiaoying Xu
University of South Florida, [email protected]
Follow this and additional works at:https://scholarcommons.usf.edu/etd
Part of theChemistry Commons
Scholar Commons Citation
Xu, Xiaoying, "Evaluation and Application of Instruments Measuring Spatial Ability and Attitude for College Chemistry Students"
(2014).Graduate Theses and Dissertations.
https://scholarcommons.usf.edu/etd/5599
This Dissertation is brought to you for free and open access by the Graduate School at Scholar Commons. It has been accepted for inclusion in
Graduate Theses and Dissertations by an authorized administrator of Scholar Commons. For more information, please contact
[email protected].
Evaluation and Application of Instruments Measuring Spatial Ability

and Attitude for College Chemistry Students

by

Xiaoying Xu

A dissertation submitted in partial fulfillment
of the requirements for the degree of
Doctor of Philosophy
Department of Chemistry
College of Arts and Sciences
University of South Florida

Major Professor: Jennifer E. Lewis, Ph.D.
Xiao Li, Ph.D.
Jeffrey R. Raker, Ph.D.
Robert F. Dedrick, Ph.D.
Robert Potter, Ph.D.

Date of Approval:
November 20, 2014

Keywords: reliability, validity, factor analysis, chemistry performance, higher education
Copyright © 2014, Xiaoying Xu
Acknowledgments
First, I would like to thank my major advisor, Dr. Lewis, for her patience and
encouragement for me to develop the knowledge and skills in educational research and
evaluation, and her tremendous amount of effort to help me accomplish this dissertation work.
Without her support, my life during the past years might not have been so meaningful, and this
dissertation work would be not possible. I also greatly appreciate the instrumental roles of the
committee members, Drs. Robert Dedrick, Xiao Li, Robert Potter, and Jeffrey R. Raker, for the
academic advice, feedback, and the commitment for my graduation work and manuscript
preparation. Similarly I acknowledge the support and recommendations from my collaborators,
Drs. Kim, Alhooshani, Southam, Brandriet, Bretz, Jiang, Garcia and Sachel Villafañe.
A special thanks to former and current chemical education research group members
Alicia, Teresa, Keily, Yujuan, Janelle, Benjamin, Li, Todd, Adrian, and Scott for their support
and friendship. Also I appreciate the help from many people including but not limited to: Drs.
Sandi-Urena, Guida, Ferron, Kromrey, general chemistry classroom TAs, chemistry office staffs
and my class instructors.
Last but not least, I am in gratitude to my family, Charley, Feng, Wangfeng, Lachun,
Ying and Zhirong for the love and encouragement during the dissertation.
Table of Contents
List of Tables v
List of Figures vii

Abstract viii

I. Introduction 1
The Need for Transforming STEM Education 1
Important Factors in STEM Education 2
Spatial Ability 2
Attitude toward Science 3
Role of Measurement Tools in STEM Education 5
Purposes of this Work 5
References 10

II. Methods 15
Instruments 15
Data Collection and Participants 16
Data Analysis Strategies 18
Psychometric Analysis 18
Internal consistency reliability 19
Internal structure 20
Relations to other Variables by Structural Equation Modeling 21
References 22

III. Sex Difference in Spatial Ability for College Students and Exploration of
Measurement Invariance 26
Note to Reader 26
Introduction 26
Research Methods 30
Instruments 31
Participants and Data Collection 32
Statistical Data Analysis 33
Results and Discussion 35
Descriptive Statistics 35
Test-retest Relationship for Temporal Stability 36
Confirmatory Factor Analysis 37
Measurement Invariance between the Sexes 42
Sex Difference in Spatial Ability 43
Conclusion 45
i
References 48

IV. The Role of Spatial Ability in Students’ Progression through Organic Chemistry 53
Note to Reader 53
Why Spatial Ability? 53
Research Questions 56
Research Methods 56
Results 60
Descriptive Statistics 60
Comparison of Students Who Can Or Cannot Mentally Visualize 61
SEM for the Effect of Mental Rotations on Organic Chemistry Course Grade 63
Discussion 67
References 72

V. Attitude toward the Subject of Chemistry in Australia: An ALIUS and POGIL
Collaboration to Promote Cross-national Comparisons 76
Note to Reader 76
Introduction 76
Instrument 78
Sample 78
Data Analysis Methods 79
Results 80
Descriptive Statistics 80
Validity Results from CFA 82
Reliability 83
Factor Score Comparison 83
Conclusions 85
Acknowledgements 86
References 86

VI. Gathering Psychometric Evidence for ASCIv2 to Support Cross-Cultural Attitudinal
Studies for College Chemistry Programs 89
Note to Reader 89
Introduction 89
The Need for Attitude 89
Measurement for Attitude 91
Research Questions 94
Settings 95
Research Methods 97
Instrument 97
Participants and Data Collection 98
Data Analysis 98
Results 100
Descriptive Statistics for Item Scores for KU 100
Two-Factor CFA Model Fit 102
Internal Consistency Reliability for ASCIv2 103
ii
Students’ Interpretation of Item 6 104
Attitudinal Profile 106
Discussion and Conclusions 108
References 111

VII. College Students’ Attitudes toward Chemistry, Conceptual Knowledge and
Achievement: Structural Equation Model Analysis 115
Note to Reader 115
Introduction 115
Student Achievement in College Chemistry 117
Attitude toward Chemistry and Relationship with Achievement 117
Prior Conceptual Knowledge and Relationship with Achievement 119
Math Ability and Relationship with Achievement 120
Conceptual Relationship among the Four Factors in Chemistry 121
Structural Equation Modeling (SEM) 121
Present Study 123
Method 124
Instruments 124
Attitude 124
Prior conceptual knowledge 125
Math ability 125
Chemistry achievement 126
Data Collection and Participants 126
Model Specification for SEM Analysis 127
Data Analysis 130
Descriptive Statistics and Assumption Checking for SEM 130
Structural Equation Model (SEM) 130
Results 130
Descriptive Statistics and Assumption Checking for SEM 130
Structural Equation Model Analysis 132
Model fit 132
Decomposition of relationships 134
Measurement model for ASCIv2 134
Measurement part for TMI 136
Relationships among the three predictors 136
Relationships with student achievement in college chemistry 137
Discussion and Implications 138
References 141

VIII. Conclusions and Implications 146
Summary Conclusion 146
Limitations and Implications 150
References 154

Appendices 159
Appendix A: Administration instruction for ROT 170
iii
Appendix B: Administration instruction for ASCIv2 174
Appendix C: Supplement for Chapter 3 175
Appendix D: Supplement for Chapter 4 177
Appendix E: Supplement for Chapter 7 177
Figure notes for Figures 7.2-7.6 177
Additional analysis 177
Measurement Invariance for ASCIV2 to Measure Attitude toward Chemistry 177
Unique Contribution of Spatial Ability When Attitude Is Controlled 178
References for Additional Analysis 179

iv
List of Tables
Table 2.1 Summary of instruments administration 17
Table 3.1 Demographic Information for the Study Sample: SAT Math by Sex 31
Table 3.2 Cronbach’s Alpha and Descriptive Statistics for ROT by Semester 36
Table 3.3 Descriptive Statistics of ROT Scores for Retakers 37
Table 3.4 Time Interval for Retaking the ROT Test 37
Table 3.5 Fit Statistics of Three Models for ROT in Spring 2010 38
Table 3.6 Fit Statistics of ROT for Three Models for Three Semesters 42
Table 3.7 Measurement Invariance Result across Sexes for CFA Bi-factor
Model 43
Table 3.8 Effect Size from MCFA Bi-factor Strong Invariance Model and
Cohen’s d 43
Table 3.9 Sex Difference by Degree of Angular Disparity for Two Item Types 45
Table 4.1 Descriptive of SAT Math, ROT, ACS, and Organic Chemistry
Grade by Sex 60
Table 4.2 Pearson Correlations among SATM, ROT, and ACS Scores, Course
Grade for Organic Chemistry I and II 61
Table 4.3 Mental Rotation vs Progression through Organic Chemistry Course
Sequence 62
Table 4.4 Standardized Path Coefficient and Effect Size for Chemistry
Performance 66
Table 5.1A Descriptive Statistics for Curtin after Recoding Items 1, 4, 5 and 7 81
Table 5.1B Descriptive Statistics for the US University for Comparison 81
Table 5.2 CFA Model Fit for the 2-Factor Solution 82
Table 5.3 CFA Item Loadings for the 2-Factor Solution 83
v
Table 5.4 Internal Consistency Reliability by Cronbach's Alpha for ASCIv2 84
Table 5.5 Factor Scores for each Subscale for US and Curtin Students 85
Table 6.1 Descriptive Statistics of Item Scores for KU 101
Table 6.2 CFA Item Loadings for the 2-Factor Solution and Item-total
Correlation within Each Intended Subscale 103
Table 6.3 Inter-rater Agreement Data For Whether an Interpretation of Item 6
Is a Good Indicator of The Intended Scale 104
Table 6.4 CFA Model Fit for the 2-Factor Solution Without Item 6 107
Table 6.5 Factor Scores for each Subscale for KU, WU and SE 107
Table 7.1 Variables in ASCIv2 Instrument 124
Table 7.2 Items in TMI Instrument 125
Table 7.3 Demographics: Number and Percentage of Students by Sex and
Race/Ethnicity 127
Table 7.4 Descriptive Statistics for each Measured Variable 131
Table 7.5 Fit Indices and Predicted Variance for Models (N = 963) 131
Table 7.6 Standardized Correlation Coefficients 137
Table 7.7 Standardized Path Coefficient and Effect Size for the Regression on
Achievement 138
Table A3.1 Item Difficulty, Item-total Correlation, and Discrimination Index by
Semester 174
Table A4.1 Descriptive Analysis for SEM Variables 175
Table A7.1 Fit Result for Multiple-group CFA Models with Measurement
Equivalence Test across Sexes 178

vi
List of Figures
Figure 2.1 Evidence Sources for Psychometric Quality 19
Figure 2.2 An Example of a Full SEM Model 22
Figure 3.1 Rot Item 9 30
Figure 3.2 Rotation Pattern for Item 9 39
Figure 3.3 Standardized Parameter Estimates for One-Factor (3A), Second-Order 42
(3B) And Bi-Factor (3C) Alternate Models
Figure 4.1 Assign the Configuration of 2-Butanol Using Visualization 54
Figure 4.2 Comparing Students Who Progressed through Organic Chemistry Or Not 62
Figure 4.3 SEM Model with Unstandardized Parameter Estimate 64
Figure 4.4 SEM Model with Standardized Parameter Estimate 65
Figure 6.1 Box-and-Whisker Plot for KU Students Overlapped with Asciv2 Items 102
Figure 7.1 Conceptual Model of Chemistry Achievement and Three Predictors 121
(Model 1)
Figure 7.2 SEM Model 2 with Three Predictors 129
Figure 7.3 SEM Model 3 with Two Predictors 129
Figure 7.4 Standardized Parameters for the Path Analysis Model 1 135
Figure 7.5 Path Diagram with the Standardized Parameter Coefficients for Model 2 135
Figure 7.6 Path Diagram with the Standardized Parameter Coefficients for Model 3 135
Figure A4.1 Alternate Models 176

vii