ERIC ED594359: Technology & Engineering Literacy Framework for the 2018 National Assessment of Educational Progress PDF
Preview ERIC ED594359: Technology & Engineering Literacy Framework for the 2018 National Assessment of Educational Progress
TECHNOLOGY & ENGINEERING LITERACY FRAMEWORK for the 2018 NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS NATIONAL ASSESSMENT GOVERNING BOARD U.S. Department of Education THE NATIONAL ASSESSMENT GOVERNING BOARD The National Assessment Governing Board was created by Congress in 1988 as an independent, nonpar tisan board to set policy for the National Assessment of Educational Progress (NAEP), also known as The Nation’s Report Card. In overseeing The Nation’s Report Card, the Governing Board identifies subjects to be tested, determines the content and achievement levels for each assessment, approves all test questions, and takes steps to improve the reporting of results. The Governing Board is responsible for communicating NAEP results to a wide range of audiences. MEMBERS Honorable John Engler, Chair Carol Jago Chasidy White Former Governor of Michigan Associate Director Director of Strategic Initiatives Former President California Reading and Literature Project at UCLA Office of the Superintendent Business Roundtable Oak Park, Illinois Alabama Department of Education McLean, Virginia Montgomery, Alabama Terry Mazany Tonya Matthews, Vice Chair Former President and CEO Joe Willhoft President and CEO Chicago Community Trust Consultant Michigan Science Center Chicago, Illinois Former Executive Director Detroit, Michigan Smarter Balanced Assessment Consortium Dale Nowlin Tacoma, Washington Dana K. Boyd Teacher and Mathematics Department Chair Principal Bartholomew Consolidated School Corporation Ex-Officio Member East Point Elementary School Columbus, Indiana El Paso, Texas Thomas Brock Honorable Jeanette Nuñez Commissioner for Education Research Alberto M. Carvalho State Legislator Delegated Duties of the Director Institute of Education Sciences Superintendent Florida House of Representatives, District 119 U.S. Department of Education Miami-Dade County Public Schools Miami, Florida Washington, D.C. Miami, Florida Joseph M. O’Keefe, S.J. Gregory J. Cizek Fellow and Scholar of Residence Guy B. Phillips Distinguished Professor of Fordham University Graduate School of Education Educational Measurement and Evaluation New York, New York University of North Carolina Chapel Hill, North Carolina Honorable Alice H. Peisch State Legislator Tyler W. Cramer Massachusetts House of Representatives, 14th CEO/Manager Norfolk District Remarc Associates, LLC Wellesley, Massachusetts San Diego, California Honorable Beverly Perdue Frank K. Fernandes Former Governor of North Carolina Principal Managing Director Kaimuki Middle School Perdue Strategy Group Honolulu, Hawaii New Bern, North Carolina Rebecca Gagnon B. Fielding Rolston Chair Chairman Minneapolis Board of Education Tennessee State Board of Education Minneapolis, Minnesota Kingsport, Tennessee Shannon Garrison Linda P. Rosen Fourth-Grade Teacher Former Chief Executive Officer Solano Avenue Elementary School Change the Equation Los Angeles, California Washington, D.C. Honorable James Geringer Cary Sneider Former Governor of Wyoming Visiting Scholar Director of Policy and Public Sector Strategies Portland State University Environmental Systems Research Institute Portland, Oregon Cheyenne, Wyoming Honorable Ken Wagner Andrew Dean Ho Commissioner for Elementary and Secondary Professor Education Harvard Graduate School of Education Rhode Island Department of Education Harvard University Providence, Rhode Island Cambridge, Massachusetts NATIONAL ASSESSMENT GOVERNING BOARD Honorable John Engler Chair Tonya Matthews Vice Chair Bill Bushaw Executive Director Lisa Stooksberry Deputy Executive Director Technology and Engineering Literacy Framework for the 2018 National Assessment of Educational Progress Developed for the National Assessment Governing Board under contract number ED08CO0134 by WestEd For further information, contact National Assessment Governing Board 800 N. Capitol St. NW Suite 825 Washington, DC 20002 www.nagb.gov January 2018 Publication Notes The 2018 NAEP Technology and Engineering Literacy Framework is the same frame work first developed for the 2014 NAEP Technology and Engineering Literacy Assess ment. Continuity in the NAEP Technology and Engineering Literacy Framework enables reporting of student achievement trends over time. This edition reflects updated dates and references to legislation, National Assessment Governing Board actions, and NAEP activities. The second cover photo is courtesy of Allison Shelley/The Verbatim Agency for Ameri can Education: Images of Teachers and Students in Action. IV TABLE OF CONTENTS NAEP Technology and Engineering Literacy Project Committees and Staff .................................................................... IX Preface ...................................................................................................................................................................................................XIII Technology and Engineering Literacy ......................................................................................................................................XV Content ............................................................................................................................................................................................ XVI Areas of Technology and Engineering Literacy...................................................................................................................... XVII Practices .........................................................................................................................................................................................XVIII Contexts.........................................................................................................................................................................................XVIII Types of Tasks and Items...............................................................................................................................................................XIX Reporting Results............................................................................................................................................................................XIX Development of the Framework ...............................................................................................................................................XIX Chapter One: Overview Need for a Framework in Technology and Engineering Literacy ......................................................................................... 1 Background of NAEP........................................................................................................................................................................ 3 Resources for Planning the Framework ....................................................................................................................................... 3 Definitions of Key Terms Used in This Framework ................................................................................................................. 5 Technology............................................................................................................................................................................................. 5 Engineering ............................................................................................................................................................................................ 6 Technology and Engineering Literacy.............................................................................................................................................. 7 Three Areas of Technology and Engineering Literacy .................................................................................................................. 7 Three Practices....................................................................................................................................................................................... 9 Educational Technology....................................................................................................................................................................10 The Relationship Among Science, Technology, Engineering, and Mathematics..................................................................11 The Framework Development Process .......................................................................................................................................12 Summary of the Steering Committee Guidelines........................................................................................................................13 Challenges of Developing the NAEP Technology and Engineering Literacy Framework .........................................15 Newness of the Endeavor ..................................................................................................................................................................15 Diffuse Curriculum ............................................................................................................................................................................15 Varying Definitions ............................................................................................................................................................................16 Measurement Constraints.................................................................................................................................................................16 Time and Resource Constraints ......................................................................................................................................................17 Designing a Computer-Based Assessment.....................................................................................................................................18 V Predicting Future Changes in Technology ....................................................................................................................................19 Overview of Framework Chapters ...............................................................................................................................................19 Chapter Two: Areas of Technology and Engineering Literacy .................................................................................................19 Chapter Three: Practices and Contexts for Technology and Engineering Literacy .............................................................19 Chapter Four: Overview of the Assessment Design ....................................................................................................................20 Chapter Five: Reporting Results of the NAEP Technology and Engineering Literacy Assessment ................................20 Chapter Two: Areas of Technology and Engineering Literacy ...............................................................................................21 Introduction .......................................................................................................................................................................................21 Technology and Society ....................................................................................................................................................................23 Design and Systems ............................................................................................................................................................................23 Information and Communication Technology (ICT) ..............................................................................................................23 Resources Used in the Development of Assessment Targets ...............................................................................................24 Technology and Society ..................................................................................................................................................................25 A. Interaction of Technology and Humans ..................................................................................................................................26 B. Effects of Technology on the Natural World ...........................................................................................................................30 C. Effects of Technology on the World of Information and Knowledge ................................................................................33 D. Ethics, Equity, and Responsibility ............................................................................................................................................36 Design and Systems ..........................................................................................................................................................................39 A. Nature of Technology ..................................................................................................................................................................40 B. Engineering Design ......................................................................................................................................................................43 C. Systems Thinking ..........................................................................................................................................................................47 D. Maintenance and Troubleshooting ...........................................................................................................................................50 Information and Communication Technology (ICT) ...........................................................................................................53 A. Construction and Exchange of Ideas and Solutions .............................................................................................................54 B. Information Research ...................................................................................................................................................................57 C. Investigation of Problems ............................................................................................................................................................59 D. Acknowledgment of Ideas and Information ...........................................................................................................................62 E. Selection and Use of Digital Tools ............................................................................................................................................65 Conclusion .........................................................................................................................................................................................66 Chapter Three: Practices and Contexts for Technology and Engineering Literacy ...........................................................67 Introduction .......................................................................................................................................................................................67 Practices ...............................................................................................................................................................................................68 Understanding Technological Principles .......................................................................................................................................69 Developing Solutions and Achieving Goals ..................................................................................................................................69 Communicating and Collaborating ................................................................................................................................................70 Examples of Practices Applied in Each of the Assessment Areas .......................................................................................72 Practices Applied in Technology and Society ...............................................................................................................................72 Practices Applied in Design and Systems .......................................................................................................................................78 Practices Applied in Information and Communication Technology (ICT) .........................................................................84 VI Contexts ...............................................................................................................................................................................................90 Contexts in Technology and Society ..............................................................................................................................................90 Contexts in Design and Systems ......................................................................................................................................................92 Contexts in Information and Communication Technology (ICT) ........................................................................................94 Chapter Four: Overview of the Assessment Design ....................................................................................................................97 Introduction .......................................................................................................................................................................................97 Overview of the NAEP Technology and Engineering Literacy Assessment ..................................................................97 Types of Tasks and Items ................................................................................................................................................................97 Scenario-Based Assessment Sets ......................................................................................................................................................98 Discrete Item Sets ...............................................................................................................................................................................99 Definitions of the Scenario-Based and Discrete Item Assessment Sets ............................................................................99 Scenario-Based Assessment Sets ......................................................................................................................................................99 Discrete Item Sets ............................................................................................................................................................................ 101 Descriptions of the Response Types Used in the Assessment Sets .................................................................................. 101 Constructed Response .................................................................................................................................................................... 102 Selected Response ............................................................................................................................................................................ 106 Ways of Measuring Student Responses .................................................................................................................................... 109 Balance of the Assessment ........................................................................................................................................................... 111 Assessment Balance by Major Assessment Area ........................................................................................................................ 112 Assessment Balance by Practice..................................................................................................................................................... 113 Assessment Balance by Set Type ................................................................................................................................................... 114 Assessment Balance by Response Type ........................................................................................................................................ 115 Assessment Design and Student Diversity ............................................................................................................................. 115 Pilot Student Samples ..................................................................................................................................................................... 116 Universal Design .............................................................................................................................................................................. 117 Accommodations ............................................................................................................................................................................. 117 Chapter Five: Reporting Results of the NAEP Technology and Engineering Literacy Assessment ......................... 119 Introduction .................................................................................................................................................................................... 119 How NAEP Results Are Reported ............................................................................................................................................ 119 Reporting Scale Scores and Achievement Levels .................................................................................................................. 120 Achievement Level Definitions .................................................................................................................................................... 121 Cut Scores .......................................................................................................................................................................................... 122 Examples of Students’ Responses .................................................................................................................................................. 122 Reporting Contextual Variables ................................................................................................................................................ 123 Uses of NAEP Reporting ............................................................................................................................................................. 124 Appendixes ............................................................................................................................................................................................ 125 Appendix A: Glossary of Abbreviations, Words, and Terms Used in the Framework ..................................................... 125 Appendix B: Steering Committee Guidelines ......................................................................................................................... 133 VII Appendix C: Alignment Table – Comparing the NAEP Technology and Engineering Literacy Assessment Areas to U.S. Source Documents ..................................................................................................... 139 Appendix D: Alignment Table – Comparing the NAEP Technology and Engineering Literacy Assessment Areas to International Source Documents .................................................................................... 143 Appendix E: Alignment Table – Comparing ICT Subareas to the ISTE NETSS and the Framework for 21st Century Learning ......................................................................................................................... 147 Appendix F: Alignment Table – Comparing Design and Systems Subareas to the ITEEA Standards for Technological Literacy ................................................................................................................................... 157 Appendix G: NAEP Technology and Engineering Literacy Preliminary Achievement Level Definitions ...................................................................................................................................................................... 159 Appendix H: NAEP Technology and Engineering Literacy Finalized Achievement Level Definitions at Grade 8 .................................................................................................................................................. 165 Bibliography ......................................................................................................................................................................................... 167 VIII NAEP TECHNOLOGY AND ENGINEERING LITERACY PROJECT COMMITTEES AND STAFF Note: This list of project committees and staff reflects professional affiliations during the project period for framework development. STEERING COMMITTEE Don Knezek, Co-Chair John Cherniavsky CEO Senior Advisor for Research International Society for Technology in Education National Science Foundation Washington, D.C. Arlington, Virginia Senta Raizen, Co-Chair Matt Dawson Director Director National Center for Improving Science Education REL Midwest WestEd Chief Officer Washington, D.C. Research Learning Point Associates Jennifer Barrett Partnership for 21st Century Skills Manager Naperville, Illinois Professional Development ASCD Heidi Glidden Washington, D.C. Assistant Director American Federation of Teachers Phillip “Scott” Bevins Washington, D.C. Director of Institutional Research The University of Virginia’s College at Wise Paige Johnson Wise, Virginia Global K-12 Education Manager Corporate Affairs Intel Corporation Laura Bottomley Portland, Oregon Director K-12 Outreach Colleen Layman College of Engineering Director of Regions North Carolina State University Society of Women Engineers Raleigh, North Carolina Manager of Water Treatment Bechtel Power Corporation Rodger Bybee Harpers Ferry, West Virginia President Rodger Bybee & Associates Johnny Moye Golden, Colorado Supervisor Career and Technical Education Shelley Canright Chesapeake Public Schools Outcome Manager Chesapeake, Virginia Elementary, Secondary, and eEducation NASA Willard (Bill) Nott Washington, D.C. ASME Engineers Without Borders Vinton (Vint) Cerf FIRST Vice President and Castro Valley, California Chief Internet Evangelist Google Reston, Virginia Ix Philip Patterson Cary Sneider, Co-Chair President Associate Research Professor National Christian School Association Portland State University Oklahoma City, Oklahoma Portland, Oregon Greg Pearson Marie Aloia, Treasurer Senior Program Officer New Jersey Society of Women Engineers National Academy of Engineering Teacher Washington, D.C. Engineering and Physical Sciences Bayonne High School Andrea Prejean Bayonne, New Jersey Senior Policy Analyst National Education Association David Ashdown Washington, D.C. Coordinator for Instructional Technology Integration Programs Washington-Saratoga-Warren-Hamilton-Essex Board of Cooperative Ryan Reyna Educational Services Policy Analyst Saratoga Springs, New York Education Division National Governors Association Carlos Ayala Washington, D.C. Associate Professor of Education Sonoma State University Linda Roberts Rohnert Park, California Senior Advisor and Consultant Former Director Susan Brooks-Young Office of Educational Technology Consultant and Author U.S. Department of Education SJ Brooks-Young Consulting Washington, D.C. Lopez, Washington Jean Slattery Per Christiansen Senior Associate Assessment Content Specialist Science Achieve Massachusetts Department of Elementary and Secondary Education Washington, D.C. Malden, Massachusetts Yvonne Spicer Gordon K. Dahlby Vice President Director of Curriculum and Technology Advocacy & Educational Partnerships West Des Moines Community School District National Center for Technological Literacy West Des Moines, Iowa Museum of Science Boston, Massachusetts William E. Dugger, Jr. Senior Fellow Kendall Starkweather International Technology and Engineering Educators Association Executive Director and CEO Hobe Sound, Florida/Blacksburg, Virginia International Technology and Engineering Educators Association Reston, Virginia Maurice T. Frazier Teacher Martha Thurlow High School Technology Education Director Chesapeake Public School System National Center on Educational Outcomes Chesapeake, Virginia University of Minnesota Minneapolis, Minnesota Camilla Gagliolo Instructional Technology Coordinator Mary Ann Wolf Arlington Public Schools Executive Director Arlington, Virginia State Educational Technology Directors Association Arlington, Virginia Jocelyn Garcia-Thome Manager PLANNING COMMITTEE Corporate Citizenship and Corporate Affairs IBM Corporation San Jose, California Edys Quellmalz, Co-Chair Director Joan Herman Technology Enhanced Assessments and Learning Systems WestEd Director Redwood City, California National Center for Research on Evaluation, Standards, and Student Testing University of California, Los Angeles Los Angeles, California x
The list of books you might like
