ERIC ED433003: The Ohio SchoolNet Telecommunity Evaluation. Year Three Evaluation Results: Examining Interactive Video Adoption and Resource Needs.

58 Pages·0.73 MB·English

by  ERIC

DOCUMENT RESUME
IR 019 727
ED 433 003
Hawkes, Mark; Cambre, Marjorie; Lewis, Morgan
AUTHOR
The Ohio SchoolNet Telecommunity Evaluation. Year Three
TITLE
Evaluation Results: Examining Interactive Video Adoption and
Resource Needs.
North Central Regional Educational Lab., Oak Brook, IL.
INSTITUTION
PUB DATE
1999-01-14
60p.; For a related document, see ED 393 859.
NOTE
North Central Regional Educational Laboratory, 1900 Spring
AVAILABLE FROM
Road, Suite 300, Oak Brook, IL 60523-1480; Tel:
630-571-4700.
-- Tests/Questionnaires (160)
Evaluative (142)
Reports
PUB TYPE
MF01/PC03 Plus Postage.
EDRS PRICE
*Adoption (Ideas); Community; Cooperative Programs; Cost
DESCRIPTORS
Effectiveness; Distance Education; Educational Technology;
Elementary Secondary Education; Faculty Development;
Instructional Innovation; *Interactive Video; Networks;
*Partnerships in Education; Program Effectiveness; Program
Evaluation; Questionnaires; School Surveys; State Programs;
Tables (Data); Telecommunications
Access to Technology; Impact Evaluation; *Ohio SchoolNet;
IDENTIFIERS
*Technology Implementation; Technology Utilization
ABSTRACT
This evaluation of the Ohio SchoolNet Telecommunity program
focused on key Telecommunity objectives of technology deployment, teacher
professional development, and student impact. Evaluation methods included use
of the Scales of Concern Questionnaire in the application of the
Concerns-Based Adoption Model, as well as visits to each of the 23 funded
project consortia, during which observation, interviews, focus groups, and
document analysis took place. The major sections of this evaluation report
address: interactive video technology adoption; perspectives of partners and
content providers to schools; impact statements; the need for resources and
availability; thoughts about value-added contributions of the technology from
the field; and examples of Telecommunity activities. The following four key
findings and related recommendations are presented in the executive summary:
(1) Telecommunities appear to be moving through similar stages of technology
(2) providers of content expertise and other resources are unsure
adoption;
how they can best participate with their school partners in deploying
(3) there are a number of significant
interactive learning technologies;
learning impact indicators that go beyond what standardized measures will
achieve; and (4) increasing pressure has been placed on the Telecommunity
program to demonstrate cost-effectiveness. A copy of the questionnaire is
appended.
(MES)
********************************************************************************
Reproductions supplied by EDRS are the best that can be made
from the original document.
********************************************************************************
cr)
O
O
cr-1
THE OH! SCHOOLNET
TELECOMMUNITY
1
U.S. DEPARTMENT OF EDUCATION
Office of Educational Research and Improvement
EVALUATION
EDUCATIONAL RESOURCES INFORMATION
CENTER (ERIC)
PERMISSION TO REPRODUCE AND
This document has been reproduced as
DISSEMINATE THIS MATERIAL
received from the person or organization
HAS BEEN GRANTED BY
originating it.
Minor changes have been made to
M. Kroeger
improve reproduction quality.
Points of view or opinions stated in this
document do not necessarily represent
Year Three
official OERI position or policy.
ITO THE EDUCATIONAL RESOURCES
INFORMATION CENTER (ERIC)
Evaluation Results: Examining
Interactive Video Adoption
and Resource Needs
_J
North Central Regional Educational Laboratory
it de";
0
2
BEST
PY AVAILABLE
L_
THE OHIO
SCHOOLNET
TELECOMMUNITY EVALUATION
Mark Hawkes
Evaluation and Policy Information Center
North Central Regional Educational Laboratory
Marjorie Cambre
Department of Educational Leadership
Ohio State University
-Morgan Lewis
Employment
Center on Education and Training for
Ohio State University
January 14, 1999
NCREL
North Central Regional Educational Laboratory
1900 Spring Road, Suite 300
Oak Brook, Illinois 60523-1480
[phone], Fax [phone]
NCREL
Jeri Nowakowski, Executive Director
Roger J.D. Chesswas, Director, Evaluation and Policy Information Center
Gil Valdez, Director, Teaching Learning of Curriculum Center
Stephanie Blaser, Production Coordinator
Melissa Chapko, Graphic Designer
Lenaya Raack, Editor
Copyright 1999 by the North Central Regional Educational Laboratory.
All rights reserved.
4
CONTENTS
Executive Summary
i
Introduction
1
Evaluation
1
Project Implementation Status
1
2
Interactive Video Technology Adoption Analysis
15
Perspectives of Partners and Content Providers to Schools
23
Field
Impact Statements: A Summary of Thoughts From the
26
Need for Resources and Availability
of the
Some Thoughts About Value-Added Contributions
30
Technology From the Field
34
Telecommunity Practice Illustrated
Appendix 43
LIST OF TABLES AND FIGURES
2
Table 1. Operating Status of Projects
4
Table 2. Intercorrelations Among Stages of Concern
7
Role in Project
Table 3. ANOVA of Technology Adoption Stage by
Project,
Table 4. Correlation Between Stage of Concern, Year in
9
and Comfort With Educational Technologies as a Whole
Table 5. Correlation of Subcomponents to
10
Interactive Video Adoption
29
Table 6. Telecommunity Directory Example
6
Funded Year
Figure 1. Profile of Technology Adoption by
Role in Project
8
Figure 2. Profile of Technology Adoption by
14
Technology Adoption
Figure 3. Revised Model of Interactive Video
The integration of technology and
EXECUTIVE SUMMARY
curriculum
The School Net Telecommunity enterprise
The impact of the technology on the
is one of the most extensive programs in the
learning experience of students and
country attempting to employ interactive
teachers
video technologies to improve the student
learning experience. A Public Utilities
The Evaluation
Commission agreement between the Ohio
Department of Education and Ameritech ini-
The partnership sought to address the
tiated the Telecommunity plan in 1995. In
issues raised in this project by retaining the
1996, several other telephone companies
evaluation services of the North Central
serving the state of Ohio joined the
Regional Educational Laboratory (NCREL).
School Net Telecommunity program, bringing
Through the three years of evaluation,
the total contributions over the six-year life
NCREL has focused on providing informa-
of the enterprise to $26 million dollars.
tion that would improve the development of
the project. Additionally, each year's evalua-
As partners in bringing interactive video
tion has attempted to build on the results of
technology to Ohio schools, the telephone
the previous one.
companies and leaders of the state Office of
Information, Learning, and Technology
In this, the third year of the evaluation,
Services considered the issues that would
several methods were used to arrive at the
confront the broad scale use of interactive
results presented here. The Scales of Concern
video in funded sites. These issues included:
Questionnaire was used in the application of
the Concerns-Based Adoption Model
The acquisition and installation of the
(CBAM). Statistical analysis followed. Each
interactive technology
of 23 funded project consortia was visited by
The adoption of the interactive
members of the evaluation team. During the
technology into the core system of
visits, observation, interviews, focus groups,
school operation
and document analysis took place. The
methods employed contributed to the sec-
The availability of technology for
tions in this report on interactive distance
instruction and learning
technology implementation and adoption,
The frequency and nature of professional
impact, value, resource deployment, and
development
content provider partnering.
6
PAGE I
Findings and Recommendations
appears to be the sudden influx of money
and the edict that accompanied the grants to
The data collection and analysis resulted in
get the interactive video technology into the
four key findings that are briefly discussed
schools. They appear to have put the
below.
Telecommunities immediately into a
Management mode. Acquiring and installing
1. Telecommunities appear to be moving
the technology was so problematic for most
through similar stages of technology
partnerships that they have only in the last
adoption. But traditional adoption
year focused on other technology concerns.
models may not explain the unique
With the management concerns generally
context surrounding the implementa-
addressed, Telecommunities are now attempt-
tion of the Telecommunity project.
ing to meet capacity-building concerns
CBAM suggests that the process of change
(Awareness, Informational, Personal) through
involves moving through a series of concerns
collaboration, which explains the high con-
about an innovation. The model identifies
cerns at CBAM's Collaboration stage.
seven stages of concern: Awareness,
Informational, Personal, Management,
3
Consequence, Collaboration, and Refocusing.
FIGURE
The profile of Telecommunities, as represented
REVISED MODEL OF INTERACTIVE
on the model, indicates that, despite year of
VIDEO TECHNOLOGY ADOPTION
funding, they have similarly high concerns in
the beginning stages of interactive technology
adoption, lower concerns through the
Management and Consequence stages, and
then extensive interest or concerns at the
Collaboration Stage.
Events of the adoption of interactive video
in Ohio schools suggests a different pattern,
like that produced in the Figure 3 below.
Participants' concerns about interactive video
intensify through the Awareness, Informational,
e
and Personal stages and become most con-
0,4
"
`t"'"
Oec'
_,en
!Pet
centrated at the Collaboration stage. The
MI Funded Year
Role in Project
reason for this departure from the model
7
PAGE II
consider these instructional materials just
Recommendation: The data implies an
right for their programs; other may feel they
extraordinary interest and demand for col-
protocol is
are too "canned." Clearly, some
laborative opportunities and resources.
and
necessary to build collaborative ventures
Because of this, the Telecommunity leader-
create mutually beneficial expectations
ship should step up its efforts to link partner-
between content providers and schools.
ships on multiple educational levels. In addi-
tion, they should make information more
Recommendation: A few examples can be
accessible (both electronically and on paper)
found around the state of how providers might
about how each Telecommunity is addressing
collaborate with schools in ways that robustly
curriculum integration, teacher training, and
technology while meeting the
use the interactive
student assessment challenges, so that like-
distinctive needs of the school. This evaluation
minded teachers in Telecommunities across
Telecommunity
suggests that the School Net
the state can link and learn with each other
directors use those institutions and experienced
in true Telecommunity form.
school personnel to outline the steps to the cur-
riculum codevelopment and support process for
2. Providers of content expertise and
other provider organizations in the state. The
other resources are unsure how they
product that arises from this work could be a
school
can best participate with their
demonstration, a conference, a paper/Web
partners in deploying interactive learn-
the three.
resource, or a combination of
ing technologies.
As Telecommunities become operational,
3. Work with Telecommunities indicate
they begin to seek resources to which they
that there are a number of significant
can link. Currently they are using programs
learning impact indicators that go
offered by the state's museums and zoos;
beyond what standardized measures
however, if the Telecommunities continue to
will achieve.
the demand will
use just these few sources,
There is an underlying energy and optimism
overwhelm the museums/zoos and force them
in the measures of impact suggested by those
with
to dramatically curtail their interactions
who participate in the implementation of Ohio
the schools. Providers have worked diligently
Telecommunity projects. Such measures
to create and improve interactive program-
include, but are not limited to, the increased
ming often attempting to tie their work to
attendance of teachers at inservice activities
Ohio proficiency standards with accompany-
related to telecommunications use, increased
ing pre- and postprogram materials and cur-
student interest in and joy for learning,
riculum guides. Some Telecommunities may
8
PAGE III
Cost-effectiveness means different things to
increased student understanding of the basic
different people under different circum-
functions of communications technology, and
stances. In this case, we analyze cost-effec-
improved student behavior. Those teachers
tiveness and benefit under the lens of value-
and administrators who have described the
added contributions of the technology to the
project's impact as they are witnessing it in
teaching and learning process.
their own settings have done so with serious-
Telecommunities prefer to be judged on a rel-
ness, sincerity, and enthusiasm. Over time and
atively diverse continuum of value and effec-
with increased frequency, the descriptors of
tiveness that asks first what the value-added
impact that emerged in this evaluation (see
contribution of the technology is to teaching
body of report for ten key impact statements)
and learning. Some of the value-added bene-
can have a cumulatively significant effect on
fits that evaluators and participants in the
schooling in Ohio.
program have observed include opportunities
Recommendation: We suggest that the
to explore and to become pioneers; increase
impact statements presented in this report be
in the number and kinds of activities offered
published and promoted as guidelines for the
and in the number of students and teachers
estimation of impact in the Telecommunities.
participating; and promotion of understand-
Although no one project should be expected to
ing, collaboration, and tolerance among stu-
show improvement on all measures, each would
dents of diverse backgrounds.
be expected to exhibit growth in four or five
areas during evaluations in the coining years.
Recommendation: We suggest that the
As projects mature, more observations of impact
measures of cost-effectiveness suggested by
may be added to the list assembled in the report
Ohio Telecommunity participants and presented
and contribute to a grounded database of deter-
in this report be considered as a set of 12
minants of success in Ohio Telecommunities.
criteria by which Telecommunities could be
assessed in the future (see body of report for
4. Increasing pressure has been placed on
measures). While no one Telecommunity
the Telecommunity program to demon-
should be expected to provide positive evi-
strate cost-effectiveness. Participants in
dence on all of these measures, perhaps it is
the program, however, indicate there are
reasonable to expect each should show activity
benefit of
a number of ways to assess the
on at least four or more if they are making
the technology, many of which may not
acceptable progress. At the very least, they
be accounted for in the traditional sense
might provide a set of guidelines to which
of cost-effectiveness analysis. It is impor-
Telecommunities could direct their efforts.
tant to acknowledge these alternatives.
9
PAGE IV
6. The connectivity supplied by the inter-
Summary
active video enables schools to raise
Taken together, the data presented in this
awareness and expectations about what
report show that the essential conditions for
others are doing in terms of improving
improvement-oriented technological application
teaching and learning standards.
are emerging in the School Net Telecommunity
With one boundary of the technology sys-
initiative. Some of these conditions include the
tem encompassing the schools and the other
following:
boundary of the system encompassing society
1. The technology is enabling the educa-
(e.g., business, industry, Peace Officer
tion system to function more effectively
Training Commission, public and commercial
by putting teachers in greater contact
broadcasting systems) the sustainability of
with each other to discuss issues related
the Telecommunity initiative improves, as
to curriculum development, delivery,
does its affordability. The need these
and assessment.
Telecommunities have is to understand how
they can better communicate and collaborate
2. Organizational changes, such as block
with each other to ensure that sustainability.
scheduling and the coordination of
teacher preparation periods, encourage
more effective use of the technology
and curriculum.
3. Teachers have access to professional
development experiences that have pre-
viously been impossible to provide
without costly outlay.
4. Students have had access to expert
resources of knowledge and informa-
tion that without the technology would
be nonexistent.
5. With video technology in place, more
than schools are considering the nature
and content of the school learning expe-
rience: Museums, universities, business-
roles
es, and others have assumed new
in the education of Telecommunity
consortia's students and teachers.
10
PAGE V