The Effects of SURWEB Hypermedia
Construction on the Development of Complex Knowledge Structures,
Creative Thinking, and Research
Process Skills of Utah Sixth Grade Social Studies Students
Dr. Lee A. Montgomery
and Graduate Students of the
College of Education
Southern Utah University
Montgomery@suu.edu
INTRODUCTION
For the past two decades, a plethora of national reports have
indicated that American public school students have been lagging
behind their international counterparts in such areas as mathematics,
science and technology (National Commission of Excellence in
Education, 1983; National Research Council, Board on Mathematical
Sciences, 1989; The United States Congress Office of Technology
Assessment, 1995). Since the early eighties, these reports have
prompted the examination of our schools curricula and fueled
the drive for school reform (Knapp & Glenn, 1996). The contention
of these and a host of other studies is that schools need to shift
their focus from transmitting information to students to enabling
them to seek information, process it and thereby construct their own
knowledge (Knapp & Glenn).
A growing recognition on the part of educators of the
inappropriateness of traditional views of teachers as all-knowing
disseminators of knowledge and students as passive recipients has
resulted in new ways of thinking about the process of teaching and
learning. New models which focus on the use of multimedia and
communications technologies as tools to assist teachers in helping
students take responsibility for their own learning, become knowledge
explorers and collaborate with others to solve real problems are a
natural outgrowth of a new emphasis on engaged and active learning.
(Garrett & Weiner, 1998; Barker, 1999).
Knapp and Glenn (1996) present such a model aimed at examining
differences between transmissive (traditional) schools and
restructured (Constructivist) ones. Central to their model is a
paradigm shift which focuses on students learning to construct their
own knowledge by engaging in activities that require them to
think critically, solve problems, and seek answers to their own
questions (Knapp and Glenn, p. 7). To accomplish this shift,
the curriculum and classroom learning environment are restructured to
facilitate student inquiry, research, and problem solving activities.
This shift in emphasis also requires a concomitant shift in
assessment techniques. In their model, traditional short answer and
essay tests are supplanted by authentic situations in which students
are required to demonstrate their ability to express, apply and
defend their knowledge and skills. Technology also plays a pivotal
role in the configuration of a Constructivist school. Knapp and Glenn
call for the integration of technologies which support
research, analysis, problem-solving, and communication
(Knapp and Glenn, p. 7). Many researchers (Beal, 1995; Johnson, 1994;
Liu, 1998) believe that, used in this context, hypermedia development
can help students construct knowledge, develop higher order thinking
skills and, possibly, promote problem-solving skills.
As new models of learning have been developed, it has been
recognized that learners act as active constructors of knowledge
(Duffy and Jonassen, 1992). Within this constructivist framework,
which is concerned with the process of how we construct meanings of
our world as well as the results of the constructive process,
cognitive tools can help learners organize, restructure and represent
what they know. Jonassen and Reeves (1996) have summarized the
foundations of cognitive tools research and have identified the
following key principles in the context of hypermedia design:
- Cognitive tools have their greatest effectiveness when they
are applied to constructivist learning environments.
- Cognitive tools empower learners to design their own
representations of knowledge rather than absorbing knowledge
representations preconceived by others.
- Ideally, tasks or problems for the application of cognitive
tools should be situated in realistic contexts with results that
are personally meaningful to learners. (p. 698).
Of central concern for educators is how to best utilize these
cognitive tools in a way that will help their students develop the
ability to think for themselves, apply concepts that they learn and
analyze and evaluate the material that they study. In a nutshell,
students need to be able to manipulate various technologies in a way
that will enable them to synthesize information, construct knowledge
and ultimately learn how to think for themselves. Montgomery (1999)
suggests that by implementing an innovative approach based on
SURWEBs media shows and learning, these goals can be positively
addressed through the use of student created hypermedia
presentations.
SURWEB (http://www.surweb.org),
a federally funded project aimed at providing resources to students
and teachers in remote, rural areas, provides application
technologies which allow students to manipulate Internet based
multimedia. In addition to acquiring a large number of pictures and
text, SURWEB has also developed a media presentation tool that
enables users to easily access and organize all forms of digital
information and then to display their products in a simple, yet
professional manner. The SURWEB presentation tool allows users to: 1)
visit a site of interest and view the relevant information available
on-line; 2) select images and text to make a media presentation
utilizing multimedia from the database of images, texts and sounds;
and 3) choose to import or upload images, sounds and text from any
source in order to customize and enhance the media presentation. The
software application has been extensively field tested and upgraded
according to formative evaluation.
The most often used SURWEB hypermedia tool feature is the import
site, where students and teachers can select data from other sites on
the Internet to build classroom projects. In the past five years,
more than 7,000 user-created projects of this type have been created
and stored for later use on the SURWEB server. The use of SURWEB
hypermedia tools encourages knowledge creation through the ability to
research, access, evaluate, organize, and display information that is
relevant to the content and performance standards for the classroom.
In addition, the tools are useful for peer coaching, collaborative
learning, and the accommodation of a greater variety of student
orientations to learning.
This action research study will explore the use of SURWEB media shows
as cognitive tools for creating interactive learning environments in
a seventh grade social studies classroom.
PURPOSE AND RESEARCH
QUESTIONS
This study focuses on attempts by sixth grade social studies
students to develop hypermedia products which fall into the knowledge
construction category of Nelson and Palumbos (1992) cognitive
framework. The premise of this study is that, by using hypermedia in
this manner, students might engage in higher levels of cognitive
skills including analysis, synthesis, and evaluation and develop
creative thinking and research process skills. In other words, by
using SURWEB to develop their own hypermedia programs, students will
be engaged in a process which requires them to externalize the
constructs of their own knowledge.
The purpose of this study was to investigate the effects of SURWEB
Media Show construction on the development of complex knowledge
structures, creative thinking and research process skills of sixth
grade social studies students. For the purpose of the study Media
Shows were defined as web-based programs created through the use of
SURWEB software which enable students to link animation, video,
graphics, sound and text together in meaningful ways. Hypermedia, as
used in this study refers to documents which seamlessly incorporate
text, graphics, sound, video and hyperlinks (links to other
documents). Reinking (1994) points out that hypermedia provides
a means to express ourselves in ways that reflect more directly the
complexity of our thinking and the inter-relatedness of
ideas.
Yankovich, Meyrowitz and van Dam (1985) observe that the use of
hypermedia allows authors and groups of authors to link
information together, create paths through a corpus of related
material, annotate existing texts and create notes that point readers
either to bibliographic data or to the body of reference
material. Hypermedia can thus allow students and teachers to
create an automated encyclopedia which enables readers to browse
through linked, cross-referenced, annotated text (hypertext) in an
orderly but non-sequential manner.
This study incorporates both traditional and non-traditional
assessment measures to determine how students cognitive
structures and thinking skills are affected when they develop
hypermedia programs. The research questions which guide the study
include:
- What is the effect of SURWEB hypermedia development on
students ability to construct complex knowledge
structures?
- What is the effect of SURWEB hypermedia development on
students acquisition of information problem solving
skills?
- What is the effect of SURWEB hypermedia development on
students creative thinking skills.
REVIEW OF RELATED LITERATURE
Hypermedia and the Construction of Knowledge
Papert (1987) and Nicol (1989) and a growing number of
educational technologists support a constructivist school
of hypermedia which is based on the belief that students should
reconstruct their own understanding of their studies and assume a
more active role in their education. Marchionini (1988) maintains
that when students create their own hypermedia stacks, they present
knowledge in a way that matches their own schematic framework of
understanding. Ashton (1992) has suggested that when students
reconstruct knowledge for themselves they tend to learn more because
the information is internalized. Bodner (1986) argued that until
recently the accepted model for instruction was based on the hidden
assumption that knowledge can be transferred intact from the mind of
the teacher to the mind of the learner. He points out that teaching
and learning are not synonymous; teachers may teach and teach well,
but students will not necessarily learn. Von Glaserfeld (1979) and
other Constructivists advocate an approach in which students identify
topics or issues, locate resources, plan investigations and
activities, and practice self-evaluation (all with teacher support).
Under this model the emphasis is shifted from activities that
teachers do, to those that students perform during the learning
process.
Hypermedia Development and Cognition
A considerable body of research exists which supports the usefulness
of hypermedia-based learning environments (Conklin, 1987; Hammond,
1989; Heller, 1990; Jonassen, 1996). The concept has existed for
quite some time as indicated by an early reference by Nelson (1967)
who defined the process as a combination of natural language
text with the computers capacity for interactive branching and
the dynamic display of nonlinear (information). According to
Jonassen, a hypermedia system consists of fragments of information
that can be text, graphics, sound or video. He refers to these
fragments of information as nodes and notes that nodes
can be linked to other nodes to create complex knowledge structures.
Jonassen considers hypermedia knowledge structures to be similar to
processes through which information is stored in the human memory
system. A user of a hypermedia system can navigate through the
knowledge structure and determine which path to follow as well as the
sequence and length of viewing. Structures are created so that the
user can jump around within the program to related or
more detailed information as desired. Much of the early literature on
hypermedia, is descriptive or theoretical and addresses users of
hypermedia systems rather than the effect of engaging in the
construction of hypermedia on students knowledge
structures.
Taylor (1980), in his pioneer work on computers in education,
described three major functions of computer usage: computers as
tutors, tutees and tools. When used as tutors and tutees, computers
help students understand difficult concepts and check student
comprehension of those concepts. Used as a tool, computers empower
students to create and manipulate knowledge, organizing it in a
variety of cognitively useful ways. Toolistic use of the computer has
gained increasing acceptance among educators with the advent of
inexpensive and easy to use hypermedia authoring software
(Hasselbring, Goin, & Wissick, 1989). Fabris (1992) found that
the use of a such software provides rich learning experiences for all
students, regardless of cultural, social, economic, or ethnic
background.
Most recent studies on hypermedia have focused on this
toolistic approach in which teachers create their own
multimedia stacks for students to use as an alternative to
traditional texts (Jonassen, 1986; Bowers & Tsai, 1990). This
type of multimedia implementation requires more knowledge on the part
of the classroom teacher since he/she is no longer merely
applying multimedia but creating multimedia
presentations and then applying them to course content in
meaningful ways (Landow, 1989).
Other studies have focused on students as hypermedia authors
(Brigham, Hendricks, Kutcka, & Schuetta, 1994). Hooley and Toomey
(1993) advocate the student use of computers to store, share, and
reconstruct knowledge through hypermedia applications. Pea (1991) has
suggested that the full educational potential of multimedia
technology will not be realized until students are empowered to
create their own multimedia projects to use them to communicate their
understanding and organization of the topic being studied. Peas
claims are supported by Stevens (1993) who studied the use of
computer software programs and student activities utilizing the
computer in secondary social studies classrooms. Stevens concluded
that the successful use of computers in social studies instruction
depends upon the design of student created projects.
Lehrer (1993) constructed a framework for the cognitive processes
involved in the development of hypermedia which identifies the four
types of cognitive processes involved. These processes include
planning, transforming, evaluating and revising the hypermedia
document. With Erickson and Connell (1994) he studied the cognitive
effects of the development of hypermedia projects on ninth grade
students in an American History class and concluded that the students
acquired skills such as interpreting information and communicating
knowledge through the development of intricate hypermedia
documents.
Lund and Hildreth (1997) investigated an instructional model that
incorporated the personal computer and a multimedia authoring tool
into an assignment to write and illustrate a multimedia story.
Subjects were 21 public school students in a fifth-grade classroom.
Students were instructed in the writing of interactive stories using
the process approach to writing and the software and a scoring rubric
was used to evaluate the finished stories. The researchers concluded
that: (1) students writing improved in quantity and quality;
(2) students were more motivated to complete the assignment; (3) the
computer allowed for greater flexibility in writing due to its nature
as a word processing tool; (4) even a single computer could be
effectively used in integrating computer technologies into the
classroom, and; (5) there were inherent difficulties associated
with integrating computer technologies into classrooms---many
students did not have experience with computers, even those who do
have experience may lack keyboarding skills, and finding time to
accomplish such a project requires outside help. These findings
suggest that, with appropriate support, computers can be used to
enhance writing instruction and improve the quality of written
work.
A number of studies have reported benefits when hypermedia authoring
tools are placed in the hands of students. Velasco and Mendivil
(1992) reported that when students use authoring tools in a
constructivist environment, motivation increases. In addition, the
use of graphics, sounds, video clips, and peripheral devices for
photographic material is very attractive for image-centered learning,
and the hypermedia environment demands careful planning which implies
the exercise of higher skills in structural design and thinking.
Reed and Rosenbluth (1992) focused on students as creators rather
than users of programs to determine whether collaboratively creating
HyperCard stacks that presented information on four decades (i.e. the
1920s, 1930s, 1945-60 and the 1960s) affected the amount of knowledge
and the inter relatedness of informational units students had on the
assigned decades. Thirteen high school seniors utilized hypermedia
authoring tools to produce a multimedia-based strand of instruction
during a four week academic enrichment program. Results indicated
that students engaging in the development of the hypermedia program
featuring the various factors affecting the values of certain decades
exhibited an increased awareness of the inter relatedness of these
factors and that these changes in perceptions were due to the linking
nature of the hypermedia authoring language.
Oughten and Clevenger (1997) examined the effects of prior computer
experiences on graduate students cognitive levels as they
developed Hyperstudio projects. Students participating in their study
were asked to describe their use of Hyperlogo, HyperStudios
programming language and to create concept maps which depicted the
inter-relationships of the languages features and processes.
The researchers found that students with more experience focused on
the procedural aspects of the Hyperlogo while students with less
experience concentrated on the declarative aspects of the
language.
Heidenreich (1997) investigated the effects of using Hyperstudio, a
hypermedia authoring tool, and concluded that the Hyperstudio method
of instruction was not effective in raising the achievement level of
participating seventh grade social studies students. However, he
noted at students who used the authoring tool had a more positive
attitude toward learning, their partners, and cooperative learning in
general.
The positive effects of hypermedia on student perceptions were also
noted by Smith (1993) who investigated the hypermedia learning
environment from the perspective of high school students learning
about U.S. culture. Qualitative methods were used to determine
student perceptions about learning with hypermedia tools. Data were
collected from daily observations, interviews, student products,
formal and informal discussions and from the use of questionnaires.
The researcher concluded that there are a number of student
perceptions that are supportive of the use of a hypermedia learning
environment. Students indicated that the nature of hypermedia
products promoted considerable motivation. Hypermedia documents were
viewed by students as much more public than traditional work.
Producing a hypermedia document promoted understanding at a deeper
level than using a traditional assignment. Teachers participating in
the study concurred that learning was enhanced and all participants
in the program indicated the use of a hypermedia learning environment
to be advantageous.
In a similar descriptive experiment involving Constructivist use of
hypermedia Volker (1992) hypothesized that students would more
readily use primary source information if their goal was to create a
hypermedia project. During the three-month study students created the
program design and content treatment for portions of a math and
science hypermedia project while teachers served as content advisors.
Prior to the implementation and field test of the project,
participants completed attitudinal surveys on technology, their fear
of it, their level of knowledge before using the materials, and their
preference for working on their own.
Following the field test, 35 student users, 3 teachers, and 3 student
producers again responded to attitudinal survey instruments.
Participants reported that, compared to traditional instruction,
hypermedia was more interesting and that they preferred using it
instead of traditional textbooks, videotapes, etc. Teachers also
expressed enthusiasm for the motivational aspects of this approach,
claiming that students showed more interest in math and science.
Nearly all teachers recognized and supported the shift in emphasis
from teacher centered instruction to student based learning.
As a follow-up to a four year longitudinal study, Tierney and Kieffer
(1997) examined the impact of hypertext on students learning in
science and English classes. The first study compared the impact of
HyperCard stacks and regular textbook presentation of ideas in
biology on ninth grade students assigned to study these materials.
Students in the second study developed their own multimedia projects
in science and English courses using HyperCard and their standard
textbooks. The researchers analyzed interviews, videotaped
observations, and outcome and process measures and concluded that
hypermedia was an effective alternative to traditional print-based
texts. Specific findings indicated that (1) students suggested that
the hypertext offered alternatives to standard print texts, (2)
students considered hypermedia projects more interesting than
conventional texts and assignments, (3) hypermedia texts created by
students contained more main ideas, enlisted more illustrations, were
more multilayered, and provided clearer links between illustration
and the text, (4) hypertext allowed a flexible exploration of ideas
across several layers simultaneously, (5) students were motivated to
explore its possibilities and were willing to share new findings with
other students and (6) the use of hypertext supported the acquisition
of complex ideas and relations between those ideas involving science
tasks. Findings suggest that students view the advantages of
hypertext as providing a way to architecture a space that
affords different opportunities for engagement by others. Tierney
concluded that students engaged in developing hypermedia
developed notable experience in problem solving, integrating
visual representations of ideas with text, and developing refining
and restructuring ideas.
A considerable body of evidence suggests that when employed as a
constructivist tool, the use of hypermedia authoring software by
students can significantly increase their ability to retain
knowledge. Classic research by Geisman (1988) suggests that students
retain 20 percent of what they see, 40 percent of what they see and
hear, and 70 percent of what they see, hear and do. Since the use of
student produced hypermedia projects facilitate seeing, hearing and
doing by students, it is reasonable to assume that students
achievement on tests and retention of information will likely improve
when these approaches to learning are utilized by teachers in
constructive ways.
Hypermedia Construction and Information Problem Solving
Many educators and researchers advocate engaging students in
activities which require more than the simple application of rules
and procedures. Collins, Brown and Newman (1989) contend that few
educational resources, including interactive multimedia, are devoted
to higher-order problem solving activities.
To date, there appears to be little research into whether creative
thinking, problem-solving and other higher order thinking skills can
be developed through the use of hypermedia.(Herrington and Oliver,
1998).
Polya (1957) developed a model for problem-solving which included
four principal processes:
- Understanding the problem
- Developing a plan
- Implementing and carrying out the plan
- Looking back or accessing the effectiveness of the
plan
In the first step, understanding the problem, the problem solver
represents the problem, identifies the unknowns of the data, and
determines the conditions of the problem. The second step of the
model consists of developing a plan to solve the problem which meets
the conditions identified in step one. In this step, the problem
solver determines the relationships between the data and the
conditions of the problem in order to formulate a plan. In the third
step, the problem solver implements and executes a plan designed to
address the conditions of the problem. The final step of the model is
assessment or looking back. In this step, the problem solver
determines the correctness of the solution and assesses
whether it adequately meets the conditions of the problem.
Lehrers (1993) four-step framework for describing the cognitive
processes involved hypermedia development are in close alignment with
Polyas problem-solving steps. Lehrers planning stage can
be seen as being equivalent to Polyas first steps of
understanding the problem and devising a plan. According to Lehrer,
planning tasks include the determining the audience, content, topics,
relationships among topics, and interface design of the hypermedia
project. Lehrers transforming stage is similar to Polyas
second step of implementing and carrying out the plan. Transforming
tasks include accessing, interpreting, and determining how to
represent the information. Lehrers evaluation and revision
stages are equivalent to Polyas looking back step. Evaluation
and revision tasks include assessing the presentation and
organization of information, testing the program soliciting the
feedback of users, and, finally, restructuring the program to make it
more accessible or meaningful.
Reed and Liu (1994) compared the effects of programming in BASIC
versus developing hypermedia with HyperCard on graduate
students problem solving skills. They found that the BASIC
group increased their problem-solving skills, whereas, the HyperCard
group did not. Liu (1998) conducted a follow-up study to the Reed and
Liu (1994) study. The treatment in the Liu study specifically focused
on using HyperCards scripting features. Findings indicated that
the students increased their problem-solving skills from pretreatment
to post treatment.
McKenzie (1995) developed a research model and rubric for assessing
students information problem-solving skills based on the work
of Lehrer and several models including standards and rubrics proposed
by NCREL. McKenzies research cycle includes seven steps:
questioning, planning, gathering, sorting and sifting, evaluating and
reporting. According to McKenzie, the student passes through several
repetitions of the first six stages until sufficient information is
gathered to form insights worthy of reporting. Such research is based
upon a decision to be made or a solution to be proposed--- research
devoted to the exploration of essential questions. During the
research process, students maintain research logs which
track the reasoning used as well as the research actions taken while
cycling through the process.
Hypermedia Construction and Creative Thinking
The term creative thinking as used in this study refers
to a set of generalized mental abilities, identified by Torrance
(1984) commonly presumed to be brought into play in creative
achievements. Other researchers (Wallach & Kogan, 1965; Guilford,
1970; Hattie, 1980) have used the term in various contexts to refer
to such constructs as divergent thinking, productive thinking,
inventive thinking or imagination. Over a period of 25 years Torrance
and his associates have developed several batteries of test
activities at predicting the ability of an individual to behave
creatively. These batteries are incorporated into the Torrance Tests
of Creative Thinking (Torrance & Ball, 1984) which approach the
construct of creativity through the theoretical rationale of a series
of criterion referenced indicators which include fluency,
originality, abstractness of titles and resistance to premature
closure.
Harkow (1996) used a combination of strategies to improve creative
thinking skills in second- and third-grade gifted students. Sixteen
students were targeted for the intervention. Over a 12-week
implementation period, students participated in 90-minute
interventions twice weekly. The intervention was comprised of
30-minute creative problem-solving encounters with peers; 30 minutes
of computer software use to produce original writing, and to
experiment and create in open-ended settings; and a 30-minute period
of activities alternating between relaxation and imagery exercises
and the use of imagery in creative writing. There were four
objectives to the intervention: (1) increasing verbal and figural
creativity; (2) increasing figural and verbal fluency; (3) increasing
figural and verbal originality; and (4) increasing verbal
flexibility. The Torrance Tests of Creative Thinking, Figural and
Verbal Models and the Inventory of Creative Behaviors were used to
assess the impact of the intervention. Informal teacher observations
were conducted throughout implementation.
The Inventory of Creative Behaviors was completed weekly and at the
conclusion of the implementation period. Findings indicated that the
proposed number of students met the projected percentage of increase
of 80 percent or above in overall figural and verbal creativity,
verbal originality, and verbal flexibility.
Liu (1998) examined whether engaging elementary school students in
hypermedia authoring would promote their creative thinking. The
researcher reported significant correlations between hypermedia
development and scores on the Torrance Tests of Creative
Thinking.
SUMMARY OF THE REVIEW OF LITERATURE
In summary, theoretical and empirical research by cognitive
psychologists suggests that higher order thinking and problem solving
are complex human activities. Researchers have demonstrated that
cognition involves processing incoming external information and
encoding this information within memory. There are many types of
knowledge structures or schemata that we rely on for thinking and
these structures are constantly being modified based on new
information that is received and processed. Learning can be defined
as the process of modifying existing knowledge structures to
accommodate new information for specific purposes. Solving problems
involves understanding and representing the problem and then applying
different knowledge structures to develop strategies to determine a
solution.
When used appropriately, student produced hypermedia projects have
the potential to enhance student learning and subsequent achievement,
as well as cognitive and social skill development (Carver, 1992;
Wisnudel, 1994). This study will specifically examine SURWEB
hypermedia construction tools and the effect these tools can have
upon the construction of knowledge, creative thinking and information
problem solving. Is is hypothesized that the use of SURWEBs
web-based authoring environment by students will produce
instructional benefits very similar to those facilitated by
Hyperstudio and other software-based hypermedia authoring tools.
STATEMENT OF THE HYPOTHESIS
While few studies have specifically focused on sixth grade social
studies and hypermedia, a considerable body of research suggests that
the use of hypermedia authoring tools such as HyperCard and
Hyperstudio (applications whose functions are similar to the
hypermedia capabilities of SURWEB media tools) have had positive
effects on the ability of students to construct knowledge and solve
problems. Therefore, it was hypothesized that sixth grade social
studies students who create their own Media Shows will achieve
statistically significant higher ratings on performance based
measures of complex knowledge structures and information problem
solving. It was also hypothesized that students actively engaged in
hypermedia construction would score higher than their traditionally
instructed peers on a norm referenced measure of creative
thinking.
DESIGN OF THE STUDY
Participants
The participants in this study were 260 sixth grade social studies
students selected from a rural Southern Utah area intermediate
school. Students were described by their teachers as being primarily
average to slightly above average in ability. The sample was drawn
from the classrooms of ten social studies teachers who volunteered to
participate in the study. Forty students were randomly assigned to a
control group and the remaining students were randomly grouped into
five pods of forty to reduce the influence of individual teacher
input upon the outcomes being assessed.
Setting for the Study
Students assigned to the control groups met in traditional seventh
grade social studies classrooms. Students were seated in rows typical
of most sixth grade classrooms but were allowed to move their desks
together to work in small groups to complete assignments with
partners.
Students in the experimental groups met in computer labs. All labs
contained Macintosh computers which were networked and connected to
the Internet via a Local Area Network. An LCD projector or large
screen monitor was available for use by the teacher and students for
presentations.
Instruction for both groups was conducted during normally scheduled
times throughout the school day and both groups spent approximately
six weeks engaged in content of the lessons. Instructional methods
for the two groups were identical except for the use of the SURWEB
media show construction tool by students in the experimental
groups.
In the school participating in the study, computer labs were
frequently used by math, science, English, social studies and other
classes. Because of this widespread use by students, it is unlikely
that the novelty of the environment exerted either a positive or a
negative effect on the performance of students in the experimental
group. In a follow up survey conducted as part of the study,
virtually all students reported that they felt comfortable in the
computer lab and considered it a part of the normal, everyday school
environment.
Assessments
A number of researchers have pointed out that specialized assessment
techniques are required to analyze student thinking and problem
solving processes (Larkin and Rainard, 1984; Schoenfeld, 1985;
Marshall, 1995). Marshall claims that the emphasis should be placed
on addressing three critical questions: 1) What is it that
individuals learn?, How do they learn and remember
it?, and Can they use it again? (Marshall, pp.
171-172). Jonassen (1996) points out that traditional measures, such
as multiple choice tests are limited because critical thinking and
problem-solving are complex skills which are difficult to assess
because they are context dependent. Several researchers (Oughton
& Reed, 1998; Reed & Rosenbluth, 1992; Liu and Pederson,
1998) advocate the use of concept maps and metacognitive learner logs
and problem sets to explore these constructs.
Assessment of Knowledge Structures
A cognitive representation or concept map can be viewed as a
structural representation of a learners knowledge in a subject
domain (Ruiz-Primo & Shavelson, 1995). Jonassen (1993) describes
concept maps as the spatial representation of ideas and their
inter-relationships that are stored in memory (p.3). Jonassen,
Beissner and Yacci (1993) claim that concept maps generated by
learners at different points in a curriculum may be used to identify
learners progression in the attainment of differentiated and
organized structural knowledge(p. 157). The intended use of
concept maps is to tap into a learners cognitive
structure and to externalize, for both the learner and the teacher to
see, what the learner knows (Novak and Gowin, 1984, p. 40).
To assess the impact of SURWEB hypermedia construction on the
development of complex knowledge structures, students participating
in the study were asked to produce concept maps of their knowledge of
states or foreign countries during a six week social studies unit. As
an indicator of cognitive growth, concept maps constructed at the
beginning of the activity were compared to maps constructed at the
end of the unit.
Assessment of the concept maps was based on an adaptation of Novak
and Gowins (1984) scoring system. This scoring system is
outlined below.
- 1 point for each example of specific and valid processes or
concepts
- 1 point for each relationship or link between two
concepts
- 2 points for each labeled link or relationship
- 3 points for each valid cross link that did not illustrate
a synthesis between concepts
- 5 points for each valid level or segment of the
schemata
- 10 points for each valid cross link which showed a
meaningful connection between one segment of the schemata and
another segment.
The concept maps were used to determine, in part, whether the
structural knowledge of the students mental models became more
sophisticated over the five-week period.
Student projects and research questions from the learner logs were
also reviewed by evaluators who assessed their content using the six
general categories of Blooms (1956) Cognitive Taxonomy. To
simplify the process of scoring and to insure a higher degree of
reliability, Blooms six categories (Knowledge, Comprehension,
Application, Analysis, Synthesis and Evaluation) were compressed into
three rankings: Lower Level Thinking (Knowledge and Comprehension),
Medium Level Thinking ( Application and Analysis) and Higher Level
(Synthesis and Evaluation). Three evaluators were employed to code
the project and learner log data. Scores of one two and three were
awarded according to the cognitive levels represented by the data
set. If the ratings of the first two evaluators were not identical,
the data sets were scored by a third evaluator. In cases where the
first three evaluators codings were not at identical
categories, a fourth evaluator coded the set until agreement was
reached. Reliability of the scores was calculated using the codings
of all evaluators. The inter-rater reliability coefficient was
.90.
These dependent measures were used to answer the first research
question: What is the effect of SURWEB hypermedia development on
students ability to construct complex knowledge
structures?
Assessing Information Problem Solving
Jonassen (1996) claimed that having students document their problem
solutions encourages them to focus on, classify and evaluate
the steps and strategies they used to solve a problem (p. 277).
By implementing this method of assessment, patterns of skill
acquisition and types of metacognitive strategies that students
employ can be monitored (Jonassen).
Learner logs which included concept maps of student research
procedures and descriptions of information problem solving processes
were used to examine the impact of SURWEB hypermedia construction on
the development of problem solving skills. Logs entries were
collected at the beginning and end of the unit and comparisons used
to assess student growth in information problem solving skill.
Students maintained learner logs throughout the study in which they
listed research questions and recorded their progress toward solving
information problems. At the end of the six week unit students were
asked to produce a research processes concept map and to describe the
stages involved in conducting their research. Students
information problem-solving skills were assessed using a seven step
rubric proposed by McKenzie (1995). Details of the rubric appear
below.
1. Questioning: A researcher recognizes decisions,
issues and problems when exploring a topic.
- 5 points: Discovers independently an issue or problem which
needs a decision or solution after exploring a topic
- 3 points: Formulates questions about topics with adult help
to evaluate the question to focus on issues and problems
- 1 point: Relies upon adults to state questions and
topics
2. Planning: A researcher identifies sources of
information likely to build understanding.
- 5 points: Selects high quality sources independently and
efficiently
- 3 points: Selects sources with mixed success
- 1 point: Wanders from source to source without questioning
which
source will be most helpful in answering research
questions
3. Gathering: A researcher collects and stores
information for later consideration.
- 5 points:Collects and organizes important information for
retrieval independently
- 3 points: Collects information with some degree of
organization
- 1 point: Loses track of most important information
4. Sorting: A researcher reorganizes information so that
the most valuable becomes readily available to support
understanding.
- 5 points: Creates structure which provides a coherent and
clear focus
- 3 points: Creates partial organization of information
- 1 point: Leaves information as gathered
5. Synthesizing: A researcher recombines information to
develop conclusions and decisions.
- 5 points: Creates an original conclusion or decision based
on information
- 3 points: Reorganizes and combines conclusions and
decisions of others
- 1 point: Restates the conclusions and solutions of
others
6. Evaluating: A researcher determines whether the
information gathered is sufficient to support a conclusion.
- 5 points: Tests solutions and decisions to see if
supporting information is adequate
- 3 points: Looks for missing information
- 1 point: Reaches a hasty conclusion
7. Reporting: A researcher translates findings into a
persuasive, instructive, or effective product(s).
- 5 points: Creates and presents an original product which
effectively addresses original problem or issues
- 3 points: Provides a product which offers some insight with
regard to the original problem or issues
- 1 point: Shares only the work or thoughts of others
Assessors reviewed the research process component of
students learner logs and evaluated their final projects using
the above rubric. This dependent measure was used to answer research
question two: What is the effect of SURWEB hypermedia development
on students acquisition of information problem solving
skills?
Assessment of Creative Thinking
The Torrance Test of Creative Thinking (TTCT), a norm referenced
test of creative thinking ability was used to assess the impact of
SURWEB hypermedia construction on the development of creative
thinking skills. The TTCT was administered to students in both
control and experimental pods and results compared to assess
differences in performance between the two groups. Several scoring
reliability studies indicate that reliability coefficients above .90
for the TTCT can be achieved if scorers are properly trained in the
use of the scoring guidelines which accompany the tests. To insure
reliability of the assessment, each test was independently scored by
two evaluators. In cases where discrepancies between scores for each
of the five scales were identified, tests were rescored by a third
evaluator. This method yielded an inter-rater reliability of .93.
The content and construct validity of the TTCT has been explored in a
factor analytic study by Mourad (1976), a comparative study by
Rungsinan (1977), and in a developmental study by Alieldin (1982).
These studies assembled considerable theoretical and empirical
research to support the content validity of each of the indicators
used in the battery.
Two major predictive validity studies have also been conducted
(Torrance, 1980; Torrance & Wu, 1981). These studies published in
1981, involved young adults tested at the time they were in
elementary school and followed up in 1980, twelve years later. Three
criteria of creative behavior were used: the number of publicly
recognized creative achievements reported, the quality ratings of
five judges of the three most creative achievements described by the
subjects and the creativity of their future career images. Pearson
product-moment coefficients of correlation were obtained for each of
the three separate criteria with each of the creativity indicators.
Measures of Fluency, Originality and Elaboration yielded a validity
coefficient of .51, a value statistically significant at the .01
level.
Scores obtained through the use of the TTCT were used to compare the
creative thinking skills of students in the control and experimental
groups at the end of the 5-week unit. This dependent measure was used
to answer research question three: What is the effect of SURWEB
hypermedia development on students creative thinking skills.
Experimental Design
Students and teachers were selected to participate in this quasi
experimental action research study which employed a pretest/post test
control group experimental design (see Table 1). Subjects in both
groups were pretested prior to the treatment and post tested
following completion of an instructional unit on states and foreign
nations.
The school participating in the study reported high attendance rates
during the period of time in which data were collected. As a control
for subject mortality, only students who completed all assessments
were included in the analysis of data. Subject mortality posed no
threat to the validity of the study in that 93 percent of the
subjects included in the original sample concluded the unit and
completed all assessments. With the exception of a few daily
absences, the consistency of the control and experimental groups
remained unchanged.
For the purpose of the study, the presence or absence of SURWEB Media
Show construction as an instructional strategy was the independent
variable. Student performance on the three cognitive assessment
instruments (TTCT, concept maps and information problem solving
rubrics) were the dependent variables. The research design is
detailed in the table 1 which appears below.
Table 1. Experimental Design
|
Group
|
Assignment
|
n
|
Treatment
|
Pretest
|
Post test
|
|
C1
|
Control
|
40
|
Regular
Instruction
|
Concept Map
Research Rubric
|
Concept map
Research Rubric
TTCT
Project Rubric
|
|
E1
|
Experimental
|
40
|
SURWEB
Instruction
|
Concept Map
Research Rubric
|
Concept map
Research Rubric
TTCT
Project Rubric
|
|
E2
|
Experimental
|
40
|
SURWEB
Instruction
|
Concept Map
Research Rubric
|
Concept map
Research Rubric
TTCT
Project Rubric
|
|
E3
|
Experimental
|
40
|
SURWEB
Instruction
|
Concept Map
Research Rubric
|
Concept map
Research Rubric
TTCT
Project Rubric
|
|
E4
|
Experimental
|
40
|
SURWEB
Instruction
|
Concept Map
Research Rubric
|
Concept map
Research Rubric
TTCT
Project Rubric
|
|
E5
|
Experimental
|
40
|
SURWEB
Instruction
|
Concept Map
Research Rubric
|
Concept map
Research Rubric
TTCT
Project Rubric
|
Procedures
In April and May of 2000, 260 sixth grade social studies students in
ten classrooms were randomly selected to one of two groups or pods.
One pod of forty students was randomly selected to be the control
group while the remaining five pods comprised the experimental
group.
Two volunteer teachers taught a unit on U.S. states and foreign
nations to students in the control group using project-based
instruction. Students in the control group utilized traditional
library resources and classroom materials to research and prepare a
report on either a state or a foreign country.
The remaining eight regular program social studies teachers taught
the same unit to students assigned to the experimental group. The
primary difference between the two groups was the use of SURWEB Media
Show construction tools by students of the experimental group.
Instead of the traditional library and classroom materials, students
in the experimental group researched the topic and prepared an
on-line report using the SURWEB tools.
Both groups were instructed by their regular program teachers. Both
groups used the same textbooks and were given equal amounts of time
to research the content of the unit and to produce a report on their
findings. Learning objectives and assessments for both groups were
identical
Due to the limited number of computers available in the labs,
students worked in small groups. In order to prevent the student
grouping variable from influencing the results of the study, students
in the control group also worked in small groups to complete their
projects and learning tasks.
At the beginning of the unit, both groups were given pretest
assessments which included the construction of concept maps of prior
knowledge, completion of an initial learner log entry and a measure
of information problem-solving skills. Upon completion of the
geography unit, all students were completed alternative form post
test assessments.
In addition to the pretest/post test data, demographic and
descriptive data were collected using the SURWEB Student Technology
Survey questionnaire designed to provide information regarding
students level of experience with technology. Data generated by
the questionnaire were used to explore relationships between levels
of experience, gender, and other factors which might influence the
dependent variables.
At the beginning of the study, students assigned to the two groups
were given a pretest in order to obtain benchmark data and to check
initial group equivalence. Scores on the pretest measures for the
control and experimental groups were compared using a two-tail t-test
for paired samples. This statistic can be used to determine if
significant differences exist between the means of two samples. The
test makes three assumptions about the scores obtained in
causal-comparative research. The first assumption is that scores form
an interval or ratio scale of measurement. The second is that scores
in the population under study are normally distributed. The third
assumption is that score variances for the population under study are
equal. Borg (1989) maintains that even if the assumptions underlying
the t-test are violated, the t-test will still provide, in most
instances, an accurate estimate of the significance level for
differences between sample means. Analysis of the data obtained by
using the pretest/post test instruments suggest that the three
assumptions were met. Examination of the means and a paired samples
t-test indicated no statistically significant difference between the
two groups (Table 2). This finding supports the assumption that there
were no initial differences between the ability levels of the two
groups.
Table 2. Comparison of Experimental and Control Group Pretest
Scores
Concept Maps (First Assessment)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG1RP
|
40
|
.259
|
14.7000
|
4.214
|
.666
|
|
E1LOG1RP
|
40
|
.259
|
14.6250
|
3.985
|
.630
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
.0750
|
4.994
|
.790
|
.09
|
39
|
.925
|
95% CI (-1.522, 1.672)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG1RP
|
40
|
.205
|
14.7000
|
4.214
|
.666
|
|
E2LOG1RP
|
40
|
.205
|
14.5250
|
3.870
|
.612
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
.1750
|
5.103
|
.807
|
.22
|
39
|
.829
|
95% CI (-1.457, 1.807)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG1RP
|
40
|
.306
|
14.7000
|
4.214
|
.666
|
|
E3LOG1RP
|
40
|
.306
|
15.1500
|
3.431
|
.542
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.4500
|
4.546
|
.719
|
-.63
|
39
|
.535
|
95% CI (-1.904, 1.004)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG1RP
|
40
|
-.001
|
14.7000
|
4.214
|
.666
|
|
E4LOG1RP
|
40
|
-.001
|
15.0250
|
3.355
|
.530
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.3250
|
5.389
|
.852
|
-.38
|
39
|
.705
|
95% CI (-2.049, 1.399)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG1RP
|
40
|
.075
|
14.7000
|
4.214
|
.666
|
|
E5LOG1RP
|
40
|
.306
|
14.9000
|
3.153
|
.498
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.2000
|
5.070
|
.802
|
-.25
|
39
|
.804
|
95% CI (-1.821, 1.421)
Concept Maps (First Assessment)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP1
|
40
|
-.076
|
37.1750
|
14.875
|
2.352
|
|
E1MAP1
|
40
|
-.076
|
36.7350
|
13.799
|
2.182
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
.4500
|
21.046
|
3.328
|
.14
|
39
|
.893
|
95% CI CI (-6.281, 7.181)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP1
|
40
|
.250
|
37.1750
|
14.875
|
2.352
|
|
E2MAP1
|
40
|
-.076
|
36.4500
|
14.102
|
2.230
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
.7250
|
17.756
|
2.807
|
.26
|
39
|
.798
|
95% CI (-4.954, 6.404)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP1
|
40
|
.060
|
37.1750
|
14.875
|
2.352
|
|
E3MAP1
|
40
|
.060
|
38.4250
|
14.753
|
3.123
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
.-1.2500
|
24.008
|
3.796
|
-.33
|
39
|
.744
|
95% CI (-8.928, 6.428)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP1
|
40
|
.053
|
37.1750
|
14.875
|
2.352
|
|
E4MAP1
|
40
|
.053
|
37.9500
|
14.158
|
2.239
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.7750
|
19.990
|
3.161
|
-.25
|
39
|
.808
|
95% CI (-7.168, 5.618)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP1
|
40
|
-.163
|
37.1750
|
14.875
|
2.352
|
|
E5MAP1
|
40
|
-.163
|
41.8250
|
18.198
|
2.877
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-4.6500
|
25.313
|
4.002
|
-1.16
|
39
|
.252
|
95% CI (-12.745, 3.445)
ANALYSIS OF THE DATA
A t-test for paired samples was used to compare the performance of
the two groups on the post tests administered at the end of the unit.
This statistical technique was deemed appropriate because the
assumptions required for the use of the parametric t-test were met,
e.g. the data were interval.
To answer research question one: What is the effect of SURWEB
hypermedia development on students ability to construct complex
knowledge structures?, a paired samples t-test was used to
compare the means of the experimental and control groups on two
assessments: 1) holistic scoring of the learner log research
questions and the completed project using a scoring model based on
the categories of Blooms Taxonomy, and 2) concept map scores
assigned using an adaptation of a rubric developed by Novak and Gowin
(1984).
To answer research question two: What is the effect of SURWEB
hypermedia development on students acquisition of information
problem solving skills?, a paired samples t-test was used to
compare the performance of students in the control and experimental
groups on information problem solving measures included in the
learner log. Student problem-solving concept maps and research
process descriptions were scored using a seven step rubric proposed
by McKenzie (1995).
To answer research question three: What is the effect of SURWEB
hypermedia development on students creative thinking skills?,
scores on the Torrance Test of Creative Thinking were compared
using a paired samples t-test to compare the creative thinking
abilities of students in the control and experimental groups
RESULTS AND DISCUSSION
Upon completion of the unit, post test scores of the two groups on
the various assessments were compared using the statistical methods
described above. As Tables 3 through 7 indicate, there were
significant differences between the performance of the control and
some of the experimental groups participating in the study.
Table 3 compares the assessment scores of the control and
experimental groups on the final concept map included in the learner
log. This map provided indication of the complexity of knowledge
structures developed by students in both groups during the five week
unit. Four of the five experimental groups scored significantly
higher on the concept map assessment than their control group
counterparts. Although not statistically significant, the mean of
experimental group three was higher than the control groups and
approached significance at the .05 level.
On the basis of the data outlined in Table 3, it was concluded that
the use of SURWEB hypermedia development tools by students resulted
in an increased ability to construct complex knowledge structures.
This conclusion supports the findings of Tierney and Kieffer (1997)
who reported that the use of hypertext supported the acquisition of
complex ideas and relations between those ideas. Tierney's conclusion
that students engaged in producing hypermedia "developed notable
experience in problem solving, integrating visual representations of
ideas with text, and developing, refining and restructuring ideas"
was also supported.
Table 3. Comparison of Experimental and Control Group Concept Map
Scores
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP2
|
40
|
-.010
|
43.4250
|
18.628
|
2.945
|
|
E1MAP2
|
40
|
-.010
|
55.2750
|
18.799
|
2.972
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-11.8500
|
26.593
|
4.205
|
-2.82
|
39
|
.008
|
95% CI (-20.355, -3.345)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP2
|
40
|
.107
|
43.4250
|
18.628
|
2.945
|
|
E2MAP2
|
40
|
.107
|
57.5250
|
22.534
|
3.563
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-14.1000
|
27.659
|
4.373
|
-3.22
|
39
|
.003
|
95% CI (-22.946, -5.254)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP2
|
40
|
.022
|
43.4250
|
18.628
|
2.945
|
|
E3MAP2
|
40
|
.022
|
51.4500
|
23.880
|
3.776
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-8.0250
|
29.963
|
4.738
|
-1.69
|
39
|
.098
|
95% CI (-17.608, 1.558)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP2
|
40
|
-.062
|
43.4250
|
18.628
|
2.945
|
|
E4MAP2
|
40
|
-.062
|
53.7000
|
21.369
|
3.379
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-10.2750
|
29.206
|
4.618
|
-2.23
|
39
|
.032
|
95% CI (-19.615, -.935)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CMAP2
|
40
|
-.002
|
43.4250
|
18.628
|
2.945
|
|
E5MAP2
|
40
|
-.002
|
58.3500
|
23.655
|
3.740
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-14.9250
|
30.144
|
4.766
|
-3.13
|
39
|
.003
|
95% CI (-24.565, -5.285)
Holistic scoring of the learner log research questions and the
completed project using a scoring model based on the categories of
Blooms Taxonomy provided additional evidence supporting the
positive effects of the SURWEB media construction tool on the
development of complex knowledge structures. Student projects and
research questions from the learner logs were scored using a three
point scale based on the six general categories of Bloom's (1956)
Cognitive Taxonomy. To simplify the process of scoring and to insure
a higher degree of reliability, Bloom's six categories (Knowledge,
Comprehension, Application, Analysis, Synthesis and Evaluation) were
compressed into three rankings: Lower Level Thinking (Knowledge and
Comprehension), Medium Level Thinking ( Application and Analysis) and
Higher Level (Synthesis and Evaluation). Table 4 compares the
cognitive level rankings of learner log research questions and final
projects developed by students included in the control and
experimental groups.
Table 4. Comparison of Experimental and Control Group Learner
Log/Project Assessment Scores
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CPROJ
|
40
|
-.063
|
1.8750
|
.463
|
.073
|
|
E1PROJ
|
40
|
-.063
|
2.1000
|
.441
|
.070
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.2250
|
.660
|
.104
|
-2.16
|
39
|
.037
|
95% CI (-.436, -.014)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CPROJ
|
40
|
.467
|
1.8750
|
.463
|
.073
|
|
E2PROJ
|
40
|
.467
|
2.0500
|
.504
|
.080
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.1750
|
.501
|
.079
|
-2.21
|
39
|
.033
|
95% CI (-.335, -.015)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CPROJ
|
40
|
.355
|
1.8750
|
.463
|
.073
|
|
E3PROJ
|
40
|
.355
|
2.0750
|
.526
|
.083
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.2000
|
.564
|
.089
|
-2.24
|
39
|
.031
|
95% CI (-.380, -.020)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CPROJ
|
40
|
.124
|
1.8750
|
.463
|
.073
|
|
E4PROJ
|
40
|
.124
|
2.0750
|
.616
|
.097
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.2000
|
.723
|
.114
|
-1.75
|
39
|
.088
|
95% CI (-.431, .031)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CPROJ
|
40
|
.131
|
1.8750
|
.463
|
.073
|
|
E5PROJ
|
40
|
.131
|
2.100
|
.632
|
.100
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.2250
|
.733
|
.116
|
-1.94
|
39
|
.060
|
95% CI (-.460, .010)
Three of the five experimental groups scored significantly higher
than the control group on the holistic assessment of questions from
the learner log and evaluations of final projects. Students in the
experimental group consistently functioned at higher levels of
Blooms Taxonomy than their peers in the control group. On the
basis of this evidence it was concluded that the use of the SURWEB
media construction tool exerts a positive effect on the development
of complex knowledge structures.
This finding is consistent with the conclusions of Smith (1993) who
reported that producing hypermedia documents promoted understanding
at a deeper level than using traditional assignments.
Table 5 provides data comparing the performance of students in the
control and experimental groups on information problem solving
measures included in the learner log. Assessments of student
problem-solving concept maps and research process descriptions were
based on a seven step rubric proposed by McKenzie (1995).
Table 5. Comparison of Experimental and Control
Group
Information Problem Solving Scores
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG2RP
|
40
|
.194
|
17.2000
|
3.532
|
.558
|
|
E1LOG2RP
|
40
|
.194
|
18.7750
|
2.983
|
.472
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-1.5750
|
4.156
|
.657
|
-2.40
|
39
|
.021
|
95% CI (-2.904, -.246)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG2RP
|
40
|
.181
|
17.2000
|
3.532
|
.558
|
|
E2LOG2RP
|
40
|
.181
|
17.8250
|
3.257
|
.515
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-.6250
|
4.348
|
.688
|
-.91
|
39
|
.369
|
95% CI (-2.016, .766)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG2RP
|
40
|
.265
|
17.2000
|
3.532
|
.558
|
|
E3LOG2RP
|
40
|
.265
|
18.5000
|
2.298
|
.363
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-1.3000
|
3.667
|
.580
|
-2.24
|
39
|
.031
|
95% CI (-2.473, -.127)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG2RP
|
40
|
.153
|
17.2000
|
3.532
|
.558
|
|
E4LOG2RP
|
40
|
.153
|
18.4000
|
2.687
|
.425
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-1.2000
|
4.096
|
.648
|
-1.85
|
39
|
.071
|
95% CI (-2.510, .110)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CLOG2RP
|
40
|
.083
|
17.2000
|
3.532
|
.558
|
|
E5LOG2RP
|
40
|
.083
|
18.8500
|
2.370
|
.375
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-1.6500
|
4.086
|
.646
|
-2.55
|
39
|
.015
|
95% CI (-2.957, -.343)
Learner logs which included concept maps of student research
procedures and descriptions of information problem solving processes
were used to examine the impact of SURWEB hypermedia construction on
the development of information problem solving skills. Three of the
five experimental group scored significantly higher on the assessment
of information problem solving skills than the control group. Though
not statistically significant, the mean scores of groups two and four
were slightly higher than those of the control group. The
comparatively low performance of group two may have resulted from
lack of familiarity with concept maps on the part of several students
in the sample. Evaluators noted that several students in this group
failed to complete the task of concept map construction. Though
inter-rater reliability was relatively high (.81) the scoring rubric
used to assign individual scores is not without a degree of
subjectivity. It is also possible that subtle variations in scoring
may have resulted in the failure to detect statistically significant
differences in group fours performance. On the basis of this
analysis, it was concluded that SURWEB hypermedia construction has a
positive impact on the development of information problem solving
skills by students.
This finding supports the conclusions of Liu (1998) and Reed and Liu
(1994) that hypermedia development by students leads to an increase
in problem-solving skills. Tierney and Kieffer (1997) also reported
that students engaged in hypermedia development exhibited gains in
problem-solving ability.
Table 6 compares the performance of control and experimental groups
on the Torrance Test of Creative Thinking. Three of the five
experimental groups scored significantly higher than the control
group on this measure. The remaining two groups produced means that
approached significance at the .05 level.
Table 6. Comparison of Experimental and Control Group Performance
on the
Torrance Test of Creative Thinking
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CTTCT
|
40
|
-.212
|
82.5050
|
10.175
|
1.609
|
|
E1TTCT
|
40
|
-.212
|
87.4000
|
9.179
|
1.451
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-4.8950
|
15.080
|
2.384
|
-2.05
|
39
|
.047
|
95% CI (-9.718, -.072)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CTTCT
|
40
|
-.089
|
82.5050
|
10.175
|
1.609
|
|
E2TTCT
|
40
|
-.089
|
87.3675
|
9.053
|
1.431
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-4.8625
|
14.211
|
2.247
|
-2.16
|
39
|
.037
|
95% CI (-9.407, -.318)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CTTCT
|
40
|
.118
|
82.5050
|
10.175
|
1.609
|
|
E3TTCT
|
40
|
.118
|
86.7475
|
10392
|
1.643
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-4.2425
|
13.658
|
2.160
|
-1.96
|
39
|
.057
|
95% CI (-8.611, .126)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CTTCT
|
40
|
-.113
|
82.5050
|
10.175
|
1.609
|
|
E4TTCT
|
40
|
-.113
|
89.1350
|
8.582
|
1.357
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-6.6300
|
14.032
|
2.219
|
-2.99
|
39
|
.005
|
95% CI (-11.118, -2.142)
|
Variable
|
Number of Pairs
|
Correlation
|
Mean
|
SD
|
SE of Mean
|
|
CTTCT
|
40
|
.176
|
82.5050
|
10.175
|
1.609
|
|
E5TTCT
|
40
|
.176
|
86.6500
|
12.423
|
1.964
|
Paired Differences
|
Mean
|
SD
|
SE of Mean
|
t-value
|
df
|
2-tail Sig.
|
|
-4.1450
|
14.611
|
2.310
|
-1.79
|
39
|
.081
|
95% CI (-8.818. .528)
Analysis of scores on the Torrance Test of Creative Thinking support
the conclusion that the creative thinking abilities of students who
use the SURWEB media tool are enhanced. This conclusion is supported
by the findings of Liu (1998) and Harkow (1996) who reported
correlations between hypermedia development and increased scores on
tests of creative thinking.
Limitations
It would be difficult to generalize the results of this study to all
classrooms due to the limitations of the research design. Future
research should address the application of the SURWEB media
construction tool in other curricular areas and explore the issue of
how hypermedia construction can best be utilized in broader classroom
contexts. In addition, issues of individual learning preferences and
gender differences should also be addressed. While further research
is called for, the potential of the SURWEB approach to enhance the
development of complex knowledge, problem-solving and creative
thinking appears promising. When used in a constructivist,
learner-centered classroom environment, SURWEB can be a powerful tool
for facilitating higher order thinking skills.
References
Aiken, M.C. (Ed.). (1992). Media use in education. The
Encyclopedia of Educational Research, 2, 805-813
Ali-el-din, M.T. Torrance indicators of creative thinking: A
developmental study (Doctoral dissertation, University of Georgia,
1978). Dissertation Abstracts International, 1979, 39 (7-A), 4129.
(University Microfilms No 70-10, 151).
Ashton, P.T. Editorial. Journal of Teacher Education, 43 (5),
322.
Barker, Bruce O. (1999). SURWEB: A Visual Literacy Tool Which
Promotes Engaged Learning for Teachers and Students. Paper
presented at the Annual National Conference on Creating the Quality
School (Memphis, TN, March 26-28, 1999).
Beal, J.W. (1995). The Civil War: An example of integrating
Linkway into the social studies curriculum. Hyper Nexus 5,
3-6.
Bodner, G.M. (1986). Constructivism: A theory of knowledge.
History of Chemical Education, 63 (10), 220-229.
Boone, R., & Higgins, K. (1992) Hypermedia applications for
content area study guides. Reading and Writing Quarterly:
Overcoming Learning Difficulties, 8, 379-393.
Borg, Walter R. and Gall, Meredith D. (1971). Educational
Research: An Introduction.
New York: David McKay and Company. 303-307.
Bowers, D. & Tsai, C. (1990) HyperCard in educational research:
An introduction and case study. Educational Technology,
19-24.
Brigham, F.J., Hendricks, P.L., Kutcka, S.M. & Schuette, E.E.
Hypermedia supports for student learning (Report No. IR 017 028).
Indianapolis, Council for Exceptional Children. (ERIC Document
Reproduction Service No. ED 378 965).
Carver, S. (1992). Learning by hypermedia design: Issues of
assessment and implementation. Educational Psychologist, 27
(3), 380-400
Collins, A., Brown, J.S., & Newman, S.E. (1989). Cognitive
apprenticeship: Teaching the crafts of reading, writing and
mathematics. In L.B. Resnick(Ed.). Knowing, Learning and
Instruction: Essays in Honour of Robert Glaser. Hillsdale, NJ:
LEA. pp. 453-494.
Conklin, J. (1987). Hypertest: An introduction and survey.
Computer, 17-41.
Davies, D. & Pritchard, A. (1993). The multimedia
teleschool&emdash; linking European initiatives. Journal of
Computer Assisted Learning, 9, 107-112.
Dempsey, J.V. & Rasmussen, K.L. (1993). Instructional design
and development competencies in a new academic program.
Erikkson, G. (1988). Thinking in visual images in the information
age&emdash;the changing faces of the school. Gifted Education
International, 5 (2), 97-103.
Fabris, M.E. (1993). Using multimedia in the multicultural classroom.
Journal of Educational Technology Systems, 21 (2),
163-170.
Forman, D.C. (1995). The use of multimedia technology training for
business and industry. Multimedia Monitor, 13 (7), 19-25.
Galloway, Jerry P. (1994) Effect of quicktime multimedia tools on
writing style and content. In J. Willis, B. Robin & D.A. Willis,
(Eds.). Technology and Teacher Education Annual, 1994. (pp.
94-97). Charlottesville, VA: Association fo the Advancement of
Computing in Education (AACE).
Garrett, L.N. & Weiner, B.J. (1998). Learning on the
internet. Teaching for Success, 10 (7), 4-