Scaffolding Students' Problem-Solving Processes in an Ill-Structured Task Using Question Prompts and Peer Interactions

Ge, X., & Land, S. A. (2003). Scaffolding students’ problem-solving processes in an ill-structured task using question prompts and peer interactions. Educational Technology Research and Development.51(1), 21-38.

指導教授： Chen, Ming-puu報 告 者： Chen, Hsiu-Ju報告日期： 2007. 04. 21

Background literature (1/2) Many researchers (e. g., Bransford, Brown, & Cocking,

2000; Bransford & Stein, 1993; Jonassen, 1997) have emphasized the importance of engaging students in complex, ill-structured problem-solving tasks, which are intended to help students see the meaningfulness and relevance of what they learn and to facilitate transfer by contextualizing knowledge in authentic situations.

Yet previous research has pointed to student deficiencies in problem solving, for instance, a failure to apply knowledge from one context to another (Gick, 1986; Gick & Holyoak, 1980), especially when solving ill-structured problems (Feltovich, Spiro, Coulson, & Feltovich, 1996).

Students’ difficulties in problem solving have been attributed to both limited domain and metacognitive knowledge (Brown, 1987).

Background literature (2/2) Vygotsky (1978), learners should be guided or scaffolded by

a “more capable peer” to solve a problem or carry out a task that would be beyond what they could accomplish independently.

Externalized support during problem solving has also been accomplished through strategies such as modeling (Schoenfeld, 1985), prompting (Scardamalia, Bereiter, McLean, Swallow, & Woodruff, 1989; Scardamalia, Bereiter, & Steinbach, 1984), and guided, student-generated questioning (King, 1991).

Such strategies have been found to be effective in fostering comprehension, monitoring, problem solving (e.g., Palincsar & Brown, 1984; Scardamalia et al., 1989), and reflective thinking (Lin, Hmelo, Kinzer, & Secules, 1999).

Previous research has seldom investigated the effectiveness of those strategies in supporting ill-structured problem solving.

Theoretical background -Ill-Structured Problem-solving process Ill-structured problems are defined as having vague goals

(Chi & Glaser, 1985; Voss & Post, 1988) that permit multiple solutions or solution paths (Kitchner, 1983).

well-structured problems have single solutions, optimal solution paths, and structured goals (Sinnott, 1989).

Voss and Post, problem representation involves examining concepts and relations of a problem, isolating the major factors causing the problem and its

constraints, and recognizing divergent perspectives. Monitoring and evaluation are required throughout the

process, from identifying the essence of the problem to selecting the best goal for solving it (Sinnott).

Cognition and Metacognition in SolvingIll-Structured ProblemsSolving ill-structured problems requires domain-

specific knowledge (Voss & Post, 1988; Voss, Wolfe, Lawrence, & Engle, 1991) as well as structural knowledge (Jonassen, Beissner, & Yacci, 1993).

Domain-specific knowledge is content knowledge consisting of cognitive components such as propositional information, concepts, rules, and principles.

Structural knowledge is knowledge of how concepts within a domain are interrelated and requires integration of declarative knowledge into useful knowledge structures (Jonassen et al.).

Question Prompts as a Scaffolding Strategy Question prompts have been found effective to help

students focus attention and monitor their learning through elaboration on the questions asked (Rosenshine, Meister, & Chapman, 1996).

King (1991)specifically emphasized the role of question prompts in scaffolding metacognition.

She grouped questions into three metacognitive categories: planning, monitoring, and evaluation, which closely paralleled the general problem- solving model (problem identification, searching for a solution, implementation of a solution, and evaluation, Bransford & Stein, 1993; Gick, 1986).

Recently, researchers have integrated prompts into computer-based instruction to facilitate metacognition (Davis & Linn, 2000; Hannafin, Land, & Oliver, 1999; Lin & Lehman, 1999).

Peer Interaction as a Scaffolding StrategyPrevious research (e.g., King, 1991; Palincsar

et al., 1987; Webb, 1989) indicated that peer interaction could be an effective scaffolding strategy.

Webb (1982, 1989) found that, when learners were required to give explanations to and ask questions of each other, learning was enhanced.

Greene and Land (2000) found that peer interaction during open-ended learning was effective when group members offered suggestions, negotiated ideas, and shared their experiences.

Purpose of the Study (1/5) To investigate the effects of :

(a) question prompts

(b) peer interactions

(c) the combined strategies of question prompts and peer interactions in scaffolding undergraduate students’ problem-solving processes in an ill-structured task.

The problem-solving outcomes and processes investigated were:

(a) problem representation

(b) problem solution

(c) making justifications

(d) monitoring and evaluation, which feature the major processes of ill-structured problem solving (e.g., Jonassen, 1997; Sinnott, 1989; Voss, 1988; Voss & Post, 1988).

Purpose of the Study (2/5)There were two versions of peer interactions:

guided (with question prompts) versus unguided.

1. Does using question prompts and peer interactions separately or in combination affect student problem-solving processes (problem representation, solution development, justification, and monitoring and evaluation of solutions) in an ill-structured task?

2. Does using question prompts and peer interactions separately or in combination influence student cognition and metacognition in the process of developing solutions to ill-structured problems?

Purpose of the Study (3/5)The following hypotheses were generated from

Question 1 and were tested:Hypothesis 1. Students receiving question prompts will

demonstrate better problem-solving performance in an ill-structured task than those who do not receive question prompts in problem representation, solution development, justification, and monitoring and evaluation of solutions.

Previous research has shown that question prompts can facilitate explanation construction (King, 1991, 1992; King & Rosenshine, 1993), planning, monitoring, and evaluation (Davis & Linn, 2000; King, 1991; Schoenfeld, 1985), and making justifications (Lin & Lehman, 1999).

Purpose of the Study (4/5)Hypothesis 2. Students working with peers will

demonstrate better problem-solving performance in an ill-structured task than those working individually in problem representation, solution development, justification, and monitoring and evaluation of solutions.

Peer modeling and interaction have been found to facilitate self-regulation (Brown & Palincsar, 1989), distribute expertise, and foster reflection on multiple perspectives (e.g., Roschelle, 1992; Webb, 1989).

Purpose of the Study (5/5)Hypothesis 3. Students working with peers and also

receiving question prompts will demonstrate better problem-solving performance in an ill-structured task than all the other treatment groups in problem representation, solution development, justification, and monitoring and evaluation of solutions.

Previous research has shown that guided peer interaction is more effective than unguided peer interaction (e.g, Johnson et al., 1990; King, 1991; King & Rosenshine, 1993; Palincsar et al., 1987).

Method- design A quasi-experimental study, supplemented by

comparative, multiple-case studies, was employed to investigate the two research questions.

Using both quantitative and qualitative methods helps a researcher to seek triangulation of the results from different data sources.

Question 1, in an ill-structured task in the four problem-solving processes: (a) problem representation (b) solution development

(c) justification (d) monitoring and evaluation of solutions. The comparative, multiple-case studies (Yin, 1989)

served two purposes: (a) to supplement and explain findings for Research Question 1 (b) to explore Research Question 2 to gain insights into students’ problem-solving processes through think-aloud protocols, interviews, and observations.

Participants and Context of the Study

117 undergraduate course in information sciences and technology (IST)

There were two lecture sessions and one laboratory session per week.

75 min lecture sessions 115-min laboratory session Two major goals of the laboratory sessions:

(a) developing basic information technology skills through skill module exercises (e.g., spreadsheets and database management systems)

(b) developing problem-solving and collaborative learning skills through case studies.

The Quasi-Experimental Study(a) peer-question (PQ)(b) peer-control (PC)(c) individual-question (IQ)(d) individual- control (IC).

Sampling and Treatment Assignment during a 115-min laboratory session. Each class section was randomly assigned as an intact

group to one or two of the treatment conditions. Because there were only three classes to be used for four

different treatment conditions, one class had to be split into two conditions.

Lou, Abrami, and d’Apollonia’s (2001) literature review, small groups of 3 to 4 members were more effective than larger groups, which suggested little difference between 3-member and 4-member groups.

Measurement and Treatment Material (1/3)

The question-prompt treatment material (Appendix A) was a list of 10 major questions generated from the problem by the course professors, the IST experts

Measurement and Treatment Material (2/3) The question prompts were then organized and categorized

into four types:

(a) problem representation prompts

(b) solution prompts

(c) justification prompts

(d) monitoring and evaluation prompts The rubric system had four major constructs, each

measuring one of the four problem-solving processes in an ill-structured task:

(a) problem representation

(b) developing solutions

(c) making justification for generating or selecting solutions

(d) monitoring and evaluating the problem space and solutions

two specific attributes:

(a) constructing argument (ranging on a scale of 0–2–4)

(b) providing evidence (ranging on a scale of 0–1–2–3)

Measurement and Treatment Material (3/3)

In evaluating providing evidence: 0: was assigned if no evidence was provided, 1: was assigned if evidence provided was not

plausible, 2: was assigned if evidence was based on

hypothetical examples 3: was assigned if evidence was based on

previous experience or real examples. In evaluating constructing argument; 0: was assigned if no argument was constructed 2: was assigned if an argument was poorly

constructed 4: was assigned if an argument was well

constructed.

Administering the Study Sessions

The first author and a colleague in three 115-min laboratory sessions in the same week.

All the study sessions were conducted in a classroom equipped with laptop computers and LCD projectors, where the participants had regular lectures and lab sessions.

In the first study session, administered to the PQ condition, the participants were told to work on the problem-solving task in their preassigned groups.

The problem task materials and the question prompts were posted on the course Web site.

In the second study session, The problem-solving task material was delivered to them in the same way as to the PQ group, but they were not provided with question prompts.

The third study session was conducted with the IQ and IC conditions.

The participants were instructed to work individually. No hints or assistance associated with the problem were

provided.

Quantitative Data Analysis 2×2 multivariate analysis of variance (MANOVA) Wilks’s Lambda F ( = .05) The use of MANOVA was justified because of an overall

correlation among the four dependent variables (problem representation, developing solutions, making justifications, and monitoring and evaluation) indicated by the results of Pearson’s correlation, which were significant at the .01 level .

As shown by the results from the Box’s M Test and the Levene’s Test, the assumption of equal variance was met at the .05 alpha level, and thus met the MANOVA testing assumption that the residual errors follow a multivariate normal distribution in the population.

Qualitative Data Analysis Pseudonyms were used for the eight selected cases

to protect the identity of the participants. The four problem-solving processes, and student

behaviors and reactions in peer or prompting conditions were organized with data display matrixes (see examples in Appendix B).

Appendix B

Result-Quantitative Outcomes Table 1 summarizes the descriptive statistics for the four

problem-solving processes (the dependent variables) by two factors:

(a) individuals vs. peers

(b) question prompts vs. no question prompts.

Question Prompting Effects

Two way MANOVA , F (4, 48) = 17.371, p < .001, ç2 = .591, which supported the hypothesis that students who received question prompts would perform significantly better than students who did not receive question prompts.

Reveal significant effect of Problem-solving processes:

problem representation, F (1, 51) = 51.051, p < .001, MSE = 2.227, ç2 = 0.500;

generating solutions, F (1, 51) =21.429, p < .001, MSE = .960, ç2 = .296;

making justification, F (1, 51) = 32.929, p < .001, MSE =1.424, ç2 = .392;

monitoring and evaluation, F (1, 51) = 21.336, p < .001, MSE = 3.658, ç2 = .295.

Peer Interaction Effects

The two-way MANOVA did not reveal significant effects for peer interactions, F (4, 48) = 2.308, p = .071, ç2 = .161, and thus failed to support the hypothesis.

To further explore the trend shown in the means, we ran a post hoc univariate test.

The result showed that the peers significantly outperformed the individuals in problem representation, F (1, 51) = 6.991, p = .011, MSE = 2.227, ç2 = .121.

Interactive Effects of Question Prompts and Peer Interactions The result did not show significant interactive effect

of the two strategies, F (4, 48) = 1.298, p = .284, ç2 = .098, and thus failed to support the hypothesis.

Qualitative Findings

Table 2 presents the raw scores of the cases on the solution report

Effects of the Question Prompts Compared with the no-question-prompt conditions, the

students who received question prompts engaged in the following cognitive and metacognitive activities:

(a) making intentional efforts to identify factors, information, and constraints during the problem-representation process;

(b) organizing and planning for the solution process and articulating solutions explicitly;

(c) constructing arguments grounded in factors identified during problem representation and providing justification for each suggestion proposed;

(d) intentionally evaluating the selected solutions, comparing alternatives, and justifying the most viable solution.

The qualitative results showed that the question prompts had an effect of directing student attention to important information they might have overlooked, thus facilitating awareness of what is known and not known.

Effects of Peer Interactions The effects of peer interactions mainly involved

(a) building upon each other’s ideas to develop solutions,

(b) providing multiple perspectives

(c) asking questions and providing feedback

(d) benefiting from distributed cognition

Discussion Both the quantitative and qualitative findings on the effects of

the question prompts support the hypothesis that question prompts can facilitate not only well-structured problem solving, as shown by the studies of Schoenfeld (1985) and King (1991), but also ill-structured problem solving.

The prompts may have provided a strong advantage for students in the question-prompt conditions to perform well on the assessment, because the prompts focused on the same problem- solving processes as the rubrics.

The quantitative results of the students’ problem- solving performance conflict with previous studies that showed positive effects of peer interactions in developing learner cognition and metacognition (Palincsar et al., 1987; Webb, 1982, 1989) and in improving performance and achievement (Johnson et al., 1985, 1986, 1990).

Modified replication of the study is necessary, with increased number of treatments and sample size, random sample assignment, and use of pretest and posttest measures to ensure equivalence of different conditions.

Thank you !